Saturday, February 11, 2012

Generating LODs for your worldspace

Edit: I have rewrite this guide a bit.

LODs are made by four file types: meshes (.btr files), textures (.dds files), normal (.dds files) and a .lod file for the world space. There are several ways to create all of these file, you can use the CK or other programs depending on what works on your computer.

The steps in this guide are:

Duplicating textures (optional, see point 2 under Making Meshes), converting to tga
Making Meshes, Textures and Normals
Making .lod file (optional)

Duplicating textures in the CK (optional)

If you have painted the terrain with any texture apart from default texture, you need to do some extra things to make Oscape work properly. This only has to be done in the file used for LOD making.

1. Open your esp and the Skyrim.esm in the CK.
2. In the Object window, under Miscellaneous, TextureSet. Duplicate every texture you would like to have as LOD.
3. In Miscellaneous, LandTexture. Open every LandTexture you duplicated, change the file to your copied TextureSet.
4. Save

Making the Meshes, Textures and Normals

I had problems with the crashes in the CK when I tried generating textures. Oscape will make meshes, normals and diffuse/color, I would recommend using it, much faster than struggling with the CK. For me the CK worked, even with a 16x16 quad world (took several hours!).
You can find Oscape here
For working with Oscape I followed this easy to follow tutorial made by Maegfaer.
(Warning, Oscape will not work on Vista. If you are running on a Vista machine, try using the CK to make the LOD files, or contact someone that can make the LOD files for you, like me. I can't promise I will always have time to make them right away.)

For landscape textures to show in LODs they need to be put in the Source folder as tga files.
1. Nico cioN over at AFKmods has been very nice and made all the files you need for Skyrim vanilla files.
http://afkmods.iguanadons.net/index.php?/topic/4079-skyrim-vanilla-lod-source-files/
2. Under Steam\steamapps\common\source\ make a new folder TGATextures. Copy the Landscape folder from the downloaded file.

If you are using custom landscape textures you will need to convert these to TGA files too.
a) Copy you dds texture files to the Source\TGATextures\Landscape folder
b) You can use a command line batch program called Nconvert to make the tga files. Download here http://www.xnview.com/en/nconvert/ (download section is at the end of the page). Put the Nconvert.exe in the folder containing the textures.
c) Make the batch file by create a txt document. Write exactly:
nconvert -out tga *.dds
Change the file extension to .bat (file should change icon to gears).
(you can also download the file here). Put the bat file among the textures to be converted together with the Nconvert.exe file. To run the bat file simply double click on it. Now you should have tga files of all your dds files and you can delete the dds files.

Continue to generate the landscape LODs with Oscape.

1. For Oscape to work, you need to convert your esp into an esm. 
You can use Oblivions Gecko to make an esm from an esp. (TES4Gecko can't do anything else that will work for Skyrim files.) Put the esm in the Data Folder.

2. Start Oscape, under Game, select Skyrim. This will give you a list of all the masters and plugins. Click in the box of your esm.
If you are using any texture apart from default texture, you need to have that TextureSet and TextureSet in your esm (see Duplicating textures in the CK) or select an esm with these resources in Oscape together with the esm you want to make LOD from. 
(Thanks to Hyperactivehild for figuring this out!)

3. Press the Fill button, and in the drop down list next to the button select your world space.

4. Check the feature-map and surface-map boxes.

5. Under Outputs, press Browse and select the output folder, can be any folder. You must rename the raw file to the same name as the worldspace. Press Extract and wait.

6. Oscape will now show you the Preview tab. The blue is the water level. No need to change anything if you can see your land (and sea) as you want, press Accept.
If you only see land or blue water you may need to change the sea level in the “Sealevel” field.

7. Oscape will now show you the Generator tab. The Target of Max resolution value is dependent on how detailed your world is, a completely flat world can have a very high number. Ethatron (the maker of Oscape) has recommended never to use a value over 1000 pts per cell in your worldspace. I have used 8 000 000 for my 16x16 quad world for test proposes, but when I want my finished world I will use more. There are of course limits to how high number you can use depending on amount of memory on your computer. You have to test what works for your world. If a value works, try something higher.
8. Check the Meshes box and then check (1/1), 1/2, 1/4, 1/8, optimize of overdraw, and NIF boxes.
9. Check the Normals box and then check 1/1, 1/2, 1/4, 1/8, and DirectX boxes. Choose low, standard or high resolution textures.

10. Check in the Colors box and then check 1/1, 1/2, 1/4, 1/8, and DirectX boxes. Choose low or standard resolution textures. (If the Color box is grey, you forgot to check the surface-map box in step 4. )
11.  Press Generate and wait for Oscape to generate the files.

12. Oscape will now show you the Installer tab. Make sure your Worldspace is selected. check the boxes of LOD Settings Generate, Level0, Level1, Level2 and Level3. This will transfer the right files to the right folders.

The Oscape settings
If you run into errors, transfer the textures and meshes manually.
The texture files (.dds) should be put in:
Skyrim/Data/Textures/Terrain/"worldspace"
and the mesh (.btr) in to
Skyrim/Data/Meshes/Terrain/"worldspace"

You might have to create the folders if you do not have them already.
Copy all the .btr and .dds files.
Meshes are made at different levels: 4, 8, 16 and 32. All are needed by the game. Level 32 makes up the map.





Alternatively: Meshes with the Creation Kit:
In the Creation kit go to World, World LOD. A new window will open where you can choose between different settings, then press Generate. One of the settings under meshes is Background or High Priority, this determines the speed of generation (use High Priority, it will save you a lot of time). My Creation kit crashed when I minimized it,  but restarting the generation picked up where it left off. There is a tool that will Clear Scratch Files, and I guess that it would clean out any unfinished LODs.The LODs are put under Skyrim/Data/Meshes/Terrain/"worldspace" as .BTR files.

Making the lod file (optional)

Without the .lod file none of the textures or meshes will show up in game. Oscape should make this file, but sometimes you might want to make a .lod file that covers larger areas that your actual worldspace. You can make the .lod file with a simple command line program that will write a lod file for you. Download it here: lodFileGenerator (download from nexus)

Well that's it, you should have LODs now :)

It's easy to see the potential beauty, even with default texture and no flora. I just love the sky :) 


Now the LODs mostly works in game, but I still have some problems with purple textures.

This is due to lacking textures. Some of the texture .dds files had black squares in them, resulting in missing textures purple.










As I said before, the world map is made from the level 32 mesh and textures. First image is with missing normal and color textures, the second with these textures.















A big thanks to all the nice modders figuring these things out at the Bethesda forums :)

Running into troubles?
I made a test on how different problems you can run into would look in the game. First off, if you see no LODs at all, you probably don't have a working YourWorldSpace.lod file in the lodsettings folder. 
1. This is how it looks when everything is working fine :)
2. Missing textures (.dds), you get a purple color over all the LODs.
3. Missing normals (_n.dds), the terrain looks diffuse with no details.
4. Missing meshes (.btr), the LODs that show are black (probably will not fit your terrain).

LODs for objects and trees
If you are making LODs with trees and objects, take a look at this video made by Alexander J. Velicky. It explains everything nicely :)

When it comes to LOD trees, beware that the CK can not do LOD trees for any trees that stand in cells beyond  -/+ 32. This will result in a wall of trees at the edge of that cells. Tes5Edit will solve this problem, have a look here.


/E

47 comments:

  1. Hello!

    Thanks for blog, it's very usefull.

    But can you tell a bit more about making new worldspace and heightmap?
    I tryed to make my own worldspace, but after making esp in tesannwyn, the lanscape become torn, many parts on landscape are flying, many mountians are in cells.

    I dont know what to do.
    Can you be more detailed on it?


    Alsom post more your landscape shotsm please.

    Thanks a lot!

    ReplyDelete
  2. I'm making a more detailed description of my workflow right now for you, post it as soon as its done :)

    From your description of how your world behaves I had one idea of what could be wrong. If the height difference is to large between two cells they can become torn. My first attempt at a height map I even got large tears within the cells. You can try two things:
    1) reduce the "height" in gray scale of you image, do this by lowering the brightness in your image editing software, alternatively use the scaling function in TESAnnwyn (-s 0.2 or lower)
    2) make the size of the image larger, this will stretch out the image and in this way reduce the height differences.

    Hope this helps
    /E

    ReplyDelete
  3. I tried Tes4qLOD and every time i try to make a world LOD it comes out screwed up

    cells are all at different heights ... and the map is fragmented.

    all i want is for my terrain to be viable at great distances ... to see mountain ranges from a valley or to stand on a mountain and view a lake in the distance ... but the Skyrim World LOD creator is broken and i cant get Tes4qlod to work right and i cant even download Oscape because the file has been removed or something like that ... any suggestions would be greatly appreciated.

    ReplyDelete
    Replies
    1. Just to make sure you know :) TES4qLOD can only make the textures, not the actual LOD mesh files, for this you need Creation kit or Oscape.

      In this forum thread Ethatron (creator of Oscape) posted a link that lets you download a good version of Oscape, were it says "I think you can try this."
      http://forums.bethsoft.com/topic/1348239-creating-new-worldspacesdiscussion-thread-3/page__view__findpost__p__20327847

      Do you could have a problem with 1) the LODs are low triangle count (or what its called), this would make a fractionated LODs with lots of holes, or 2) that you have an issue with broken cells due to to high height differences in the height map.

      /E

      Delete
  4. Oh, okey.

    Well, I did it!
    Thanks fot helping, Bisen.

    ReplyDelete
  5. This is by far the best treatment of this subject matter I've seen! Will you please spam this everywhere on the creation kit forum? A quick read of this and the heightmap post would save a lot of people a lot of trouble.

    ReplyDelete
  6. you say that if the height difference is too large between two cells they can become torn, so what's a good way to make cliffs?

    ReplyDelete
    Replies
    1. You can still make a cliff, you can have a lot of height difference before you get a tear between cells. There is no mountain rock textures in the game that you can paint the terrain with, so right now the only way to make cliffs are using placeable objects. And Skyrim has some different cliff objects that you can. I really miss the variation if ground textures available in Oblivion.
      /E

      Delete
  7. How exactly did you fix the missing texture problem? I'm using tes4qlod to make textures, but I'm getting that problem. It also seems like the normal maps are facing the wrong way or something, because everything in LOD is dark, but I can make out details as if light is coming from the bottom. It also looks like the normal maps aren't assigned to the right LOD meshes, but maybe that's because of the missing textures problem?

    Here's a screenshot of it:
    http://i.imgur.com/MEBLf.jpg

    ReplyDelete
    Replies
    1. Nevermind my post, I just decided to use Oscape for textures only, and that worked. It looks like the blue and green channels are switched in the normal maps created by tes4qlod. Everything worked fine with Oscape.

      Delete
  8. I have the missing textures too like above created by tes4qlod. I can't use Oscape because i get an error message every time saying cannot load map output or something. Any other ideas on how to fix this?

    ReplyDelete
    Replies
    1. At what stage when working with Oscape do you get this error message?
      I haven't tried Tes4Lod in a long while. If you are getting purple texture you are missing the color/texture map for that cell.

      Delete
  9. Whenever I try using Oscape, the plugin list shows up blank. I've been told that this is because of Vista. Is there any workaround, or a different program that can work instead?

    ReplyDelete
    Replies
    1. No work around that I know of. If you want I can make the LOD files, just send me the esp/esm in an email or a downloadable link: My email is: elin.overnas(at)gmail.com.

      Delete
  10. I am wondering the same thing, I got an amazing world built but can't get the LOD to work because Oscape won't work.

    ReplyDelete
    Replies
    1. I'm afraid I don't have a good answer for you. I haven't heard of any other program apart from the CK. The CK can work for some people but not others. I can run Oscape for you if that will help, sent the esp/esm to elin.overnas(at)gmail.com and I will see what I can do :)

      Delete
  11. Just made an awesome worldspace, but the LOD looks foggy and the landscape seems to turn black at a semi-short distance from me, see it here - http://cloud.steampowered.com/ugc/559817385110939153/B0A3C0E9982049AED9CF59F9F62C3823E79094A6/

    ReplyDelete
    Replies
    1. Hello,

      Thanks you very much Bisen for your tutorial!

      I have the same problem on my new word and i don't know why...
      And in game, when i leave in a new cell, the black texture come again!
      Sorry for my bad english.

      Delete
    2. I think the black you see is a meshes problem, that is the brt files. Why I don't really know, could be that Oscape made something wrong or that the name of the worldspace is not correct or that the meshes are in the wrong folder.
      I added a section in the end of the guide where you can see how different LOD problems will look.

      Delete
  12. Hi.

    I did something stupid... ^^'

    I've manually changed the "Sealevel" and "Landlevel" values in the Oscape "Preview" tab and I cant get to know what the default one are.

    In TESAnnwyn I've set my water level at -19100 and in FNVEdit I've set -14000, -19100 and -14000....

    ReplyDelete
  13. Hi!
    First i want to thank you for this great tutorial.
    But i have a question since im a noob at this :)
    I cant seem to find a way to get my eps. file that i get from TESannwyn to become a esm. so i can use it in oscape. Is there something that im missing?

    Thanx again for this great tutorial!

    ReplyDelete
    Replies
    1. Actually I can successfully make LODs from my .esp's, with beta6 version of Oscape.

      Anyway, if you really want to turn your .esp to .esm, download TES4Gecko; I have version 15.2, which cant manage Skyrim files, but can still change esp files to esm.
      Remember that TES4Gecko doesnt rewrite the original file, instead it makes a copy of it and changes it from esp to esm. So you start with a .esp file and end up with a .esp and a .esm.
      Same thing backwards.

      Delete
  14. Hi, thanks for the tutorial, I wouldn't have been able to make any LOD's without it. One small question - when I do this, my world map goes invisible in the new area. Do you have to do something special to create it?

    ReplyDelete
    Replies
    1. What do you exactly mean with "goes invisible in the new area"?

      Delete
  15. That wasn't very descriptive, sorry. What I mean is this: http://i48.tinypic.com/142v2tk.jpg

    ReplyDelete
    Replies
    1. The world map is the same as the level 32 LODs generated by Oscape. Looks like you have a mesh but no color map for that level. Opened the textures for the LOD (Data/Textures/terrain/"worldspace") and see how they look. The level 32 LODs color textures should be something like 128x128px in size. I remember I just to had problems with 1x1px color images when I let Oscape install the LOD, but that should be fixed in the latest version.
      The texture should also be gray or another color if you used other than default textures.

      Delete
    2. I opened the 32s up and they were ok, 120x120 as you say. Turns out the problem was that I had made my LOD file to small, missing out the -64 cells in the x-axis. Regenerating the LOD file fixed this.

      Thanks again for both your amazing tutorials and for your fast replys!

      Delete
  16. I followed your tutorial but no matter what i do my LOD stays purple/black

    ReplyDelete
    Replies
    1. have you looked at the .dds files Oscaped generated?

      Delete
    2. I'm not really sure what you mean by looked at, but i tried to both manually and Oscape install them but neither worked.

      Delete
    3. Well what I was angling for is if the files are broken in some way. My texture and normal files are 11 or 12 kb in size, and are 128 x 128 pixels large. Open the dds files in an image viewer that supports that format, and have a look, if they only 1x1 pixels, then you would get the problem you describes. If that's not the case, I have to think some more on what could be the problem.

      Delete
  17. You were right, all of the .dds files are only 1 pixel and only 1kb in size.

    ReplyDelete
    Replies
    1. Ok, then something went wrong with Oscape. I remember reading about this problem earlier this spring, but I thought it was solved in a later version of Oscape. Right now I don't remember what was the source of this issue.

      Delete
    2. I tried to re-generate the files with oscape but now it is impossible for me to "check" the colors field. It is simply greyed out.

      Delete
    3. ^Sorry for that one, i discovered what i did wrong

      Delete
    4. Ok so now the LOD is no longer purple, now it is grainy and no water is generated in the LOD.

      Screenshot: http://i.imgur.com/Ii6dp.jpg

      Delete
    5. The LODs looks to me like they should.
      What water level do you have in you worldspace? You should have water level at -14000 and default land level below that, say -16000 (that is the sea floor level). For me Oscape can only do water LODs if the sea level is at -14000, others have made it work for other levels as well but i do not know how.

      Delete
    6. It seems like i was missing the "-" which then forced me to recreate the .esp with TESAnnwyn because of the waterlevel-terrainlevel mismatch.

      But it works now thanks, and great tutorial :D

      Delete
    7. Yes, I have done that mistake lots of times too :)

      Delete
  18. Hey Bisen, I'm trying to generate LOD for my world space. Its far from finished, but I'd like to see it with LOD to make sure I'm completely happy with the some of the areas I've created before I move on.

    I'm on vista so can't use oscape. I've had the ck running, but it's been going non-stop for 4 days now. My estimates are that it will be weeks before its finished.

    It only seems to be using 600Mb of RAM, is there someway to make it use more to speed it up?

    Note:It's a 16x16 quad world (large - I wanted vast forests to get lost in) ^.^

    Thanks in advance.

    ReplyDelete
  19. I can do the LODs for you if you like, for a 16x16 quad it will take some hours, but that's defiantly faster than weeks...
    Send the esm to elin.overnas@gmil.com, and I will fix it :)

    ReplyDelete
    Replies
    1. Better if I actually type the right adress... elin.overnas@gmail.com

      Delete
    2. Hello, thank you that would be really great of you if you could do the LODs for me. It'll maybe be a few days before I can email my esm to you. My internet isn't working very well at the moment, but someone will be coming out to fix it soon. I'll email you when it's fixed! Thanks so much, your first city is looking good too!

      Delete
  20. Hey Binsen,i've a big problem with my LOD.
    With my unexisting LOD -.-
    I tried everything...your guide...Alexanders video...the guide on the creationkit.com site...
    Either i've purple objekt LOD and no terrain, or working objekt LOD and purple terrain, or neither objekt LOD nor terrain.
    If i use the CK generator, it destroys my whole worldspace.

    And there is also a weird thing. My map size is only 512x512. But Oscape shows 2048x4096.

    ReplyDelete
  21. Hi Binsen
    Ive been working on a worldspace for 2 months now and I've run across an LOD issue. It seems my land LOD is missing textures (purple)
    I've tried using Oscape's install, and manually dragging the .dds to the directory, but it still not working

    attached is picture
    http://i1279.photobucket.com/albums/y540/arquey/ScreenShot68_zpsf0c99bd0.jpg

    ReplyDelete
    Replies
    1. Hi sorry to doublepost. But I've solved my problem.
      Must have been something to do with generating CK tree and static LOD AFTER already having done land LOD with Oscape

      If anyone else has this problem the solution is to move the .dds manually

      The missing textures can be copied from the steamapps/common/source/ddst/terrain/"worldspace" and pasted into your data/textures folder.

      Delete
  22. Duplicating textures in the CK (optional)
    If you have painted the terrain with any texture apart from default texture, you need to do some extra things to make Oscape work properly. This only has to be done in the file used for LOD making.

    1. Open your esp and the Skyrim.esm in the CK.
    2. In the Object window, under Miscellaneous, TextureSet. Duplicate every texture you would like to have as LOD.
    3. In Miscellaneous, LandTexture. Open every LandTexture you duplicated, change the file to your copied TextureSet.
    4. Save


    Wouldn't this overwrite all of those textures for other mods if they used the same LandTexture? Isn't that bad...?

    ReplyDelete