From Heightmap to Worldspace in Skyrim

Update (23 April 2013)
I’m sorry to be the bearer of bad news to all of you who wish to do really large worlds. There is a bug in the game engine that limits the functional size of a worldspace to 4096×4096 pixels (128×128 cells centered around cell 0,0). You can read more about what this bug effect here.
___________________________________________________________________

GK asked if I could outline the world and height map creation in more detail. I will try to make an in depth description of my workflow. There will be repetitions of some of the things I have described earlier, but perhaps in a more collected manner :)

(Updated 07 Dec 2013)
First a brief summary of the steps I go through from image to worldspace:

• make Height map in Photoshop, save as TIFF
• run it in Geocontrol 2 to make natural looking erosion
• back to Photoshop to lower the gray scale color range, save as RAW
• run TESAnnwyn to make an .esp
• change LOD water, default water and land levels in CK
• (convert esp to esm with CK)
• Previous post: Make LOD meshes and Textures 

Making the height map: 
(Updated 07 Dec 2013)
The size in pixels of the image will determine the size of the world in Oblivion/Skyrim. The game is built up by quadrants, with 32 x 32 cells in each. One quadrant is 1024 x 1024 pixels. As a reference, Skyrim is 3808 x 3008 pixels (not all of which is playable). A world can be up to 16 x 16 quadrants (16384 x 16384 pix) in size, but due to the bug I mentioned in the header, worlds larger than 4 x 4 quadrants (4096x4096) will have huge problems.

There are several ways to make the height map, paint it yourself or use a landscaping software (like GeoControl, EarthEdit, etc).

EDIT

Don't have Photoshop? Well I think I found a way to still make a 16bit height map using Gimp, ImageJ and GeoControl. See new post.

This is my workflow for a 4096 x 4096 pixel sized world:

Photoshop
1.  I have made my original height map in Photoshop. Make a new image with the size you want, File -> New, size 4096 x 4096 pix, 16-bit and gray scale.
(Alternatively change the settings of an image you already have: Image -> Mode -> gray scale and 16-Bit.)
2.  Paint your terrain, I did this in layers, first black for the lowest to almost white as the top of a huge mountain. In the color tab you can chose the % gray scale. 100% is Black and 0% is White, so 50% gray has 50% black in it and so on. Whenever I talk about % gray, this is what I mean. 

(Oblivion and Skyrim can not handle a full range black to white height map like this, but I will fix this in a later step)
What can be good to know when you paint your world is where the sea level will be. When I paint my height map in 100-2% gray scale, the sea level will be between 89% and 90% gray scale. (This will change later to 99-98% when I lower the gray scale range to fit what Skyrim can handle.)

3.  Make sure to flatten the image if you worked in layers, Save file as TIFF

GeoControl
4.  Import Image to GeoControl2. Generation -> Terrain Import -> Image Import
GeoControl2 can only handle images up to 4096 x 4096 pixels. There are a lot of settings in GeoControl2, DON'T PANIC :) 
5.  Under the General tab -> Filter tab -> Add -> erosion system or erosion system inverse -> choose the erosion typed you want. I have used: Thin flow deep - inverse (all levels, strength 100), Thin flows sediment (only levels 1024 and 2048, strengths 40), Thin flow deep (all levels, strength 100) and Mountain ridged (all levels, strength 100).
I have uploaded some of my more used GeoControl settings if anyone want them.

EDIT:

Under the tabs Generation, General, General, General Settings, set "Random Noise up to level" to 0 and "Protect Terrain up to Level" to at least 64.

A high number for "Random Noise up to level" will introduce features that are not in your original map and a low number at "Protect Terrain up to Level" will lead to a flat terrain.

Some ideas on what the different settings can do

6.  Press Generate, depending on you computer this can take a lot of time.

7.  When it's done and you like the results go to Generation -> Terrain Export ->chose TIFF 16 bit greyscale.

Using GeoControl this way gives me less control over the final outcome of my world, but for me that's okay, in fact I have been happily surprised by my landscape when I'm in the game, as it turned out completely different to what I expected sometimes. 

Photoshop

8.  Open file in Photoshop, to flip the image vertical (TESAnnwyn will flip it back) go to Image -> Image Rotation -> Flip Canvas Vertical 
9.  The last thing we need to do is change the grayscale range. This is because Oblivion/Skyrim can only handle images that is from 0-76% gray (see previous post where I try to explain why and how I think it works, 1, 2, 3)
Go to Image -> Adjustments ->  Brightness/Contrast change to -150 brightness, and repeat again with -150 brightness. 
10.  Check with the Eyedropper tool that the lightest parts of the height map is not above 76% gray (With "above" I mean anything in the range 0-75%)
11.  Save as RAW, with the settings : Header 0, IBM PC
Now your height map is ready :)

Note: Important rules for the Heightmap
(Updated 07 Dec 2013)
1. If the change in height is too great there will be tears between cells!

This image posted by someone that had problems with this issue. Here you can see a classic example of this problem. Try making the height map image darker by lowering the brightness even more.

I have made some tests to see when I have too much height differences and get breaks between cells, I apologize if this is getting a bit technical. In the image below I have different gray scale color ranges from 100-95%, 100-90%, 100-85% and 100-80%. The color range is spread out over approximately 3 or 5 cells, and distinct color borders is where a break between cells will happen (*). Note that a change in height from 100-90% over 3 cells is almost a vertical wall :)  

2. Sea levels will be between 99% and 98% gray color in the finished height map (When using the setting -h -22192 in TESAnnwyn, and setting the default land levels to -27000, default water levels to -14000, see description below) 

TESAnnwyn: making the esp

I use the TESAnnwyn (Skyrim compatible version) to make esp from raw files, much easier than using the Creation kit. TESAnnwyn is a command line program that is really easy to use.
TESAnnwyn.exe -i Skyrim -p 1 -b 16 -d 4096x4096 -x -64 -y -64 -h -22192 -w NameOfWorldspace YourImage.raw
 -i sets the game, -p RAW format, -b 16 bit image, -d size in pixels, -x and -y cell offset in x and y directions, -h -22192 lowers the land, specific to work with water LODs in Skyrim.

h -22192 will give about 8000 units of the land under sea level, that is a lot. I like to have more above the sea level, so I use -16048 (2048 units under sea level) or -18096 (4096 units under sea level).

Now move the .esp to you Skyrim/Data folder.

(Updated 07 Dec 2013)
To make LODs work we have to change the default water and land settings for the worldspace. This can be done with the Creation kit (alt 1).
There are also other ways to do this, but you will only need them if you are doing a world that is larger that 4096 x 4096, and as mentioned above larger world spaces will have huge bugs.
The alternative methods are: Creation Kit with version control (alt 2), or an editor for Fallout New Vegas, FNVEdit (alt 3, you need a registered copy of Fallout New Vegas for it to work).

Alt 1) using the Creation kit (for .esp files)
For a world up to 4096x4096 pixels you can open and edit the esp in the Creation kit.
1.  In the Creation kit, open your esp (set as active).
2.  Under World -> Worldspace
3.  Select the worldspace, edit: LOD water height to -14000.0000, Default Land Height to -27000.0000 and Default Water Height to -14000.0000
Set Default Land Height to different thing dependent on value given in TESAnnwyn: -18096.0000 for h -18096 or -16048.0000 for h -1604.
4.  Save the esp

Update 2012-07-28
Alt 2) using Version Control (with the Creation Kit)
For a large worldspace like my 106384x16384 pixels there is no way I can open it as an active esp in the Creation kit, but you can be opened as an esm. What you need is to set up Version Control, follow Meagfaer's guide on how to do this: http://www.creationkit.com/Version_control

1.  Make the esp in to a esm, I use TES4Gecko. You also have to set up this esm according to the version control guide.
2.  In the CK, open only your esm (not the Skyrim.esm).
3.  Under World -> Worldspace
4.  Select the worldspace, edit: LOD water height to -14000.0000, Default Land Height to -27000.0000 and Default Water Height to -14000.0000
Set Default Land Height to different thing dependent on value given in TESAnnwyn: -18096.0000 for h -18096 or -16048.0000 for h -16048.
5.  Save the esp
6.  Load your esm and the new esp as active.
7.  Click on the version control botton (left of the load botton)
8.  Select the worldspace entry, click Check out. Select the worldspace entry again, click Check out.
9.  You can close the new window that pops up, and then select the your esm that the changes will be saved in to.
10. I was also asked if I would set the ID for WaterType and LOD WaterType, choose Yes.
11. Choose NO when asked about Check In Data files.
Now the esm has changed water height and so on.

Alt 3) using FNVEdit (for esp and .esm files)

1.  Put the esp file in the FalloutNewVegas/Data folder, open FNVedit and select the file
2.  Change the LOD water height to -14000.0000, Default Land Height to -27000.0000 (alt -16048.0000 or -18048.0000 ) and Default Water Height to -14000.0000.

3.  Close the FNVEdit and wait for it to save. It will create a backup and a save file. Rename the save file to .esm again.

(Updated 07 Dec 2013)
Optional: Converting the esp to an esm
You can trick the Creation Kit into making a functional esm.
1. Make a copy of your esp file and rename the extension to .esm
2. Open the new file as active in the CK, the file will still be listed among the esp as a plugin.
3. Save. The CK will now make the adjustments needed to make the file a proper esm. Next time you open the CK, the file will be listed as an esm.

Now you are ready to make LODs (described in a previous post).

Hopefully this procedure will work for you too :)

EDIT 2012-03-03

After Skyrim updates some esm file will no longer work, the game will crash.
* esm that is dependent on the Skyrim.esm will work (example, the esp was loaded together with Skyrim.esm and then saved, then converted to an esm)
* esp will work
* As default the CK can only open one esm at the time, but you would like to open both your esm and Skyrim esm. You need to make a change in the SkyrimEditor.ini file in your /Skyrim folder. Add bAllowMultipleMasterLoads=1 under [General].


/E