A solution it seams is a good but ugly program I used to work with called ImageJ. It is a Java program developed for scientific image analysis, and I used to measure root growth of mutant Arabidopsis with it. But it can also do what I need for Skyrim world creation, and it is free :)
Work flow
First make the height map in Gimp, save as 8 bit Tiff (see previous post for some guidelines)
Download ImageJ.
Open in ImageJ, change to 16 bit under Image -> Type, save as RAW.
After you run the filters and have a terrain you like, export as a 16 bit RAW file.
To make the esp run TESAnnwyn, and because the image is to bright you have to use the scaling function. Exactly what scaling factor to use depends on how bright/high your world is, try what works for your world.
TESAnnwyn command line:
TESAnnwyn.exe -i Skyrim -p 1 -b 16 -d 4096x4096 -s 0.2 -x -64 -y -64 -h -22192 -w NameOfWorldspace YourImage.raw
-i sets the game, -p RAW format, -b 16 bit image, -d size in pixels, -s scales the height range of the image, -x and -y cell offset in x and y directions. (See original post on -h values and the effect on sea levels.)
If anyone have found a better program or solution let me know.
/E