• Welcome to SC4 Devotion Forum Archives.

How can I ensure I'm creating a valid 16 png for use with SC4Mapper?

Started by Han Solo, April 11, 2022, 01:11:29 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Han Solo

I'm getting back into SC4 after a long hiatus. I decided to finally make my dream region (and have it be my first upload to Simtropolis) but I'm stumbling when it comes to using SC4Mapper.

There's a website called https://heightmap.skydark.pl/, that lets you download 16 bit png height maps. I've downloaded a bunch of height map png's of the area I'm creating a map from, and then I stiched them all together using GIMP. I then exported the stiched images to png, using various settings.

When I check the image properties using the Linux command line utility, pngcheck, it says the png is 16 bit but when I try to use SC4Mapper to create a region... it says that the image doesn't appear to be a valid 16 bit png.

Any advice that can be shared would be greatly appreciated.

Odainsaker

Hiya!  I have been doing map renderings recently with GIMP 2.10.18 on Windows, and the default export settings for PNG do not work for me for SC4 Mapper 2013.  Fortunately, there are specific steps and settings I have found that do allow me to export 16-bit PNG images that will work with SC4 Mapper.

First, check to make sure in GIMP we are indeed working with a 16-bit image in grayscale mode in the right format:
Image --> Mode --> select "Grayscale"
Image --> Precision --> select "16 bit integer"

Next, make sure you have no current or floating selections:
Select --> None

Then, make sure you have a flat image with no layers:
Image --> Flatten Image
or, after choosing a higher layer:
Layer --> Merge Down

Now, export as PNG with your selected filename, path, and file type:
File --> Export As... --> Export button

The "Export Image as PNG" dialogue box should appear and needs two changes amid all the default settings:
Interlacing (Adam7) - default OFF
Save background color - default OFF
Save Gamma - default OFF
Save layer offset - default OFF
Save resolution - default ON
Save creation time - default ON
Save comment - default OFF
Save color values from transparent pixels - default OFF

Change "automatic pixelformat" dropdown to "16bpc GRAY"
Change Compression level slider or numerical ticker from "9" to "0"

Save Exif data - default ON
Save XMP data - default OFF
Save IPTC data - default OFF
Save thumbnail - default ON
Save color profile - default ON

The Export button should then save an image workable with SC4 Mapper.

______________


Some other pointers:

If you haven't already done so, you will want to be sure your image has valid height and width dimensions in pixels for SimCity 4, where an additional 1 pixel has to be added to the height and width of the planned map tile size.  For example:  a 2 x 2 grid of 4 large city tiles will need an exported map of 256 + 256 +1 pixels x 256 + 256 +1 pixels, or 513 pixels x 513 pixels.

Also, 16-bit PNG images are processed by SC4 Mapper such that each increment in the 16-bit grayscale corresponds to only 0.1 meters in DEM height in SimCity.  Hence, a 16-bit grayscale value of 250 actually corresponds in SC4 Mapper to an elevation of only 25 meters, which in SimCity is well below the default sea level of 250 meters.  Instead, in 16-bit grayscale a value of 2500 will correspond in SC4 Mapper to an elevation of 250 meters, or sea level in SimCity.

If you find your map in SC4 Mapper is oddly all under sea level, or if you notice in GIMP with the Color Picker Tool that all your grayscale values are too low by a magnitude of 10, or if the DEM image displays so dark that you can barely see what should be bright grayscale values for high mountains, you can quickly change all your values with the Color Curves function:
Colors --> Curves...

The "Adjust Color Curves" dialogue box will appear showing a grid display of Input to Output with a smooth line and 2 points.  Select the point at the top-right of the grid.  When the point is selected the Input and Output fields just below the grid should activate, showing that "Input: 100.00" corresponds to "Output: 100.00".  Change the Input field to "Input: 10.00".  Do not change the Output field, so that now "Input: 10.00" corresponds to "Output: 100.00".  All your color values should adjust by a magnitude of 10, and this can be visually confirmed if your preview map noticeably brightens.  After hitting the OK button, use the Color Picker Tool to confirm that areas of the map expected to be above sea level have 16-bit grayscale color values greater than 2500 and that areas of the map expected to be below sea level have 16-bit grayscale color values lower than 2500.


I hope this helps!

Han Solo

Apologies for the slow reply Odainsaker, I thought I'd set an reply notification up but evidently not.

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMFirst, check to make sure in GIMP we are indeed working with a 16-bit image in grayscale mode in the right format:
Image --> Mode --> select "Grayscale"
Image --> Precision --> select "16 bit integer"


This turned out to be a the first major stumbling block. I'm using Linux as my OS, and a software application called "Synaptic Package Manager" for installing various applications. One of which was GIMP. As it turns out, Synaptic's version of GIMP is outdated and completely lacking the Precision menu. So I've updated and that's one hurdle... erm, hurdled. 

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMNext, make sure you have no current or floating selections:
Select --> None

Then, make sure you have a flat image with no layers:
Image --> Flatten Image
or, after choosing a higher layer:
Layer --> Merge Down

Is flattening the layers actually necessary if you're just going to export it as a PNG?

What I've been doing is using the Selection tool to select the area I want to export, copying the selection to a new layer, hiding all other layers, and then shrinking the canvas to the layer.

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMThe "Export Image as PNG" dialogue box should appear and needs two changes amid all the default settings:
Interlacing (Adam7) - default OFF
Save background color - default OFF
Save Gamma - default OFF
Save layer offset - default OFF
Save resolution - default ON
Save creation time - default ON
Save comment - default OFF
Save color values from transparent pixels - default OFF

Change "automatic pixelformat" dropdown to "16bpc GRAY"
Change Compression level slider or numerical ticker from "9" to "0"

Save Exif data - default ON
Save XMP data - default OFF
Save IPTC data - default OFF
Save thumbnail - default ON
Save color profile - default ON

The Export button should then save an image workable with SC4 Mapper.

This section was especially helpful, thank you. I am now able to export images that SC4M will accept. Now I just need to up the quality of the images I'm creating. By which I mean, I'm still having numerous issues with the end product being what I had in mind.

I'm pretty sure all the issues can be fixed in the GIMP step of the work flow.

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMIf you haven't already done so, you will want to be sure your image has valid height and width dimensions in pixels for SimCity 4, where an additional 1 pixel has to be added to the height and width of the planned map tile size.  For example:  a 2 x 2 grid of 4 large city tiles will need an exported map of 256 + 256 +1 pixels x 256 + 256 +1 pixels, or 513 pixels x 513 pixels.

Do you know why the extra pixel is required? I was actually laying in bed thinking about it last night!

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMAlso, 16-bit PNG images are processed by SC4 Mapper such that each increment in the 16-bit grayscale corresponds to only 0.1 meters in DEM height in SimCity.  Hence, a 16-bit grayscale value of 250 actually corresponds in SC4 Mapper to an elevation of only 25 meters, which in SimCity is well below the default sea level of 250 meters.  Instead, in 16-bit grayscale a value of 2500 will correspond in SC4 Mapper to an elevation of 250 meters, or sea level in SimCity.

This is one of the key areas I need to learn about. I understand how elevations are calculated from 16bit grayscales... but I don't fully understand where the figure of 2500 comes from. Does the figure actually appear in the GIMP interface somewhere or is it the sum of all the colour channels?

If I wanted to create a colour of a given value, such as 2500, is there a simply way of doing it?

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMIf you find your map in SC4 Mapper is oddly all under sea level, or if you notice in GIMP with the Color Picker Tool that all your grayscale values are too low by a magnitude of 10, or if the DEM image displays so dark that you can barely see what should be bright grayscale values for high mountains, you can quickly change all your values with the Color Curves function:
Colors --> Curves...

The "Adjust Color Curves" dialogue box will appear showing a grid display of Input to Output with a smooth line and 2 points.  Select the point at the top-right of the grid.  When the point is selected the Input and Output fields just below the grid should activate, showing that "Input: 100.00" corresponds to "Output: 100.00".  Change the Input field to "Input: 10.00".  Do not change the Output field, so that now "Input: 10.00" corresponds to "Output: 100.00".  All your color values should adjust by a magnitude of 10, and this can be visually confirmed if your preview map noticeably brightens.  After hitting the OK button, use the Color Picker Tool to confirm that areas of the map expected to be above sea level have 16-bit grayscale color values greater than 2500 and that areas of the map expected to be below sea level have 16-bit grayscale color values lower than 2500.

So, does the Curves tool just change the way things are displayed or does it actually change the colour's values?

Quote from: Odainsaker on April 11, 2022, 04:09:04 PMI hope this helps!

You and CorinaMarie over at Simtropolis have been hugely helpful and welcoming, thank you!

Odainsaker

Quote from: Han Solo on April 20, 2022, 03:46:45 PMApologies for the slow reply Odainsaker, I thought I'd set an reply notification up but evidently not.

Worry not...look how slow my now reply is!

.
.
.

QuoteIs flattening the layers actually necessary if you're just going to export it as a PNG?

What I've been doing is using the Selection tool to select the area I want to export, copying the selection to a new layer, hiding all other layers, and then shrinking the canvas to the layer.

It might not be, and I admit I might be carrying over a trait from dealing with other filetypes where selections, layers, transparencies, and other options were saved into the file or affected settings in a way that affected compatibility that now I instinctively try to box out all the variables.  As long as your process is working comfortably for you, go with it.


QuoteDo you know why the extra pixel is required? I was actually laying in bed thinking about it last night!

I'm not sure on that one yet as the terrain generation process in SimCity 4's code is still mysterious.  Like many others, I had just assumed that it was so each DEM pixel correlated to each vertex in the map grid, but that apparently is not actually the case.  In "The Mystery of +1 for Grayscale SC4 Maps Solved" thread, CorinaMarie worked out that it may be a shortcut for readability of the code underlying the image array, and has clearly demonstrated that both the last row and last column of pixels in a grayscale DEM are ignored when importing to SimCity.  Rivit further teased out that there may be replication of edge rows and columns across city tile boundaries to match boundaries elevations before smoothing.  I have a suspicion that this is somehow involved in why I get a widening visual seam between city tiles on increasingly high sloping elevations, most prominently on one side of my rendering of Mount Fuji.  Most of the time we don't notice such mountain seams, but Mount Fuji has such a smoothly symmetrical figure that the seam stands out like a seppuku belly slash, and now that I am more aware of these seams, I can see them in other players' high mountain regions.

EDIT:  I am reminded that SRTM, ASTER, and Copernicus DEM tiles also have an extra row and column of repeated pixels that function as single-pixel overlap assists.  When merging these DEM tiles in MicroDEM, the overlaps are accounted for and overlapped automatically.  If instead we attempt to paste together individual tiles as raster images in GIMP, we would need to manually align the tile to do an overlap.  Interestingly, ALOS DEMs do not appear to use tile overlaps.


QuoteThis is one of the key areas I need to learn about. I understand how elevations are calculated from 16bit grayscales... but I don't fully understand where the figure of 2500 comes from. Does the figure actually appear in the GIMP interface somewhere or is it the sum of all the colour channels?

If I wanted to create a colour of a given value, such as 2500, is there a simply way of doing it?

This is a tough part because unfortunately in GIMP there is not a 16-bit grayscale swatch or other palette means to specify a 16-bit grayscale value with which to directly paint.  Frustratingly, there is nowhere you can type in "2500" as a value for a 16-bit grayscale, and there is no combination of RGB or other value notation to create one.  However, we can see those values in our images and use the color picker tool to select such a gray with the value if we already have that grayscale value in our images, and then paint with it.  Therefore, we are left to either randomly pluck with the color picker against a dark slope until we get lucky, or we have to somehow generate the desired values into our images.

One elaborate way to generate the value is to force a program like MicroDEM to create the grays we want, which also means that certain types of editing might actually be better done in MicroDEM.  If I open in MicroDEM a throwaway DEM that depicts an ocean coastline with shallow topography but otherwise has no bathymetry, then that map will have areas that have an elevation value of 0 meters, as MicroDEM treats sea level as elevation 0.  We can use the single grid arithmetic toolset to artificially raise, lower, or flatten z values to whatever arbitrary value we like, and so we can raise our entire map by 2,500 meters and then save it as a throwaway 16-bit DEM image for GIMP.  GIMP doesn't see elevation or sea level, it only sees grayscale values, and your throwaway image will have swaths of area with grayscale value 2500.  You can then use the color picker tool on this to put it in the active or foreground color.

Of course, unless creating a mask for later editing, we probably shouldn't necessarily be painting with the selected grayscale of 2500, as that is just the water surface at sea level and will have no depth.  Moreoever, painting in 16-bit for SimCity elevations can be unwieldy as all the swatches for most elevations would be nearly imperceptible variations of black-on-black.  Instead, we might raise, lower, stretch, or compress our DEM in MicroDEM using the single grid arithmetic toolset, taking into consideration which parts are above the 250 meters elevation and will be seen in SimCity as above water, and which parts are below 250 meters elevation and will be seen as underwater.  More intuitive perhaps, especially for freeform editing of terrain, is to later import our 16-bit PNG DEM into SC4Terraformer, whose popular compressor tool essentially does the same thing as MicroDEM's single grid arithmetic tools.

I admit, it took me a while to realize the useful potential of MicroDEM's single grid arithmetic tools, because there was so little documentation or guidance online and practically nothing readily available in the SimCity community.  Even the name "single grid arithmetic" made the tools seem like something in GIS-nightmare-techno-jargon best avoided, but now they are the top functions I use.  Worryingly, while the current standard, primary MicroDEM download of build 2020.8.20.1 still has the functional tools as I describe them here, the most current update options break those tools.  I learned the hard way not to apply the optional updates.


QuoteSo, does the Curves tool just change the way things are displayed or does it actually change the colour's values?

The Curves tools actually uniformly changes all the colors' values, in this case by multiplying them by a factor of 10.  Value 0 remains 0, value 1 becomes 10, value 100 becomes 1000, and value 250 becomes 2500, value 1000 becomes 10000.

In SC4 Mapper, 1 single interval of grayscale value of a 16-bit DEM translates into a 0.1 meter elevation.  So value 0 is still the 0 elevation of the seafloor, value 10 is only the 1-meter elevation above the seafloor, value 100 is the 10-meter elevation, and value 2500 is the 250-meter elevation, which in SimCity 4 is the default sea level.  Value 2510 becomes 1 meter above sea level, value 10000 is 750 meters above sea level, and value 12500 is 1,000 meters above sea level.  Value 40260 is 3,776 meters above sea level, roughly the height of Mount Fuji.

Additionally, because the Curves function works into the 16-bit grayscale range, it too can be use on a throwaway image with directly specifiable grayscale values in order to transform them into values that we cannot specify.  Perhaps make an 8-bit grayscale image with a swaft of value 250, then convert it to 16-bit grayscale and use the Curves function to output it with value 2500 on which you can then use the color picker tool.

__________


Incidentally, mapping software like MicroDEM will commonly interpret and save 16-bit files within the signed value range of -32768 to 32767, with the negative signed values used to encode bathymetric depths.  MicroDEM even somewhere explicitly indicates it uses such signed 16-bit integers.  In a graphics program like GIMP, negative color value is nonsensical, and it may interpret the 16-bit file as having an unsigned range of 0 to 65535, with the negative values instead interpreted as continuing beyond the 0 to 32767 positive value range as the further upper positive range values of 32768 to 65535.  This can have bizarre translation effects in the DEM image if the mapping software saved negative values or if it tried to save an elevation into a value higher than the maximum 32767 value of the signed range.  The DEM image can end up with black holes for mountaintops or inverted 65K white plateaus for oceans.

How does that gobbledygook come into play for SimCity 4 mapmaking?  Just like uniformly raising a z values with the single grid arithmetic tools, it is possible in MicroDEM to simply multiply all the DEM values by a factor of 10 with those tools, rescaling all the elevation.  Hence, with a couple of clicks you can easily scale an elevation of 250 meters into 2,500 meters, and then have the program save a 16-bit image so that the 2,500-meter elevation would have the grayscale value of 2500.  Just in terms of elevation scaling, that image is ready for SC4 Mapper, which will then interpret the 2500 value as the 250-meter sea level.

However, this only works when the elevations are below 3,276.7 meters, give or take for integer rounding, which when scaled by a multiple of 10 would have grayscale value 32767, the maximum of the 16-bit signed range of -32768 to 32767.  Anything above will be out of range, and could be reinterpreted as value 0, or null, or something else.  When I try this single grid arithmetic multiplication shortcut with 3,776-meter Mount Fuji, its summit needs a grayscale value of 37760, which is out of range, and the mountaintop gets cut off and distorted in the saved DEM image.

Instead, when there is a high mountain like Mount Fuji, the better option is to adjust the color Curves in GIMP.  The DEM z values would not be multiplied by 10 in MicroDEM, and the DEM image handed over to GIMP would show the top of Mount Fuji with the grayscale value 3776.  If this went in SC4 Mapper as is, it would interpret the DEM to give mighty Mount Fuji a diminutive elevation of only 377.6 meters.

With the Curves function, we can convert values as Input by some factor to generate an Output.  Rather than entering a direct multiplication factor, GIMP represents the Input and Output like percentages.  If the expected Output has a maximum of 100, and we want the factor to be a multiplication of 10, then the Input must be 1/10 the Output, or an Input of 10.  Plugged into the function, an Input grayscale value of 3776 is 1/10 of the Output, which then must generate an Output grayscale value of 37760.  GIMP uses the 16-bit unsigned range of 0 to 65535, and as this grayscale value is still within this range, the DEM image will show and save correctly.  The entire image will have its values recalculated this way, and, when handed over to SC4 Mapper, the 37760 grayscale value will be interpreted as the desired elevation of 3,776 meters.

By the way, if I recall correctly, SC4Terrafomer has a built-in limit of 6,000 meters, regardless of any DEM's unsigned 16-bit integer range, while SC4 Mapper may be able to use the full range.  Care should also be taken to ensure imported heights work with the MaxTerrainHeight exemplar of the installed terrain mod, with an exemplar value of 2500 being the default of SimCity 4, and some mods overriding this with 5300 or even 6000.

Yeah, that is a lot of convoluted explanation for two easy steps:  multiply all elevations in MicroDEM by 10, or change the color Curve to an Input of 10 in GIMP.  Both methods have the same ultimate effect, but either one or the other might be better to use depending on the elevations of a given map.  I admit it took me a long time to work these kinks out.

__________


Yikes, this was yet another lengthy reply!  May it again be useful.

dougr_w

Hi just getting back into Simcity 4 Deluxe Edition after years away. I bought my version from COG Games. Am trying to use you SC4Mapper to create new regions but when i try to save the region it seems to go through the process but does not add the new region in the Regions folder. The directory structure where the SimCity 4 games is installed is as follows

Top Directory     ----->   COG GAMES
                               ------>   SimCity 4 Deluxe Edition
                                      ----------> Regions


Please advise where your program creates the new region


Many Thanks in advance



Many thanks in advance

mgb204

The location shown is part of the game's installation and is not intended to be used for user regions.

You should instead use the Regions folder inside Documents\SimCity 4\.