Extracting the game's textures.

[jeronij]

Hi, I just answered somebody`s private question, and I though the answer could be useful for other players as well, so I post (and extend) it here:

This is a tutorial for newbies who want to start dealing with custom textures and BAT models using the original Maxis textures.

TRANSIT TEXTURES ( The easy ones)

You need the Ilives Reader installed and linked to the game folders  .

Open Ilives Reader and go to : Tools --> Texture Viewer.
Then select the desired transit network ( tree on the left) and the textures appear on the right. Zoom 5 for the right textures.
Right click on the texture you want to export and choose "Save FSH". Two files will be created wherever you saved the texture.

Now you need to open the new fsh file with Ilives Fishman tool to convert it into a bitmap file

Creating and editable bitmap file with Fishman.
Go to File --> Open and open your file. In the tree on the left select the Bitmaps folder. Right click on the small bitmap icon and select --> Export --> Color.
Select bmp type and give it a name. Save and you have your texture in a useful filetype for any software.
You can also extract the alpha maps ( for transparency) using the same method. Just select "alpha" instead "color"

You can find both programs, Ilives Reader and Fishman, in the Modds and Downloads section at Simtropolis.

OTHER TEXTURES ( Rock, water, custom textures, etc...)

You need to start Ilives Reader. Open the dat file containing the textures. The Maxis textures are found in the 5/6 original Maxis files. SimCity_1.dat to SimCity_5.dat. Also EP1.dat (in my version) contains textures (mainly buildings).
Once you have found your texture ( you can use Ilives Reader filters to help you with this), make sure you select the biggest one in the series.

Extra quick explanation about textures IDs.

Normally you will find until 5 textures for each final texture. One per zoom level. Its Instance number keeps the following patterns:

For Lot Editor Base and Overlay textures and Water textures:
XXXXX001 to XXXXX004  being XXXXX004 the biggest texture ( closest zoom level)

For Rock textures:
XXXXX044 to XXXXX444 being XXXXX444 the biggest texture ( closest zoom level)

Quick Info:
The TGI for rock mods is:    7AB50E44     891B0E1A     00000444
The TGI for water mods is:  7AB50E44     891B0E1A     09187304

You can already see the idea, right ?¿

Back to topic, once you have found your texture, select it and right click on it. Click on : " Save decoded file(s)" and save your file. You can give it a .fsh extension or not. It will work anyway.
Now see the Transit textures section above to remember how to deal with Fishman to convert it into a .bmp.

Another extra tip:

Correct the textures for the BAT /  GMAX

How do I correct the game textures before using them in the BAt or GMAX ?¿.
After several try and error I did not find the right answer. I still have to play with the brightness and contrast until I get the right combination. I use Paint Shop Pro and usually something like: Brightness -20/25 and contrast -5/10.

I have also noticed that if i save the texture as a grey scale bitmap, it usually fits better. This can not be done with the USA version roads.
There is a tutorial that explains how to configure Photoshop for this purpose. You can find it here:
Calibrating Photoshop for gmax

I hope this tutorial is useful for somebody. It wasnt for me. Too complicated for a non native english speaker      

I hope you find this post useful.



EDIT 

OK, if you are shocked by the complexity of what is proposed here, you can also follow mjig_dudy's technique, which is much quicker and useful most of the times. Not so useful if you plan to reinsert the textures with the same IDs...

I would just like to adda couple of things that some people might find usefull:

personally when getting textures I simple take a screenshot of the texture while in the reader and just copy and paste the texture i want out. I find this a lot easier than messing around with fishman and all that.

I you want to use the orginanal maxis ground and overlay textures (most are also found in the lot editor, although for some strange reason not all) for a modd or something (my sandstone pavement modd for example) most of these are found in simcity_2.dat and have the group id 0986135e so if you sort the files by group id then you can find all of the ground textures in the same place. This means it makes it easier to simply copy and paste the files you want out of simcity_2 and into your own modd.

Hopefully some pople will find this helpfull   

Great adverstisement  !!!!

[TheTeaCat]

Wow Another useful tutorial.

Thanks Jeronij.

You are a font of knowledge

lol there are too many new things for me to try, its like being a kid in a sweetshop again 

[mjig_dudy]

Great tutorial this, thanks.

I would just like to adda couple of things that some people might find usefull:

personally when getting textures I simple take a screenshot of the texture while in the reader and just copy and paste the texture i want out. I find this a lot easier than messing around with fishman and all that.

I you want to use the orginanal maxis ground and overlay textures (most are also found in the lot editor, although for some strange reason not all) for a modd or something (my sandstone pavement modd for example) most of these are found in simcity_2.dat and have the group id 0986135e so if you sort the files by group id then you can find all of the ground textures in the same place. This means it makes it easier to simply copy and paste the files you want out of simcity_2 and into your own modd.

Hopefully some pople will find this helpfull

[Alfred.Jones]

Thanks for the tutorial Jeronij Thanks Mjig_dudy

[bat]

Yes, it's another very useful tutorial!

[City Builder]

I don't quite get this.  I open up EP1.dat with the reader and it shows the following files as loaded:
Blank.dat and
EP1.dat

Now I go to Tools and choose texture viewer but Im still only shown transportation textures.  How can I get to individual building textures or are they not stored except the whole building fsh file itself.  So if I wanted to grab a texture that was used on the side of one of the buildings in EP1.dat, do I have to grab the whole building looking file in Reader and then photoshop it to get just the wall without all the other stuff?

As an example, if I want to pull that green roof texture used in the screenshot below, is that possible? Is it stored seperately in the dat file or is it just permanantly attached to the image seen on the right in the screenshot?

[Swamper77]

City Builder,

The Texture Viewer in the Reader will only show network, bridge, and vehicle textures.

As an example, if I want to pull that green roof texture used in the screenshot below, is that possible? Is it stored seperately in the dat file or is it just permanantly attached to the image seen on the right in the screenshot?

The roof is part of the image that you see there. It is not stored separately anywhere in the Maxis DAT files. It is part of the building image. Maxis' models are much like the models that the BAT produces when you render them from GMAX. BAT pre-renders the images of the models and then maps them to LODs generated by the BAT. The LODs are usually boxed shaped except for special cases.

So all the buildings and props you see in the game are actually made of a collection of models and images for each zoom and rotation.

-Swamper

[SimFox]

I've been messing with game textures for couple of days now and have few påints I would like to add to this tutorial.

First of all a confession of being stupid... I should have looked into here first... Funny enough I had to walk my own way just to invent the wheel... Oh, well...

Ok to the points.

1.  Mjig-Dudy suggestion of taking snapshots of the screen.
Although it may sound quick but in the end it involves just 1-2 steps less than one suggested by Jeronij.
But as you know the free cheese (or a cheaper one) normally found in mouse traps... The "cost" in this case is that you will not get the entire texture. Difference is small - 1 pixels on height and 1 pixel on width but may be noticeable on some tiling textures (particularly on those with diagonal lines).

2. Using in game textures (not the building ones, but all sorts of ground etc) in BAT (any).
Jeronij had mentioned the issue of BAT lighting altering the look of the texture, but that is not the full story. Unfortunately game isn't innocent in this process either. It , in fact also heavily changes the look of the texture, not less than GMAX BAT does.
This fact should also be taken into account when you are set to make a texture (pavement, grass etc) for the game.
why does game do such a thing? I believe this is a conscious decision aiming at providing "common denominator" fro all different textures/colors etc. Putting common color overlay binds them all together. even most different and "incompatible" colors after such a procedure will look like they belong together since now all of them care common element. This possibly means that you may want to consider  leaving all you homemade textures as is and let game do it's thing.  Or, if confident enough do something to offset it.

Here is an example of how game treat texture and what happens to it in GMAX Bat (right square - showing all 3 sides):




3. Finding the texture
I think one of the easies way to find the texture (at least as far as base and overlay ones are concerned) is to find it ID number in LotEditor. Here you can visually browse through all abvailable base and overlay textures and see the ID number associated with it and then search by this number in iLive Reader.

[Diggis]

3. Finding the texture
I think one of the easies way to find the texture (at least as far as base and overlay ones are concerned) is to find it ID number in LotEditor. Here you can visually browse through all abvailable base and overlay textures and see the ID number associated with it and then search by this number in iLive Reader.

you just need to make sure you don't have any extra files in your plugins when doing this, otherwise you could end up looking for a custom file. 

[Ennedi]

I've been messing with game textures for couple of days now and have few påints I would like to add to this tutorial.
(...)
2. Using in game textures (not the building ones, but all sorts of ground etc) in BAT (any).
Jeronij had mentioned the issue of BAT lighting altering the look of the texture, but that is not the full story. Unfortunately game isn't innocent in this process either. It , in fact also heavily changes the look of the texture, not less than GMAX BAT does.
This fact should also be taken into account when you are set to make a texture (pavement, grass etc) for the game.
why does game do such a thing? I believe this is a conscious decision aiming at providing "common denominator" fro all different textures/colors etc. Putting common color overlay binds them all together. even most different and "incompatible" colors after such a procedure will look like they belong together since now all of them care common element. This possibly means that you may want to consider  leaving all you homemade textures as is and let game do it's thing.  Or, if confident enough do something to offset it.

Here is an example of how game treat texture and what happens to it in GMAX Bat (right square - showing all 3 sides):


(...)

SimFox, you started very interesting theme. Maybe it deserves a separate thread (for example in "Brainbusters" section) - let Mods decise.

1. What does game do with textures colors? It uses global lighting (something about it in point 3) and in addition it uses an Environment Map.
What is Environment Map? Please look at these pictures.

1.1. Water. As you can see the water is brighter in the center. This effect is visible on large water areas, no matter which water mod do you use. I find this effect rather positive, it enables making interesting pictures.



1.2. Land. You can see pinky color on some rock areas. I placed on the first picture an original rock texture No 44 of the same zoom level as on the picture.





This is Environment Map in action.

2. Can we see the same on other textures? Yes.

2.1. Here is one of popular Maxis textures placed on the slope in various directions. Unedited picture...



You can see that pinky shadow on tiles to the right.
Now the same texture with slightly changed levels to make the effect better visible



2.2. Can we do something with the Environment Map? Yes, we can remove it.  How to do it? Very simple. We must go to the "Lighting" exemplar in simcity1.dat, save it as a separate dat and change the property "Use Environment Map" from True to False.
But will it be better? Let's see 



Do you see this horrible yellow color and deep blue shadows? This is an original game Global Lighting.

3. Changes in the terrain look after turning the Evironment Map off are even more dramatic.

3.1. A mountainous terrain from South - normal view...



...And after removing Environment Map



3.2. The same terrain from North - normal view



And without an Environment Map:



This last picture leads to an interesting conclusion: It seems that Global Lighting is directional (its obvious, there are "Sun Direction" and "Sun Pitch" properties in the Lighting exemplar), and the environment Map not only improves the colour palette, but it also equalize lighting from all directions.
However, the whole mechanism of these two kinds of lighting is not quite clear for me - and this is the reason I'm very glad that one of the most experienced 3D graphic specialists started this theme. It would be even better if more good  BATers would research this.
I was going to present these effects in my MD in the near future, but this discussion is in my opinion a great opportunity to exchange knowledge between BATers and terrain textures makers. So I decided to present it here.

4. As we see, it is not easy to remove the game impact on our textures. Maybe, as you suggested, we should learn to live with it?
The pinky rock is sometimes unwanted. But if we know about this effect, we can make use of it. I'd like to show you the panoramic view of the Mount Robson - I must say I like this rock! And the original rock texture for Zoom1 is pasted into this picture for comparation...



Adam

Edit 1: As I said Global Lighting is directional. But the Environment map looks the same from every direction.
Edit 2: We can make experiments with sun direction and sun pitch, but we should turn the Environment Map off to see any effect.

[SimFox]

that is a very interesting discovery Adam! Thank you for bringing it to my attention.
I'll have to take a closer look at it to say something more or less definitive.

[jeronij]

It would be great to know more about this indeed 

[SimFox]

After taking a look at the issue here is what I have to report.

Game apparently uses multilayered lighting/color-correcting setup. This isn't all that surprising given the mix of 2 and 3d elements in it. Fact that pre-rendered buildings and props and "real" 3d terrain call for separate systems in order to establish "sunny"/"shadow"  illusion is understandable and expected. What is not all that expected is that Flora type object have they own, independent lighting subroutine as well. Probably use of term "independent" was a step too far, let's call it semi-independent, or autonomous, but more about it later.

So, how is the game light? Let's deconstruct operations of the games engine:
The very idea of maps gave me a clue of how exactly the shadows are produced in game. Prior to that all I and JasonCW, with whom we have investigated the issue trying to fix the discrepancy between the on building and in-game on-ground shadows, knew just that these shadows in reality aren't shadows at all. I mean what we see on Games ground isn't result of interaction of a light source and the object in game. This came clear when we tried to adjust Sun Direction Property in Lighting Exemplar. Would it be a real shadow setting new, correct angle would resulted in alignment with on-building shadows. However this wasn't the case. This adjustment resulted in mere rotation of the shadow image around the center that is located (speculatively) in the middle of the LODs base. This made it 100% clear that we are dealing with another ready and superimposed image. However this didn't reveal exactly how was that image generated. We thought that this may be FSH Alpha turned and /or skewed. But no this sort of manipulation in Photoshop produced result like those in game. Particularly when those numbers  - 67,5 and 45 were involved. I thought that there may be some other data hidden in the depths of Exemplar forest that would explain what have happens to the "shadow" but wasn't prepared to dig deeper.

Now, thinking of maps made it clear to me that there we are also dealing with a map – Shadow map – sort of the projection mask slapped on the main light casting the shadow onto the ground plane. I'm pretty sure now that this is the way shadows are regenerated by the game and stored as ready maps. I also know what is the sun "direction" is said 67,5 degrees – cause this would make the Right Angle with view cameras. This is quite an essential part since this ensures that shadows of all vertical lines will be strictly horizontal – removes tons of headache in dealing with aliasing problems of diagonal lines.

But talk is cheap and here is a "proof" this is an superimposition of such a "shadow map" over the real shadow – as you see glove DOES fit.
In-game image with shadow:



Shadow map generated in MAX:


It is blurry cause I had to scale it up (keeping all the proportions and shape of the original)

Shadow map over imposed on top of the in-game pic:



One point I've forgotten to mention earlier. To generate the "shadow" Alpha from previous rotation is used e.g. to shadows supposedly cast by south view are in fact generated from alphas of West view, etc.

The practical value of this "discovery" unfortunately is little and rather negative...
There is no way to fix this shadow discrepancy issue. Altering the Sun Direction would break the right angle between "sun" and view cameras and lead to misfit between shadows and LOds base. Simply put the shadow would stick from under the building in the front right corner. There will be some other issues but this one puts fat black cross on whole idea. Sad... but knowledge is its own reward!
Of course, it leaves you wondering were those 10 degrees (67,5 vs. 57.5 in BAT) of a difference so important... and the answer, given the age of the game is most probably yes they were. Choosing games sun direction for buildings would result in lighting too inferior (particularly given those days visualization solutions) and choosing the building "sun" direction would put to much of the burdent on CPUs to calculate thousands of shadows in game or result in ugly stair stepped shadows. So given the possibilities at the disposal of the creative team that was the best solution. My hat's off to the team!

Next ...
environment maps (yeah! right! MAPS not just map) Darker nights and brighter/redder (for Sim Mars adepts) days and other fun stuff that may have practical applications!
stay tuned...

[jeronij]

... but knowledge is its own reward

Very well said 

The information so far is quite interesting... lets see what those environement maps are hiding for us...   

[Ennedi]

I'm sorry for late comment, but too much interesting things happen at this site in the same time 

SimFox, I am really interesting in your findings. Especially your words about environment maps are very intriguing for me. Do you think we can have any influence on the ingame environment map parameters?
I spent last few weeks on creating various terrain textures and and all these issues we are talking about here are extremely important for me. Tell me please, would you like to suggest me any experiments? or could I help you in another way?

Adam

[dedgren]

Following this with interest, and a bit of awe.  SimFox, that was a excellent and extraordinarily clear explanation of things on top of Adam's already brilliant exploration.  When you guys have it at the point where I feel like I can grasp this stuff, you can pretty much figure you are connecting with others as well


David