SC4 Dat Packer

[GMT]

i tried it again, and instead of crashing and giving error messages, it just stopped at a certain file, so it was easy to my to point it out.
it was a file in the bsc/parks subfolder called temp_save.dat
I opened it with ilive's reader and it contained lots of fsh, examplars and so on.
I intended to see what's what but immediatly after opening it with the reader, the reader crashed aswell and told me some wierdo error message bout compression, corrupt anythings or bad weather or whatever, don't know anymore...

anyway, I just dumped the file and then the datpacker worked just perfectly fine 

[Serkanner]

Thank you very much for a fast response ... 

[comptongb]

I don't know what's wrong, but my Datpacker always crashes when I have Force Recompression and Include only useful TFI selected? Should I only pick one?

[dioangel]

GMT - I've been having the same problem compressing the BSC folder the program will crash.. 

[ltrskn]

I have the same problem as comptongb in post #102. Also, for me it crashes at different subfolders every time. 

[snorrelli]

GMT - I've been having the same problem compressing the BSC folder the program will crash.. 

@dioangel - pay attention to which file the packer is stopping at and either dump it or re-download and replace it. I had the same problem and it seemed to be a corrupt .dat file.

On another note, I notice regional tracking has disappeared since compression. Will have to figure that one out... #?¿?¿?$

The positive side is that, with my 3GB of plugins, game loading has gone from about 8 minutes to about 3 and a fully-developed large city has gone from about 14 minutes also to about 3. Moreover, my parks menu will now open without me having to leave the room or go do something else while waiting for it. 

[BarbyW]

There are certain files that must remain in the root of plugins and as such should not be packed. The Regional Tracker file is one; any Essentials files including the BSC Reward Essentials should not be packed into a large dat.

[Elektra]

Oops!.....guess I better go fix that one and repack...
Thanks Barby, that is good information to know.

[jeronij]

I was wondering if it is possible to add some sort of checker to the tool, that first scan the files to detect corrupted ones ?¿ Just a quick test to see if it can open it. Maybe it would mean to double the time it takes to compress... but i dont think you will use this tool every day 
Just a suggestion of course. I love the tool the way it is now 

[snorrelli]

There are certain files that must remain in the root of plugins and as such should not be packed. The Regional Tracker file is one; any Essentials files including the BSC Reward Essentials should not be packed into a large dat.

I didn't have it packed before, but just in case I re-installed the mod and even deleted the apps files to reset the tracking. Still no region tracking (or at least no window at city start-up). Does it need to read certain files that shouldn't be packed?

[CabraBuitre]

How does a "packed" plugins folder with the lot editor?  I haven't tried it yet... but, if one would want to simply create new lots from already existing props, etc, shouldn't the dat packer speed up the lot editor speed as well?  ... then, after creating your lots, just re-run the dat packer to add the new lots to their respective subfolder dats...?

Or am I completely off base on this, and will it not work altogether?

[BarbyW]

I keep all my unpacked folders in PluginsDisabled so when I am lotting I use the ones I want from there. Using packed dats will not show separate prop packs.

[dioangel]

@dioangel - pay attention to which file the packer is stopping at and either dump it or re-download and replace it. I had the same problem and it seemed to be a corrupt .dat file.

I've been kept an eye on it.. even if I'm not in my bedroom. But everytime It crashes, it's always stops to CSX folder..  I don't know what caused it to happen.. even saying low virtual memory (top me messing with memory is dangerous for me btw).

[BourneID]

If any of you are disorginized like me... your plugin subfolders my not be as orginized as it should be,...  meaning a large number of subfolders.... one for each lot and such...  (sept for say BSC... who graciously orginize themselves on my PC thanks   )


heres an idea... I had been having trouble with the program... ie Memory issues was key among them....   heres my workaround...  after removing often replaced files... terrain, water texture, and such.... including all the essesials for the diffrent groups... I opened my plugins folder.... and in the folder options menu... clicked order by name.... and checked show as groups...  now they are separated in abc order... on my desktop... I make a new folder... named a for the A lettered lots... move all folders in the a section... into that folder and place back in the plugins folder... from 666 folders... I now have 27...  and the dat packer went to town... turning my 4.5gig plugins folder into a 2.1gig folder....  thats almost half... but with the amount of ram I have, my load time.... region... 8 minutes... city (larger map, 100,000+) 14minutes......  you say thats still along time...  well just loading the region used to leave me time to go make dinner, watch one of my recorded daily show or Cobart Report shows.... and still have time for a smoke....


so.... in case it hasnt been said... woung... oh crap... "really hard to spell guys name"   

IS A Sim City GOD![/shadow]


havnt been this excited sense Will Wight pee'd on me......      (dont ask...   )

[dioangel]

I've heard if you moved your plugin files... your game will crash and that's what happen to me. So I'm afraid it's going to do that again if I moved it. 

[wouanagaine]

I don't know what can cause the regional tracking to not work anymore. I'll have to call for Dealey expertise

I will try to make an update this week regarding the various 'corrupted' files issues, so you'll know which files on your system are actually causing the hanging
In the same time, if I find a good way to do it, I'll try to resolve the RAM issue

[shanghai kid]

This is a great program, but i've got a question #?¿?¿?$

To all user of SC4 Dat Packer:does the traffic in SC4 go fast even at slowest speed mode(turtle mode) in your game to, or is it just on my computer, as the traffic(cars, buses, trucks, trains, airplanes, glr/sub-trains) speed is very fast in my game and i don't have any special mods for faster speed on traffic.

1 example is the airport and airplanes, as i barely see the plane on the runway before it disapper again very fast(it's there, it's gone again).

Further checking reveals that the fountains and peg's schovels(boats on the canal sets) is also going way to fast & appearing/disappering to fast as if i'm running on the fastest speed mode + a little faster.

Ps: i tried to take the compressed folder out of pluggins and the original files back in the pluggins folder, and now the automata/traffic speed is normal again, though starting up slower as before.

[soulstealer]

I must make this post and thank you guys for making this amazing tool!!! My load time is now a couple minutes when before it was well above 10 minutes for a large city with 1.5 Gig of plugins. So far I haven't encountered any problems and everything is working smoother than ever before, no crashes and menus show up in 1-2 seconds when before it was more like 8-10 seconds for my over populated LM and Park menu. Now I can really play the game as before I would work on something for half an hour the game would crash and I would lose my patience and not play again for that day.

This tool should be brought to the attention of any SC custom content freak out there. I see how it's exclusive to this site and I'm not suggesting it should be posted at STEX. But, please don't forget there are SC fans back at ST that need this tool and have no idea about this place.

For people having problems and crashes let me suggest reordering and cleaning your folders before you make the pack.

1. Look for and remove duplicates (I use a tool called HCCLITE http://www.heatsoft.com/has/HASindex.html).
2. Use BSC Cleanit to remove garbage files (like .sav, .htm, .txt, .jpg etc') this files shouldn't be in your plugins folder.
2. Install the props and textures mega packs and remove old files carefully (Use BSC Cleanit).
3. Look for files with no extension and add the DAT extension (it seems the packer ignores files without extension like the Mntoes - Bosham Church I had installed and after packing did not show up).
4. Look for files with 0 size they seem to cause errors (0 size shouldn't be, it's probably due to corruption in a zip file or a mistake done by the lot/bat maker - try to dl again or remove it all together).
5. Move all mods away (Terrain and tree mods, nam, support files, essentials, stuff in zzz folders, slope mod, basically anything that is not a lot or a bat) - For me I always have these files in my maxis/simcity/plugins folder and not in mydocuments/simcity/plugins - see further explanation below).
6. Move all the lots/bats you have into a subfolder in your plugin folder like ...\plugins\mainplugins).
7. Fire up the packer (kick back - for me it took about 10-12 minutes to pack 1.5 Gig).
8. Move the stuff you moved away back in and remove the unpacked mainplugins away (Don't delete it just keep it somewhere else as you will update it with new stuff) (Now your plugin folder should have only the plugins_compressed folder and the mod folders like the terrain mod, nam, zzz etc').
9. Start the game and witness with awe and respect what a miracle DATPacker is

Step 5.5 (Optional)
I move all texture, prop files I can find into my "program file/maxis/simcity/plugins/props_and textures/" folder as these get updated often and its easier to find duplicates and remove oldies when necessary. I also use different sets of plugin according to the city I'm making (like Urban city and Rural town) so each time I load a different set of BATs and lots but the props and textures are shared by them both so it's better to keep them in a central place e.g the "program files\maxis..." folders).

So suggested folder structure should be like this:

Program files\maxis\simcity\plugins
Props and textures folder - You can pack this folder too to improve performance a bit
Mods, terrain, trees, essentials etc' folders
zzz folder

My documents\simcity\plugins
Main lots and bats folder - This is the folder we are packing
Mods folders
zzz folders

If you want to organize even further, inside the main lots and bats folder you can reorganize everything into their category.
This is basically how my Urban folder structure looks like:

Residential
- High rise
- R$
- Small Homes
- Mansions
Commercial
- Suburbs
- High rise
- Mid rise
Industrial
- ID and IM
- HT
Recreational
Civic
Landmarks
Network
-Rail
-Sea
-Land
-Air
zzz

All of this will take you about an hour to do, but believe me it's well worth the effort. When you have a problem like a missing prop/texture or a lost file or you want to update or remove something then in a matter of seconds you can locate what you need and fix it.

Hope this helps out.

Regards, Soul.

[beskhu3epnm]

Thank you for posting that, soulstealer. This will help a great number of people with crash issues!

[jeronij]

As a main rule, I would keep all the modding files outside the compressed plugins folders.
I always install the mods in the program files folder, and the normal buidings and lots in the MyDocs plugins. this allows me to enjoy SC4Datpacker, without loosing the modding capabilities. Just my suggestion 
Ooops, I just read soulstealer post mentioning this. I agree with what he said. I also think that the primary 9 points should be included in the readme file 

Oh, and I am sure this tool is announced at ST. But maybe the public there did not understand its potential. Of course you can make any aditional advertisement you find adequate .