SC4 Devotion Forum Archives

SC4Evermore Welcome Portal => LEX File Exchange Discussion => Topic started by: Sorengaard on July 10, 2010, 12:24:52 AM

Title: Can All Duplicate Files Be Removed
Post by: Sorengaard on July 10, 2010, 12:24:52 AM
Hi, Can all duplicate files be removed or are there necessary reasons for more than the same file to be in different places.  For example: I have the file "Extra Street texture LD" located in folders "JRJ Sidewalks_OPTIONAL" in  "JRJ Street Side Mod" and also in folders "JRJ Sidewalks_OPTIONAL_LW" and "JRJ Sidewalks_OPTIONAL_MODULAR" in folder "JRJ Sidewalks" and finally in folder "JRJ Sidewalks_OPTIONAL_MODULAR" in folder "JRJ CDS Streets Plop SSM compatible" in folder "JRJ Street Cul de Sac".  So "Extra Street texture LD" is located 4 times in different locations. It's only 10 KB's so size isn't a major concern in this case. But can that folder be removed and put in one place or is it better to leave it where it is in each location.  This is just an example.  Obviously there are all kinds of different files.  So my questions doesn't just focus on this one example but on the overall principle. Obviously there are some files that should be removed.  But can all duplicate files be removed and if it "depends" what would be the criteria.  Thanks Steve
Title: Re: Can All Duplicate Files Be Removed
Post by: Sorengaard on July 11, 2010, 12:18:13 PM
Hi again,  Let me re-ask in hopefully an easier way to understand. If for example I have 3 different houses with the same swimming pool file do I: One, just leave the 3 files of the pools where they are, or Two, randomly delete 2 of the pool files and leave 1 where it is, or Three, delete all 3 pool files and put another 1 in a different place assuming the 3 houses can then access it?  Hopefully this makes sense.  Thanks again, Steve
Title: Re: Can All Duplicate Files Be Removed
Post by: MOREOPTIONS on July 11, 2010, 12:24:33 PM
many times people may see a file with the same name, but you also need to see the size of the file. as the contens may differ in minuet ways.   most cases if the file with the same name and size, than it's a dup and would be able to be removed without affecting the rest and play.  you can also test by deletting It not empty out the trash. and see if you see any changes...  one safe way to do it.  bottom line.. the file name and size are what you need to look for!!!
Title: Re: Can All Duplicate Files Be Removed
Post by: Lowkee33 on July 11, 2010, 12:51:13 PM
I am not too sure about the "Extra Street Texture LD".  It may be that each of the files contains that specific shape of texture (i.e. cul de sac), in which case all shapes would be needed.  Do you have PIMX (the x-tool)?  This and other Tools that can be found in this forum will help you understand more about the file than just size.

When you are talking about pools you are talking about props/models.  You would only need one of a prop, because if they were the same they would have the same reference numbers that the game would use.  That would be like having three BSC_essentials.dat in your plugin folder.

I may be wrong about this, so you would have to look into it, but I think the Datpacker will remove duplicates.  I imagine that idea could be tested pretty easy (data pack two identical files and see if the result is twice as big).  To answer your question though, identical files can/should be removed.  You really want to do your best at keeping your plugins folder slim.  By selective dat packing my game no longer crashes, and that relates to the amount of files in the plugins folder, not the size.       
Title: Re: Can All Duplicate Files Be Removed
Post by: Sorengaard on July 11, 2010, 02:41:41 PM
I checked out Dat Packer and it does appear to eliminate duplicates.  It isn't really possible to tell by the size of the files.  In all the trials I checked each one showed an over all reduction in size of just about 1.2%.  In some cases the total duplicate files were larger than the reduction and in other they weren't.  But in the log it does say that it replaced the duplicate files without re-adding them so to me that says it eliminates duplicate files.  I was surprised in how small the size of reduction was.  Just 1.2%.  Over a 2 GB file that would be a reduction of 24 MB if I calculated that correctly.  I assumed it would be more.  Concerning my original question we seem to be in agreement.  I just wasn't sure.  Thanks for your help.  Steve
Title: Re: Can All Duplicate Files Be Removed
Post by: SC4BOY on July 11, 2010, 02:59:03 PM
DatPacker is NOT a file compressor.. it is in effect a "pre-scanner" and "working assembly" generator. There is much info available on it (including the fact that it eliminates duplicates at the GAME level, not the "dat file" level). That thread is available HERE (http://sc4devotion.com/forums/index.php?topic=204.0) rather than generating a redundant thread. It is not necessary to read every message in detail, but certainly worth a scan.
Title: Re: Can All Duplicate Files Be Removed
Post by: Sorengaard on July 11, 2010, 09:44:43 PM
Thanks for the point to the thread.  I wasn't originally asking about Dat Packer but I planned on using it this go around so all the more info the better.  Thanks Steve