• Welcome to SC4 Devotion Forum Archives.

What causes Prop Pox (and how to avoid it)

Started by bap, February 24, 2009, 08:37:13 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Livin in Sim

Hi, everyone,

You can't imagine how tickled I was to find this thread.  I have a city that is infected.  I saw the original thread quite some time before it happened, so that is how I recognized that it was probably the Pox.  That thread had gone quiet, by then, however.  I'm convinced that there are more than a few of us.  I see threads about this fairly often at ST.  The usual consensus is that "It is probably the Prop Pox, but what to do ()what() --the cause is an unsolved mystery..."  So the questioner usually deletes or gives up on that city.  And it doesn't reoccur again soon, because, as has been determined, it takes a good bit of content, detailing, time, etc., on one tile to cause it.

The city of mine that is infected is a large tile, low density, rural, with many farms and seasonal trees.  It is almost covered with lots, and is over 500 years old.  I do have the Pegasus Trail Parks, the CDK file, and the OWW file.  I also have the BSC Mega Pack Vol 2, but downloaded it after I last played the city.  I just checked the size of the 2977aa47 file.  It is 21,913,616 bytes, which would be close to 22MB, I think?  Another thing I remember was that, as well as lots, the avenue dividers were also affected.  I have since reorganized and cleaned prior to starting a fresh region, determined to test and research all content, so I don't know if the city is playable.  I do have Plugins backed up on disc from when I was playing it.  If any other info is needed, please let me know.

I would like to thank bap so much for all his research.  He has brought a very scientific mind and process to address the problem.  I have learned a great deal from his methods.  I now have hope that this problem will not reoccur, and plan to use bap's method of celebration, too.    I would also like to thank all the great "SC4 Technicians", wouanagaine, RippleJet, barbyw, callagrafx, Andreas, Ennedi, and others who have also spent precious time on this and take the difficulties we players have to heart.  It is only a game, but a most delightful hobby as well.  When one has expressed themselves artistically through playing this game, it becomes rewarding to see the result of one's efforts.  It is especially gratifying to realize there are others who not only share that enjoyment, but want to help enhance and preserve those efforts.  In other words, you guys rock.   ()flower()

Bye for now.
--Liv

RippleJet

Liv, let me also take the chance to thank you for all help you're giving, especially to newcomers, both here and on Simtropolis. :)


Quote from: catty on March 02, 2009, 10:30:25 PM
I had to wait till I got home from work before posting a picture, but this is what I see ...

It's basically the same that Reader or DatGen would show, however with one important difference.
Game Extractor doesn't show the Type, Group and Instance ID's of the subfiles,
and also doesn't show the structure of those files whose structure is known (e.g. exemplars).

BarbyW

This is going to be a long post so bear with me. In the early days of SC4 - 2003 - there were clever people who decoded parts of the game and also who provided mods of various types. TWrecks mentioned one - the RCI Blank Lots file from ralphaelninja - that has caused pain in various ways and is actually still being used so still causing problems. There were only modders and lotters in those early days as BAT did not exist. Colyn and mrbisonm were two of the most innovative and productive lotters at the time and GrampaAl worked with both as a modder. When Maxis released the two buildings as props files they left out the Query as Main property which means that if, as was done frequently in 2003, props were joined to make bigger lots then the props didn't show the name of the lot when queried. As one of the earliest respected modders GrampaAl found this unacceptable and to get round the problem he copied the exemplars from the buildings as props files into the lot files and added Query As Main BUT he did not change the TGI of the exemplar.
I have looked at Colyn's lots - sorry I can't check everyone's from way back then - and found all the following have modded Maxis exemplars with no TGI change. Fortunately most of these lots are now withdrawn from the STEX and only available on the STEX disks. I have started to update them but am waiting for Colyn to tell me which he wants to be uploaded apart from the ones that I have posted for him. I have no idea if these lots could be the cause of the Pox but would rather err on the side of caution and advise against using them.
CSX Abbatoir_6c31d500.SC4Lot
CSX Admin Centre_4c39087f.SC4Lot
CSX Agricultural Centre District Office RH_6ca4a64d.SC4Lot
CSX Agricultural School RH_4ccb9b02.SC4Lot
CSX Centennial Plaza RH_ed09e452.SC4Lot
CSX Commercial - Ali Baba RH_2c81a24f.SC4Lot
CSX Commercial - Keg and Tankard RH_cc991a3d.SC4Lot
CSX Commercial - Vivaldi RH_8ca834fe.SC4Lot
CSX Corporate HQ RH_2c9ef1b2.SC4Lot
CSX Farming - Fuel Depot_ed15ffb7.SC4Lot
CSX Farming - Welding Works RH_ed1bb9eb.SC4Lot
CSX Fun Palace_RH_0c9cf9aa.SC4Lot
CSX GA Farm - Bob's Chili Peppers_cd663162.SC4Lot
CSX GA Farm - Don's Produce RH_0d018b4b.SC4Lot
CSX GA Farm - Grampa's Barley_6d6c7eaa.SC4Lot
CSX GA Farm - Hopeless RH_4cf214f3.SC4Lot
CSX GA Farm - Nexis Chickens RH_ecf64d62.SC4Lot
CSX GA Farm - Nexis Hops RH_ed303d0b.SC4Lot
CSX GA Farm - Nexis Strawberries RH_ace4e7c9.SC4Lot
CSX GA Orchard - Owensby Apples RH_0d15b3a1.SC4Lot
CSX GA Orchard - Washington Cherries RH_cd33cbaa.SC4Lot
CSX GA Vineyard - Tuscany Merlot RH_4d045aa6.SC4Lot
CSX GA Wine Estate - Nexis RH_aced43ec.SC4Lot
CSX GM Farm - Coconuts RH_ac940fa4.SC4Lot
CSX GM Farm - Eggs_6d1b2e4c.SC4Lot
CSX GM Farm - Mystery RH_8cb0cc52.SC4Lot
CSX GM Farm - Rainbow Trout RH_8d1870a6.SC4Lot
CSX GM Research Centre_ac23463b.SC4Lot
CSX GM Test Centre RH_4cc725df.SC4Lot
CSX Hi Tech Coal Power_6c452095.SC4Lot
CSX Hitech Security HQ RH_4c2571b8.SC4Lot
CSX Home - Sandton_cc7926b7.SC4Lot
CSX Industrial - Bloodsweat RH_4c6f176b.SC4Lot
CSX Industrial - Cement Plant RH_8caee7cf.SC4Lot
CSX LSU Crop Duster RH_acce044d.SC4Lot
CSX Mini Generator_0cc1301b.SC4Lot
CSX Mini Power Plant RH_ec56b1eb.SC4Lot
CSX Nexis Cattle Auction_ac40d4d8.SC4Lot
CSX Planning - Business Affairs_4cbef037.SC4Lot
CSX Produce Market RH_0cc42ab8.SC4Lot
CSX Produce Packaging RH_6c5d05d6.SC4Lot
CSX Resort - Black Rock RH_0cc41637.SC4Lot
CSX Restaurant - Al Fresco RH_ccafa5a9.SC4Lot
CSX Rising Sun Oil Refinery RH_ecc00109.SC4Lot
CSX Rural 911 Centre - RH_ac3aef5f.SC4Lot
CSX School of Design and Architecture_0ce7124e.SC4Lot
CSX Services - DustBusters RH_0ccc0118.SC4Lot
CSX Services - Girl Friday RH_4ccccf9e.SC4Lot
CSX Services - The Mews_2d129b17.SC4Lot
CSX Unemployment Office_2c5c7e2d.SC4Lot
FirstNationalBank_BSCLarz_ad6dc33e.SC4Lot

If bap and anyone else who is testing out the theories would care to try with one or more of the above lots in I have them all in the original format and can send for testing.

Now to the second part of this long post. In one of his posts Pegasus said this:
Quote<snip>
and obviously influenced by the BSC.
<snip>
Fortunately, the majority of the community can see it for the senseless & petty attack that is it is. You further the BSC agenda of trying to spread fear of using material from any developer other than their own. Meanwhile, the material you attempt to spread fear about is being used extensively by the entire community without any problems whatsoever. Despite the personal and vicious nature of your attack, the facts and reliablity of the material does not support any of it.

Andreas is first and foremost a member of the SFBT community. He is BSC affiliated but does not contribute to BSC other than to make translations and offer advice. BSC have no agenda and never try to prevent anyone using any GOOD game assets. The original concept of the BSC was to help, through the use of experienced modders and lotters, all BATters to provide the best possible upload. This concept is still the underlying reason for the existence of BSC and many in the community can testify to that.
Last year Ripplejet tried to help Pegasus with the seaports as originally he announced that his new one would only replace the Maxis one; there being no way to make custom seaports without causing CTD problems. The result was a ban from his site for Ripplejet, jplumbley, wouanagaine and me. At no stage did any of us do other than try to help but, as with Andreas, telling the truth or trying to correct erroneous statements at SimPeg is not welcome. Andreas and Warrior tried to correct Pegasus' idea that XML and Hex are the same thing and that .sc4 files are XML. As has been stated they are in a proprietary format DBPF and have never been completely decoded. I shall return to the comments about his material later.

This was followed by:
QuoteThe BSC is not even remotely interested in this pox issue. They have been aware of it for years... and have never made a single effort, as a group... to do anything about it. But as soon as one unknown person indicated that PEG material might be involved, they throw 100% of their effort into the issue with a full focus only on manufacturing evidence that supports that.

The one verifiable fact with this issue is that it is extremely rare and random... and only strikes very large cities with a very large number of props in it. City size actually does not appear to be factor... but apparently only large cities can contain enough props to trigger it.

No player that uses PEG material exclusively has ever gotten this prop issue. And the simple and obvious reason for that is that almost all of the PEG material is ploppable. PEG material is geared more toward the sandbox player and for development of CJs.  Growable material has always been more a BSC thing.

Ploppable lots do not add many (overall) props to a city... because players normally don't plop very many of them in a single city. Growables, however, can show up hundreds and even thousands of times in a large city... with each occurrence adding new props to the city and inflating the overall prop data size to well beyond the 6 meg pox-trigger mark.

And who are the kings and queens of Growables... with more mega prop pack dependencies than all developers combined? The BSC, of course.  To use any one BSC lot may require installing up to 7 or more large dependencies files.

It appears that every documented "poxed" city was loaded to the hilt with BSC material. So if that is true, then any one with a poxed city should try uninstalling all of the BSC material, loading up their poxed city, deleting all the BSC lots in that city and allow default lots to redevelop. Save the city, reload and see if the "pox" is still there. If the assumption that "once poxed, always poxed" is true, it won't make any difference. But that still doesn't mean that all those extra props were not the cause.

* In fact... if someone would zip up and email me a poxed city (normal large city map), I'd like to test that myself.

Prior to this post Pegasus had denied categorically the existence of the Prop Pox. Now he has once again done a U turn and decided that maybe it does exist. Unfortunately he has omitted to read carefully the whole testing report from bap. He states:
Quoteran SC4Tools (merge file option) on all custom prop packages in search for multiply defined props. It was able to find several conflicts and hundreds of multiple definitions of props. I eliminated all conflicting and doubly defined props from the prop packages, installed the corrected files back on PP, and ran another test with City#2.1. Prop Pox sets in when the city reaches 16 Mby (again, network subfile about 6 Mby). Multiply defined props per si are not the cause of Prop Pox.

The 'virus' was still there. In order to find it, I performed binary search (take half of the prop package files out, test with the other half; if you don't find Prop Pox, you can eliminate the tested half from the exercise; take half of the other half and proceed with the testing). After several such tests, I was able to narrow down the search to only one file: a blend of props from Pegasus CDK3 and OWW2 packages. Only when this package was in PP folder, did the city developed Prop Pox.

I tested developing the City #2.1 with only the texture packs and Maxis plugins in PP folder. No sign of Prop Pox. I then added all other prop packages except for the suspect one, and installed back all lots in MP (except for the lots which depend on the excluded prop package). Almost all my custom content back. No sign of Prop Pox. (another box of bier to celebrate.)

I broke my Pegasus mega prop into its CDK3 and OWW2 parts and performed a few more tests to find that I only got Prop Pox when the peg-OWW2_BDK_RESOURCE.DAT file was installed. Removing everything except this file from the plugins, I got Prop Pox. Including everything in the plugins except this file (and the associated lots), I got no Prop Pox.

This is the conclusion from many months of testing and his poxed test cities were not "loaded to the hilt" with BSC material. In one test he had no custom lots but only custom props. Once he had narrowed down the cause by binary search he replaced all his custom content except for anything that depended on the now excluded RESOURCE packs. No sign of Prop Pox. The problem seems to occur when props used by Maxis on their own lots have been modded and included in a custom prop or RESOURCE file without giving the exemplars a new IID.
At this point in time no one is saying that Pegasus is the only creator whose props may cause problems. BSC found one occurrence and have corrected and updated that file. Unfortunately if it has poxed your city all we can do is apologise. As you can see from the first part of this epic I am spending a good deal of my time checking and have now given a second heads up with that list of Colyn's 2003 lots and I also noted that the Snake Petting Zoo from Pegasus also has the same characteristics. No has pointed fingers or tried to pass blame apart from one person who continues his vituperative tirades against one group. At no stage did Andreas make any personal attacks other than to correct him on one point and then to demonstrate that Pegasus had modded his Mayor Mode Ploppables incorrectly and using a cycledogg - who is BSC - base. This was known to BSC some weeks ago but as none of us can post there we could not help and we could not correct his mods and upload as that is against his EULA.

It is up to all creators to check their material, no matter how old, to see if they may have contributed to this problem.


Inside every old person is a young person wondering what happened. TP



Barbypedia: More alive than the original

jablme

RE: the CSX lots mentioned----do these lots all need to be bulldozed before removal, or is it safe to remove and just clean up the empty lots as I encounter them, if not might as well just start over   :'(


jeronij

#144
Quote from: jabl0me on March 03, 2009, 07:00:49 AM
RE: the CSX lots mentioned----do these lots all need to be bulldozed before removal, or is it safe to remove and just clean up the empty lots as I encounter them, if not might as well just start over   :'(

As a general rule, it is a good practice to remove all the lots in your cities before removing the files from your plugins folder ;)
I am currently not active - Please, contact Tarkus for any site related matter. Thanks for enjoying SC4D :D


Autism Awareness;  A Father Shares
Mallorca My Mayor Diary


jablme

Quote from: jeronij on March 03, 2009, 08:06:51 AM
As a general rule, it is a good practice to remove all the lots in your cities before removing the files from your plugins folder ;)


I know it's a general rule but do they NEED to be? I've seen occasionally that some mods/lots say that no bulldozing is necessary or they just need to be deleted and re-plopped. I just want to know if they absolutely have to be deleted first. Thanx!

wouanagaine

Definitly needed
This will at least remove the prop from those lots from the savegame


New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

jeronij

Quote from: jabl0me on March 03, 2009, 08:18:08 AM

I know it's a general rule but do they NEED to be? I've seen occasionally that some mods/lots say that no bulldozing is necessary or they just need to be deleted and re-plopped. I just want to know if they absolutely have to be deleted first. Thanx!

so, you are a man of short answers then....  $%Grinno$%

YES


Sorry wou, perhaps your answer was still open to doubts...   :D  :D
I am currently not active - Please, contact Tarkus for any site related matter. Thanks for enjoying SC4D :D


Autism Awareness;  A Father Shares
Mallorca My Mayor Diary


bap

#148
Barbyw, I much appreciate your considerate policy of choosing to err on the side of caution.
I haven't been able to finish the further tests I planned to do. But even though, I can add a few comments to the discussion now.

Among the other prop packs I have in my plugins folder there are several Maxis modded props which keep their original IID. In most cases, the modding is either including the prop in a prop family (by adding the Building/Prop Family property) or adding the prop to another prop family (to an already existing Building/Prop Family property). This leads to increases in exemplar size by 4-17 bytes. None of these props lead to Prop Pox. I do have [url=http://sc4devotion.com/forums/index.php?topic=7066.0]Pegasus[/url]' cdk-industrial package. It also doesn't lead to Prop Pox. I also found one Maxis modded prop in [url=http://sc4devotion.com/forums/index.php?topic=7066.0]Pegasus[/url] cdk-oww package (not the oww2 package) which is a timed prop for which he replaced the "Nighttime Change" property by a "Prop Time of Day" one. This also leads to a small <10 bytes increase in the size of the prop exemplar.  This is identical to what he did with the umbrella prop. As I do have the cdk-www package and it does not cause Prop Pox, I am convinced that the Maxis modded umbrella prop (IID 29000000)  does not cause Prop Pox. I could confirm that by performing another experiment with only that prop exemplar in the plugins folder, but I did not have time yet.

This weakens my original hypothesis that is the increase in size of the modded Maxis props that lead to Prop Pox. The important feature which is exclusive to the other 3 Maxis modded props in the bdk-resource file is being originally a static prop modded into a timed one (this is something no other prop in my plugins folder has). This strengthen the idea that what causes Prop Pox is changing an originally static prop into a timed one without changing its IID. This narrows the search for possible infected props in other prop packs.

For instance, I do have some of the CSX lots you listed (GA Farm - Nexis Chickens, GA Vineyard - Tuscany Merlot, CSX Produce Packaging and others) together with GrampaAl modded props and they also do not lead to Prop Pox.

So, Jabl0me, I suggest you keep your CSX lots and give us a little bit more of time to confirm the findings.

Bap

BarbyW

Thank you for the quick response, bap. It is good to know that the list I posted and the BSC MEGA props Misc Vol02 are unlikely to cause the Prop Pox in the same way that the other Peg files are probably fine as well. On the other hand the Prop Pox was known well before the release of the BDK file so there must be some other instances in the community. I have checked all the BSC prop packs that I know have Maxis modded props in and have not yet found any others without changed TGIs. I shall continue to search.
Inside every old person is a young person wondering what happened. TP



Barbypedia: More alive than the original

bap

Snorrelli could be of great help in this regard, as he is one who was infected prior to the release of the oww2-bdk pack. There must be at least one static prop modded into a timed prop without chaging IID sitting in his plugins folder waiting to be uncovered. I note that it may not necessarily be a Maxis prop. It may just be a custom static prop with a later (in term of reading sequence) redefinition as a timed prop.

Bap

z

Oh wow, this is a known bug with a known fix!  The Near West Side of Chicago can be saved!  &hlp

I guess this makes it about a dozen and one affected players.

Thanks everybody!  &apls

BarbyW

One possible known cause but to date there is no fix if you have the Prop Pox, sorry. Some of us are working to try to find the exact problem as there are other causes than one file and once we have more information it will be posted.
Inside every old person is a young person wondering what happened. TP



Barbypedia: More alive than the original

FrankU

Dear friends,

I accidentally found this thread very shortly after I discovered some pox also in one of my cities.
After having opened I found the Schönbrunn palace incomplete. The central prop was missing. I pityfully did not make a screenshot, but you can believe me.
This city, though, was not very large. It inhabits at the moment around 100.000 sims. The 2977aa47 file is 3.355.919 bytes large (so not even close to the 6Mb) and the cityfile as a whole measures 8.296.228 bytes.

I use extensive amounts of plugins. Some of the old CSX ones, a lot of PEG's, a lot of BSC's, and more.

The city is a large tile in a region of 18 (6x3) large tiles.
Here is an overview.



Would this information give new light on the subject?

wouanagaine

FrankU
First make a backup
then Make sure that by saving / loading multiple time that city,  others props are disappearing. When prop pox strikes, it , as this date,  usually removed more than one prop the first time.
If no, then you aren't prop poxed ( yet, as you use files that trigger the prop pox )
If yes, I'll definitely need the savegame




New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

bap

#155
Brief report

I performed an additional test. I changed the IIDs of the three Maxis modded static into timed props of the bdk-resource file and updated the corresponding lots to show the new IID props. Then I developed test city #2.1 with all my custom content, the props at PP and the buildings, lots and modds at MP, including the changed props. Just to make sure people understand the test, let me explain that, up to this point, city #2.1 has been developed exclusively with Maxis vanilla lots, buildings and props. No custom props, no modding. The city was developed up to the point where it has a 15.9 Mb savegame file with a 5.9 Mby 2977aa47 subfile size. So, it is close to the critical point where Prop Pox may manifest itself. If developed further exclusively with Maxis content, it shows no Prop Pox. Developing this city with all custom content and the corrected props also leads to no Prop Pox (this is the new result). Developing this city with all custom content and the infected props with their original Maxis IIDs lead to Prop Pox.

Because the only difference between the two latter tests is the change in IID of the three Maxis static props that were modded into timed props, one must conclude that the cause of the Prop Pox is having a static and a timed prop sharing the same IID in one's plugins, the static prop being read first by the program. By changing the IID of the timed prop one eliminates the conflict, and avoids Prop Pox. Beware: this statement does not mean we found a cure for an already infected city, just that we confirmed how can we avoid Prop Pox for brand new cities.

Cheers,
Bap

callagrafx

Can't fault the logic there....thank you Bap for your efforts.
The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it

Tarkus

bap, thank you so much for all the time and energy you've spent on illuminating the cause of this issue. :thumbsup:

Upon reading your latest report just now, I had an idea--what happens if the original Maxis Static Prop Exemplar is copied into a new .dat and loaded after the problematic Timed Prop?  

-Alex

cowcorn

Well I'm glad to see some light shed on this problem. This problem along with others inherent to collecting custom content for so long is what had caused me to get sick of the game.

Searching my memory about the state of what was my infected city I think I can bring some more information. The city had lots of medium and low density residentials, mots of which were custom content. However like all my cities in this region nearly all undevelopped space was covered by peg seasonnal trees which are timed props.

Knowing that, did anybody examine the possibility that those or any older version of them could also be associated with the pox.

I would also ad that I had a mod that added peg seasonal trees on the streets instead of the palms. I don't remember what the mod is named nor whose it was.

Swamper77

Cowcorn,

The problem isn't caused by timed props in general. The problem is caused by static Maxis props that get modded into timed props and still have the original ID. Those seasonal trees were created from scratch and have their own ID's. None of them have Maxis ID's. If they had Maxis ID's, then there will be a problem. I've checked the ID's myself.

That particular mod was Ardecila's and it used the same seasonal tree props, which shouldn't cause a problem.

-Swamper
You can call me Jan, if you want to.
Pagan and Proud!