• Welcome to SC4 Devotion Forum Archives.

NAM Traffic Simulator Development and Theory

Started by z, August 02, 2008, 05:07:50 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Shadow Assassin

Well, my mayor rating at the time was +50, and yet I somehow got a riot happening. I have no idea how that happened...
New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dedgren ♦ dmscopio ♦ Ennedi
emilin ♦ Heblem ♦ jplumbley ♦ moganite ♦ M4346 ♦ papab2000
Shadow Assassin ♦ Tarkus ♦ wouanagaine
See my uploads on the LEX!

z

It's not your global mayor rating, it's the local mayor rating of the individual tiles, which varies greatly.  You need to check it out in the Data Views.

ldog

#242
Quote from: Shadow Assassin on October 28, 2009, 06:54:19 PM
Just a question - it seems using Z Classic had an unintended consequence: are people supposed to riot as a result of traffic?

Or is this unrelated?

That's awesome!  &apls
*visions of LA and NY traffic riots danced through his head*
:'( I can't stop laughing  :'(

Z of course answered this well, but I got such a good laugh out of this I had to say something.

Shadow Assassin

#243
Quote from: z on October 28, 2009, 10:03:47 PM
It's not your global mayor rating, it's the local mayor rating of the individual tiles, which varies greatly.  You need to check it out in the Data Views.


I forgot to check that particular data view - but when I re-checked it, the mayor rating was quite red in that immediate area... perhaps b/c of traffic? There was also a major fire in the area which spread like... wildfire... just recently which would've dropped the rating some. But that is interesting how people just up and riot because of a combination of factors  - it only went away when I put a police station down.
New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dedgren ♦ dmscopio ♦ Ennedi
emilin ♦ Heblem ♦ jplumbley ♦ moganite ♦ M4346 ♦ papab2000
Shadow Assassin ♦ Tarkus ♦ wouanagaine
See my uploads on the LEX!

z

This post is in response to a request by jplumbley.  It concerns the effect of the pathfinding heuristinc parameter on the game.  Briefly, this parameter determines whether or not a traffic simulator has Perfect Pathfinding, merely good pathfinding, or just plain lousy pathfinding (though you won't find a traffic simulator bearing the latter name).

When I first built Simulator Z more than a year ago, I verified experimentally that Tropod's value of .003 for Perfect Pathfinding was correct, at least to within a few percent, and I have used this value in Simulator Z ever since then.  I also found through experiments that when this value was raised above .003 by any significant amount, abandonment due to commute time occurred, even when the maximum commute time in the simulator would be effectively unlimited, and even when the Sims were able to reach jobs for which they were qualified.  This effect was quite noticeable at a value of .009, which is one of the values I tested, and which is the value that is used in Simulator A.  I have mentioned this effect occasionally, and recently jplumbley challenged me on it:

Quote from: jplumbley on October 27, 2009, 09:12:48 PM
I would like you to proove this...  I have not had any problems what-so-ever with abandonment in my cities due to the PH [pathfinding heuristic] being higher than yours.  If you are so confident that the PH causes abandonment, proove it with Simulator Z and raise the value to 0.009 and show me the abandonment.

And later:

Quote from: jplumbley on October 28, 2009, 04:11:59 AM
You cant make that claim publically and then hide the test in a private board.  That is not right.  You should post it for everyone to see, since you have claimed it publically you should proove it publically.

So here is the post, showing exactly the test that jplumbley requested.

I'll begin with a view of the South Side of Chicago, taken in January of the year 431, during the last years of the Roman Empire.  ???  Many of the other pictures have their dates showing, so it's easy to see the timeline here.  The city has been running Simulator Z v1.2 (Low) for several decades; this version of Simulator Z was released as a beta version in August, and will be released unchanged as the official version of Simulator Z in the next NAM release.  Meanwhile, everything in the city is quite stable, with one notable exception, which I will discuss below.  Here's a picture of the city at this point; it's slightly smaller than subsequent pictures so that I could get a more panoramic view in, as I will be referring to different parts of it later.



As you can see the city looks quite healthy; if you look closely, you can see two or three buildings that have been downgraded from their initial wealth level, but other than that, all RCI buildings are occupied at their maximum wealth level.  There are no abandoned buildings.

Now let's take the exact same version of Simulator Z and modify it the way jplumbley requested.  We raise the Pathfinding Heuristic from .003 to .009, which is the value used in Simulator A.  That is absolutely the only thing we change in the simulator.  Now we run the city for an additional 19 years.  Here is a close-up view of one part of the previous picture after that amount of time has passed:



As you can see, this picture looks quite a bit different from the previous picture.  Buildings are starting to turn dark.  Usually, when buildings turn very dark, it's because they're abandoned.  But sometimes they're merely downgraded.  So let's query them:



And:



I could go on, but I think you get the idea.  I queried them all, and altogether, there are about a dozen abandoned buildings in this picture.  You may have noticed a big new building under construction in the second picture.  The construction is a result of the abandonment; there are now jobs looking for workers to replace those who left town.  At this point, this neighborhood enters a cycle of construction and abandonment, as there is demand for workers, but the pathfinder can't find a way to get them all to work.  As you might imagine, cyclic construction and abandonment is not good for a city's health.

Now we run the city for an additional 13 years; it has now been a total of 32 years that it's been running with the degraded simulator.  This is where I was able to verify what I had recently started noticing in other experiments; other problems begin to show up from the increased Pathfinding Heuristic as well.  Specifically, the population has started to rise:



You might wonder why a rising population is a problem.  Normally, it isn't in a growing city.  But this city isn't growing.  The only new construction is that which replaces abandoned buildings in the cycle I described above.  So where's the extra population coming from?  The following graph should make it clear:



Notice the two diverging lines.  They start to diverge 17 years before now, when the degraded simulation had already been running 15 years.  The top line is the population of R-$ Sims, which is increasing rapidly.  The bottom line is the population of R-$$$ Sims, which is declining.

In other words, the high-wealth Sims are leaving town and the low-wealth Sims are moving in.  This can be done with a stable number of buildings, since RCI buildings hold far more low-wealth Sims than high-wealth Sims.  And in fact, this is exactly what's happening.  Querying a lot of high-wealth buildings that have obviously been downgraded shows that they are now filled with low-wealth Sims, unlike the first picture.

To my knowledge, this effect has not been noticed anywhere before.  People often wonder why high-wealth residential buildings won't grow in their cities, and I've seen many valid explanations advanced.  But I have never seen it mentioned that the traffic simulator may be at fault, which is clearly what's happening here.

Meanwhile, earlier in this post I mentioned that there was an exception to the general stability of this city before I switched simulators.  You can see it in the first third of the above graph, which represents the period before the switch.  Here you see the exact opposite from what you see in the last third of the graph, although the effect is not as dramatic.  The high-wealth population of the city is increasing, while the low-wealth population is decreasing.  But here, the high-wealth population is increasing faster than the low-wealth population is decreasing, which would seem to imply that new high-wealth construction is occurring.  I've witnessed some of this construction, and I've seen this effect in all my cities; it started when I switched to the v1.2 version of Simulator Z.  Why is this increase happening?  I'm not sure; perhaps someone more familiar with the demand aspect of this game would know.

Finally, let's run the simulator a final four years and take a final look around:



Here we see more abandonment, although it's impossible to say exactly when it happened.  You can see many abandoned buildings near the bottom of the picture, to the left of the high school, and another group of abandoned buildings at the upper right of the picture, near the top edge of the city.

But have things leveled off?  After 36 years, is abandonment restricted to those areas where it's already occurred, or is it still spreading?  The following picture should answer that question:



Here we see five buildings with no-job zots.  $%Grinno$%  We know what's going to happen to them.

I should also mention at this point that I have meticulously kept up city services throughout this whole period.  It's obvious that there are no power or water problems.  But I have also maintained education, health, and other services at full capacity as well.  I have taken just as good care of this city when it was running the degraded simulator as I did when it was running the standard simulator.  And overall the other data graphs show little or no change; the biggest change is the Education and Life Expectancy graphs, which are both down about a couple of percent.

Before closing, I'd like to review one of the statements I quoted from jplumbley above:

QuoteI have not had any problems what-so-ever with abandonment in my cities due to the PH [pathfinding heuristic] being higher than yours.

At first, I was willing to take this statement at face value - I certainly have no reason to doubt jplumbley's experience.  But as I reflected on the the quote, I thought, How would he know?  For that matter, how would anyone running Simulator A or B know if "Abandoned due to commute time" was due to commute time limitations, or a pathfinding heuristic that was too high?  In truth, there is no way to know.  The information just isn't there.  So players who use traffic simulators with a pathfinding heuristic greater than .003 may experience the ill effects that I've shown above without having any idea (or any way to know) that it's the pathfinding heuristic that's causing these problems.  And in the cases where that is the cause, there is nothing the players can do to make their city work better, short of rebuilding it in a different way.

Just how serious is this problem?  As you might expect, it depends on your city.  The problem has to do with the traffic simulator's ability to find complex paths, and the complexity of paths rises exponentially with the size of cities.  My experience has been that all my Chicago cities (which are all on large tiles) experience this problem to a similar extent when played with a higher pathfinding heuristic, even though they are very different in nature.  These cities are all modeled on the actual city of Chicago, using all the city's streets and built exactly to scale.  Zoning is the same as Chicago's wherever possible; in some cities (including the Near South Side), I had to put in some extra commercial zones to balance things a bit, as the game at the time I built the cities couldn't handle the large distances that people regularly commute in the real city of Chicago.

So for some large cities, such as mine, the problem can be fairly serious, as you can see above; even after 36 years, abandonment was continuing.  For other large cities, where zones are built much closer together, the problem could be expected to be less serious.  However, it's hard to predict by how much.  In the second picture in this post, the buildings that are abandoned face buildings with suitable open jobs just a few blocks away.

With medium-size tiles, the problem could be expected to be far smaller, due to its exponential nature, and it could be expected to be far smaller still for small tiles.  But much more research would be needed to quantify exactly what this means.

In the mean time, I think it is safe to answer the original question in the following way:

Using a pathfinding heuristic of .009 or higher can lead to abandonment due to commute time, and this effect can be significant in large cities.  Such values for the heuristic can also make a city far less attractive to high-wealth Sims.  These effects become more pronounced both with increasing city size and increasing values for the pathfinding heuristic.

For reference, the pathfinding heuristic used in both Simulator B and Cogeo's proposed Universal Simulator are .025; the original Maxis simulator uses .09.  Tropod's personal traffic simulator used .003, the same as all versions of Simulator Z.  Tropod called this Perfect Pathfinding, and through experiments, I have verified that it is.  But description of those experiments will have to wait for another day...

jplumbley

Quote from: z on October 30, 2009, 03:55:47 AM
Before closing, I'd like to review one of the statements I quoted from jplumbley above:

At first, I was willing to take this statement at face value - I certainly have no reason to doubt jplumbley's experience.  But as I reflected on the the quote, I thought, How would he know?  For that matter, how would anyone running Simulator A or B know if "Abandoned due to commute time" was due to commute time limitations, or a pathfinding heuristic that was too high?  In truth, there is no way to know.  The information just isn't there.  So players who use traffic simulators with a pathfinding heuristic greater than .003 may experience the ill effects that I've shown above without having any idea (or any way to know) that it's the pathfinding heuristic that's causing these problems.  And in the cases where that is the cause, there is nothing the players can do to make their city work better, short of rebuilding it in a different way.

I appreciate that you posted your test.  I need some time to read it much closer though, later.

I do have a response on this quote though...  I know I do not get abandonment from the PH, because I do not get abandonment in my cities, especially not in the way you have shown in your city on this test.  The only time I start getting abandonment in my cities is when I have over-congested a section of my city, and in that case it's easy to fix by building another road, MT line, etc.

Maybe, it is just the way I play that I do not get abandonment in the way you do...  I dont know that answer.  But I do not have the same experiences you do.
"You learn something new everyday."

http://img517.imageshack.us/img517/169/nhpjplumbleykv3.gif
Bringing the new horizons closer to reality.

Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio jplumbley ♦ moganite ♦ M4346 ♦ Dedgren ♦ Ennedi Shadow Assassin ♦  Tarkus ♦ wouanagaine
Street Addon Mod - SAM

z

That sounds quite reasonable.  I know that it's certainly quite possible to use Simulator A without getting abandonment.  I think what you said in your last line sums it up - the way a person plays is a very big factor here.

Shadow Assassin

What would happen if the PH was reduced further? Would it start to degrade (ie, same way as increasing it to .009) or would it simply just slow the computer down even more as it tries harder to calculate a route accurately?

QuoteThe only time I start getting abandonment in my cities is when I have over-congested a section of my city, and in that case it's easy to fix by building another road, MT line, etc.

The problem is that some people don't like having to add an extra road, etc because it doesn't fit into their playing style. Which makes perfect sense, really. Everybody plays differently, and as a consequence, the simulator may not always be able to cope with the circumstances that are generated.

It's possible that your city was grown and geared to the particular PH that you originally set for Z (0.003), and once that was changed, the city no longer could cope as well with being able to find a way to and from work for its individual citizens. It's also possible that the simulator you begin your city with may affect a city more than originally thought when it is changed or altered.

Cyclic construction and abandonment is never good - and I've realised how useful a tool having the dilapidation occur is... if you use the No Abandonment mod, for instance, that doesn't actually take away the building's ability to abandon, which can be a problem because you don't get the visual confirmation that there is a problem in that area of the city.
New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dedgren ♦ dmscopio ♦ Ennedi
emilin ♦ Heblem ♦ jplumbley ♦ moganite ♦ M4346 ♦ papab2000
Shadow Assassin ♦ Tarkus ♦ wouanagaine
See my uploads on the LEX!

SC4BOY

Quote from: Shadow Assassin on October 30, 2009, 06:15:56 AM
...- and I've realised how useful a tool having the dilapidation occur is... if you use the No Abandonment mod, for instance, that doesn't actually take away the building's ability to abandon, which can be a problem because you don't get the visual confirmation that there is a problem in that area of the city....

I certainly agree with this part of your statement.. I never use that mod..

jplumbley

Quote from: Shadow Assassin on October 30, 2009, 06:15:56 AM
The problem is that some people don't like having to add an extra road, etc because it doesn't fit into their playing style. Which makes perfect sense, really. Everybody plays differently, and as a consequence, the simulator may not always be able to cope with the circumstances that are generated.

It's possible that your city was grown and geared to the particular PH that you originally set for Z (0.003), and once that was changed, the city no longer could cope as well with being able to find a way to and from work for its individual citizens. It's also possible that the simulator you begin your city with may affect a city more than originally thought when it is changed or altered.

I went looking for a quote I couldn't find because I feel I have said this before in some other post.  My goal for Simulator A was not to make the "perfect" Simulator fo ALL styles of play, simply because I do not believe that there can be one Simulator that suits everybody's needs or wants out of thier style of gameplay.  Some people want to build the pretty city for their MD.  For me, and some others out there, I like the chellenge of figuring out how to re-route my Sims to make better use of my transit network.  I could care less about population or how pretty everything is, my enjoyment comes from the transit system.  My Simulator was built to tune and make that part of the game still enjoyable, not remove it.


The Simulator you play with has great implications on how you build your city from the start.  This was one thing I tried to teach when I released Simulator A....  I suggested for people starting a new City that they start with the "Hard" version for two reasons, the first being with lower capacities you will see issues with your cities sooner as they grow and second so that people realize your cities will still work with "low" capacities in them, you just have to plan a little differently and actually think about how traffic moves in your city.
"You learn something new everyday."

http://img517.imageshack.us/img517/169/nhpjplumbleykv3.gif
Bringing the new horizons closer to reality.

Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio jplumbley ♦ moganite ♦ M4346 ♦ Dedgren ♦ Ennedi Shadow Assassin ♦  Tarkus ♦ wouanagaine
Street Addon Mod - SAM

sumwonyuno

#250
EDIT:  Ok, I got answers for the following post, from the other thread.

I realized something in another thread.  Referring back to my region, Sims weren't commuting to the industrial areas along the coast because they weren't the closest jobs.  Isn't the traffic simulator based on a residential building as the start point and the closest available job as the end point?  It makes sense when considering the case where commuters have jobs within the same city tile.

Neighbor connections are the closest jobs for many areas of a city tile.  If the game only considers the closest job for a commuter, why aren't "neighboring" commuters not going to the same neighbor connection?  There seems to be a bit of randomness, or some sort of determination that another "job" is closer.

For example, a ground highway neighbor connection is the closest job.  Sims that live right next to that neighbor connection are crossing over the highway and using a surface street neighbor connection instead.  There is an onramp somewhat further away, and the Sims living close to the on-ramp choose the ground highway neighbor connection.  But some of their neighbors don't, and choose the surface streets neighbor connections instead.  Also, when I have two neighbor connections side by side (to simulate a wide one-way street), the commuters choose either neighbor connection tile.




I used to have that cyclic dilapidation and construction before using Simulator Z, and I don't remember having that now.  I do have the CAM, and it's been present before I used Simulator A.

I've said the following in ldog's thread:
QuoteThere are instances where there are jobs and residential demand in the same city tile, but new housing development abandons.  

The latest example I had of this is when I widened a street and had to destroy a line of residential houses.  The area is desirable, and it's quite close to available jobs.  R$ Sims came, they had the no-job zots, and then their buildings abandoned.  So, I demolished those abandoned buildings and planted trees to jumpstart the area.  Sometimes new development comes, it's successful, other times nothing wants to get built (R, C and I).  I do the unzone-rezone-tree trick and it won't develop.  That may be another problem.  But, after many years later, successful development may occur without abandonment.  The only thing in the game that I know takes years to update is the traffic simulator.


The City & County of Honolulu, a Mayor Diary based on Honolulu, Hawai'i.

mark's memory address - I've created a blog!

z

Quote from: Shadow Assassin on October 30, 2009, 06:15:56 AM
What would happen if the PH was reduced further? Would it start to degrade (ie, same way as increasing it to .009) or would it simply just slow the computer down even more as it tries harder to calculate a route accurately?

I tried this when building Simulator Z, and the results were exactly as predicted by the A* algorithm.  Pathfinding accuracy did not increase at all, but CPU time rose significantly.  This is one of the factors that led me to believe that .003 was indeed the Perfect Pathfinding heuristic.

Quote
It's possible that your city was grown and geared to the particular PH that you originally set for Z (0.003), and once that was changed, the city no longer could cope as well with being able to find a way to and from work for its individual citizens. It's also possible that the simulator you begin your city with may affect a city more than originally thought when it is changed or altered.

It actually works the other way around.  When I originally built the Near South Side, I started out with a modified version of the CAM simulator, which uses a pathfinding heuristic of .0465.  The city was in much worse shape than shown in any of the pictures above.  I expanded the network capacities and then the commute times to those of the current Simulator Z.  This helped, but the city was still in a lot worse shape than shown above.  Finally, I started working with the pathfinding heuristic.  First, I dropped it to .025, which helped a lot.  Then I dropped it to .009, which helped even more, and in fact left me with a city looking very much like the one shown above for this value.  (Even the abandonment was in the same places.  And traffic jams that had existed for centuries in the area above where I first showed abandonment disappeared literally overnight.)  Then I gradually reduced the value all the way down to zero.  After .003, there was no further benefit though, and CPU times just started rising.  (With this prototype version of Simulator Z, there was still some abandonment due to commute time at .003, but it was caused by other factors, which have since been largely or completely fixed.)

Also, following the tie-breaking strategies that help A* work better, I was able to keep CPU time constant between .009 and .003.  This shift from exponential growth to constant use of the CPU was another indication that I had hit the perfect pathfinding number at .003.

So that's the story of how I arrived at .003 as being the Perfect Pathfinding heuristic.

When building the city, the changes that appeared from changing the simulator appeared quite rapidly.  However, repeating those tests by rebuilding the city from scratch would have taken a very long time.  Instead, jplumbley's suggested testing method was much faster, and ended up yielding the same results.  The tests themselves had to run longer, though, due to the way the traffic simulator works.  Once Sims have jobs, they hold onto them until something affecting their employment changes.  This means that once a city has been built with Simulator Z, you can switch to any other simulator, and the city will run just as well - for a while.  For example, I have taken the Near South Side and switched from Simulator Z to the Maxis simulator, which could never have been used to build that city.  Yet everything runs fine after I switch - for a while.  Even though the existing routes are largely in excess of what is permitted by that simulator, they persist, because the game sees no need to recalculate them.  Eventually, though, things begin to degrade, and as you would expect, they degrade much worse than in the example shown above.

So if you switch from some other simulator to Simulator Z, you will see the benefits quite quickly.  On a large tile, traffic patterns should show a major change within two years, and should be complete within eight.  On the other hand, if you move from Simulator Z to any other simulator, it will take substantially longer for the limitations of the new simulator to completely manifest in your city.

Quote
Cyclic construction and abandonment is never good - and I've realised how useful a tool having the dilapidation occur is... if you use the No Abandonment mod, for instance, that doesn't actually take away the building's ability to abandon, which can be a problem because you don't get the visual confirmation that there is a problem in that area of the city.

Agreed.  Between what is provided by Simulator Z and the CAM, I see no need for what is provided by the abandonment mod and the various demand mods, and I would recommend against them.

Quote from: sumwonyuno on October 30, 2009, 02:43:33 PM
Neighbor connections are the closest jobs for many areas of a city tile.  If the game only considers the closest job for a commuter, why aren't "neighboring" commuters not going to the same neighbor connection?  There seems to be a bit of randomness, or some sort of determination that another "job" is closer.

There actually is some randomness in determining whether or not a Sim will cross over into a neighbor tile for a job.  More specifically, my experiments with the Simulator Z prototypes showed that the probability that a Sim will do so is related to the product of the maximum commute type and the Sim's travel type speed.  That's why the maximum commute time is set so high in Simulator Z; it's to enable effective intercity commuting down to the level of car and bus speeds.  Of course, this can cause pedestrians living near the edge of a tile to be more willing to cross into a neighbor tile as well, but the result is usually much smaller than what you observed in your cities. 

ldog

I've got 2 new simple questions and being they are directly Z sim questions were more appropriate back here than in my thread.
Also I did not miss or discount above posts but I haven't had time to digest them so am not gonna comment(yet).

1. The clean air ordinance. A prime example of the kind of out of the box thinking that proves a lot of thought and effort went into sim Z. Now I understand why Maxis made it work the way they did. I also understand why you chose to make the modification you did. What I don't understand is why Maxis didn't make the car smogging ordinance work the way you did the clean air? And do you think it would be safe to make it work the way you did the clean air? I can explain what I mean in better detail but trying to keep it brief(er) :P

2. The traffic item (one of the other extra exemplars in Z). What exactly does it do? I mean I can see it adjusts scaling on one of the data graphs but I don't understand which or how much.

Also having been once again more closely re-reading some posts and reading some new ones besides.
Uh...wow...Steve I have seen so many instances of people being downright belligerent with you and you try to help them not once, not twice, but three times on average.
So while I "personally" didn't deserve the attitude you gave me initially, I can see how that would tend to wear on you after a year of putting up with it and I got the results of "this is what happens when people mess up things for the next guy to come along". I really cannot blame you for your initial reluctance to help me. This isn't an apology on my part by any means but...I what I am trying to say is that I have a newfound extra measure of gratitude to you for the time you have taken to answer my questions.

I hope no one here mistakes my enthusiasm for learning as trying to just start trouble or be a jerk. I know sometimes I get so into an argument (most people put a negative connotation on the word but I don't see it that way) that I forget my manners, and while I may not be getting upset, I can lose sight that the people on the other end might be.

jplumbley

Quote from: z on October 30, 2009, 05:36:51 PM
So if you switch from some other simulator to Simulator Z, you will see the benefits quite quickly.


I dont think this has been tested enough to claim that.  You've only tested on your city, a style where you have built the city to "scale".  It would be interesting to see how this one change in the PH effects other play styles.  There is a big question of why do I not get abandonment with a PH of .009 but you do?  Maybe there was an anomoly in your experiment?  I simply have not seen anything like that happen in my cities other then when I was using the MAXIS Default Simulator.

I am not trying to challenge your test, but I think it needs more testing.  There were numerous test of Simulator A before it was officially released and these tests were published in this thread:

http://sc4devotion.com/forums/index.php?topic=3508.60

Only one or two of them even mentioned having abandonment problems which were determined to be from congestion problems in their cities.  The one that would stand out most was the test by Nate (xxdita), he had about half of his city go abandoned due to the layout of his zones with all the residential in the corner and the jobs both Ind and Com on the opposing sides forcing all the Sims to snowball through the City before any of them reached thier destinations.  After some re-zoning, his abandonment issue was gone and his medium tile City rose from under 800k in population to over 1.8 million.  After the thread was closed, I still talked with Nate on MSN from time to time never once did he have abandonment problems after the re-zoning and his medium tile city reach an impressive 2.8 million Sims while he was testing the CAM.

There were about 8 or 10 people who provided testing results in that thread, I find it hard to discount that they did not report any abandonment problems that could not be attributed to simple congestion.
"You learn something new everyday."

http://img517.imageshack.us/img517/169/nhpjplumbleykv3.gif
Bringing the new horizons closer to reality.

Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio jplumbley ♦ moganite ♦ M4346 ♦ Dedgren ♦ Ennedi Shadow Assassin ♦  Tarkus ♦ wouanagaine
Street Addon Mod - SAM

z

#254
Quote from: ldog on October 30, 2009, 06:36:36 PM
What I don't understand is why Maxis didn't make the car smogging ordinance work the way you did the clean air?

The Car Smogging ordinance is a strange bird indeed.  Its effect on the various types of air pollution is identical in every way to the unmodified Clean Air Act.  Your question is one of the first ones that came to me when I looked at its insides.  I don't know the answer.

QuoteAnd do you think it would be safe to make it work the way you did the clean air?

It would certainly be safe.  The reason I chose the Clean Air Act is that the property that I modified affects all vehicle pollution, not just cars.  So I saw it as essentially an omission from the Clean Air Act.  The Car Smogging ordinance just seemed to me to be a mistake that was better left alone.

QuoteThe traffic item (one of the other extra exemplars in Z). What exactly does it do? I mean I can see it adjusts scaling on one of the data graphs but I don't understand which or how much.

It scales the Commute Time Graph to be about as accurate as it can be (which is not very accurate).  See this post for the gory details.

Quote from: jplumbley on October 30, 2009, 06:38:16 PM
I dont think this has been tested enough to claim that.  You've only tested on your city, a style where you have built the city to "scale".

I also tested this on my New York cities, which were not built to scale, were built on medium and large tiles, and were originally built in a completely different style, as they were the first cities I built in SC4.  My conclusions were also based on the feedback of my testers, and later from users.

Since I so often refer to "feedback from users," "quotes from users throughout the threads," etc., and since these would take a lot of work for any individual to find, rather than have everyone who is interested do all that work, I will put these all together in a single post.  They will be numbered, and linked back to their original post.  That way, it will be much easier to cite the data behind many of my conclusions, and much easier for people to verify it.

QuoteThere is a big question of why do I not get abandonment with a PH of .009 but you do?

That's actually very easy to explain.  As you have correctly stated previously, Simulator A allows a Sim to drive across an entire large tile during a commute period.  This means that on a medium tile, a Sim can drive from any point to any point during a commute period, as long as there is not excessive congestion.  So for most medium tiles, there should never be any abandonment due to commute time caused by exceeding the maximum commute time.

As for the pathfinding heuristic, I described how the complexity of paths varies exponentially with the size of a tile, which means that it declines exponentially when going from a large tile to a medium tile.  As the complexity of paths declines, the need for perfect pathfinding declines, and a value of .009 is sufficient for a much larger percentage of cases.  (But not all, as I will soon show.)  And if you build with your zones close together, you reduce the path complexity further, making it even more unlikely that a value of .009 will give you trouble.

QuoteMaybe there was an anomoly in your experiment?

I don't think so, especially since I ran two types of experiments (forwards and backwards) a year apart, and I also ran experiments in different cities with different styles, and still got identical results.  Also, I am able to explain both my results and yours without contradiction.  Instead, I think that this effect appears much more often than people realize, and has simply gone unrecognized until now.  For example:

QuoteThere were numerous test of Simulator A before it was officially released and these tests were published in this thread:

http://sc4devotion.com/forums/index.php?topic=3508.60

Only one or two of them even mentioned having abandonment problems which were determined to be from congestion problems in their cities.  The one that would stand out most was the test by Nate (xxdita), he had about half of his city go abandoned due to the layout of his zones with all the residential in the corner and the jobs both Ind and Com on the opposing sides forcing all the Sims to snowball through the City before any of them reached thier destinations.

This is just the type of example I was looking for.  First, let's look at Nate's introductory comment:

QuoteSuddenly, I'm facing abandonment & my traffic network just isn't able to deal with the higher stages of CAM. Can't get anything above a stage 13 to grow, even with ample Regional Capacities, and all other caps met. Played for 20 years, and routes changed consistently.

The last sentence is what's most significant - the routes changing consistently.  Why should they do that if the pathfinding is correct?  For that matter, why should he get any abandonment at all?  The one thing that could cause Sims to run past the maximum commute time on a medium tile running Simulator A is excessive congestion.  Fortunately, Nate posted his congestion map:


That doesn't look too bad.  If you think that the congestion on the diagonal is a problem, remember that there are no true diagonals, and that any combination of orthogonal routes is just as efficient.  In fact, the congestion on the diagonal and the uneven congestion elsewhere is in itself evidence that the pathfinding heuristic is set too high.  If you look at all the examples of Simulator Z, you see that congestion is spread out evenly to minimize travel times.  And it's Perfect Pathfinding that allows that even spreading.

If you don't accept this explanation, how do you explain the abandonment?  There's enough commute time and enough network capacity; the pathfinder simply isn't finding workable routes.  That's purely a function of the pathfinding heuristic.  Your solution was to have Nate move his zones closer together.  That fixed his problem because it simplified the paths.  A value of .009 was now sufficient for workable paths to be found.

I remember when this all happened, and Nate really became a convert to your simulator after this.  There was one post that particularly stuck in my mind.  One user was complaining that his city used to work, but now with Simulator A, he was getting abandonment.  Nate's reply was that this person had to be better at zoning, and as good mayors, we should be up to the challenge.

In other words, we were supposed to change our cities to fit the simulator.

This was the first time I began to question what was happening with Simulators A and B.  It seemed to me that the traffic simulator should be designed for the city, not the other way around.  So this is when I began work on what eventually became Simulator Z.

QuoteAfter the thread was closed, I still talked with Nate on MSN from time to time never once did he have abandonment problems after the re-zoning and his medium tile city reach an impressive 2.8 million Sims while he was testing the CAM.

This is not surprising.  If you build cities that are designed to work well with Simulator A, then they will work well with Simulator A.

Quote
There were about 8 or 10 people who provided testing results in that thread, I find it hard to discount that they did not report any abandonment problems that could not be attributed to simple congestion.

As I have tried to show, some abandonment problems, such as Nate's, were caused by pathfinder inefficiency, but were not recognized as such.  I see no evidence in that thread that such a possibility was even considered, much less tested.

jplumbley

#255
Quote from: z on October 31, 2009, 02:02:29 AM
This is just the type of example I was looking for.  First, let's look at Nate's introductory comment:

The last sentence is what's most significant - the routes changing consistently.  Why should they do that if the pathfinding is correct?  For that matter, why should he get any abandonment at all?  The one thing that could cause Sims to run past the maximum commute time on a medium tile running Simulator A is excessive congestion.  Fortunately, Nate posted his congestion map:


That doesn't look too bad.  If you think that the congestion on the diagonal is a problem, remember that there are no true diagonals, and that any combination of orthogonal routes is just as efficient.  In fact, the congestion on the diagonal and the uneven congestion elsewhere is in itself evidence that the pathfinding heuristic is set too high.  If you look at all the examples of Simulator Z, you see that congestion is spread out evenly to minimize travel times.  And it's Perfect Pathfinding that allows that even spreading.

If you don't accept this explanation, how do you explain the abandonment?  There's enough commute time and enough network capacity; the pathfinder simply isn't finding workable routes.  That's purely a function of the pathfinding heuristic.  Your solution was to have Nate move his zones closer together.  That fixed his problem because it simplified the paths.  A value of .009 was now sufficient for workable paths to be found.

Do you remember that Congestion Data View mod I made that didn't really work as well as I thought it did....  Guess what, it was used in that experiment.

The problem was congestion, you can tell by the quarter circle of red routes that are shown around the Residential area.  And the fact that the problem was fixed simply by re-zoning and spreading out the Sims through the jobs.  Like I said, Nate has not complained about abandonment problems since, and in fact after talking to him yesterday still uses Simulator A with medium tile cities in the millions for population.  If he hasn't reported abandonment problems in the 2 years he has been using Simulator A, then maybe there is something else that is effecting the way the PH works.

You agree that when you change one value in the Simulator that you will need to change a number of other things as well...  Maybe something you have changed in your Simulator Z that is different from Simulator A has made it so that the PH is more sensitive to these changes than it is in Simulator A.  It just does not make sense that I amoungst many others do not experience the same abandonment problem you do, unless there is something you have changed with Simulator Z that makes it more sensitive to the changes in PH.

Quote
I remember when this all happened, and Nate really became a convert to your simulator after this.  There was one post that particularly stuck in my mind.  One user was complaining that his city used to work, but now with Simulator A, he was getting abandonment.  Nate's reply was that this person had to be better at zoning, and as good mayors, we should be up to the challenge.

In other words, we were supposed to change our cities to fit the simulator.

This was the first time I began to question what was happening with Simulators A and B.  It seemed to me that the traffic simulator should be designed for the city, not the other way around.  So this is when I began work on what eventually became Simulator Z.

This is not surprising.  If you build cities that are designed to work well with Simulator A, then they will work well with Simulator A.

Let me ask you, what happens in the real world?

Does traffic flow in the way "WE" tell it to?
No, it doesnt...  We provide a number of routes for the population to use, but ultimately it is thier choice how they drive to work.

Can we change a factor in the "Simulator" of life that will make it all better?
Of course we cant...  There is no Simulator to modify to make our cities work.  They will just keep getting more and more congested.

What did they have to do in Boston when thier highway was over-crowded causing 10-hour a day traffic jams?
Boston had to re-build thier highway underground in a project called the Big Dig, because they did not have the option to expand the old one.

What is the trend in Europe in regards to zoning?
Due to the space requirements in Europe, they are forced to bring residents and work closer together not only for the reason of limited space but because it reduces overall traffic in the cities.

What is happening in the downtown cores around the world which used to be reserved for commercial towers?
At least in Toronto anyways and I am sure in other parts of the world, there are more and more condos (residential units) being built right in the middle of the downtown core.  The initiative is there for the population to live closer to work instead of commute from the "bedroom" communities so that we can reduce traffic, promote walking or MT usage.

So, do I think Simulator A has addressed real life situations and kept them part of the game?
Most definitely and there is no way you are going to convince me otherwise.  Simulator A is NOT a Sandbox Simulator like Simulator Z and it never will be.

I think you need to accept the fact that our goals are different and finally admit that Simulator A has done what was intended and works fine for what it was intended to do.  At the same time you need to stop bad mouthing Simulator A and putting it down for simply being a more challenging Simulator than Simulator Z.

Quote
As I have tried to show, some abandonment problems, such as Nate's, were caused by pathfinder inefficiency, but were not recognized as such.  I see no evidence in that thread that such a possibility was even considered, much less tested.

The users were asked to report abandonment.  If they did not report it, then where was the inefficiency?  Nate and SC4BOY were the only ones, SC4BOY was because he had 1000s of Sims trying to use 1 street to enter his CBD.  Nate was because he need to re-zone.
"You learn something new everyday."

http://img517.imageshack.us/img517/169/nhpjplumbleykv3.gif
Bringing the new horizons closer to reality.

Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio jplumbley ♦ moganite ♦ M4346 ♦ Dedgren ♦ Ennedi Shadow Assassin ♦  Tarkus ♦ wouanagaine
Street Addon Mod - SAM

xxdita

#256
Quote from: z on October 31, 2009, 02:02:29 AM

This is just the type of example I was looking for.  First, let's look at Nate's introductory comment:

The last sentence is what's most significant - the routes changing consistently.  Why should they do that if the pathfinding is correct?  For that matter, why should he get any abandonment at all?  The one thing that could cause Sims to run past the maximum commute time on a medium tile running Simulator A is excessive congestion.

As I said in the testing, play was for 20 years before my initial report. I don't think it was so much as the PH not working as it was the city & the game itself adjusting to the new traffic simulator. Which was a fairly significant adjustment...

Quote
Fortunately, Nate posted his congestion map:


That doesn't look too bad.  If you think that the congestion on the diagonal is a problem, remember that there are no true diagonals, and that any combination of orthogonal routes is just as efficient.  In fact, the congestion on the diagonal and the uneven congestion elsewhere is in itself evidence that the pathfinding heuristic is set too high.  If you look at all the examples of Simulator Z, you see that congestion is spread out evenly to minimize travel times.  And it's Perfect Pathfinding that allows that even spreading.

Even more fortunately, I also posted all of my congestion views. The only diagonals used in the city are inmy subway system. So Simulator A worked like a charm, increasing the usage of the mass transit.


Quote
If you don't accept this explanation, how do you explain the abandonment?  There's enough commute time and enough network capacity; the pathfinder simply isn't finding workable routes.  That's purely a function of the pathfinding heuristic.  Your solution was to have Nate move his zones closer together.  That fixed his problem because it simplified the paths.  A value of .009 was now sufficient for workable paths to be found.

First, let's take a look at the city in question.



That was my playing style at the time, and it worked for me then. It really shouldn't have, and I doubt that it would have in the vanilla version of the game. For a small city, sure. But not a medium tile. I've never claimed to be any good at actually playing the game, but I've improved a bit, thankfully.

Quote
I remember when this all happened, and Nate really became a convert to your simulator after this.  There was one post that particularly stuck in my mind.  One user was complaining that his city used to work, but now with Simulator A, he was getting abandonment.  Nate's reply was that this person had to be better at zoning, and as good mayors, we should be up to the challenge.

In other words, we were supposed to change our cities to fit the simulator.

But our playing styles had already adapted to the traffic simulators previously available, however inept they've been since proven to be. Out of all currently or previously publicly available traffic sims, I would still recommend A at least 9 times out of 10. Not because I consider Jason a friend, which I do, but because it has worked the best, for me, with the least amount of problems. But no publically available traffic sim suits my personal needs when it comes to large city tiles, which is why I've designed and use my own, based on Simulator A Easy.

Quote
This was the first time I began to question what was happening with Simulators A and B.  It seemed to me that the traffic simulator should be designed for the city, not the other way around.  So this is when I began work on what eventually became Simulator Z.

This is not surprising.  If you build cities that are designed to work well with Simulator A, then they will work well with Simulator A.

As I have tried to show, some abandonment problems, such as Nate's, were caused by pathfinder inefficiency, but were not recognized as such.  I see no evidence in that thread that such a possibility was even considered, much less tested.

Notice that in the test city, my transportation system consisted of streets, roads, and avenues, mixed with buses & subways. The abandonment occured because this system was inadequate. Though it had worked quite well with the previous traffic sim I had used, which I should point out was one that was included with CAM (the easiest one, with promote biking). So adding in a new traffic sim completely changed the network capacities, and all that jazz. No longer could my Sims get to work very effectively. Also, the buildings that were upgrading, fairly quickly, closer to the jobs, caused an even more instense traffic nightmare for the Sims that lived farther away from work.
Not only was my style of playing in grids no longer acceptable, I had to seriously improve my mass transit system, going from:



To:



But I had played like an idiot because the traffic simulator I had used allowed me to do so. And now I've learned how to actually play the game, based on what I want my city to look like as it grows, and what it will take to make that happen. Granted there have been other factors that have helped as well, like a much better overall understanding of the game mechanics as a whole.
Oh, and cheating.

z

Quote from: jplumbley on October 31, 2009, 09:47:47 AM
Do you remember that Congestion Data View mod I made that didn't really work as well as I thought it did....  Guess what, it was used in that experiment.

Oops.  Well, I guess that makes that graph pretty worthless, then.  And the conclusions I drew from it can't be supported without additional data.  And the volume data graphs are the standard Maxis ones, which top out at a volume of 1200, so no congestion information can be gleaned from them.

However, I have designed some new experiments to address your questions.  They take a long time to run, so the final results won't be ready for a while.  But the preliminary results already show some rather interesting things.  I will address  your other comments at that time.

Quote from: xxdita on October 31, 2009, 10:47:15 PM
I would still recommend A at least 9 times out of 10. Not because I consider Jason a friend, which I do, but because it has worked the best, for me, with the least amount of problems. But no publically available traffic sim suits my personal needs when it comes to large city tiles, which is why I've designed and use my own, based on Simulator A Easy.

I appreciate your contributing here, Nate, especially as I consider this a real opportunity to talk to a very skilled player who is quite enthusiastic about Simulator A.  This most recent discussion started out with some tests trying to determine what the differences between Simulators A and Z actually are in a quantifiable way.  The general subject of this thread is the development of Simulator Z, as I'm always looking for ways to improve it.  I'd like to ask you a few questions here, basically for research purposes.  You are obviously quite happy with your current simulator, and I assure you that I am not here to debate you on any points.  But I think that knowing what a player such as yourself looks for and likes in a simulator could be very useful information for further simulator development, especially since you have gone to the trouble to design your own simulator.  So if you wouldn't mind answering the following questions, I would be very grateful.  And this is a one-time request; I do not intend to turn this into a discussion.

You've obviously used other traffic simulators.  Could you please list which ones (and the capacity levels, where appropriate it?)

If the above list contains the Low version of Simulator Z, what didn't you like about it?  (Again, there will be no response from me.   $%Grinno$%)

What do you see as the biggest strength(s) of Simulator A?

What are the problems with other simulators that you referred to?

Were you ever able to build large cities (1 million Sims or more) on large tiles with Simulator A that had no abandonment due to commute time?

Which factors led you to design your own simulator, and what does it do?

Would you be willing to attach a copy of your simulator to your reply?  I always like looking at other simulators...  ;D

Quote
As I said in the testing, play was for 20 years before my initial report. I don't think it was so much as the PH not working as it was the city & the game itself adjusting to the new traffic simulator.

The new tests I am currently running have been designed to separate out many of these factors.  It appears that the PH is involved a lot more than anyone thought, including me.  Part of the reason that this has been missed is that the PH affects congestion, which of course affects commute time.  So the utlimate effect may be that the Sims do exceed their maximum commute time, but the original cause of all this may be that the PH is too high.

There's not enough data at this point from your old city to say how much of a part the PH played.  (Unless you've still got the old CAM version around, which I realize is pretty unlikely.)  However, the experiments I'll be displaying in the next day or so should give a good idea of how this functions in general.

Thanks for taking the time to read this, and thanks for whatever input you can give.

ldog

Ok, what I don't really get is how in Z you could ever not be able to find a valid route.
Valid route defined as "A there and back route you can complete without running out of time"

Even if we had to go the full diagonal across a large tile and back with your max commute time of 60...
1024/60=17

So as long as we can do at least 17 kph on average we can make it.
At maximum congestion all trains are still much faster, so is the highway.
Driving on roads, ave, owr at 30% speed we come up with 15, also your walking speed.
Since 1024 is absolute longest distance most of the time it is of course going to be much less, even if we have to drive out of our way to catch a train or go around a lake, mountain, etc.

So pretty much no matter what we should be able to find a valid route.

Has to be something outside the traffic simulator.

SC4BOY

Don't forget the "across the border" commuters .. and especially if your network design allows "commuter loops" you can have HUGE commute times