NAM Traffic Simulator Development and Theory

[z]

Release of the NAM Unified Traffic Simulator v2.4

As I mentioned in my previous post, I've had some features that I've been thinking of putting into the NAM traffic simulator for a while, so I finally put them in.  Here is what they are:

• The speed of el rail trains has been raised from 105 kph to 115 kph.  This now makes them a little bit faster than subways, which remain at 105 kph.  This attracts more traffic to el rails; this was done because in cities with extensive subway networks, it was very hard to get much traffic at all on el rail networks.  This change will also increase the traffic on trams, since they use the same underlying network.  However, since trams and el rail account for just a small portion of a city's overall transportation (especially compared to the road networks), this change does not have a noticeable impact on the usage of the other transportation networks.
• The monthly maintenance cost of subway lines has been reduced from §1.8 per tile to §1.2 per tile.  This is still four times the cost per tile in the standard Maxis simulator.  The reason for this change is described below.

The new version of the traffic simulator has been integrated into the NAM Traffic Subsystem.  Additionally, the Traffic Simulator Configuration Tool has been updated with both new values.  However, if you use the TSCT with an existing NAM traffic simulator, it won't automatically update the subway maintenance cost, since that is one of the fields it allows you to edit, and it can't be sure that you don't like the number as it is.  In any case, you are free to change it.  For new installations of the NAM traffic simulator, the value of §1.2 per tile per month will be used.

Both the new traffic simulator and the new TSCT will be integrated into NAM 31 when it is released.

For those who would like more information on the rationale behind these changes, the following is a slightly edited version of the proposal I gave to the rest of the NAM Team:



There hasn't been much news out of Traffic Simulator Land recently.  However, there are a couple of simple things I would like to do.  Both of them have to do with the destination finder.

A more complete description of the destination finder and its workings can be found in A Guide to the Operation of the Traffic Simulator.  But the part that I would like to address here is its notorious habit of restricting the maximum distance of Sim residences from possible jobs, in the process ignoring the "Commute Trip Max Time" property.  This means you can get a lot of abandonment due to commute time regardless of what you set the maximum commute time to be.

One saving grace, though, is that the destination finder assumes that if you have rapid transit in your city, Sims will be able to travel farther, and it increases its limits accordingly.  The more rapid transit you have, the more these limits are increased.  It is possible to get rid of abandonment due to commute time completely or nearly completely in any city, but in big cities this requires a lot of rapid transit and stops.  Subways are the only feasible mode of rapid transit if you want to do this in big cities, because anything else would take up far too much real estate.

Initially, these were merely my own observations, and they actually date back to SC3K, which apparently used a similar type of traffic simulator, at least in this respect.  But since the initial release of Simulator Z (now the NAM Unified Traffic Simulator), I've recommended this approach to people who are using the NAM simulator and still having abandonment problems, and it has worked very well for those people who have tried it.  So this does not appear to be anything unique to my cities, but instead appears to be a fundamental principle of the way the game works.

This brings me to my proposed modifications.  Having a lot of subways in a city does impact other travel types of course, but this impact is unavoidable and still leaves most travel types with plenty of traffic.  However, there is one travel type that suffers dramatically from the presence of a lot of subway lines, and that is the el train.  El trains have the same speed as subways, but in the scenario I've described above, there are a lot fewer of them compared to subways, and their stations also tend to be spaced farther apart.  As a result, el train usage suffers dramatically from the introduction of a large subway network.

Early on in the development of Simulator Z, I tried boosting the speed of el trains slightly to counteract this effect, but this strategy did not yield noticeable benefits.  However, this was before I discovered what the true perfect pathfinding heuristic for the traffic simulator was.  Unfortunately, I did not revisit the issue until recently.  However, true perfect pathfinding makes differences in travel times of even a fraction of a second count.  So when using perfect pathfinding, a small (9.5%) increase in the speed of el trains is enough to generate some traffic on lines that previously had none, and increase traffic by 50% to 100% on lines that already had traffic.  This is still not a lot of traffic in absolute terms, but it is enough so that the existence of the el rail lines seems a lot more justified.  So my first proposal is to raise the speed of el trains from 105 kph to 115 kph.  Although speeds in SC4 are measured in kph, these numbers shouldn't be taken too literally; instead, their importance is their relationship to other speeds and other parameters in SC4.  For reference, subways currently have a speed of 105 kph, commuter rail is 140 kph, highways are 150 kph, and monorail is 225 kph.  (In the Maxis traffic simulator, subways, el rail, and commuter rail were all 150 kph, highways were 82 kph, and monorail was 200 kph.)

My second proposal is also related to the usefulness of large numbers of subway lines in large cities.  In the Maxis traffic simulator, although subways are somewhat expensive to build, they're still incredibly cheap compared to the cost to build them in RL.  I wanted to remedy this, but the cost of building subway tiles was not located in the traffic simulator exemplar; instead, it is in the Subway Placement Tuning Parameters exemplar, which is also affected by various other mods (such as slope mods).  So instead of increasing the cost of building subway lines, I increased the cost of maintenance, which is located in the traffic simulator.  The rationale here is that in the real world, major projects such as subways are often paid for by municipal bonds, which cost the city a fixed amount over a number of years.  So the increase in monthly maintenance cost was effectively equivalent to financing a large part of the construction of subways through the issuance of interest-only bonds.  In this way, the cost of subways became more realistic.

Initially, I quadrupled the monthly maintenance cost, from 0.3 simoleons per tile per month to 1.2 simoleons.  In a later release of Simulator Z, I increased this further to a cost of 1.8 simoleons.  However, this was before the value of large subway networks was fully realized.  It turns out that at a rate of 1.8 simoleons, creating enough subway lines to avoid the abandonment problem can be extremely difficult in certain cities without busting the budget.  Therefore, I propose going back to my original cost of 1.2 simoleons per tile per month, which is still four times the Maxis cost.  And as this cost is one of the properties that can be tweaked in the Traffic Simulator Configuration Tool, sophisticated players can increase it if they want more of a challenge.  But this way, subway costs aren't too onerous for people who are less expert at playing the game.

[b22rian]

First Steve,

                       Thanks for your continued commitment to keep improving
the NAM traffic simulator. Especially with as much time as you are
giving to such a massive undertaking as the next release of RTMT.

         For sure the best change you are making in this latest version for
me is increasing the El train speed. 
I think the reason why subways get used so much is mostly because there
so easy to build underground. So you end up with ideal efficient paths down
there that the Traffic Sim makes use of as it choice a lot of times. Although
I am sure part of this for me is my personal preferences in game for using
the GLR. But I approve fully of making GLR + El train traffic a bit faster
than Subways because of the reasons you cited in your post.

Thanks, Brian

               

[Frex_Ceafus]

I think this would be a great idea Steve, I always like the el-rail and Tram more, as you get to actually see the trains heh.

Thanks for your hard work and dedication,


Clayton

[pierreh]

I agree too, and I add my thanks for this good work.

When I still had time to play the game, I ran into some interesting findings about using trams (aka GLR) in large cities, and I wanted to present them, discuss and get advice. These days I cannot find enough free time for the game, so this will have to wait. But this remains for sure an area of great interest to me.

[catty]

I agree too, and I add my thanks for this good work...

ditto ... anything that will give me more trains, I love trains, nearly bought a house last year that had the main railway line for the town I live in at the bottom of the garden, the trains would have been going by ... about 30 feet from the back bedrooms   

No guest bedroom and in the end ... no trains   

[nettronic]

How is traffic calculated in game? Is one car = 1 traffic point?

If so than the traffic configuration tool does not appear to function properly. It says it increases capacity, however I am getting traffic alerts (and red highlights) on an avenue with only 3000 cars, a few dozen pedestrians and a handful of freight trucks. (Barely breaking 3K not even close to 4K listed.

Win 7x64 (I have run the tool as administrator as well).

the tool appears to work overall though as I am having problems getting enough customers to my CBD now that the coefficient was adjusted.

(Which brings up another question, how can my green street residential neighborhood have "high traffic noise" while my CBD located next to a red highway have low customers???)

[Tarkus]

From the image I saw on your other posting, that's happening at intersections.  The traffic simulator has a property, the Intersection and Turn Capacity Effect, that decreases the capacity/speed at intersections, to simulate delays and queueing from traffic control devices (e.g. signals).  The NAM simulator calculates this effect a bit differently, shifting the burden away from the actual intersection itself (if you're waiting for a signal to turn green, why would the actual intersection itself be congested?) and onto the tiles leading up to the intersection.  The side-effect of this is that things show up a good bit redder on Congestion View.

You can dial down this effect by lowering the value of the Intersection Effect property in the Traffic Simulator Configuration tool.  The minimum value (0.2) eliminates it entirely.

-Alex

[nettronic]

From the image I saw on your other posting, that's happening at intersections.  The traffic simulator has a property, the Intersection and Turn Capacity Effect, that decreases the capacity/speed at intersections, to simulate delays and queueing from traffic control devices (e.g. signals).  The NAM simulator calculates this effect a bit differently, shifting the burden away from the actual intersection itself (if you're waiting for a signal to turn green, why would the actual intersection itself be congested?) and onto the tiles leading up to the intersection.  The side-effect of this is that things show up a good bit redder on Congestion View.

You can dial down this effect by lowering the value of the Intersection Effect property in the Traffic Simulator Configuration tool.  The minimum value (0.2) eliminates it entirely.

-Alex

Thanks Alex, I wish there was a thanks/rep system on SC4D you guys have been very helpful answering all my questions

[z]

Release of NAM Traffic Simulator v2.5 Beta 1

Perfect Pathfinding Revised Again

With this post, I am reviving an old tradition of releasing open beta versions of the NAM Traffic Simulator for widespread testing before they are formally adopted.  In the early days of the NAM Traffic Simulator (especially when it started out as Simulator Z), such open beta testing was routine, as the traffic simulator was venturing into many previously unexplored regions, and open beta testing was a necessity for making sure that new versions of the traffic simulator did not contain any regressions.  This was especially important considering the central role that the traffic simulator plays in the game.  In the last few years, the NAM traffic simulator has reached a point of great stability, and changes to it have been relatively minor and limited in scope.  Because of this, the NAM Team's internal testing has been sufficient to validate the few changes that have been made in the traffic simulator over this time.

This new version of the NAM Traffic Simulator features a change in one of the fundamental constants of the simulator - in fact, it is likely the most fundamental constant in the entire simulator, and one of the most important constants in the entire game.  It affects virtually all aspects of the traffic simulation, and as a result, has a large effect on much of the operation of the game.  For this reason, an open beta test was deemed necessary before this version of the traffic simulator is included in the next NAM release (NAM 32).

The constant that has been changed is the Pathfinding Heuristic, which determines the accuracy of the traffic simulator's pathfinding.  Basically, with perfect pathfinding, the Sims always choose the fastest route from their homes to their jobs.  The last time this constant was changed was almost four years ago, and is described in this post, which announced the release of Simulator Z v2.1.  Throughout the history of the NAM Traffic Simulator, Simulator Z, and various other (but not all) traffic simulators released by the NAM, a major goal has been to find what is technically known as the perfect pathfinding heuristic for the Maxis traffic simulator engine.  For much of the history of the NAM, this value was thought to be .003, which is the value used in Tropod's perfect pathfinding simulators.  For reasons that are detailed in the post mentioned above, the correct value for this constant is notoriously difficult to determine, which is one of the main reasons that earlier versions of Simulator Z used Tropod's value of .003.  However, for Simulator Z v2.1, I found an indirect method for getting a much more precise value of the perfect pathfinding heuristic, and tests and experience quickly showed that this new value (.005797) was a significant improvement over the previous value of .003.

The indirect method that I used, described in the above-referenced post, allowed me to get an extremely accurate value for the perfect pathfinding heuristic, which is why it contains four significant figures.  And until recently, there were no indications that this constant did not indeed provide perfect pathfinding for the traffic simulator.  However, the NAM Team has recently been doing extensive testing of various transit stations, and in the course of this testing, we found a few paths that the Sims were taking to work that were not the fastest available.  This meant that either the traffic simulator was broken, or that we had an incorrect value for the perfect pathfinding heuristic.  Further testing showed that there was a nearby value for the perfect pathfinding heuristic where these pathfinding errors disappeared.  At the same time, certain other problems in the traffic simulation disappeared, while no new problems appeared.  Therefore, it was clear that the new value for the perfect pathfinding heuristic was a better value than the old one.  The new value is .0054.

You'll notice that there are fewer significant digits in the new value than in the old one.  The main reason for this is simple:  We are working with fewer data points.  However, the data we're working with here are actual paths, whereas the previous perfect pathfinding heuristic was developed by using the level of abandonment due to commute time as a proxy for the level of pathfinding accuracy.  It is possible that with time greater accuracy can be determined for this number, but for now, we know that it is a clear improvement over the previous number.  You can also see that the degree of change in these numbers has decreased significantly; we first went from Tropod's .003 to .005797, and then from .005797 to .0054.  This also means that changes in game play will generally be fairly subtle, although in certain situations, major problems may disappear.  However, most people will not notice much if any change between this version of the NAM Traffic Simulator and the previous one.

I would like to encourage all those who are willing to give this version of the NAM Traffic Simulator a try in their cities.  It is attached below; the attachment contains the five main capacity levels of the simulator.  Choose the one you want, and use it to replace the one in your NAM folder, which you should keep in a safe place, outside of your Plugins folder.  The Traffic Simulator Configuration Tool can even be used on this new version of the traffic simulator to customize it as you like.  In order for this beta test to be worthwhile, please report in this thread any changes you see in your game once you start using the new traffic simulator, whether they are positive or negative.  I say "any changes" here because it often is not obvious that certain changes in your city are caused by changes in the traffic simulator, even though they often are.  For example, a better pathfinding heuristic can cause not only a reduction in abandonment due to commute time, but also a healthier distribution of wealth levels among Sims in your city, with the Sims' wealth levels being better matched to the available jobs.

This is beta software, and there is a slight possibility that using it may cause minor problems in your city.  However, this is rather unlikely as we have already tested it extensively.  For those who are willing to try it, please do, and let us know what you find, positive or negative.  Thanks!

[b22rian]

So Steve ?

Was the PH setting the only one you modified here.
Or did you also modify any other properties ?

thanks, Brian

[cmdp123789]

Ok, I downloaded, and I copied the ultra from the folder, and then pasted it on the NAM folder in my plugins, basically replaced it. Now, I do see less abandonment on a test city I created... and I also noticed an improvement on "job finding"?? I dont know if thats part of the deal... but somehow I saw that. Hmmm, I didnt see anything negative yet, but then again, I tested it on a little city I created for this, I suppose I should test it on a big one... right? But good job so far 

[z]

Was the PH setting the only one you modified here.
Or did you also modify any other properties ?

Just the PH.

Now, I do see less abandonment on a test city I created... and I also noticed an improvement on "job finding"?? I dont know if thats part of the deal...

It certainly is.  Both of these effects would be expected if our pathfinding heuristic were actually better than the previous one.  Thanks for the feedback!

[rivit]

Steve,

  here are some results from a large tile city I've had for some years but I had not played a lot in recent times due to it just being too big for my old machine.

As it happens I still have the data of when 2.1 was implemented in the data trace so it offers a good view of the contrasting effects of the new value. These graphs were the only ones showing any real deviations - no other changes were made to the city (although its the first run with 31.2 and its repaired Maxis stations) - I just let it run for 15 or so years with the new Medium simulator.

I think this gives some food for thought.

Crime down and Population/Jobs up would suggest abandonment has been suppressed. The Traffic volumes have resumed the pre- 2.1 pattern but commute times are not as bad despite the city being bigger. Pleasing to see traffic car volumes have fallen in favour of pedestrians, train, subway (its only an old-fashioned subway, no El-Rail, GLR).

Hope this helps some evaluation.

[z]

Release of NAM Unified Traffic Simulator v2.5

Although the response to the Beta 1 version of NAM Traffic Simulator v2.5 was positive in terms of producing good results, I was not happy with the accuracy of the new pathfinding heuristic.  The new PH was .0054, and as explained in my original post on this topic, the lack of greater precision was due to the sparsity of data points.  This also left questions as to whether the new PH would really perform better than the old one in all circumstances; it might merely perform better than the old one in most circumstances, but worse in others.  This was the case, for example, with Tropod's value of .003 for the perfect PH when compared with the PH of .009 used in the old Simulator A.  I needed a much more methodical and systematic method of determining the perfect PH.

The method used to produce the previous perfect PH of .005797 was a good method, but clearly the PH it produced wasn't as perfect as I thought.  Why?  I underestimated three things:

• The large variability from one test to another where starting parameters were identical.  To get really precise results, I needed to do many more tests at a given PH than I had done the first time.
• The variability of a single test over time.  Whereas the first time, my tests had typically run 10 to 20 years each, this time all tests were run for 50 years.
• The way that pathfinding efficiency, as measured by abandonment due to commute time as well as traffic simulator speed, varied as a function of the pathfinding heuristic.  In fact, this relationship turned out to be a fractal function.  This meant that the perfect PH could be hiding between two PHs that were relatively close to each other and yet were both far from perfect.  So much finer testing of PH ranges was required here.

After adopting the above changes, I calculated a new perfect PH of .00560239 based on a total of 110 tests for the PH, with 20 tests alone on the value .00560239; all of these tests were 50 years long in a city of two million Sims.  Although the precision of this perfect PH is now two digits more precise than the previous one, I found that the test results justified this precision.  And if you multiply out all the numbers involved here - 110 tests of two million Sims running for 50 years per test - you see that eleven billion Sim-years were involved in this testing.  (That's most of the age of the universe (ours, that is), for those who are counting.)  For this reason, among others, I have a lot of confidence in the number .00560239.  It may not be exactly the perfect PH (which may very well contain many more digits), but it's significantly better than anything else we've seen in this game.  This PH also resulted in the fasted test I've run, which was 7% faster than the next fastest (which was for the adjacent .00560240).  According to both theory and my experience, speed is probably the single best indicator of when the perfect PH has been reached, although many tests need to be performed to get an accurate speed rating for a given PH.  The other indicator I've been using, which was the main indicator used to determine the old perfect PH, is the amount of abandonment due to commute time.  The new perfect PH definitely came out on top here, and furthermore was the only run to feature multiple test runs in which abandonment due to commute time at the end of the 50-year test was zero.  Further use of this new perfect PH in MTA station testing confirmed that it was producing slightly better paths than the old one.

So where are the dancing elephants?  Although we will continue to call this the "perfect PH", it is not necessarily the theoretically perfect PH (which would go out to many more decimal places in any case); it's simply by far the best PH we've ever seen in the game.  However, as the traffic simulator has matured, changes in the value of the perfect PH have gotten smaller.  Consider just the last two changes:  The previous perfect PH of .005797 replaced Tropod's .003, while the new perfect PH of .00560239 replaces .005797.  The difference between the second set of two numbers is only about 6% of the difference between the first two.  The effects are a fair amount greater than that 6%, although they're still much less than the previous change.  Nevertheless, using the new traffic simulator containing this new perfect PH, depending on the city, there should be noticeable improvements in abandonment due to commute time and the runtime speed of the traffic simulator, and therefore the runtime speed of the game itself.

The new traffic simulator will be released as a standard part of NAM 32.  However, since the implementation of a new perfect PH is rather significant (even without the dancing elephants), I am also unlocking the NAM Traffic Subsystem file and updating it with the new traffic simulator.  This has already been done, so those who wish to use the new traffic simulator in their cities may do so.

[khendricks456]

I just downloaded the new version, but after install it still shows as version 2.4.1. Please check...

[z]

I just downloaded the new version, but after install it still shows as version 2.4.1. Please check...

Thanks for the report!  There are several places in the package where the version number appears, and I missed one of them.  It's fixed now.  To be safe, it would be a good idea to download and install the revised package to be sure that the newer version overrides your old traffic simulator.

[Wilfried]

The current NAM 32 still comes with a PH of .005797 instead of .00560239. Is there any reason for that?

[z]

The current NAM 32 still comes with a PH of .005797 instead of .00560239. Is there any reason for that?

Yes. Further research and  experimentation showed that the former value of .005797 was indeed the best pathfinding heuristic of all those that we had tested, and so the PH was reverted to this value for NAM 32.

[Wilfried]

I see. Thank you for the quick reply. 

[HappyDays]

The current NAM 32 still comes with a PH of .005797 instead of .00560239. Is there any reason for that?

Yes. Further research and  experimentation showed that the former value of .005797 was indeed the best pathfinding heuristic of all those that we had tested, and so the PH was reverted to this value for NAM 32.

Could you elaborate? How did you go from "11 billion sim years shows this is the best we've come up with" to "Nope, the older one's better"?