• Welcome to SC4 Devotion Forum Archives.

SC4Reader Tool

Started by Tropod, February 05, 2014, 01:32:52 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

Tropod

Hi all,

This tool is for editing SimCity 4 files. It includes some functionality similar to ilives Reader & also Region Census tool (ie full Region Viewing). For a full list of features, including known issues, please see the included Readme.txt.

Link:
SC4Reader
(locate relevant version, unzip & run, no install required).

The program also needs the following Microsoft's mfc.dll if you don't have it already, need to install vcredist_x64.exe &/or vcredist_x86.exe redistributable package;
http://www.microsoft.com/en-au/download/details.aspx?id=40784

This is a WIP so any bugs/issues etc, please post detailed system specs including a detailed description of what you were doing.
Any other feedback/suggestions is more than welcomed.

Program source code link:
source code
A very special thanks to ilive & his reader source code, some of which this program is based. Feel free to ask any questions or give suggestions regarding source code. Program is written in c++/MFC.

cheers :).

Some basic pictures & notes now added. In the following for each (note images may not be upto date);

Main Window
1. Make sure options are set before using program as some functions rely on folder settings.
2. Loaded *dat is for anything that's been opened, inserted, or virtual dat. Other is for Find functions but only for non-loaded file.
3. Just shows file names of files in plugins, install & other etc folders as per options settings. These can be selected & file loaded.
4. Virtual *dat; loads install, plugins, &/OR other as per options settings. Note: this loads header & index info only. Upon selecting item in main list it will load what it needs for item selected.
5/5b; Displays item info for item selected in main TGI List. 5b can edit TGI.
6/6b; cross-checks TGIs against Install &/or Plugins as per options settings. Depending on size of plugins & PC specs, full scan may take 30+minutes & generates a *.txt file with possible conflicts. 6 does a comprehensive scan, 6b only scans what's been selected in a loaded *dat.



Options Window
1. Add folder/file type; you can add multiple folders/files here. For folders, the program will automatically search subfolders so you don't need to add them separately & doing so may cause problems.
The rest of the options should be self explanatory. Changes to settings do require program be restarted.




Display Type:
Default/Hex/Text; Default = default mechanism how the program handles TGI item. Files like RUL are not handled like a text file by the program, but you can choose Text here to view/edit like a text file.



Error Message;
Shows error message on exit if program had an issue loading a *dat type file. You should check the logging.txt for details if you encounter this.



TGI/Plugins scanner;
1. Is the main file being checked against.
2. Shows other info including (other) files currently cross checking against 1 above. These will get repeated (in case of Plugins scanner) as program loops through each file cross checking against each other file.




TGIs.ini file (located in settings folder)
This file contains settings relating to assignment & descriptions of TGI items for the program (including for the main list, Filter, Search functions & Tree). This file can be added to &/or changed by end-user if necessary.


Crash Reporting
If program crashes the following should appear;
You'll need to click on link in the 1st window to display 2nd;



Export function will save contents of dump, XML & screenshot to zip file. Feel free to upload that here. This should help me pinpoint any issues with crashing/hanging.

memo

Tropod, this is great! What language is this? I started something similar a couple of months ago, at least I think it is, but stopped working on it when I couldn't come up with suitable concurrency measures. Now, seeing that you provide us with a new tool is a relief.

I can't even start it though – it exits with "ERROR Creating InitContext", followed by "Error: Could not create the shader program". My System is Win7 Professional SP1, 32bit.

memo

I still get the same error with the updated version though. My graphics card is Intel HD 3000; I am running Bootcamp on a MacBook Pro, if it matters. I don't know which version of OpenGL I have.

Tropod

Quote from: memo on February 06, 2014, 12:11:16 AM
I still get the same error with the updated version though. My graphics card is Intel HD 3000; I am running Bootcamp on a MacBook Pro, if it matters. I don't know which version of OpenGL I have.

I've done a quick google search & I'd say that's the most likely issue, your HD3000. I have same problem on another WinXP PC. At the moment it doesn't support OpenGL 3.2 or lower, next version I'll probably just take that out for now as the paths are the only thing using it & they're buggy atm.

droric

#4
I just attempted to run this on my Windows 8.1 machine and it shows SC4Reader is trying to recover your information... for 10-20 milliseconds then the program just closes out.  It was quite difficult to get a screenshot of it.  My specs are in my sig :)

I also tried to modify \Settings\Settings.ini to match my system but I still receive the same result.



EDIT:  This only occurs when launching the application from within an RDP session, it works properly when launching from VNC or such.  I do alot of my SC4 stuff at work ;) shh don't tell


PC Specs: i2700k @ 4.9 Ghz, 16 GB @ 1600-8-9-8-16, 2X GTX 580 SLI, Gigabyte Z68XP-UD3R, 120 GB Vertex 3 SDD, 3 TB RAID1, 1200W SilentGold, Antec P180B case, Win 8 x64 Pro

mrbisonm

........it says: * Can't find mfc100u.dll *


?????


....Uploading the MFP 1.... (.........Finishing the MFP1)

droric

Quote from: mrbisonm on February 06, 2014, 02:42:15 PM
........it says: * Can't find mfc100u.dll *


?????

I believe you need the Microsoft Visual C++ 2010 SP1 Redistributable
http://www.microsoft.com/en-us/download/details.aspx?id=8328

PC Specs: i2700k @ 4.9 Ghz, 16 GB @ 1600-8-9-8-16, 2X GTX 580 SLI, Gigabyte Z68XP-UD3R, 120 GB Vertex 3 SDD, 3 TB RAID1, 1200W SilentGold, Antec P180B case, Win 8 x64 Pro

mrbisonm

#7
Quote from: droric on February 06, 2014, 05:54:19 PM
Quote from: mrbisonm on February 06, 2014, 02:42:15 PM
........it says: * Can't find mfc100u.dll *


?????

I believe you need the Microsoft Visual C++ 2010 SP1 Redistributable
http://www.microsoft.com/en-us/download/details.aspx?id=8328



Thanks, appreciated. ;) Now it works.


....Uploading the MFP 1.... (.........Finishing the MFP1)

droric

#8
Quote from: mrbisonm on February 06, 2014, 07:22:58 PM
Quote from: droric on February 06, 2014, 05:54:19 PM
Quote from: mrbisonm on February 06, 2014, 02:42:15 PM
........it says: * Can't find mfc100u.dll *


?????

I believe you need the Microsoft Visual C++ 2010 SP1 Redistributable
http://www.microsoft.com/en-us/download/details.aspx?id=8328



Thanks, appreciated. ;) Now it works.

The amount of runtimes made available over the years has gotten a little bit out of hand  :D

http://i.imgur.com/OMVoBOo.png

PC Specs: i2700k @ 4.9 Ghz, 16 GB @ 1600-8-9-8-16, 2X GTX 580 SLI, Gigabyte Z68XP-UD3R, 120 GB Vertex 3 SDD, 3 TB RAID1, 1200W SilentGold, Antec P180B case, Win 8 x64 Pro

cogeo

There is an option in the Project Settings that cause the MFC code to be included in the executable. The executable gets bigger (at least 4MB), but the application has no external dependencies. Maybe you should consider setting this option, at least for the release version.

Tropod


@droric
Thank you for pointing this out (ie mfc100u), I'd forgotten about it.
Regarding the RDP session: are you referring about remote desktop connection? I'm not overly familiar with this in programming environment, I also don't have Win7/Win8 (maybe in few months). I take it it's working ok now?

@cogeo:
There's pros n cons for including redistributables & at this stage I don't have any plans to include that. Perhaps down track though.



I'd be interested in any performance issues/bottlenecks anyone encounters, particularly with the viewing/editing of large RULs. I know ilives reader has some issues handling the larger ones so I'm hoping this tool helps alleviate that somewhat.


memo

Quote from: Tropod on February 24, 2014, 02:16:23 AM
I'd be interested in any performance issues/bottlenecks anyone encounters, particularly with the viewing/editing of large RULs. I know ilives reader has some issues handling the larger ones so I'm hoping this tool helps alleviate that somewhat.

While it would be admirable if an application scales nicely, you don't need to worry about the RULs in particular. Nobody in the NAM team actually edits the large RUL files directly anymore, let alone in the Reader. They are all split up into multiple files and get compiled using this tool. This has several advantages, among others: smaller files that are easier to edit and to track, and everyone can use one's favourite text editor with regex searching, syntax highlighting and whatnot.

Tropod

Quote from: memo on February 24, 2014, 04:03:58 AM
....They are all split up into multiple files and get compiled using this tool.

I figured something like that may of been the case; there's nothing like last minute changes :D. I've been out of the game for awhile & so this is just something I'm doing in my spare time for the fun of it really ()testing().

Tropod

#13
an update has been done. see my first post for link.

just summary of this update;
Exemplar/Cohort editing.
File listings pane added for all SC4 type files (excluding city type files).
Plugins scanner improved. Can now also scan individual/multiple TGIs for conflicts.
Tree pane & TGI.ini updated. Can now use custom labels.
Various fixes.


EDIT: Just a note about plugins scanner & TGI checker: depending on PC specs & size of plugins:
-full scan may take 30mins + or so.
-individual/multiple TGI checker: anywhere from a minute upwards.


Any questions around these let me know.
cheers  :)

Egi

Nice tool :)
What I miss so far is an option to cancel the conflict scanner.
May I ask why you made you own tool instead of improving the ILives Reader?

Tropod

Thank you.

I've made note about conflicts scanner. Will try to have that in next version.

ilives Reader is wonderful/awesome/priceless tool, using & improving upon the source code on the other hand is easier said than done & is quite challenging to get ones head around, hence why making my own tool. This also gives me more control over what I want to do. For example, for the next release I hope to allow (full) editing of the effects dir & I am 4/5ths the way there  ;).

Tropod

#16
update done, see first post for link. Summary of this update as follows;

-RUL files; Search & Goto within RULs themselves.
-dat/hex file comparison.
-scanner_compare_list.ini file now in use (used in conjunction with TGI/plugin scanner, if needed). See file for example.
-Various fixes/improvements (including cancel function for scanner  ;) ).

(basic) Editing now possible for:
-ATC files.
-AVP files.
-HLS files.
-Effects Dir - a word of caution though; Nearly all sections/sectors should be editable, however, there are still a lot of unknowns in this file & so I strongly advise referring to specs for this & ensure you back up file before doing so. Also there is no validation. Additionally, any "REP" type items should not be changed as it is currently not handled (ie don't increase Rep value items). String type items though should be safe to increase/decrease.

EDIT:
I've just updated this again, & link will be updated shortly, after discovering a critical bug with find/search function  :bomb:.

memo

I have just tried version 124B and I absolutely like what I saw.

A few random bugs (i.e. unreplicable) occured though:
• Sorting by filetype in sc1.dat yielded "SC4Reader has stopped working".
• Cross-Checking TGIs in sc1.dat did not complete. (Not sure what it does anyway. If it completes, it says that no files were scanned)
• Opening any files did not display any content, until restarting the program.

Also, when opening a file, the window size changes.

Loading a 60 MiB RUL file works flawlessly. I could do without the line-based editing approach, but a monospace font in the text editors would be immensely useful, as well as an option to select a fixed column number in the hex editor.

The EffDir editing functionality will be very convenient for creating puzzle pieces.

Tropod

@memo
Thank you for the feedback.

With the bugs; I'll take a closer look at these to see if I can't pinpoint any issues. I'm currently in the process of moving to Windows 7 (finally :'( ), so that may help pinpoint these bugs.

TGI checker & Plugins Scanner:
The plugins/conflict scanner scans & cross references all files (ie installation &/or custom plugins) whereas the TGI checker only checks TGIs selected inside a dat you've got loaded (against all install/plugin files). Both are based on file/folder settings (so 1 possible reason no files were scanned?), as is the File Listings pane. Both load *.dat header & *.dat index, read data & do a TGI compare for possible conflicts.

RULs;
If you go to Options menu there is a text option (default, hex, text), however you may encounter out of memory issue with the larger RULs when saving. Not sure if I can improve upon that or not at this stage, which is why I've take the approach I have. The default view will allow you to save multiple lines of data in the lower section, it will basically insert/append it to the current line/area you're working on.

EffectDir;
Along with Exemplar/Cohort & other files, I hope to allow extra editing capabilities along the way in due course.

monospace font & fixed columns; added to my list. Will see what I can come up with  ;D.

cogeo

Just out of curiosity, which library did you use to access DBPF (.dat ) files, perform compressions/decompressions, and read/write exemplars? Did you take code fragments out of the Reader, make your own or find another one?