SC4 Devotion Forum Archives

SC4D Off Topic Section => Computer Hardware and Software - Technical Discussion and Support => Topic started by: Bdswim on March 25, 2013, 10:29:31 AM

Title: Python Woes
Post by: Bdswim on March 25, 2013, 10:29:31 AM
(Okay, I don't know if this is the best place to ask this question, but I don't see any forums for programming here, so...)

I have this C code for compressing QFS files, and I've tried to translate it to Python. I'm not sure what I'm doing wrong, but the code won't run.

The code in question is from ILive Reader (http://ilive-reader.svn.sourceforge.net/viewvc/ilive-reader/trunk/or_dat/common.cpp?view=markup#l154).

This (http://pastebay.net/1192840) is the Python code, and this is the error:
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    test()
  File "C:\...\qfs.py", line 147, in test
    len(FILE_TO_COMPRESS))
  File "C:\...\qfs.py", line 68, in compress_data
    outbuf[outpos] = 0xE0 + _len
IndexError: list assignment index out of range


What could be going wrong here? My Python code should be doing the same exact thing as the C/C++ code, but apparently, it's not.
Title: Re: Python Woes
Post by: ACEfanatic02 on March 25, 2013, 11:40:38 AM
Well, to state the obvious, you're writing outside the bounds of the list.  Without some sample input to trace through it's hard to be more specific.

You can get away with this (usually) with C arrays because C does no bounds checking at runtime -- as long as you don't blow the stack or write outside your allocated memory C could care less (until you *do* and you segfault).  Python, on the other hand, will crash.

The block that's throwing the error is a pretty tight loop with no checks on outpos.  Try this experiment: in the original C code, store the length of the output buffer, and then add an assert in that block to notify you if outpos ever exceeds it.  If this doesn't bring up anything, try wrapping the Python statement in try: except: blocks, catch the IndexError, and log outpos and the output buffer length at that point.
Title: Re: Python Woes
Post by: wouanagaine on March 26, 2013, 12:14:19 AM
Why don't you use the qfs C code and use it as a python module ?
Given the source of qfs and its algorithm, I expect python not to be the best tool to make it efficient (but I can be wrong)

If you need such code, https://github.com/wouanagaine/SC4Mapper-2013/tree/master/Modules there is a qfs.c that can be made to qfs.pyd with setuptools (thanks to JoeST)