• Welcome to SC4 Devotion Forum Archives.

SC4D LEX API Source-code on Github

Started by CasperVg, June 13, 2014, 09:32:42 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

CasperVg

Source code and documentation for the SC4D LEX API (a REST server for LEX functionality in mobile apps) can be found on Github now.

https://github.com/caspervg/SC4Devotion-LEX-API

Developers, feel free to post suggestions for features or code improvements either here or in the Github issues system.
Follow my SimCity 4 Let's play on YouTube

Yild

#1
regarding search functionality... there is an option to search by author... but... where can I get list of all authors/creators to get their IDs...
actually I don't think that API user need to know other users IDs... for this we need only name (already unique (I assume) in db table)

so allow search by authors name and give us an option to retrieve list of authors :)

edit: lex_category, lex_type field values... I can get this from html source but it should be in documentation ;)
DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

CasperVg

#2
There's actually a way to get a list of authors. I just forgot to document it (forgot to document the entire Category endpoint actually). I'll add documentation later today.
I've also added a link to a few wrapper libraries (LEX4J, SharpLEX) in the API readme.

If you call the "/category/author" endpoint, it will return an array like this:
[
   {
      "id":1,
      "name":"ADMIN"
   },
   ...
   {
      "id":5275,
      "name":"zero7"
   }
]


Other categories (lex_category, lex_type) have a similar structure.


EDIT: Full documentation for the /category endpoint is now available: check it out at https://github.com/caspervg/SC4Devotion-LEX-API/blob/master/documentation/Category.md.
Follow my SimCity 4 Let's play on YouTube

Yild

Thanks :)

I didn't check endpoint folder :/
DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

Yild

today at any API request I get an exception - something is broken :/
DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

Tarkus

The databases for the forums were offline for about 30 minutes just now, and the LEX database went down at the same time, and is still down for the moment.  It'll sort itself out shortly.

-Alex

Yild

hehe ok, so my new request is to have some db status info via API - ie test if mysql_connect != false :)
DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

CasperVg

#7
Alright, the API will now return:

If the database is unavailable:
HTTP Status 503 - Service not available
{"status":503,"error":"Could not connect to database","suggestion":"Try again later"}

Other processing exceptions:
HTTP Status 500 - Internal Server Error
{"status":500,"error":"Message explaining the exception that occurred","suggestion":"Report to a site administrator"}

Documentation for the error modes and the basic endpoints (the API index and the version endpoint) has been added as well: https://github.com/caspervg/SC4Devotion-LEX-API/blob/master/documentation/Base.md

This change will also be backported to api/v1 and api/v2
Follow my SimCity 4 Let's play on YouTube

Yild

DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

Yild

Would it be possible to add image file size to primary branch of GET /api/<version>/search (images->primary) and/or to GET /api/<version>/lot/:lotid:, and add other images urls/sizes like secondary/right (where primary is left one), maybe only with specific param like getfullimageinfo=0/1 ?

Atm my app implements images cache - when search is executed images are downloaded to my HDD so next time I don't need to download them again but I wont know if they haven't change except url was changed by image reuploaded with different name. So next best thing to verify is file size.

I don't think that image file size information is necessary but I wanted to point some issues that only url info may cause.

ps. some images are over 2MB like here http://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=3098, so this method is great at reducing server/end user bandwith in exchange of some disk space ;)

DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

CasperVg

#10
While it's a cool idea (and reducing bandwidth usage is always good), I'm afraid implementing it could cause some issues. The biggest problem is that we don't currently store any image metadata in the database, so looking up things like filesize (or a file hash) would mean polling the file system to gather data like that each time a request is made, which has the potential to slow down the server response quite a bit. We could of course run a separate program to look up image filesizes/hashes/etc. periodically (on a cronjob or something) and have that put the image data in a new database table. But honestly, that seems like a lot of work for a feature that wouldn't be extremely useful in the API, and rather unnecessary for the LEX website.

You could probably store the images for a set amount of time (say 10 days) and download them again after that period. That should probably be enough for most applications - after all, I don't think many file authors ever change their file images.

However, I have added the two extra image types ('secondary' and 'extra') to the output of both the /search and /lot/:lotid: endpoints a few days ago in api/v3 ;)
See http://sc4devotion.com/csxlex/api/v3/lot/3129 for example.
Follow my SimCity 4 Let's play on YouTube

Yild

That's good idea with cache stored by specific amount of time, I will use this method - its easy and not time consuming to check file creation date/time.

So documentation was again not updated... next time I will check source code first :P

Thanks for your patience ;)
DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

CasperVg

Heh, I'll admit documentation isn't one of my strong points. Then again, I don't know many people who do enjoy keeping documentation up-to-date. I'll try to keep them updated better from now on though ;)
Follow my SimCity 4 Let's play on YouTube

Yild

its me... again :P

I have found an error in search functionality

https://github.com/caspervg/SC4Devotion-LEX-API/blob/master/api/endpoints/search.php - line 253:

$sql = "SELECT * FROM LEX_LOTS WHERE " . $select;

this code assume that $select will have valid WHERE options but in fact there are possible outcomes of $select value that will generate invalid SQL query:

sc4devotion.com/csxlex/api/v3/search?order_by=recent
sc4devotion.com/csxlex/api/v3/search?order_by=popular
sc4devotion.com/csxlex/api/v3/search?order_by=download

will generate invalid sql: "SELECT * FROM LEX_LOTS WHERE "

atm only order_by=update return 'correct' output



btw atm order_by=updated returns entries that was updated at any time (regarding atm the date/time of update)... so is it actually correct to just output limited entries or it should be something like 2012 2013 2014 null null ?
query
select * from order by isnull(`data`), `data` asc

will return
2010
...
2014
null
...
null

- just a thought

DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev

CasperVg

Quote$sql = "SELECT * FROM LEX_LOTS WHERE " . $select;

this code assume that $select will have valid WHERE options but in fact there are possible outcomes of $select value that will generate invalid SQL query:

sc4devotion.com/csxlex/api/v3/search?order_by=recent
sc4devotion.com/csxlex/api/v3/search?order_by=popular
sc4devotion.com/csxlex/api/v3/search?order_by=download

will generate invalid sql: "SELECT * FROM LEX_LOTS WHERE "

atm only order_by=update return 'correct' output

That is actually the way it is supposed to work (if you try sc4devotion.com/csxlex/api/v3/search?order_by=recent for example, it will fail with 400 - Bad Request (and never actually execute the 'broken' SQL), because there are no actual "filters" in such a request, only an ordering. ?order_by=update, however, does work, because ordering by update also implies a filter (the file was updated at least once).

However, looking at it now, that approach is a pretty useless limitation without any good reason, so I will be changing that soon. Hopefully today, but it could be later next week (have some important Uni stuff coming up).

Quotebtw atm order_by=updated returns entries that was updated at any time (regarding atm the date/time of update)... so is it actually correct to just output limited entries
Yes, this is also intended behavior. I modeled it (like most of the API) after the website functionality of the LEX. If you check the "Recent Updates" page in the LEX itself, you will see that it too only returns files that were updated at least once. Files that were never updated do not appear in that list.
Follow my SimCity 4 Let's play on YouTube

CasperVg

Quote from: CasperVg on August 16, 2014, 11:43:09 PM
Quote$sql = "SELECT * FROM LEX_LOTS WHERE " . $select;

this code assume that $select will have valid WHERE options but in fact there are possible outcomes of $select value that will generate invalid SQL query:

sc4devotion.com/csxlex/api/v3/search?order_by=recent
sc4devotion.com/csxlex/api/v3/search?order_by=popular
sc4devotion.com/csxlex/api/v3/search?order_by=download

will generate invalid sql: "SELECT * FROM LEX_LOTS WHERE "

atm only order_by=update return 'correct' output

That is actually the way it is supposed to work (if you try sc4devotion.com/csxlex/api/v3/search?order_by=recent for example, it will fail with 400 - Bad Request (and never actually execute the 'broken' SQL), because there are no actual "filters" in such a request, only an ordering. ?order_by=update, however, does work, because ordering by update also implies a filter (the file was updated at least once).

However, looking at it now, that approach is a pretty useless limitation without any good reason, so I will be changing that soon. Hopefully today, but it could be later next week (have some important Uni stuff coming up).

This change has now been pushed to api/v3. ?order_by now always counts as a filter, not only when it's ?order_by=updated. Relevant documentation has also been updated, and a few other small backend changes have been made to the api.
Follow my SimCity 4 Let's play on YouTube

Yild

#16
I didn't check all updates but order_by works ok (with a few queries) :)

Thanks


any chance to get result file from "Download dependencies" button via api ?

------------
regarding value 'rating' from

POST /api/<version>/lot/:lotid:/comment

what's this 'rating' value? is it value of votes green/cream/pink like in http://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=2876 ?

if yes how can I retrieve it by API ?

DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev


CasperVg

#18
Seems like I was retrieving only comments for non-locked files by accident. I'll fix that as soon as possible.

As for the rating numbers (god I'm 1 year late to replying to that - sorry!), the 1 to 3 do indeed correspond to the LEX "Feedback for the author". With 1 being "Thank you very much" and 3 being "I don't like this". I don't believe this is currently retrievable, but it makes sense to have that functionality, so I will add it soon as well. However to preserve "anonymity" it will only be possible to retrieve the total votes for each category and an average.

To get the dependency list, all you need to do is call /lot/<lotid>. If it supports the dependency tracker, it will return something like this:
"dependencies":{
      "status":"ok",
      "count":2,
      "list":[
         {
            "internal":false,
            "link":"http:\/\/www.simtropolis.com\/forum\/files\/file\/11421-porkie-props-vol1-european-street-accessories\/",
            "name":"Porkie Props Vol 01"
         },
         {
            "internal":true,
            "id":1263,
            "name":"BSC Mega Props - JES Vol05",
            "status":{
               "ok":true,
               "deleted":false,
               "superceded":false,
               "superceded_by":-1,
               "locked":false
            }
         }
      ]
   }


To get the actual response html, you can request http://sc4devotion.com/csxlex/lex_deptracker.php?lotID=<lotid>.
Follow my SimCity 4 Let's play on YouTube

Yild

Quote from: CasperVg on July 20, 2015, 11:36:57 PMgod I'm 1 year late to replying to that - sorry!

better now then never ;)
I was inactive with my SC projects for a year too ;)

Thanks
DAMN Manager download: at LEX at STEX
DAMN Manager support thread: at SC4Dev