Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed TheGamesDB scraper by adding a document tree segment 'scrapers' to the config file #802

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

ocket8888
Copy link

@ocket8888 ocket8888 commented Jul 2, 2019

This allows the user to pass an API key for the gamesdb scraper. It works by letting users add scraper configurations to es_systems.cfg (maybe not the right place?)

Here's an example of what the segment looks like:

<scrapers>
	<scraper>
		<name>gamesdb</name><!-- only valid scraper name atm -->
		<path>api.thegamesdb.net/Games/ByGameName</path><!-- currently unused -->
		<APIKey>REDACTED/APIKey>
	</scraper>
</scrapers>

The idea is try to make scrapers as configurable as possible, but at the moment the only valid name is gamesdb and the only used child element of a scraper is <APIKEY>. The code I have works, but obviously this needs to be cleaned up. For one thing, ratings, developers, publishers and genres aren't yet supported. For another, it's a long way from truly configurable. But hopefully this is a good enough starting point for some feedback.

Update

Basic support for scraping developers, publishers and genres has been added. Currently, there doesn't appear to be a way to access ratings through the TheGamesDB API. Fetching each requires a minimum of 3 synchronous API requests, at the moment. I expect that I'll want to make those asynchronous - similar to the image/thumbnail fetches - in the future. But it's a bit more complex than those because the fetched information is share-able between all games scraped with that scraper.

But I digress. Next up is refining the configuration files a bit.

… to the config file

This allows the user to pass an API key for the gamesdb scraper
@ocket8888
Copy link
Author

I guess this shoulda been a draft :P

@ocket8888 ocket8888 mentioned this pull request Jul 3, 2019
@Zer0xFF
Copy link

Zer0xFF commented Oct 6, 2019

Hi there, I'm currently the lead dev behind TGDB, just a small comment about this PR, we're not handing out keys to individual users, since we expect you to use the public key, since the key is limited per IP, so you can have as many users using it as you'd like. it also allows us some basic statistics on how and by which project the requests are made. Note, public key doesn't need to be obscured, since we don't expect malicious use, nor do we have an hard to meet requirement to obtain one.

P.S, Ideally the project owner would use their own key, but that's not set in stone.

@ocket8888
Copy link
Author

Unfortunately I think the project owner has left this project.

Good to hear that I can just put my public key in there, thanks for letting me know.

joolswills pushed a commit to joolswills/EmulationStation that referenced this pull request Sep 18, 2022
ViewController: improves bulk removal of entries
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants