Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add titles and base_url to onions table, populated automatically by backend. #42

Open
haxys opened this issue Apr 5, 2018 · 1 comment
Labels
enhancement New feature or request

Comments

@haxys
Copy link
Contributor

haxys commented Apr 5, 2018

I would like to add title to the onions table, so that we might obtain a solid title for every onion domain. It can be populated from the root url, or if the root url redirects to another url, it can be populated from that url instead. An easy way to populate this field would be to check if the page is / (and to always be sure to add / to the list of urls to scan if it doesn't already exist) and to add the title of the page if this is the case. If / redirects to another page, then use that page's title instead. We might also store an additional column in the onions table for the base_url.

All of this can be done in the backend.

This change will require us to modify the database structure. We don't necessarily need to start the database from scratch to instantiate this change; all the urls will be scanned again eventually. But if we start from a blank slate before launch, then that will take care of it too.

@haxys haxys added the enhancement New feature or request label Apr 5, 2018
@haxys haxys added this to the Stable Version 1.0 milestone Apr 5, 2018
@haxys
Copy link
Contributor Author

haxys commented Apr 5, 2018

This has been completed in the offload_to_backend pull request. Just needs testing and merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant