Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor using JSON, and multiple smaller additions/changes/improvements #56

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

lethuan007
Copy link

@lethuan007 lethuan007 commented Oct 14, 2024

Added function for multiple areas #55
Added min/max plot_area
Added 'property type'
Added sleep-timer to reduce the chance of getting locked out of Funda

Added min/max plot_area
Added function for multiple areas
Added 'property type functionality'
@lethuan007 lethuan007 changed the title Update scrape.py Added multiple areas and minor tweaks Oct 14, 2024
… db, ppm2.5 and no2, as well as additional data like average neighbourhood price / m2, %families.
- seperated Search-request in seperate class for readability
- changed to json retreival and extraction of data!
- save each json used in /data/id/*.json
- added option to download photos to (date/id/photo)
- added the polution (sound/ppm2/no2)
- added Neighbourhood data from json
- added most fields from funda-json
 - usage for json selectors:
   - 1 element -> finds the content of the key
   - 2 elements -> finds the content of the 'Value' key by searching in the (1) Key, (2) Values.
   e.g. Id, ["overdracht-huurprijs", "overdracht-vraagprijs"], searches for 'Id' containing either of "overdracht-huurprijs", "overdracht-vraagprijs", and returns the content of 'Value' within that element.
@lethuan007 lethuan007 changed the title Added multiple areas and minor tweaks Refactor using JSON, and multiple smaller changes/improvements Oct 18, 2024
@lethuan007 lethuan007 changed the title Refactor using JSON, and multiple smaller changes/improvements Refactor using JSON, and multiple smaller additions/changes/improvements Oct 18, 2024
@G-ster
Copy link

G-ster commented Nov 10, 2024

@lethuan007
How can I test your changes.
I am getting errors regarding the arguments:
FundaScraper.init() got an unexpected keyword argument 'area'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants