The goal of this project is to create a tool for easier data collection.
The data collected with this tool includes user opinions about products.
This tool is very useful for data scientists.
Instructions for using this tool are explained below in gettinh started section.
open your terminal and clone the repository
Inline code clone code
clone with HTTPS
git clone https://github.com/SaoshyantOfficial/Digikala-Data-Collector.git
clone with SSH
git clone [email protected]:SaoshyantOfficial/Digikala-Data-Collector.git
Install prerequisites if you dont have them
- selenium
- flask
- bs4
Inline code
install prerequisites
selenium
pip install selenium
flask
pip install flask
BeautifulSoup
pip install bs4
Inline code run the code in terminal
python user_interface.py
open your browser and type this address
http://127.0.0.1:5000/
go to your considered product page in digikala.com website and copy main url
Once you have added all the desired urls, click on the scrap button
Web scraper starts to extract all reviews about your considered product
Then all reviews about your choosen product are extracted and you can download them in a ad .csv
file from result page
It is an open source tool and is available for all interested people so they can do their own research or projects with the help of this tool faster and easier.
The use of this tool is free and open, and also developers can upgrade this tool.
Hoping to influence the progress of the data scientist community.
Enjoy using this tool
Saoshyant Team
The first release in Spring 2023