This project involves creating a scraper for products from an eCommerce site using Scrapy.
The scraper needs to deliver the following fields from product pages on the site:
- Product Title
- Product Description
- Product Price
- Product Image URL
Export should be to a CSV file.
The project needs to be able to collect all product page URL’s from the site and then parse them using the fields mentioned above.
The successful bidder will have in depth knowledge of Python and Web scraping
The project needs to be completed quickly
1. Install Python 3
2. Install Google Chrome
3. Install python packet (run these command on command line tool)
pip install Selenium
pip install requests
How to run the program
1. Open command line tool. Change directory to project folder
2. Run the program with command “python getproduct.py”
3. Waiting for result. The result is written to file product_result.csv at project folder
If the program is crashed for any reason (ex: connection lost), let’s run program again to continue previous job
If you want to run new job, let’s delete temp folder inside project folder.
Demo video[youtube https://www.youtube.com/watch?v=gqP7qi6W4hM&w=560&h=315]
I am Tuan from Vietnam. I am a programmer Freelancer at web scraping, web automation, Python scripting. I have worked for 7+ years in these fields.