Using a custom proxy in a Scrapy spider

Make use of Scrapy’s standard HttpProxyMiddleware by specifying proxy meta value and the autherization header in a Scrapy Request, for example:

Xem tiếp >> “Using a custom proxy in a Scrapy spider”

How to import files from different folder python

If there are many folders in your project. It would difficult to import from this file to other file. At this article, I will give you 2 ways to do this Xem tiếp >> “How to import files from different folder python”

Scraping data from Gumtree

I will help you to scrape data from http://gumtree.com/  with phone number.

The fields to get: category, title, full_name, phone, street_address, city, town, lifecycle_state, avaiable, posted, bedrooms, price, url Xem tiếp >> “Scraping data from Gumtree”

Python printing to screen and writing to a file at the same time

You create a Python script. If you need print to screen and writing to a log file at the same time, you could use the under source code. Xem tiếp >> “Python printing to screen and writing to a file at the same time”

Get product infomation from Kmart.com

Project requrements

This project involves creating a scraper for products from an eCommerce site using Scrapy.
The scraper needs to deliver the following fields from product pages on the site:

  • Product Title
  • Product Description
  • Product Price
  • Product Image URL
  • SKU

Export should be to a CSV file. Xem tiếp >> “Get product infomation from Kmart.com”

Automatic check, download, analyse expired domains

Main purpose of program is get latest expired domains and then analyse these domain.

Working steps: All below steps are run automatically

Step 1: Open brower and login to https://www.expireddomains.net

Step 2: Get latest expired domains excel file

Step 3: Read the excel file which have latest expired domains

Step 4: Go to http://ahrefs.com. Using bulk analytic feature to check all above expired domains

Stop4: Download the analytic results to defined folder with specific filename. (add current date, index,…)

Xem tiếp >> “Automatic check, download, analyse expired domains”