Sometime, we need send result item to pipeline manually ex, entering input data to the database. This article will detail the step to do it
Step 1: Create a function to get pipeline instance
|
1 2 3 4 5 6 7 |
def get_pipeline_instance(spider, pipeline_class): manager = spider.crawler.engine.scraper.itemproc for pipe in manager.middlewares: if isinstance(pipe, pipeline_class): return pipe else: return None |
Step 2: Call the pipeline to send an item
|
1 2 3 4 5 6 |
try: # Manually call the filter f = get_pipeline_instance(self, YourPineLineClass) f.process_item(data_item, self) except: pass |
Hope it help. Thanks!!!
I am Tuan from Vietnam. I am a programmer Freelancer at web scraping, web automation, Python scripting. I have worked for 7+ years in these fields.
