[Python] Ajutor Scrapy !

icezone · Posted: 24-07-2016, 08:13:03 | Translate post to: ... (Click for more languages)

Salut ! Nu am mai facut un post de mult ....
Acum cateva ore cautam un program care scaneaza siteurile si fura continul ,si am gasit acest Scrapy folosit prin Python.
NU stiu nimic in acest program , si as vrea putin ajutor !

As dorii putin ajutor la un script ...

Code:

# -*- coding: utf-8 -*-
import scrapy
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
from lxml import html

class Scrapy1Spider(CrawlSpider):
name = "craiglist"
allowed_domains = ["sfbay.craigslist.org"]
start_urls = (
'http://sfbay.craigslist.org/search/npo',
)

rules = (Rule(LinkExtractor(allow=(), restrict_xpaths=('//a[@class="button next"]',)), callback="parse_page", follow= True),)

def parse_page(self, response):
site = html.fromstring(response.body_as_unicode())
titles = site.xpath('//div[@class="content"]/p[@class="row"]')
print len(titles), 'AAAA'

Acum as putea face ca acest script , sa citeasca pagina , sa intre intr-un post , sa se intoarca si sa citeasca alt post ?

Multumesc !

hast