Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

w3school不知道为什么爬不出东西 #11

Open
wawawawo opened this issue Jun 21, 2017 · 1 comment
Open

w3school不知道为什么爬不出东西 #11

wawawawo opened this issue Jun 21, 2017 · 1 comment

Comments

@wawawawo
Copy link

不知道为什么我的爬不出东西来,json文件是0kb的。。其中spider里面我改了一点:from scrapy.spiders import Spider(因为报错说要用spiders)。还有log改logging了,然后运行的结果看不大懂,望大佬指正

D:\LZZZZB\w3school>scrapy crawl w3school
2017-06-21 22:33:03 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: w3school
)
2017-06-21 22:33:03 [scrapy.utils.log] INFO: Overridden settings: {‘BOT_NAME’: ‘
w3school’, ‘NEWSPIDER_MODULE’: ‘w3school.spiders’, ‘ROBOTSTXT_OBEY’: True, ‘SPID
ER_MODULES’: [‘w3school.spiders’]}
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled extensions:
[‘scrapy.extensions.corestats.CoreStats’,
‘scrapy.extensions.telnet.TelnetConsole’,
‘scrapy.extensions.logstats.LogStats’]
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled downloader middlewares:
[‘scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware’,
‘scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware’,
‘scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware’,
‘scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware’,
‘scrapy.downloadermiddlewares.useragent.UserAgentMiddleware’,
‘scrapy.downloadermiddlewares.retry.RetryMiddleware’,
‘scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware’,
‘scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware’,
‘scrapy.downloadermiddlewares.redirect.RedirectMiddleware’,
‘scrapy.downloadermiddlewares.cookies.CookiesMiddleware’,
‘scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware’,
‘scrapy.downloadermiddlewares.stats.DownloaderStats’]
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled spider middlewares:
[‘scrapy.spidermiddlewares.httperror.HttpErrorMiddleware’,
‘scrapy.spidermiddlewares.offsite.OffsiteMiddleware’,
‘scrapy.spidermiddlewares.referer.RefererMiddleware’,
‘scrapy.spidermiddlewares.urllength.UrlLengthMiddleware’,
‘scrapy.spidermiddlewares.depth.DepthMiddleware’]
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled item pipelines:
[‘w3school.pipelines.W3SchoolPipeline’]
2017-06-21 22:33:03 [scrapy.core.engine] INFO: Spider opened
2017-06-21 22:33:03 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pag
es/min), scraped 0 items (at 0 items/min)
2017-06-21 22:33:03 [scrapy.extensions.telnet] DEBUG: Telnet console listening o
n 127.0.0.1:6023
2017-06-21 22:33:03 [scrapy.core.engine] INFO: Closing spider (finished)
2017-06-21 22:33:03 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{‘finish_reason’: ‘finished’,
‘finish_time’: datetime.datetime(2017, 6, 21, 14, 33, 3, 262577),
‘log_count/DEBUG’: 1,
‘log_count/INFO’: 7,
‘start_time’: datetime.datetime(2017, 6, 21, 14, 33, 3, 252576)}
2017-06-21 22:33:03 [scrapy.core.engine] INFO: Spider closed (finished)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@wawawawo and others