Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document stats produced by Scrapy #6351

Open
mohmad-null opened this issue May 9, 2024 · 0 comments
Open

Document stats produced by Scrapy #6351

mohmad-null opened this issue May 9, 2024 · 0 comments

Comments

@mohmad-null
Copy link

Documentation bug

Scrapy exposes stats through the crawler.stats, however the stats themselves are not documented.

I have these values:

'log_count/DEBUG' = {int} 983
'log_count/INFO' = {int} 201
'start_time' = {datetime} 2024-05-09 14:51:37.927572+00:00
'scheduler/enqueued/memory' = {int} 1238
'scheduler/enqueued' = {int} 1238
'scheduler/dequeued/memory' = {int} 793
'scheduler/dequeued' = {int} 793
'downloader/request_count' = {int} 183
'downloader/request_method_count/GET' = {int} 183
'downloader/request_bytes' = {int} 61189
'downloader/response_count' = {int} 182
'downloader/response_status_count/200' = {int} 163
'downloader/response_bytes' = {int} 814553
'response_received_count' = {int} 182
'downloader/response_status_count/410' = {int} 19
'request_depth_max' = {int} 3
'log_count/WARNING' = {int} 19

Some of them are fairly self evident (i.e. log counts), but there's nothing anywhere in the docs for the scheduler related ones: dequeued, or enqueued, or the memory subsets.
https://docs.scrapy.org/en/latest/search.html?q=dequeued&check_keywords=yes&area=default
https://docs.scrapy.org/en/latest/search.html?q=enqueued&check_keywords=yes&area=default

@wRAR wRAR changed the title Docs - stats Document stats produced by Scrapy May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants