Callback method not called all the time while using js rendering

[pkmishra]

I am facing this weird issue where the data goes missing randomly for some pages. Upon debugging I found that callback method is not called all the time though Get request is made for all the links given. There is no error logged so I am not sure how to give you more data points. Only option is too attach debugger and see if I am able to replicate it.

[musabgultekin]

I have a guess. It could be some middleware canceled your requests. There are 3 middlewares do this

1. Allowed Domains (cancels requests on hosts that are not on this list)
2. Duplicate Requests
3. Robots.txt file

I commented out logging on 1 and 2 . It could be them. Can you check those? If one of those, I must add logging for future debugging issues.

[musabgultekin]

I updated Chrome module and added logs on middlewares. Can you check again

[pkmishra]

Doesn't look like it falls under any categories mentioned but let me check it.

[pkmishra]

no luck yet. intermittently getting ERROR: encountered exception 'Uncaught' (9583:29) and also observed js rendering doesn't honor allowed domain e.g. https://www.facebook.com/ pixel is rendered with message Crawled: (200) <GET https://www.facebook.com......>

[musabgultekin]

I cannot replicate errors you have encountered. I need more details, Real code snippet would be awesome.
Allowed domains only applied on first request. If some website makes external requests on rendering, we dont block them. It could cause not loading pages properly on chrome.