python - How to implement pre_navigation_hook of PlaywrightCrawler in crawlee py? - Stack Overflow

admin管理员组

文章数量:1321250

I've been using crawlee for python recently to build reliable web scrapers, but sometimes I needed to execute some code before visiting the page url like:

• blocking specific resources (ex: imaged, media, etc)
• passing the created page to playwright-stealth like this: await stealth_async(page)

I searched the Docs for a parameter or a class with a similar functionality & found that PlaywrightCrawler has a param called pre_navigation_hook. However, I couldn't find how to use it or a tutorial on their site demonstrating that.

Please provide an example how to use it or any other way to achieve the points above.

Note: pre_navigation_hook accepts a PlaywrightPreNavCrawlingContext as mentioned in docs not PlaywrightCrawlingContext like almost all other request_handlers if this makes any hint.

本文标签： pythonHow to implement prenavigationhook of PlaywrightCrawler in crawlee pyStack Overflow