javascript - is it possible to do some simple web scraping in chrome extension? - Stack Overflow

admin管理员组

文章数量:1353658

thanks in advance and Im sorry if this might not be a well formed question, I am relatively new to CS and stackoverflow.

I am hoping to make a simple chrome extension, which overrides the new tab page to display some simple data collected from a couple websites. I am wondering if it is possible to web scrape within the basic JS or chrome API's? Any information or guidance would be greatly appreciated, I have been trying to do research on the subject and haven't found any recent or clear answers to this.

Thanks for your help!

Here is an older stackoverflow question asking the same question but I wasn't able to make any progress from the answers.

Web Scraping in a Google Chrome Extension (JavaScript + Chrome APIs)

thanks in advance and Im sorry if this might not be a well formed question, I am relatively new to CS and stackoverflow.

I am hoping to make a simple chrome extension, which overrides the new tab page to display some simple data collected from a couple websites. I am wondering if it is possible to web scrape within the basic JS or chrome API's? Any information or guidance would be greatly appreciated, I have been trying to do research on the subject and haven't found any recent or clear answers to this.

Thanks for your help!

Here is an older stackoverflow question asking the same question but I wasn't able to make any progress from the answers.

Web Scraping in a Google Chrome Extension (JavaScript + Chrome APIs)

• javascript
• google-chrome
• web-scraping
• google-chrome-extension

Share Improve this question Follow asked Jul 20, 2020 at 0:56 Brian WardBrian Ward 6322 silver badges44 bronze badges

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 11

Absolutely and not just simple scraping.

If you think about it using the browser itself is as close as possible to replicating a real user session. You don't have to care about manually setting cookies, discover and construct json http requests. The browser does it all for you. After a page has been rendered (with or without javascript) you can access the DOM and extract any content you like.

Take a look at https://github./get-set-fetch/extension, an open source browser extension that does more than just basic scraping. It supports infinite scrolling, clicking and extracting content from single page javascript apps.

Disclaimer: I'm the extension author.

If you're serious about the subject start by developing a simple chrome extension (from my own experience Chrome throws more verbose extension errors than Firefox): https://developer.chrome./extensions/getstarted

Take a look afterwards at the main get-set-fetch background plugins: FetchPlugin (loads an url in a tab and waits for the DOM to stabilize), ExtractUrlPLugin (identifies additional urls to scrape from the current url), ExtractHtmlContentPlugin (actual scraping based on CSS selectors).

There are downsides though. It’s a lot easier to run a scraping script in your favorite language dumping the scraped content into a database than automatically starting the browser, loading the extension, controlling the extension, exporting scraped data to a format like csv, importing that data into a database.

In my opinion, it only makes sense to use a browser extension if you don’t want to automate the data extraction or the page you’re trying to scrape is so javascript heavy it’s easier to automate the extension than writing a scraping script.

I am wondering if it is possible to web scrape within the basic JS or chrome API's?

Yes, use fetch to call REST APIs.

本文标签： javascriptis it possible to do some simple web scraping in chrome extensionStack Overflow