selenium webdriver - How do I scrape data from Google Find My Device - Stack Overflow

admin管理员组

文章数量:1123362

I am trying to scrape the website data over at / in order to obtain the data about device battery. However, I am unable to directly get this data using a session on selenium driver, I am trying to first login using my cookie and then open the website in the session.

I get logged in using this way but the data for devices is not available. Moreover, when a device is clicked on, it fetches the data for a while before presenting it which might also be the issue in my case

So, my question is that how can I scrape dynamic websites like this? Answers showing how to do so in python would be very much appreciated. Any other logging-in methods/suggestions which work would also be appreciated.

How I create the session and login:

let service = new chrome.ServiceBuilder('path-to-chrome-driver')
    .build();
let options = new chrome.Options();
options.addArguments('headless');
driver = chrome.Driver.createSession(options, service);
await driver.navigate().to('/');
for(cookie: cookies) {
    driver.manage().addCookie({name: name, value: value});
}
await driver.navigate().refresh();
await driver.navigate().to('/');

then get the page source by

driver.getPageSource()

I have read this but it is not useful and outdated for my purposes.

I have tried out creating a session on selenium which allows me to login but I cannot get the data I need only from that.

本文标签： selenium webdriverHow do I scrape data from Google Find My DeviceStack Overflow