How to know if a page has failed to load while using SeleniumPython - ChromeEdge? - Stack Overflow

admin管理员组

文章数量:1419223

I am trying to program a bot using Selenium/Python - Chrome/Edge web driver that changes the section of a course that the university has assigned me to from my university's registration page. However, in the registration period, there are lots of people who are trying to add/drop courses or change sections; therefore, the site becomes overloaded with people, causing unpredictably long loading times for the registration website. The problem here is that sometimes the website does not load and gives an error or even worse, sometimes it looks like its loading, but the page is not actually loading and will fail to load after about 1-2 minutes. This means that the bot will probably waste lots of time trying to load a page that will not load in the end, and I would like to prevent this by refreshing the page if the program sees that the page will not load. My question is, is there any way to know if a page will load or will fail to load or has failed to load with Selenium Python? I know there is a "complete" condition for checking if the page has been successfully loaded; however, I haven't found a way to see if a page has failed to load etc. Also, the "loading" condition from my understanding, is not very reliable in my case since the program also does not know exactly what to look for to check if a page will actually load or is doing one of those "fake" loads I have talked about above, which will fail after some time. I should also add that I tried to use explicit wait - until and check if the login box is available to see if it would work; however, since the site's loading time is unpredictable during the registration period, it also does not work. Could anyone with knowledge in how sites actually load and knows when a site will load or fail could help me figure this out? P.S. I did not include any code here since the registration site is closed outside the registration period and the code is pretty much useless at the time that I'm asking this question.

I am trying to program a bot using Selenium/Python - Chrome/Edge web driver that changes the section of a course that the university has assigned me to from my university's registration page. However, in the registration period, there are lots of people who are trying to add/drop courses or change sections; therefore, the site becomes overloaded with people, causing unpredictably long loading times for the registration website. The problem here is that sometimes the website does not load and gives an error or even worse, sometimes it looks like its loading, but the page is not actually loading and will fail to load after about 1-2 minutes. This means that the bot will probably waste lots of time trying to load a page that will not load in the end, and I would like to prevent this by refreshing the page if the program sees that the page will not load. My question is, is there any way to know if a page will load or will fail to load or has failed to load with Selenium Python? I know there is a "complete" condition for checking if the page has been successfully loaded; however, I haven't found a way to see if a page has failed to load etc. Also, the "loading" condition from my understanding, is not very reliable in my case since the program also does not know exactly what to look for to check if a page will actually load or is doing one of those "fake" loads I have talked about above, which will fail after some time. I should also add that I tried to use explicit wait - until and check if the login box is available to see if it would work; however, since the site's loading time is unpredictable during the registration period, it also does not work. Could anyone with knowledge in how sites actually load and knows when a site will load or fail could help me figure this out? P.S. I did not include any code here since the registration site is closed outside the registration period and the code is pretty much useless at the time that I'm asking this question.

• python
• selenium-webdriver
• selenium-chromedriver

Share Improve this question Follow asked Jan 29 at 11:33 GokalpGokalp 344 bronze badges 1

• 1 It is hard to read such a long text without any proof. Proof is usually a code. In your case it could be some HTML or debugging info. – vitaliis Commented Jan 31 at 5:03

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

What JeffC answered is correct. Either you check if a specific element on the page has loaded or you wait for the timeout(which you can set yourself to however long you want).

A page "not loading" will display the browser timeout window, which you can check for, but even if the timeout is reached, it doesn't mean the page wouldn't have loaded eventually so it's impossible to tell that it won't load, it all depends on how patient you want to be in waiting for it to do so.

Since you can only check and wait for one element at a time in selenium, what you want to do if you don't want to wait for the timeout of each sequentially is to manually do a loop(with a custom timeout checking the current time + however long you want) that constantly search for each element presence on the page and once one is found, you break the loop and you know which page loaded(what you want, the website error or the browser timeout or whatever else you want to check for).

Edit: Here's an example of how I handled looking at multiple things at once(feed the good xpath to the element you're expecting to see and add additional xpath for failures scenario after):

# Returns True if the first element found is the first parameter
def MultiWait(*xpaths):
    start_time = time.time()
    xpath_indices = {xpath: index for index, xpath in enumerate(xpaths)}
    while time.time() - start_time < 90:
        for xpath in xpaths:
            if xpath:
                elements = driver.find_elements(By.XPATH, xpath)
                if elements:
                    first_element_index = xpath_indices[xpaths[0]]
                    current_xpath_index = xpath_indices[xpath]
                    if first_element_index == current_xpath_index:
                        return True  # First XPath was found first
                    else:
                        return False  # Another XPath was found first
        time.sleep(1)  # Wait for 1 second before checking again
    return False  # Timed out

本文标签： How to know if a page has failed to load while using SeleniumPythonChromeEdgeStack Overflow