Java HtmlUnit not finding an HtmlTable with specific role - Stack Overflow

admin管理员组

文章数量:1296318

If you go to / , type 3020702601 in the text field and click on Submit, you will see $514.20 at the (4,6) position in a table. If you click on that $514.20, you will get a new page that has a new table. In you inspect any element of that table, you will see that it has role="treegrid".

I am trying, and so far failing, to access, using HtmlUnit 4.7.0, the table with role="treegrid".

Below is my code in Apache Netbeans 24 (a single Java file):

package cochise3020702601;

import java.util.*;
import .htmlunit.*;
import .htmlunit.html.*;

public class Cochise3020702601 {
    public static void main(String[] args) {
    try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
            webClient.getOptions().setCssEnabled(false);
            webClient.getOptions().setThrowExceptionOnScriptError(false);

            HtmlPage page = webClient.getPage("/");
            webClient.waitForBackgroundJavaScriptStartingBefore(10_000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();

            List<HtmlForm> forms = page.getByXPath("//form[@method='post']");
            Iterator<HtmlForm> formsIterator = forms.iterator();
            HtmlForm form = formsIterator.next();

            List<HtmlInput> his = form.getByXPath("//input[@value='Submit']");
            Iterator<HtmlInput> hisIterator = his.iterator();
            HtmlInput hi = hisIterator.next();

            HtmlTextInput textField = form.getInputByName("parcelNumber_input");
            textField.type("3020702601");

            HtmlPage page2 = hi.click();
            webClient.waitForBackgroundJavaScriptStartingBefore(10_000);
            List<HtmlTable> accountTables = page2.getByXPath("//table[@role='grid']");
            Iterator accountTablesIt = accountTables.iterator();
            HtmlTable accountTable = (HtmlTable) accountTablesIt.next();
            List<HtmlElement> clickables = accountTable.getCellAt(5,5).getByXPath("//a[@class='gridlink']");
            Iterator clicks = clickables.iterator();
            HtmlElement he2 = (HtmlElement) clicks.next();
            HtmlPage page3 = (HtmlPage) he2.click();
            List<HtmlElement> hts = page3.getByXPath("//a[@role='treegrid']");
            System.out.println("hts.size() = " + hts.size());
            System.out.println("--------------------------------------------------------");
    }
    catch (Exception e) {
            System.out.println("Exception:  " + e.toString());
    }    
    }    
}

When I run the above program, I expect to see "hts.size() = <some number that is at lease 1>" printout. Instead, I see hts.size() = 0. Which means that my Java code doesn't find any HtmlTable with a role="treegrid".

Any hints on what I might be doing wrong will be appreciated.

If you go to https://parcelinquirytreasurer.cochise.az.gov/ , type 3020702601 in the text field and click on Submit, you will see $514.20 at the (4,6) position in a table. If you click on that $514.20, you will get a new page that has a new table. In you inspect any element of that table, you will see that it has role="treegrid".

I am trying, and so far failing, to access, using HtmlUnit 4.7.0, the table with role="treegrid".

Below is my code in Apache Netbeans 24 (a single Java file):

package cochise3020702601;

import java.util.*;
import .htmlunit.*;
import .htmlunit.html.*;

public class Cochise3020702601 {
    public static void main(String[] args) {
    try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
            webClient.getOptions().setCssEnabled(false);
            webClient.getOptions().setThrowExceptionOnScriptError(false);

            HtmlPage page = webClient.getPage("https://parcelinquirytreasurer.cochise.az.gov/");
            webClient.waitForBackgroundJavaScriptStartingBefore(10_000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();

            List<HtmlForm> forms = page.getByXPath("//form[@method='post']");
            Iterator<HtmlForm> formsIterator = forms.iterator();
            HtmlForm form = formsIterator.next();

            List<HtmlInput> his = form.getByXPath("//input[@value='Submit']");
            Iterator<HtmlInput> hisIterator = his.iterator();
            HtmlInput hi = hisIterator.next();

            HtmlTextInput textField = form.getInputByName("parcelNumber_input");
            textField.type("3020702601");

            HtmlPage page2 = hi.click();
            webClient.waitForBackgroundJavaScriptStartingBefore(10_000);
            List<HtmlTable> accountTables = page2.getByXPath("//table[@role='grid']");
            Iterator accountTablesIt = accountTables.iterator();
            HtmlTable accountTable = (HtmlTable) accountTablesIt.next();
            List<HtmlElement> clickables = accountTable.getCellAt(5,5).getByXPath("//a[@class='gridlink']");
            Iterator clicks = clickables.iterator();
            HtmlElement he2 = (HtmlElement) clicks.next();
            HtmlPage page3 = (HtmlPage) he2.click();
            List<HtmlElement> hts = page3.getByXPath("//a[@role='treegrid']");
            System.out.println("hts.size() = " + hts.size());
            System.out.println("--------------------------------------------------------");
    }
    catch (Exception e) {
            System.out.println("Exception:  " + e.toString());
    }    
    }    
}

When I run the above program, I expect to see "hts.size() = <some number that is at lease 1>" printout. Instead, I see hts.size() = 0. Which means that my Java code doesn't find any HtmlTable with a role="treegrid".

Any hints on what I might be doing wrong will be appreciated.

• java
• htmlunit

Share Follow edited Feb 15 at 12:52 Mark Rotteveel 109k229229 gold badges156156 silver badges220220 bronze badges asked Feb 11 at 23:05 God's Gift To JavaGod's Gift To Java 4544 bronze badges 5

• Do you need to wait for the table to be loaded before you can get it? Like in Selenium, you would wait until the element is clickable for example. – pebble unit Commented Feb 12 at 0:51
• How do I do that? ... I have just added webClient.waitForBackgroundJavaScriptStartingBefore(100_000); immediately after HtmlPage page3 = (HtmlPage) he2.click(); and I am still getting the same result: hts.size() = 0 – God's Gift To Java Commented Feb 12 at 3:36
• Then, you might want to try some Thread.sleep() as explicit waits as I m not sure how its done in HtmlUnit. You may also want to try printing out the page source as well to take a look at what your code is getting – pebble unit Commented Feb 12 at 3:46
• I don't know how to do the equivalent of Thread.sleep() in HtmlUnit. I have added System.out.println("page3.asXml() = " + page3.asXml()); immediately after webClient.waitForBackgroundJavaScriptStartingBefore(100_000);. It does show, as expected, Payment History. But a search for treegrid didn't return any result. So, the table isn't being loaded into Page 3 after webClient.waitForBackgroundJavaScriptStartingBefore(100_000);. – God's Gift To Java Commented Feb 12 at 16:54
• Update: I have filed this as an issue at HtmlUnit's GitHub: github/HtmlUnit/htmlunit/issues/927 – God's Gift To Java Commented Feb 13 at 23:28

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

Please have a look at https://github/HtmlUnit/htmlunit/issues/927 for details how the make this work.

本文标签： Java HtmlUnit not finding an HtmlTable with specific roleStack Overflow