javascript - How to find all elements containing the word "download" using Selenium x-path? - Stack Overflow

admin管理员组

文章数量:1317909

I'm using Selenium to do some webscraping and I now want to find all elements on which the user can click and which contain the word "download" (in any capitalization) in either the link text, the button text, the element id, the element class or the href. This can include both links, buttons or any other element.

In this answer I found the an xpath for somebody looking for an xpath to search for buttons based on a certain text (or non-case-sensitive and partial matches):

text = 'download'
driver.find_elements_by_xpath("(//*[contains(text(), 'download')]")

but on this page that returns no results, even though the following link is in there:

<a id="downloadTop" class="navlink" href="javascript:__doPostBack('downloadTop','')">Download</a>

Does anybody know how I can find all elements which somehow contain the word "download" in a website?

[EDIT] This question was marked as a duplicate for a question which gets an answer in which it is suggested to change it to "//*[text()[contains(.,'download')]]". So I tried the following:

>>> from selenium import webdriver
>>> d = webdriver.Firefox()
>>> link = '.aspx?x=492449&y=8687&px=92AD8EAA22C9223FBCA3102EE0AE2899510C03E398A8A08A222AFDACEBFF8BA95D656F01FB04A1437669EC46E93AB5776A33951830BBA97DD94DB1729BF42D76&rand=a17cafc7-26fe-42d9-a61a-894b43a28046&utm_source=PurchaseSuccess&utm_medium=Email&utm_campaign=SystemMails'
>>> d.get(link)
>>> d.find_elements_by_xpath("//*[text()[contains(.,'download')]]")
[]  # As you can see it still doesn't get any results..
>>>

Does anybody know how I can get all elements on which the user can click and which contain the word "download" in either the link text, the button text, the element id, the element class or the href? All tips are wele!

I'm using Selenium to do some webscraping and I now want to find all elements on which the user can click and which contain the word "download" (in any capitalization) in either the link text, the button text, the element id, the element class or the href. This can include both links, buttons or any other element.

In this answer I found the an xpath for somebody looking for an xpath to search for buttons based on a certain text (or non-case-sensitive and partial matches):

text = 'download'
driver.find_elements_by_xpath("(//*[contains(text(), 'download')]")

but on this page that returns no results, even though the following link is in there:

<a id="downloadTop" class="navlink" href="javascript:__doPostBack('downloadTop','')">Download</a>

Does anybody know how I can find all elements which somehow contain the word "download" in a website?

[EDIT] This question was marked as a duplicate for a question which gets an answer in which it is suggested to change it to "//*[text()[contains(.,'download')]]". So I tried the following:

>>> from selenium import webdriver
>>> d = webdriver.Firefox()
>>> link = 'https://www.yourticketprovider.nl/LiveContent/tickets.aspx?x=492449&y=8687&px=92AD8EAA22C9223FBCA3102EE0AE2899510C03E398A8A08A222AFDACEBFF8BA95D656F01FB04A1437669EC46E93AB5776A33951830BBA97DD94DB1729BF42D76&rand=a17cafc7-26fe-42d9-a61a-894b43a28046&utm_source=PurchaseSuccess&utm_medium=Email&utm_campaign=SystemMails'
>>> d.get(link)
>>> d.find_elements_by_xpath("//*[text()[contains(.,'download')]]")
[]  # As you can see it still doesn't get any results..
>>>

Does anybody know how I can get all elements on which the user can click and which contain the word "download" in either the link text, the button text, the element id, the element class or the href? All tips are wele!

• javascript
• python
• html
• selenium
• xpath

Share Improve this question Follow edited May 23, 2017 at 11:59 CommunityBot 111 silver badge asked Nov 20, 2015 at 17:02 kramer65kramer65 54k133133 gold badges331331 silver badges523523 bronze badges 0

Add a ment  | 

8 Answers 8

Sorted by: Reset to default 3 +150

Try this:

//*[(@id|@class|@href|text())
       [contains(translate(.,'DOWNLOAD','download'), 'download')]]

This Xpath 1.0 expression selects: all elements that have an id or class or href attribute or text-node child, whose string value contains the string "download: in any capitalization.

Here is a running proof. The XSLT transformation below is used to evaluate the XPath expression and to copy all selected nodes to the output:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3/1999/XSL/Transform">
 <xsl:output omit-xml-declaration="yes" indent="yes"/>

  <xsl:template match="/">
    <xsl:copy-of select=
    "//*[(@id|@class|@href|text())
       [contains(translate(.,'DOWNLOAD','download'), 'download')]]
    "/>
  </xsl:template>
</xsl:stylesheet>

When we apply the transformation to the following test-document:

<html>
  <a id="downloadTop" class="navlink" 
    href="javascript:__doPostBack('downloadTop','')">Download</a>
  <b id="y" class="x_downLoad"/>
  <p>Nothing to do_wnLoad</p>
  <a class="m" href="www.DownLoad.">Get it!</a>
  <b>dOwnlOad</b>
</html>

The wanted elements are selected and then copied to the output:

<a id="downloadTop" class="navlink" href="javascript:__doPostBack('downloadTop','')">Download</a>
<b id="y" class="x_downLoad"/>
<a class="m" href="www.DownLoad.">Get it!</a>
<b>dOwnlOad</b>

Since you need a case-insensitive match and the XPath 1.0 does not support it - you'll have to use translate() function. Plus, since you need a wildcard match - you need to use contains(). And, since you also want to check the id, class and href attributes, as well as a text:

from selenium import webdriver

driver = webdriver.Firefox()
driver.get("https://www.yourticketprovider.nl/LiveContent/tickets.aspx?x=492449&y=8687&px=92AD8EAA22C9223FBCA3102EE0AE2899510C03E398A8A08A222AFDACEBFF8BA95D656F01FB04A1437669EC46E93AB5776A33951830BBA97DD94DB1729BF42D76&rand=a17cafc7-26fe-42d9-a61a-894b43a28046&utm_source=PurchaseSuccess&utm_medium=Email&utm_campaign=SystemMails")

condition = "contains(translate(%s, 'DOWNLOAD', 'download'), 'download')"
things_to_check = ["text()", "@class", "@id", "@href"]
conditions = " or ".join(condition % thing for thing in things_to_check)

for elm in driver.find_elements_by_xpath("//*[%s]" % conditions):
    print(elm.text)

Here we are basically constructing the expression via string formatting and concatenation, making a case insensitive checks for text(), class, id and href attributes and joining the conditions with or.

Well, the answer you found already tells you how to do what you want. The problem I see is that text = 'download' starts with lower case while the text in <a id="downloadTop" class="navlink" href="javascript:__doPostBack('downloadTop','')">Download</a> starts with upper case.

Start by changing your text to text = 'Download' and see if it finds your element now. If that was the problem then you can use a little trick like

text = 'ownload'

driver.find_elements_by_xpath("(//*[contains(text(), '" + text + "')] | //*[@value='" + text + "'])")

to ignore the first character.

EDIT: Yes you can make it case insensitive.

driver.find_elements_by_xpath("(//*[contains(translate(text(), 'DOWNLOAD', 'download'), 'download')])")

You can use the translate function as below, it is not case sensetive for any words:

driver.find_elements_by_xpath("//*[translate(text(), 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') = 'download']")

>>> driver.find_elements_by_xpath("//*[translate(text(), 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') = 'download']")
[<selenium.webdriver.remote.webelement.WebElement (session="0b07fcba-86ee-3945-a0ae-85619e97ca31", element="{4278753b-8b59-bf45-ae3b-f60f40aed071}")>, <selenium.webdriver.remote.webelement.WebElement (session="0b07fcba-86ee-3945-a0ae-85619e97ca31", element="{8aed425c-063e-7846-915d-d8948219cc12}")>]

If you still want more generalization of xpath and do not want to use that translate function, you can use itertools.product and generate all variant of the string download as node text-attribute as below.

from  itertools import  product
from selenium import webdriver

driver = webdriver.Firefox()
driver.get("https://www.yourticketprovider.nl/LiveContent/tickets.aspx?x=492449&y=8687&px=92AD8EAA22C9223FBCA3102EE0AE2899510C03E398A8A08A222AFDACEBFF8BA95D656F01FB04A1437669EC46E93AB5776A33951830BBA97DD94DB1729BF42D76&rand=a17cafc7-26fe-42d9-a61a-894b43a28046&utm_source=PurchaseSuccess&utm_medium=Email&utm_campaign=SystemMails")
txt = 'Download' # text to be searched
#Generate variants of that txt
l = [(c, c.lower()) if not c.isdigit() else (c,) for c in txt.upper()] #make tuple of upper and lower of each lettern that string (Download)
variants = ["".join(item) for item in product(*l)] # make all variant of the string Download
anchors = ["text()", "@class", "@id", "@href"] #node attribute to be searched
#Generate xpaths
xpaths_or = " or ".join(["contains(%s,'%s')"%(i,j) for i in anchors for j in variants])
xpaths = "//*[%s]" %xpaths_or
for download_tag in driver.find_elements_by_xpath(xpaths):
    print(download_tag.text)
driver.quit()

Output-

Download
Download

N.B. isdigit function to avoid changing case of the numbers if exists.

but on this page that returns no results, even though the following link is in there:

Its because of there is different text. Look:

Download
download

one letter is in the uppercase. So you need to use case insensitive xpath for this:

driver.find_elements_by_xpath("(//*[contains(lower-case(text()), 'download')]")

its must work good enough for you

Well, I don't know selenium very well, but I can suggest a solution, that will work. You can use regular expressions to parse entire page source first. For example, if you need just elements with attributes, containing 'download' substring, use this regexp:

<\w*([a-zA-Z]+).*\w+([a-zA-Z]+)="(.*?download.*?)"?\/?>

Then find all mathes with re.finditer function, every match object will contain tag name (group(1)), attribute name (group(2) and attribute value (group(3))

import re

# wd == webdriver

for m in re.finditer('<\w*([a-zA-Z]+).*\w+([a-zA-Z]+)="(.*?download.*?)"?\/?>', wd.page_source):
    tag, attr, val = m.group(1), m.group(2), m.group(3)

Then, you can use wd.find_elements_by_css_selector (or something else) to find all tags in selenium tree structure:

wd.find_elements_by_css_selector('{0}[{1}={2}]'.format(tag, attr, val))

When using Selenium and finding web elements its better to always search first for "ID" or "Class Name" since its more reliable and easier than using XPath, usually XPath is only used when you cant find your element using the first 2 methods mentioned.

In this case you have a very clear ID tag in the download element of that website.

Try using this instead:

downloadButton = driver.find_element_by_id('downloadTop')

And then you can use this to click it:

downloadButton.click()

本文标签： javascriptHow to find all elements containing the word quotdownloadquot using Selenium xpathStack Overflow