admin管理员组文章数量:1323335
There is a webpage which have a link "Click to Download" Clicking which a file is downloaded . I can download this file manually by going to webpage and clicking on this link however I need to download this file via a python script .
If i see the source i can see the anchor tag is will run a js function
<a class="download-data-link1" onclick=" document.forms['dataform'].submit()" style="cursor:pointer; vertical-align: middle;">Download in csv</a>
There is a webpage which have a link "Click to Download" Clicking which a file is downloaded . I can download this file manually by going to webpage and clicking on this link however I need to download this file via a python script .
If i see the source i can see the anchor tag is will run a js function
<a class="download-data-link1" onclick=" document.forms['dataform'].submit()" style="cursor:pointer; vertical-align: middle;">Download in csv</a>
But i dont know the url of csv file and i am looking for a way to download it via python .
I know we can download a file if we have url using httplib but couldnt understand how to get a file without url .
Tried few things like in header added 'Content-Disposition': 'attachment;filename="data.csv"'}
but it dosent seems to work . Any ideas ?
Share asked Jan 26, 2015 at 9:09 paarth batrapaarth batra 1,4125 gold badges32 silver badges54 bronze badges 4- It has to python? Because links have a tendency (especially nowadays) to be added to the DOM later on, or to respond to certain events, triggered by the user, which makes a headless browser a better option here (IMHO). I'd look into PhantomJS or similar tools – Elias Van Ootegem Commented Jan 26, 2015 at 9:13
- Well even if its not python maybe something which i can invoke from python script . Just saw phantomjs as per your suggestion . Need to study it further – paarth batra Commented Jan 26, 2015 at 9:18
-
PhantomJS
can be automated throughselenium
- so it can be python. – alecxe Commented Jan 26, 2015 at 9:19 -
@paarthbatra: Selenium + chromedriver would be an option, too (link is to 2 scripts: get-drivers downloads what you need,
codeception-start
starts/stops selenium), you can then use that setup with a python script quite easily – Elias Van Ootegem Commented Jan 26, 2015 at 9:26
2 Answers
Reset to default 3Thanks all for your answers but I want to add, how i implemented it.
- First of all you can create a firefox profile. To do that:
- Close all firefox browsers
- go to cmd prompt and execute firefox.exe -P
- create a profile and note down the name of the folder where new profile is created
You can set some options for your profile here, like - automatically download these kind of files from content etc.
Now Download selenium for python and use below code
import os
from selenium import webdriver
download_dir="D:\a"
fp = webdriver.FirefoxProfile(<profile directory here as in step 4>)
fp.set_preference("browser.download.dir", download_dir)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/octet-stream");
browser = webdriver.Firefox(firefox_profile=fp)
browser.get("http://pypi.python/pypi/selenium")
# you can use your url here
browser.find_element_by_partial_link_text("selenium-2").click()
# Use your method to identify class or link text here
browser.close();
Hope this may help others :)
Two basic options can be applied here:
- mimic the logic involved in the
onclick()
call - in your case, make thedataform
form submission usingrequests
, ormechanize
high-level approach - automate a real browser, headless (
PhantomJS
) or not, usingselenium
- find the link and click it:from selenium import webdriver driver = webdriver.PhantomJS() driver.get('url here') driver.find_element_by_class_name('download-data-link1').click()
Though, as far as I understand, clicking the link would trigger a "Download" browser dialog to appear - then PhantomJS
is not an option since it doesn't support downloads. In case of Chrome
or Firefox
you would need to tweak browser capabilities to automatically download files without opening the popup, see:
- Access to file download dialog in Firefox
- Firefox + Selenium WebDriver and download a csv file automatically
本文标签:
版权声明:本文标题:javascript - downloading a file from a webpage using python script without url , calling onClick function - Stack Overflow 内容由网友自发贡献,该文观点仅代表作者本人, 转载请联系作者并注明出处:http://www.betaflare.com/web/1742137903a2422451.html, 本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容,一经查实,本站将立刻删除。
发表评论