web crawler - Simulate human click in JavaScript - Stack Overflow

admin管理员组

文章数量:1392099

I have a small scraper where I need to click an anchor link using JavaScript. I've tried a few ways: jQuery.click(), document.createEvent('MouseEvents') etc. They all sort of worked, however they don't fully execute like a human click (they open a tab like they should but don't start a download).

The anchor tag has this attribute:

onclick="if (document.getElementById('ReportViewer_ctl01_ctl05_ctl00').selectedIndex == 0) return false; 
if (!ClientToolbarReportViewer_ctl01.HandleClientSideExport()) __doPostBack('ReportViewer$ctl01$ctl05$ctl01','');return false;"

I've also tried running to crux of this in the mand line :

 __doPostBack('ReportViewer$ctl01$ctl05$ctl01','')

this also sort of works but not fully like a human click.

I can go into more detail if required however at the moment I am looking for a magic bullet which I think should exist.

I have a small scraper where I need to click an anchor link using JavaScript. I've tried a few ways: jQuery.click(), document.createEvent('MouseEvents') etc. They all sort of worked, however they don't fully execute like a human click (they open a tab like they should but don't start a download).

The anchor tag has this attribute:

onclick="if (document.getElementById('ReportViewer_ctl01_ctl05_ctl00').selectedIndex == 0) return false; 
if (!ClientToolbarReportViewer_ctl01.HandleClientSideExport()) __doPostBack('ReportViewer$ctl01$ctl05$ctl01','');return false;"

I've also tried running to crux of this in the mand line :

 __doPostBack('ReportViewer$ctl01$ctl05$ctl01','')

this also sort of works but not fully like a human click.

I can go into more detail if required however at the moment I am looking for a magic bullet which I think should exist.

• javascript
• web-crawler
• web-scraping

Share Improve this question Follow edited Sep 17, 2011 at 22:17 Luwe 3,03211 gold badge2222 silver badges2222 bronze badges asked Sep 17, 2011 at 20:34 henry.oswaldhenry.oswald 5,4341313 gold badges5353 silver badges7676 bronze badges 5

• Why don't you access the desired uri with PHP cURL (in case you're using PHP as server-side language)? – yoda Commented Sep 17, 2011 at 20:42
• I think there's a deliberate security-based reason for the dissimilarity; it seems to be to prevent drive-by downloads/EULA-acceptance and so forth. Though I could be hideously, hideously wrong. – David Thomas Commented Sep 17, 2011 at 20:43
• @yoda thanks, not using php, its a mishmash of odds and ends. This link actually submits a form to simulate a download. – henry.oswald Commented Sep 17, 2011 at 20:46
• @beck then how are you injecting javascript on the page? It's always usefull to mention everything that is relevant to the subject. – yoda Commented Sep 17, 2011 at 21:19
• this also sort of works but not fully like a human click. - can u explain what you mean by "sort of works" ? b/c this approach should work. – ampersand Commented Sep 17, 2011 at 21:41

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 6

I keep a pastebin saved of two programmatic ways to do it. It's only ever failed me when google decided to strip the window object (and every other object) of their default functions >.>

http://pastebin./VMHvjRaR

function callClickEvent(element){
    var evt = document.createEvent("HTMLEvents"); 
    evt.initEvent("click", true, true); 
    element.dispatchEvent(evt);
}

function callClickEvent2(element){
    var evt = document.createEvent("MouseEvents");
    evt.initMouseEvent("click", true, true, window,
    0, 0, 0, 0, 0, false, false, false, false, 0, null);
    element.dispatchEvent(evt);
}

callClickEvent(document.getElementById("myElement"))
callClickEvent2(document.getElementById("myElement"))

MDN documentation:

• document.createEvent
• event.initEvent
• event.initMouseEvent
• element.dispatchEvent

本文标签： web crawlerSimulate human click in JavaScriptStack Overflow