admin管理员组文章数量:1356104
When I am using curl in order to retrieve a html page, I face with the following message:
Please turn JavaScript on and reload the page.
I am not sure how to handle this, hence I can open the same page on my web-browser.
[Q] How could I fix this in order retrieve html-page's information only using terminal?
$ curl /
<html>
<head>
<script src=".js" async defer></script>
<script>
function onSubmit(token) {
document.getElementById("recaptcha-form").submit();
}
</script>
</head>
<body>
<div id="recaptcha-loading" style="margin: 0px; padding: 0px; position: fixed; right: 0px; top: 0px; width: 100%; height: 100%; z-index: 30001; opacity: 0.8;">
<p style="position: absolute; color: White; top: 30%; left: 40%;">
<img src=".rotating-balls-spinner.gif">
</p>
</div>
<center><noscript><h1 style="text-align:center;color:red;"><strong>Please turn JavaScript on and reload the page.</strong></h1></noscript>
<form id='recaptcha-form' action="/captcha" method="POST">
<button id='submitbutton' style="visibility:hidden;" class="g-recaptcha" data-badge=bottomright data-sitekey="6LcigjgUAAAAACyu9edrmWKmIce8h0kIFQz7iyRo" data-callback='onSubmit'></button>
<script>
window.onload = function(){
document.getElementById('submitbutton').click();
}
</script>
<br/>
</form>
</center>
</body>
</html>
If you do inspect element on the site(/) you can see that more detailed html code.
Thank you for your valuable time and help.
When I am using curl in order to retrieve a html page, I face with the following message:
Please turn JavaScript on and reload the page.
I am not sure how to handle this, hence I can open the same page on my web-browser.
[Q] How could I fix this in order retrieve html-page's information only using terminal?
$ curl http://bsod.pw/
<html>
<head>
<script src="https://www.google./recaptcha/api.js" async defer></script>
<script>
function onSubmit(token) {
document.getElementById("recaptcha-form").submit();
}
</script>
</head>
<body>
<div id="recaptcha-loading" style="margin: 0px; padding: 0px; position: fixed; right: 0px; top: 0px; width: 100%; height: 100%; z-index: 30001; opacity: 0.8;">
<p style="position: absolute; color: White; top: 30%; left: 40%;">
<img src="https://250410.selcdn.ru/antiddos/lg.rotating-balls-spinner.gif">
</p>
</div>
<center><noscript><h1 style="text-align:center;color:red;"><strong>Please turn JavaScript on and reload the page.</strong></h1></noscript>
<form id='recaptcha-form' action="/captcha" method="POST">
<button id='submitbutton' style="visibility:hidden;" class="g-recaptcha" data-badge=bottomright data-sitekey="6LcigjgUAAAAACyu9edrmWKmIce8h0kIFQz7iyRo" data-callback='onSubmit'></button>
<script>
window.onload = function(){
document.getElementById('submitbutton').click();
}
</script>
<br/>
</form>
</center>
</body>
</html>
If you do inspect element on the site(http://bsod.pw/) you can see that more detailed html code.
Thank you for your valuable time and help.
Share Improve this question edited Jan 22, 2018 at 20:33 alper asked Jan 22, 2018 at 18:30 alperalper 3,42013 gold badges62 silver badges121 bronze badges 1- 1 Wget does not have support for executing Javascript. – esqew Commented Jan 22, 2018 at 18:39
2 Answers
Reset to default 6There is no "error". You make a GET request using curl. It returns you some HTML. The HTML happens to contain mostly links to JavaScript code that your browser is supposed to load and to execute. Your browser (with JS activated) could load the .js scripts and run them. Those scripts would generate some neat web page. If you don't load the linked scripts, and do not execute them, then you don't get much out of the page. Consider using a proper headless browser instead (see example below).
Here is a small example that is supposed to demonstrate the point:
<!DOCTYPE html>
<html>
<head>
<title>Source code empty, page full!</title>
</head>
<body>
<div id="fillThis">
<p>Almost nothing there in the source code!</p>
<p>... but inspect this div after JS is executed.</p>
</div>
<script>
var fillThis = document.getElementById("fillThis");
for (i = 0; i<1000; i++) {
var child = document.createElement('p');
child.innerHTML = "tons of content " + i;
fillThis.appendChild(child);
}
</script>
</body>
</html>
Just save this as "something.html", and open it in the browser. When you ask you browser to show page source, this is exactly what you will get. However, when you inspect the div by right-clicking on it, it will show a that it has >1000 child elements appended to it. Those are generated by JS in your browser, they do not e from the server in form of HTML.
Edit
I tried to access the page using PhantomJS, it almost worked. Here is what I did:
#!/bin/bash
cat <<HereDoc > /tmp/phantomjsScript.js
var page = require('webpage').create();
page.open('http://example.', function(status) {
if(status === "success") {
console.log(page.frameContent);
}
phantom.exit();
});
HereDoc
phantomjs /tmp/phantomjsScript.js
This is a bash script that generates a helper script in /tmp, which is then executed by phantomjs. PhantomJS loads the website, and also executes the JavaScript. Unfortunately, the website that you've linked to is protected by a captcha-mechanism, and is not directly accessible, so the above example uses example. instead. If you can somehow work around the captcha, you probably can use a similar script to load the HTML, run the JS, and then dump the rendered DOM to the console.
Try running the code on chrome. Actually the error is due to captcha connection and the error says "Cannot contact reCAPTCHA. Check your connection and try again."
本文标签： htmlcurl How to fix quotPlease turn JavaScript on and reload the pagequotStack Overflow
版权声明：本文标题：html - curl: How to fix "Please turn JavaScript on and reload the page" - Stack Overflow 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://www.betaflare.com/web/1743949585a2567039.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。
发表评论