admin管理员组文章数量:1323730
I'm simply trying to make an HTTP request to this product on the Pokemon Center website. If you have cookies cleared and open the URL then the request will succeed (200 status), but without any cookies Pokemon only responds with a Set Cookie
(no HTML body) and then the webpage reloads with the received cookies and the GET request succeeds and returns the actual HTML.
I'm having trouble replicating this in Golang: my first request responds with 200 and the set cookies, the second request should have the cookies but responds with the same thing instead of actually returning the HTML.
There's a bunch of requests that take place between the first product page GET request and the second one. You can simply see them by making sure you have cookies cleared and then load that URL with the Dev Network tool open.
It seems like an incapsula block, how can I get past this? The images attached are for the first GET request with NO cookies, and the second is for the GET request WITH cookies.
First request:
Second request:
func FetchPokemonURL(client *http.Client, productUrl string, firstTry bool) (string, bool, string) {
req, err := http.NewRequest("GET", productUrl, nil)
if err != nil {
return "", false, "Failed to create request: " + err.Error()
}
req.Header.Set("Host", "www.pokemoncenter")
req.Header.Set("Sec-Fetch-Dest", "document")
req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15")
req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
if firstTry {
req.Header.Set("Sec-Fetch-Site", "none")
} else {
req.Header.Set("Sec-Fetch-Site", "same-origin")
}
req.Header.Set("Sec-Fetch-Mode", "navigate")
if !firstTry {
req.Header.Set("Referer", productUrl)
req.Header.Set("Cache-Control", "max-age=0")
}
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
req.Header.Set("Priority", "u=0, i")
req.Header.Set("Accept-Encoding", "gzip, deflate, br")
req.Header.Set("Connection", "keep-alive")
resp, err := client.Do(req)
if err != nil {
return "", false, fmt.Sprintf("Error making request: %v\n", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return "", false, fmt.Sprintf("Error reading response body: %v\n", err)
}
strBody := string(body)
if firstTry {
return FetchPokemonURL(client, productUrl, false)
} else {
return strBody, false, ""
}
}
本文标签: goHTTP request fails for Incapsula blockStack Overflow
版权声明:本文标题:go - HTTP request fails for Incapsula block - Stack Overflow 内容由网友自发贡献,该文观点仅代表作者本人, 转载请联系作者并注明出处:http://www.betaflare.com/web/1742122080a2421761.html, 本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容,一经查实,本站将立刻删除。
发表评论