Python Requests Disable zstd Encoding - Stack Overflow

admin管理员组

文章数量:1287104

My Synology DS418play recently updated to the latest version of DSM7 that is available. I noticed that a lot of the python scripts that I had have started returning weirdly encoded data. Here is an example of the code:

requests.get("/").content

returns

b'(\xb5/\xfd\x04X\x1c&\x00\xb6\xbd\xba50\x8b&\x0f\xc0\xc0@\xc7\xb000P\x15\x7fkd\xe1\x8eRJ\x1d\xa4MC\x8bw;\xacv/Ln\x804E\xe7i\xf2\xff\x00U\x11Y\xd9n\x98C\xbe\xcc\xa0\x8ce\x15\xb1\x00\xab\x00\xa5\x00\xd5\xbf\xda\xd8Kl\xa7\x8ds(\x8aK\xb06|\x97\x9a{Tk\x154T\xa7d+\xed?\x15<\xa7?\xdfy\x12z\xe4\x9c\xb5\x1e\xae\xbb\xfb\xad\xf5p\x0f\x82\x05\xc6#\x12\x99\x98\xe8~kA\xd8\x98\xb2\xfa\x83\x87\xeb\xa7\xa8\xf4\x91\xa6E"\x11\x08%WiZI\xf8T\x94\x9c!\x8dM\xa5\x8f\xdc\x83 \xd1\x16\x18\xbd1\x1f\xac\xf5p\xceS\xf2%\xf3l-m\x10T\xfa\xa8%\xb84\x08[\xf60\xb1i\x9aZ\x93\xdc\xffH\xb5:`\xd1\x1a\x85\xd5\xce\x9f\xb9B|i\xc8\xc3 ......'

and it looks like this is because the request headers is 'Accept-Encoding': 'gzip, deflate, br, **zstd**'.

Running requests.get("/", headers={'Accept-Encoding': 'deflate'}).content returns the proper data. I am trying to avoid changing each of my python requests to explicitly set Accept-Encoding. Is there a way to prevent requests from using zstd compression? The output breaks a lot of my scripts.

My Synology DS418play recently updated to the latest version of DSM7 that is available. I noticed that a lot of the python scripts that I had have started returning weirdly encoded data. Here is an example of the code:

requests.get("https://www.23andmedatasettlement/").content

returns

b'(\xb5/\xfd\x04X\x1c&\x00\xb6\xbd\xba50\x8b&\x0f\xc0\xc0@\xc7\xb000P\x15\x7fkd\xe1\x8eRJ\x1d\xa4MC\x8bw;\xacv/Ln\x804E\xe7i\xf2\xff\x00U\x11Y\xd9n\x98C\xbe\xcc\xa0\x8ce\x15\xb1\x00\xab\x00\xa5\x00\xd5\xbf\xda\xd8Kl\xa7\x8ds(\x8aK\xb06|\x97\x9a{Tk\x154T\xa7d+\xed?\x15<\xa7?\xdfy\x12z\xe4\x9c\xb5\x1e\xae\xbb\xfb\xad\xf5p\x0f\x82\x05\xc6#\x12\x99\x98\xe8~kA\xd8\x98\xb2\xfa\x83\x87\xeb\xa7\xa8\xf4\x91\xa6E"\x11\x08%WiZI\xf8T\x94\x9c!\x8dM\xa5\x8f\xdc\x83 \xd1\x16\x18\xbd1\x1f\xac\xf5p\xceS\xf2%\xf3l-m\x10T\xfa\xa8%\xb84\x08[\xf60\xb1i\x9aZ\x93\xdc\xffH\xb5:`\xd1\x1a\x85\xd5\xce\x9f\xb9B|i\xc8\xc3 ......'

and it looks like this is because the request headers is 'Accept-Encoding': 'gzip, deflate, br, **zstd**'.

Running requests.get("https://www.23andmedatasettlement/", headers={'Accept-Encoding': 'deflate'}).content returns the proper data. I am trying to avoid changing each of my python requests to explicitly set Accept-Encoding. Is there a way to prevent requests from using zstd compression? The output breaks a lot of my scripts.

• python
• python-requests
• zstd

Share Improve this question Follow asked Feb 24 at 19:15 BijanBijan 8,6702020 gold badges102102 silver badges160160 bronze badges

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 2

I was able to find a commit that basically says if zstandard module is installed, add zstd to the Accept-Encoding.

Running pip uninstall zstandard fixed my issue.

Running requests.get("https://www.23andmedatasettlement/", headers={'Accept-Encoding': 'deflate'}).content returns the proper data. I am trying to avoid changing each of my python requests to explicitly set Accept-Encoding.

I suggest taking look at functools.partial, in this particular case

import functools
import requests
requests.get = functools.partial(requests.get, headers={'Accept-Encoding': 'deflate'})
r = requests.get("https://httpbin./get")
print(r.json())

gives output

{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'deflate', 'Host': 'httpbin.', 'User-Agent': 'python-requests/2.31.0', 'X-Amzn-Trace-Id': 'Root=1-67bcd5ea-7dbedeec1a12af8b7ed3f74e'}, 'origin': '213.134.176.24', 'url': 'https://httpbin./get'}

本文标签： Python Requests Disable zstd EncodingStack Overflow