admin管理员组文章数量:1313161
I am trying to sync a local directory to an S3 bucket and the set commands are taking me in an erroneous circle.
(I've scrubbed the personal directory and bucket names)
Command for the simple sync function I am using:
aws s3 sync . s3://<BUCKET NAME>
Result:
An error occurred (MissingContentLength) when calling the PutObject operation: You must provide the Content-Length HTTP header.
I added the "content-length" header in the command:
DIRECTORY=.
BUCKET_NAME="BUCKET NAME"
# Function to upload a file with Content-Length header
upload_file() {
local file=$1
local content_length=$(stat -c%s "$file")
local relative_path="${file#$DIRECTORY/}"
aws s3 sync "$file" "s3://$BUCKET_NAME/$relative_path" \
--metadata-directive REPLACE \
--content-length "$content_length" \
--content-type application/octet-stream \
--content-disposition attachment \
--content-encoding identity
}
export -f upload_file
# Find and upload files in the local directory
find "$DIRECTORY" -type f -exec bash -c 'upload_file "$0"' {} \;
Result:
Unknown options: --content-length,1093865263
I try a simple CP command
aws s3 cp . s3://BUCKETNAME
upload failed: ./ to s3://BUCKETNAME Need to rewind the stream <botocore.httpchecksum.AwsChunkedWrapper object at 0x72351153a720>, but stream is not seekable.
Copying a single file:
aws s3 cp FILENAME s3://BUCKETNAME
Result:
An error occurred (MissingContentLength) when calling the UploadPart operation: You must provide the Content-Length HTTP header.
I am at a loss as to what exactly AWS S3 CLI is looking for from me at this point. Does anyone have any direction to point me to? Thanks!
I am trying to sync a local directory to an S3 bucket and the set commands are taking me in an erroneous circle.
(I've scrubbed the personal directory and bucket names)
Command for the simple sync function I am using:
aws s3 sync . s3://<BUCKET NAME>
Result:
An error occurred (MissingContentLength) when calling the PutObject operation: You must provide the Content-Length HTTP header.
I added the "content-length" header in the command:
DIRECTORY=.
BUCKET_NAME="BUCKET NAME"
# Function to upload a file with Content-Length header
upload_file() {
local file=$1
local content_length=$(stat -c%s "$file")
local relative_path="${file#$DIRECTORY/}"
aws s3 sync "$file" "s3://$BUCKET_NAME/$relative_path" \
--metadata-directive REPLACE \
--content-length "$content_length" \
--content-type application/octet-stream \
--content-disposition attachment \
--content-encoding identity
}
export -f upload_file
# Find and upload files in the local directory
find "$DIRECTORY" -type f -exec bash -c 'upload_file "$0"' {} \;
Result:
Unknown options: --content-length,1093865263
I try a simple CP command
aws s3 cp . s3://BUCKETNAME
upload failed: ./ to s3://BUCKETNAME Need to rewind the stream <botocore.httpchecksum.AwsChunkedWrapper object at 0x72351153a720>, but stream is not seekable.
Copying a single file:
aws s3 cp FILENAME s3://BUCKETNAME
Result:
An error occurred (MissingContentLength) when calling the UploadPart operation: You must provide the Content-Length HTTP header.
I am at a loss as to what exactly AWS S3 CLI is looking for from me at this point. Does anyone have any direction to point me to? Thanks!
Share Improve this question edited Feb 1 at 23:25 John Rotenstein 270k28 gold badges446 silver badges530 bronze badges Recognized by AWS Collective asked Feb 1 at 15:51 Ben MurphyBen Murphy 111 bronze badge 6 | Show 1 more comment1 Answer
Reset to default 0This is a new breaking change feature in aws cli: https://github/boto/boto3/issues/4392 Many 3rd party services not supported this yet.
Try downgrade aws cli version
curl "https://awscli.amazonaws/awscli-exe-linux-x86_64-2.17.13.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update
Or just add 2 final lines to your ~/.aws/config
profile
[profile my_account]
request_checksum_calculation=WHEN_REQUIRED
response_checksum_validation=WHEN_REQUIRED
本文标签: amazon web servicesAWS CLI copy local file to S3 cloud giving errorsStack Overflow
版权声明:本文标题:amazon web services - AWS CLI copy local file to S3 cloud giving errors - Stack Overflow 内容由网友自发贡献,该文观点仅代表作者本人, 转载请联系作者并注明出处:http://www.betaflare.com/web/1741871506a2402222.html, 本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容,一经查实,本站将立刻删除。
aws s3 sync
. Is it just one file that is causing the problem or is it all files? You might want to update your AWS CLI. Or, useaws s3 cp --recursive
. – John Rotenstein Commented Feb 1 at 23:27botocore
versions (aws cli
usesbotocore
). The suggested workaround is to downgradebotocore
to a version before1.36
. – Man made of meat Commented Feb 2 at 4:48