c# - AWSSDK.S3 file upload corrupted on .NET 5 - Stack Overflow

admin管理员组

文章数量:1406312

Anyone got a problem on upload on s3 that the file uploaded is corrupted.

Uploaded Sample Preview:

00000000  31 34 30 30 30 3B 63 68  75 6E 6B 2D 73 69 67 6E  14000;chunk-sign
00000010  61 74 75 72 65 3D 30 38  65 37 65 34 31 61 35 65  ature=08e7e41a5e
00000020  33 32 30 62 38 37 33 39  34 66 36 35 37 33 34 35  320b87394f657345
00000030  61 32 31 62 66 30 33 32  37 33 33 31 35 62 64 36  a21bf03273315bd6
00000040  64 31 32 30 35 38 37 63  61 62 65 30 36 39 34 61  d120587cabe0694a
00000050  38 66 33 30 30 63 0D 0A  89 50 4E 47 0D 0A 1A 0A  8f300c...PNG....

But the file uploaded directly to my solution is okay and this is my code implementation.

Startup.cs:

var awsOptions = Configuration.GetSection("AWSSettings");
var credentials = new BasicAWSCredentials(awsOptions["AccessKey"], awsOptions["SecretKey"]);

AWSConfigsS3.UseSignatureVersion4 = true;

var awsConfig = new AmazonS3Config
            {
                ServiceURL = awsOptions["ServiceURL"],
                UseHttp = false,
                ForcePathStyle = true,
                SignatureVersion = "4"
            };

awsConfig.GetType().GetProperty("DisablePayloadSigning")?.SetValue(awsConfig, true);

services.AddSingleton<IAmazonS3>(sp => new AmazonS3Client(credentials, awsConfig));

Controller

[AllowAnonymous, HttpPost("upload")]
public async Task<ActionResult<CustomResponse<UploadFileResponse>>> Upload([FromForm] TestFileRequest request, CancellationToken cancellationToken)
{
    try
    {
        var response = await _s3Service.UploadFileAsync(request.File, FileDirectory.AWSBucket, FileDirectory.AWSDirectory, cancellationToken);
        return ResponseHandler.CreateResponse(response.StatusCode, response.Data, response.Message);
    }
    catch (Exception ex)
    {
        return ResponseHandler.CreateErrorResponse<UploadFileResponse>(ex);
    }
}

Service

public async Task<CustomResponse<UploadFileResponse>> UploadFileAsync(
            IFormFile file,
            string bucketName = AWSBucket,
            string directory = AWSBucketDirectoty,
            CancellationToken cancellationToken = default)
{
    try
    {
        if (file == null || file.Length == 0)
            return new CustomResponse<UploadFileResponse>(400, null, "File is empty or not selected.");

        var id = Guid.NewGuid();
        var fileExtension = Path.GetExtension(file.FileName);
        var newFileName = $"{id}{fileExtension}";
        var key = $"{directory}/{newFileName}";

        using var memoryStream = new MemoryStream();
        await file.CopyToAsync(memoryStream, cancellationToken);

        memoryStream.Position = 0; // Reset stream position

        // Debug: Save file locally to verify it's not corrupt
        await File.WriteAllBytesAsync("debug_image.png", memoryStream.ToArray(), cancellationToken);

        // Use TransferUtility to Upload
        using var transferUtility = new TransferUtility(_s3Client);
        await transferUtility.UploadAsync(new TransferUtilityUploadRequest
                {
                    BucketName = bucketName,
                    Key = key,
                    InputStream = memoryStream,
                    ContentType = file.ContentType ?? "application/octet-stream",
                    CannedACL = S3CannedACL.Private, // Adjust permissions if needed
                    StorageClass = S3StorageClass.Standard // Optional: Set storage class
                }, cancellationToken);

        var preSignedUrl = GetPreSignedURL(key, bucketName);

        var uploadFileResponse = new UploadFileResponse
                {
                    Id = id,
                    FileName = newFileName,
                    Extension = fileExtension,
                    ContentType = file.ContentType,
                    PreSignedUrl = preSignedUrl
                };

        return new CustomResponse<UploadFileResponse>(200, uploadFileResponse);
    }
    catch (Exception ex)
    {
        return new CustomResponse<UploadFileResponse>(500, null, $"Unexpected error: {ex.Message}");
    }
}

Anyone got a problem on upload on s3 that the file uploaded is corrupted.

Uploaded Sample Preview:

00000000  31 34 30 30 30 3B 63 68  75 6E 6B 2D 73 69 67 6E  14000;chunk-sign
00000010  61 74 75 72 65 3D 30 38  65 37 65 34 31 61 35 65  ature=08e7e41a5e
00000020  33 32 30 62 38 37 33 39  34 66 36 35 37 33 34 35  320b87394f657345
00000030  61 32 31 62 66 30 33 32  37 33 33 31 35 62 64 36  a21bf03273315bd6
00000040  64 31 32 30 35 38 37 63  61 62 65 30 36 39 34 61  d120587cabe0694a
00000050  38 66 33 30 30 63 0D 0A  89 50 4E 47 0D 0A 1A 0A  8f300c...PNG....

But the file uploaded directly to my solution is okay and this is my code implementation.

Startup.cs:

var awsOptions = Configuration.GetSection("AWSSettings");
var credentials = new BasicAWSCredentials(awsOptions["AccessKey"], awsOptions["SecretKey"]);

AWSConfigsS3.UseSignatureVersion4 = true;

var awsConfig = new AmazonS3Config
            {
                ServiceURL = awsOptions["ServiceURL"],
                UseHttp = false,
                ForcePathStyle = true,
                SignatureVersion = "4"
            };

awsConfig.GetType().GetProperty("DisablePayloadSigning")?.SetValue(awsConfig, true);

services.AddSingleton<IAmazonS3>(sp => new AmazonS3Client(credentials, awsConfig));

Controller

[AllowAnonymous, HttpPost("upload")]
public async Task<ActionResult<CustomResponse<UploadFileResponse>>> Upload([FromForm] TestFileRequest request, CancellationToken cancellationToken)
{
    try
    {
        var response = await _s3Service.UploadFileAsync(request.File, FileDirectory.AWSBucket, FileDirectory.AWSDirectory, cancellationToken);
        return ResponseHandler.CreateResponse(response.StatusCode, response.Data, response.Message);
    }
    catch (Exception ex)
    {
        return ResponseHandler.CreateErrorResponse<UploadFileResponse>(ex);
    }
}

Service

public async Task<CustomResponse<UploadFileResponse>> UploadFileAsync(
            IFormFile file,
            string bucketName = AWSBucket,
            string directory = AWSBucketDirectoty,
            CancellationToken cancellationToken = default)
{
    try
    {
        if (file == null || file.Length == 0)
            return new CustomResponse<UploadFileResponse>(400, null, "File is empty or not selected.");

        var id = Guid.NewGuid();
        var fileExtension = Path.GetExtension(file.FileName);
        var newFileName = $"{id}{fileExtension}";
        var key = $"{directory}/{newFileName}";

        using var memoryStream = new MemoryStream();
        await file.CopyToAsync(memoryStream, cancellationToken);

        memoryStream.Position = 0; // Reset stream position

        // Debug: Save file locally to verify it's not corrupt
        await File.WriteAllBytesAsync("debug_image.png", memoryStream.ToArray(), cancellationToken);

        // Use TransferUtility to Upload
        using var transferUtility = new TransferUtility(_s3Client);
        await transferUtility.UploadAsync(new TransferUtilityUploadRequest
                {
                    BucketName = bucketName,
                    Key = key,
                    InputStream = memoryStream,
                    ContentType = file.ContentType ?? "application/octet-stream",
                    CannedACL = S3CannedACL.Private, // Adjust permissions if needed
                    StorageClass = S3StorageClass.Standard // Optional: Set storage class
                }, cancellationToken);

        var preSignedUrl = GetPreSignedURL(key, bucketName);

        var uploadFileResponse = new UploadFileResponse
                {
                    Id = id,
                    FileName = newFileName,
                    Extension = fileExtension,
                    ContentType = file.ContentType,
                    PreSignedUrl = preSignedUrl
                };

        return new CustomResponse<UploadFileResponse>(200, uploadFileResponse);
    }
    catch (Exception ex)
    {
        return new CustomResponse<UploadFileResponse>(500, null, $"Unexpected error: {ex.Message}");
    }
}

• c#
• -5
• asp-core-5.0

Share Improve this question Follow edited Mar 6 at 5:27 marc_s 757k184184 gold badges1.4k1.4k silver badges1.5k1.5k bronze badges asked Mar 6 at 2:11 AngelicaAngelica 31711 silver badge1414 bronze badges 2

• Is the file size different between AWS and debug_image.png ? – ProgrammingLlama Commented Mar 6 at 2:16
• @ProgrammingLlama the debug_image.png got 88,069 bytes and the file uploaded to s3 86.38KB then if I upload the file directly to S3 the file size is 86.00KB – Angelica Commented Mar 6 at 2:23

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

I just found the solution that you need to set false the UseChunkEncoding

本文标签： cAWSSDKS3 file upload corrupted on NET 5Stack Overflow