Uploading file using ftp failed because of ArgumentNullException

[techiechelle]

I'm trying to upload a content from ftp to S3 via standard storage type, but I keep getting ArgumentNullException. No where in my code that I'm using IEnumerable object (see below for sample code).

Error: System.ArgumentNullException: Value cannot be null.
Parameter name: key
at System.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
at System.Collections.Generic.Dictionary2.FindEntry(TKey key) at System.Collections.Generic.Dictionary2.TryGetValue(TKey key, TValue& value)
at Amazon.Runtime.Internal.ErrorHandler.d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__91.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.Signer.<InvokeAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.EndpointDiscoveryHandler.<InvokeAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Amazon.Runtime.Internal.EndpointDiscoveryHandler.d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at Amazon.Runtime.Internal.CredentialsRetriever.d__71.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at Amazon.Runtime.Internal.RetryHandler.d__101.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__101.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__91.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__91.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.S3.Internal.AmazonS3ExceptionHandler.<InvokeAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Amazon.S3.Transfer.Internal.SimpleUploadCommand.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at S3Demo.Program.d__6.MoveNext() in C:...\S3Demo\Program.cs:line 131

I wrote a sample code to see if I can upload file using stream from local dir and from an ftp server via stream. The test for upload from local dir works, but not from FTP. I modeled my code after the S3 Upload example (https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html). Here is an excerpt of my code

public class S3Info {
      public string accessKeyId;
      public string secretAccessKey;

      public S3Info() {
          this.accessKeyId = "InsertAccessKeyId";
          this.secretAccessKey = "insertSecretAccessKey";
          init();
      }
      private S3Info init() {
          return this;
      }
  }
  public class Program
  {
      private const string bucketName = "myBucket";
      private const string key = "simpleStreamUpload.png";
      private static S3Demo.S3Info s3Info;
      public static async Task Main(string[] args)
      {
          s3Info = new S3Demo.S3Info();

          string filename = @"C:\FTP\someImg.png";

          using (FileStream SourceStream = File.Open(filename, FileMode.Open))
          {
              await UploadObject(SourceStream);
              Console.WriteLine("Uploading using FTP ------------------------------------------");
              await UploadObjectFromFTP();
          }
      }

      // This upload, directly reading the file using System.IO, was successful
      public static async Task UploadObject(Stream stream, CancellationToken token) {

          AmazonS3Client client = new AmazonS3Client(s3Info.accessKeyId, s3Info.secretAccessKey, RegionEndpoint.GetBySystemName("ca-central-1"));
          PutObjectRequest request = new PutObjectRequest
          {
              BucketName = bucketName,
              Key = key,
              InputStream = stream
          };

          try {
              PutObjectResponse response = await client.PutObjectAsync(request, token);
          }
          catch (AmazonS3Exception e)
          {
              Console.WriteLine(
                      "Error encountered ***. Message:'{0}' when writing an object"
                      , e.Message);
          }
          catch (Exception e)
          {
              Console.WriteLine(
                  "Unknown encountered on server. Message:'{0}' when writing an object"
                  , e.Message);
          }

      }

      // This upload request failed with the above error. This request was made by 
      public static async Task UploadObjectFromFTP() {
                // calling methods to download file over ftp. stream.Length is non-zero
            try
            {
                using (Stream stream = /* custom ftp download in bytes via System.Net.Sockets.Socket.Receive()*/) {
                    // if i do Console.WriteLine, the stream.Length is still > 0.
                    await sendUploadRequestToS3(stream); 
                }
            }
            catch (Exception ex) {
                Console.WriteLine(ex.Message);
            }

        }
      publicstatic async Task sendUploadRequestToS3(Stream stream) {
            AmazonS3Client client = new AmazonS3Client(s3Info.accessKeyId, s3Info.secretAccessKey, RegionEndpoint.GetBySystemName("ca -central-1"));
            TransferUtility transferUtility = new TransferUtility(client);
            TransferUtilityUploadRequest uploadRequest = new TransferUtilityUploadRequest
            {
                BucketName =bucketName,
                InputStream = stream,
                Key = "ftpUpload.png",
                StorageClass = "STANDARD"
            };

            try
            {
                long percent = 0;
                uploadRequest.UploadProgressEvent += new EventHandler<UploadProgressArgs>((sender, progress) => {
                    percent = progress.TransferredBytes * 100 / progress.TotalBytes;
                });

                await transferUtility.UploadAsync(uploadRequest); // error thrown here
            }
            catch (Exception e)
            {
                Console.WriteLine("Error: " + e.ToString());
                throw e;
            }
        }

I've also tried using PutObjectAsync and the issue is the same. Is the issue with ftp or am I missing something in TransferUtilityUploadRequest?

[ashishdhingra]

@techiechelle Good afternoon. Could you please share the following:

• Version of Nuget packages used.
• Target .NET Framework / .NET Core version
• Environment
• Is the issue reproducible using latest version of AWSSDK.S3?

Thanks,
Ashish

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.