To configure and use multipart uploads for large objects with s3cmd, follow these steps:
Install s3cmd: If you haven't already, install s3cmd on your system. You can find the installation instructions on the s3cmd official website.
Configure s3cmd: Run the s3cmd --configure command to configure s3cmd. You will be prompted to enter your AWS access key, secret key, default region, and a few more settings. Provide the required information based on your AWS account.
Enable multipart uploads: By default, s3cmd tries to upload files in a single operation. To enable multipart uploads, you need to modify the s3cmd configuration file. Open the configuration file located at ~/.s3cfg in a text editor. Add the following lines under the [default] section:
enable_multipart = True
multipart_chunk_size_mb = <chunk_size>
Replace <chunk_size> with the chunk size in megabytes, e.g., 10 for a 10MB chunk.
Upload large objects: To upload a large object using multipart uploads, run the following command:
s3cmd put --multipart <local_file> s3://<bucket_name>/<object_key>
Replace <local_file> with the path to the file on your local system, <bucket_name> with the name of your S3 bucket, and <object_key> with the desired key for the object in S3.
The --multipart flag enables multipart uploads for the specific object.
Monitor and manage multipart uploads: You can monitor and manage your multipart uploads using the following commands:
s3cmd multipart [s3://<bucket_name>/]: List all ongoing and unfinished multipart uploads in the specified bucket (or all buckets if not specified).s3cmd abortmp <s3://<bucket_name>/<object_key>>: Abort a multipart upload for the specified object.s3cmd complmp <s3://<bucket_name>/<object_key>>: Complete a multipart upload for the specified object.Replace <bucket_name> and <object_key> with relevant values.
By following these steps, you can configure and use multipart uploads for large objects with s3cmd.