1. Initiate multipart Upload
Post request headers:
POST /ObjectName?uploads HTTP/1.1
Host: BucketName.s3.amazonaws.com
Date: date
Authorization: signatureValue
Data that to be encoded with secret key
final StringBuffer buf = new StringBuffer();
buf.append("POST").append("\n");
buf.append("\n");
buf.append("\n");
buf.append(date).append("\n");
buf.append("/" + bucket + key + "?uploads");
return buf.toString();
Java code snippet to set header:
final PostMethod filePost1 = new PostMethod("http://" + bucket + "." + "s3.amazonaws.com");
filePost1.setPath(key + "?uploads");
filePost1.addRequestHeader("Authorization", "AWS " + awsAccessKeyId + ":" + signature);
filePost1.addRequestHeader("Date", date);
Note: No data to post.
Response
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Bucket>example-bucket</Bucket>
<Key>example-object</Key>
<UploadId>
VXBsb2FkIElEIGZvciA2aWWpbmcncyBteS1tb3ZpZS5tMnRzIHVwbG9hZA
</UploadId>
</InitiateMultipartUploadResult>
Extract UploadId from response xml, which is needed while uploading parts
2. Upload parts
Post request headers:
PUT /ObjectName?partNumber=PartNumber&uploadId=UploadId HTTP/1.1
Host: BucketName.s3.amazonaws.com
Date: date
Content-Length: Size
Authorization: Signature
Data that to be encoded with secret key
final StringBuffer buf = new StringBuffer();
buf.append("PUT").append("\n");
buf.append("\n");
buf.append("\n");
buf.append(date).append("\n");
buf.append("/" + bucket + key + "?partNumber=" + String.valueOf(partnumber) + "&uploadId=" + uploadID);
return buf.toString();
Java code snippet to set header:
final PutMethod filePut = new PutMethod("http://" + bucket + "." + "s3.amazonaws.com");
filePut.setPath(key + "?partNumber=" + String.valueOf(partNumber) + "&uploadId=" + uploadId);
filePut.addRequestHeader("Authorization", "AWS " + awsAccessKeyId + ":" + signature);
filePut.addRequestHeader("Date", date);
...
final int dateRead = randomAccess.read(buffer, 0, upLoadlimit);
filePut.addRequestHeader("Content-Length", String.valueOf(dateRead));
Note: post part of file.
Response
Extract ETag header form response header
HTTP/1.1 200 OK
x-amz-id-2: Vvag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==
x-amz-request-id: 656c76696e6727732072657175657374
Date: Mon, 1 Nov 2010 20:34:56 GMT
ETag: "b54357faf0632cce46e942fa68356b38"
Content-Length: 0
Connection: keep-alive
Server: AmazonS3
We need map of partid and ETag of all parts to Complete multipart upload
Java code snippet:
final Header etag = filePut.getResponseHeader("ETag");
if ( null != etag )
{
upTag = etag.getValue();
}
3. Complete Multipart Upload
Post request headers:
POST /ObjectName?uploadId=UploadId HTTP/1.1
Host: BucketName.s3.amazonaws.com
Date: Date
Content-Length: Size
Authorization: Signature
Data that to be encoded with secret key
final StringBuffer buf = new StringBuffer();
buf.append("POST").append("\n");
buf.append("\n");
buf.append("\n");
buf.append(date).append("\n");
buf.append("/" + bucket + key + "?uploadId=" + uploadID);
return buf.toString();
Java code snippet to set header:
final PostMethod filePost1 = new PostMethod("http://" + bucket + "." + "s3.amazonaws.com");
filePost1.setPath(key + "?uploadId=" + uploadID);
filePost1.addRequestHeader("Authorization", "AWS " + awsAccessKeyId + ":" + signature);
filePost1.addRequestHeader("Date", date);
filePost1.addRequestHeader("Content-Length", String.valueOf(postData.length()));
filePost1.setRequestEntity(new ByteArrayRequestEntity(postData.getBytes()));
Note: postData here is XML format as below.
<CompleteMultipartUpload>
<Part>
<PartNumber>1</PartNumber>
<ETag>"a54357aff0632cce46d942af68356b38"</ETag>
</Part>
<Part>
<PartNumber>2</PartNumber>
<ETag>"0c78aef83f66abc1fa1e8477f296d394"</ETag>
</Part>
<Part>
<PartNumber>3</PartNumber>
<ETag>"acbd18db4cc2f85cedef654fccc4a4d8"</ETag>
</Part>
</CompleteMultipartUpload>
Response
<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Location>http://Example-Bucket.s3.amazonaws.com/Example-Object</Location>
<Bucket>Example-Bucket</Bucket>
<Key>Example-Object</Key>
<ETag>"3858f62230ac3c915f300c664312c11f-9"</ETag>
</CompleteMultipartUploadResult>
4. Notes & References
1) Except last part all parts must be greater than 5mb
2) Date format should "EEE, dd MMM yyyy HH:mm:ss GMT"
E.g.
Sun, 05 Aug 2007 15:57:11 GMT
References
Function used to Encode data with Secret key.
public static String sign(final String data, final String key) throws SignatureException
{
String result;
try
{
final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
// get an hmac_sha1 key from the raw key bytes
final SecretKeySpec signingKey1 = new SecretKeySpec(key.getBytes(), HMAC_SHA1_ALGORITHM);
// get an hmac_sha1 Mac instance and initialize with the signing key
final Mac mac1 = Mac.getInstance(HMAC_SHA1_ALGORITHM);
mac1.init(signingKey1);
// compute the hmac on input data bytes final byte[] rawHmac = mac1.doFinal(data.getBytes());
// base64-encode the hmac
result = Base64.encodeBase64String(rawHmac);
result = result.trim();
}
catch ( final Exception e )
{
throw new SignatureException("Failed to generate HMAC : " + e.getMessage());
}
return result;
