Amazon S3 Assigned URLs Allow Slash In Key

I use the Amazon Java SDK to work with S3 to store downloaded files. I would like to keep the original file name, and I put it at the end of the key, but I also use the virtual directory structure - something like <dirname>/<uuid>/<originalFilename>.

The problem is that when I want to create a designated download URL using api, for example:

URL url = s3Client.generatePresignedUrl(generatePresignedUrlRequest);
return url.toExternalForm();

The sdk url goes beyond the entire key, including slashes. Although it still works, this means that the name of the downloaded file includes the entire key, not just the original bit of the file name at the end. I know that this should be possible to do without escaping slashes, but I try not to rewrite a lot of code already in the SDK. Is there a general solution to this? I know that I used web applications that follow the same pattern and do not have a slash problem.

+5
source share
3 answers

, , , , @aKzenT , , . AmazonS3Client. , , , , . , . gist, :

import com.amazonaws.AmazonWebServiceRequest;
import com.amazonaws.HttpMethod;
import com.amazonaws.Request;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.handlers.RequestHandler;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.Headers;
import com.amazonaws.services.s3.internal.S3QueryStringSigner;
import com.amazonaws.services.s3.internal.ServiceUtils;

import java.util.Date;

/**
 * This class should be a drop in replacement for AmazonS3Client as long as you use the single credential
 * constructor. It could probably be modified to add additional constructors if needed, but this is the one we use.
 * Supporting all of them didn't seem trivial because of some dependencies in the original presignRequest method.
 *
 * The only real purpose of this class is to change the behavior of generating presigned URLs. The original version
 * escaped slashes in the key and this one does not. Pretty url paths are kept intact.
 *
 * @author Russell Leggett
 */
public class PrettyUrlS3Client extends AmazonS3Client{
    private AWSCredentials awsCredentials;

    /**
     * This constructor is the only one provided because it is only one I needed, and it
     * retains awsCredentials which might be needed in the presignRequest method
     *
     * @param awsCredentials
     */
    public PrettyUrlS3Client(AWSCredentials awsCredentials) {
        super(awsCredentials);
        this.awsCredentials = awsCredentials;
    }

    /**
     * WARNING: This method is an override of the AmazonS3Client presignRequest
     * and copies most of the code. Should be careful of updates to the original.
     *
     * @param request
     * @param methodName
     * @param bucketName
     * @param key
     * @param expiration
     * @param subResource
     * @param <T>
     */
    @Override
    protected <T> void presignRequest(Request<T> request, HttpMethod methodName, String bucketName, String key, Date expiration, String subResource) {

        // Run any additional request handlers if present
        if (requestHandlers != null) {
            for (RequestHandler requestHandler : requestHandlers) {
                requestHandler.beforeRequest(request);
            }
        }
        String resourcePath = "/" +
                ((bucketName != null) ? bucketName + "/" : "") +
                ((key != null) ? keyToEscapedPath(key)/* CHANGED: this is the primary change */ : "") +
                ((subResource != null) ? "?" + subResource : "");

        //the request apparently needs the resource path without a starting '/'
        request.setResourcePath(resourcePath.substring(1));//CHANGED: needed to match the signature with the URL generated from the request
        AWSCredentials credentials = awsCredentials;
        AmazonWebServiceRequest originalRequest = request.getOriginalRequest();
        if (originalRequest != null && originalRequest.getRequestCredentials() != null) {
            credentials = originalRequest.getRequestCredentials();
        }

        new S3QueryStringSigner<T>(methodName.toString(), resourcePath, expiration).sign(request, credentials);

        // The Amazon S3 DevPay token header is a special exception and can be safely moved
        // from the request headers into the query string to ensure that it travels along
        // with the pre-signed URL when it sent back to Amazon S3.
        if (request.getHeaders().containsKey(Headers.SECURITY_TOKEN)) {
            String value = request.getHeaders().get(Headers.SECURITY_TOKEN);
            request.addParameter(Headers.SECURITY_TOKEN, value);
            request.getHeaders().remove(Headers.SECURITY_TOKEN);
        }
    }

    /**
     * A simple utility method which url escapes an S3 key, but leaves the
     * slashes (/) unescaped so they can stay part of the url.
     * @param key
     * @return
     */
    public static String keyToEscapedPath(String key){
        String[] keyParts = key.split("/");
        StringBuilder result = new StringBuilder();
        for(String part : keyParts){
            if(result.length()>0){
                result.append("/");
            }
            result.append(ServiceUtils.urlEncode(part));
        }
        return result.toString().replaceAll("%7E","~");
    }
}

UPDATE , , ~. , . . Gist / , .

+1

Java SDK:

https://github.com/aws/aws-sdk-java/blob/master/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L2820

presignRequest, , :

    String resourcePath = "/" +
        ((bucketName != null) ? bucketName + "/" : "") +
        ((key != null) ? ServiceUtils.urlEncode(key) : "") +
        ((subResource != null) ? "?" + subResource : "");

, , , .

, AmazonS3Client , .

url.getQuery() awsURL (https://forums.aws.amazon.com/thread.jspa?messageID=356271). , , , .

, :

URL- Amazon S3 , amazon sdk?

Amazon : https://forums.aws.amazon.com/thread.jspa?messageID=418537

, .

+7

version 1.4.3 of the Java SDK seems to fix this problem. This may have been fixed earlier, but I can confirm that it works correctly in 1.4.3.

+1
source

All Articles