Uploading Files Directly to Amazon S3 Using Uppy

This guide will help you to create a user-friendly image upload feature with image processing.

Requirements

This is an advanced tutorial. To follow it, you should be familiar with basic platformOS concepts, HTML, JavaScript, Liquid, GraphQL, and some topics in the Get Started section.

• platformOS Workflow
• Get Started

Steps

Uploading images directly to Amazon S3 is a four-step process:

Step 1: Create Table and image processing configuration

There will be only one property in Table used in this example (photo), but it will generate 4 different versions of the image uploaded by the user. Apart from explicitly defined versions, there is also an original version that is always there.

modules/direct_s3_upload/public/schema/uppy.yml

name: uppy properties: - name: photo type: upload options: content_length: lte: 2048 # maximum file size will be 2048KB versions: - name: large_bw # version name output: format: jpeg # output format quality: 70 # output quality resize: width: 1600 # width of the image - in this case max width height: 1200 # height of the image - in this case max height fit: inside # preserve aspect ratio, do not crop the image, make sure all pixels are within viewport without_enlargement: false # if image is smaller than width/height, do not enlarge it manipulate: greyscale: true # make image greyscale (similar to css "filter: grayscale(1)") blur: 20 # blur using gaussian mask (similar to css "filter: blur(1.5rem)") flatten: red # merge alpha transparency channel (if exists) with given color - name: small output: format: webp quality: 80 resize: width: 640 height: 480 fit: cover position: top # define the gravity direction or strategy when cropping the image using fit cover or contain - name: thumb output: format: png quality: 80 resize: width: 200 fit: contain background: "rgba(255, 255, 255, 0.5)" # if resized image is not covering whole viewport, set background color - name: center output: format: jpeg quality: 100 resize: height: 400 width: 400 fit: cover 

Step 2: Get presign data and expose it using a form

To upload a file to AWS S3 directly, you will need the server-generated data called signature. This information is from GraphQL. It takes table and property_name as arguments. Other options like versions, content_length, ACL are taken from property configuration.

modules/direct_s3_upload/public/views/pages/images.liquid

--- slug: direct-s3-upload/images --- {% graphql data %} mutation presign { property_upload_presigned_url(table: "modules/direct_s3_upload/uppy", property_name: "my_upload") { upload_url upload_url_payload } } {% endgraphql %} {% assign data = data.property_upload_presigned_url %} <form action="{{ data.upload_url }}" data-s3-uppy="form" data-s3-uppy-user-id="{{ context.current_user.id }}" hidden> {% for field in data.upload_url_payload %} <input type="hidden" name="{{ field[0] }}" value='{{ field[1] }}'> {% endfor %} </form> 

Below the form, paste the code responsible for containing Uppy.

modules/direct_s3_upload/public/views/pages/images.liquid

<div class="row"> <div class="col-5"> <div class="card"> <div class="card-body"> <div id="drag-drop-area"></div> <!-- This is where Uppy will inject itself --> </div> </div> </div> <div class="col-7"> <div class="card"> <div class="card-body"> <p>URLs to image returned by AWS (input):</p> <small> <ol data-s3-uppy="log"></ol> </small> </div> </div> </div> </div> 

Step 3: Configure Uppy and use presigned data

Uppy is a JavaScript widget that helps to handle uploads. With its S3 plugin on board, you can make the implementation much shorter and easier.

However, before you can use any scripts, you need to load them.

modules/direct_s3_upload/public/views/pages/images.liquid

<link href="https://transloadit.edgly.net/releases/uppy/v1.6.0/uppy.css" rel="stylesheet"> <script src="https://transloadit.edgly.net/releases/uppy/v1.6.0/uppy.js" defer></script> <script src="https://cdn.jsdelivr.net/npm/sweetalert2@9" defer></script> <script src="{{ 'modules/direct_s3_upload/images.js' | asset_url }}" defer></script> 

This is the final images.js file, responsible for uploading the image, logging its URL, creating the record, and showing the confirmation box.

modules/direct_s3_upload/public/assets/images.js

const _form = document.querySelector('[data-s3-uppy="form"]'); const _log = document.querySelector('[data-s3-uppy="log"]'); const uppy = Uppy.Core({ autoProceed: true, restrictions: { maxFileSize: 2097152, // Limit size to 2 MB on the javascript side maxNumberOfFiles: 3, allowedFileTypes: ['image/png', 'image/jpeg', 'image/webp'] } }) .use(Uppy.Dashboard, { inline: true, target: '#drag-drop-area', note: 'Images only, up to 3 files, 2MB each', width: '100%', height: 350 }) .use(Uppy.DragDrop) .use(Uppy.GoldenRetriever) .use(Uppy.AwsS3, { getUploadParameters() { // 1. Get URL to post to from action attribute const _url = _form.getAttribute('action'); // 2. Create Array from FormData object to make it easy to operate on const _formDataArray = Array.from(new FormData(_form)); // 3. Create a JSON object from array const _fields = _formDataArray.reduce((acc, cur) => ({ ...acc, [cur[0]]: cur[1] }), {}); // 4. Return resolved promise with Uppy. Uppy it will add file in file param as the last param return Promise.resolve({ method: 'POST', url: _url, fields: _fields }); } }); uppy.on('upload-success', (_file, data) => { const li = document.createElement('li'); li.textContent = data.body.location; _log.appendChild(li); }); uppy.on('complete', ({ failed, successful }) => { /* For every successfully uploaded image to S3, send request to the Instance that will create a record with the uploaded image's URL as direct_url param. */ Promise.all(successful.map(({ response }) => createImage(response.body.location))) .then(() => { // Show confirmation box (SweetAlert2) after all the images has been created Swal.fire({ title: 'Images uploaded', icon: 'success', text: 'Press refresh to display the results', confirmButtonText: 'Refresh', showCloseButton: true }).then(result => { if (!result.value) { return; } // Reload page after "Refresh" button has been clicked inside Sweet Alert2 window.location.reload(); }); }); }); const createImage = imageUrl => { // Get logged in user id const userId = _form.dataset.s3UppyUserId; // Create record for this user with s3 image url return fetch('/direct-s3-upload/images/record_create', { method: 'POST', headers: { 'Content-Type': 'application/json', Accept: 'application/json' }, body: JSON.stringify({ direct_url: imageUrl, user_id: userId }) }).then(response => response.json()); }; 

Step 4: Create page that will create the record with the image

For fetch to succeed, you need to create a page under the /direct-s3-upload/images/record_create path. It will execute the record_create mutation with params passed in the XHR request.

modules/direct_s3_upload/public/views/pages/record_create.liquid

--- slug: direct-s3-upload/images/record_create method: post --- {% if context.params.user_id %} {% graphql record, direct_url: context.params.direct_url, user_id: context.params.user_id %} mutation record_create($direct_url: String!, $user_id: ID!) { record_create(record: { table: "modules/direct_s3_upload/uppy" user_id: $user_id properties: [{ name: "my_upload" value: $direct_url }] }) { id } } {% endgraphql %} {{ record | fetch: "record_create" }} {% endif %} 

Step 5: Show the results

To check the results of image processing, get some of them from the database and display them.

First you need a query to fetch records.

modules/direct_s3_upload/public/graph_queries/get_records.graphql

query get_records($per_page: Int = 5, $user_id: ID!) { records( per_page: $per_page sort: { created_at: { order: DESC } } filter: { table: { value: "modules/direct_s3_upload/uppy" }, user_id: { value: $user_id } } ) { total_entries results { id my_upload: property_upload(name: "my_upload") { url versions } } } } 

Then you can use it to display images.

modules/direct_s3_upload/public/views/pages/images.liquid

<!-- Get recently uploaded images for this user, in all versions --> {% graphql records = "modules/direct_s3_upload/get_records", user_id: context.current_user.id | dig: "records" %} {% if records.total_entries > 0 %} <hr> <div class="my-2 row"> <div class="col-9"> <h3>Last 5 processed images:</h3> </div> </div> <!-- Show all versions next to each other --> {% for record in records.results %} {% assign versions = record | dig: "my_upload", "versions" %} <div class="card my-4"> <div class="card-body"> <div class="row"> <div class="col-sm-2"> <h4>Original</h4> <a href="{{ record.my_upload.url }}"><img src="{{ record.my_upload.url }}" alt="Original" width="150"></a> </div> {% for version in versions %} <div class="col-sm-2"> <h4>{{ version[0] }}</h4> <a href="{{ version[1] }}"><img src="{{ version[1] }}" alt="{{ version[0] }}" width="150"></a> </div> {% endfor %} </div> </div> </div> {% endfor %} {% endif %} 

Important notes / troubleshooting

General

• Instead of sending an image to our server you will send a url (returned by S3 response) to the image inside the text field for given image. Image URL should look similar to https://near-me-oregon-staging.s3-us-west-2.amazonaws.com/uploads%2Fbc50d6da-c17e-477d-b143-c65422c221fd%2Ftest.3+%282%29.png
• After form submission give our server couple of seconds to pickup your image, generate size versions, process them and upload to the new S3 location so the CDN could pick it up for you and datatabase to be updated with CDN links

If you have problems that you cannot figure out, go to live example page, inspect whats being sent to S3 and our server respectively and find differences between your requests (params, headers, method, form encoding) and those that work - the difference usually is whats broken.

AWS

• Make sure your form is using POST method
• Make sure your form is sending data as enctype="multipart/form-data"
• Do not include any fields that are not listed in the upload_url_payload key, otherwise AWS will return error 403 with message Invalid according to Policy: Extra input fields: xxx
• Make sure file input is sent last in the form
• Remember to set your file input name to file

Read more about aws requirements.

Uppy

If you use Uppy and its S3 plugin as we did in this tutorial, you dont have to worry about most AWS notes.

Live example and source code

To play with a live example, create a developer account at https://examples.platform-os.com and go to the Direct S3 upload images page.

Source code can be found on GitHub.

Additional resources

• Example: Browser-Based Upload using HTTP POST (Using AWS Signature Version 4)
• Creating an HTML Form (Using AWS Signature Version 4)
• Upload a file with $.ajax to AWS S3 with a pre-signed url