Skip to content

codeit-ninja/simple-s3-uploader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

S3 Upload Test Server

A simple Express.js server with S3-compatible file upload functionality. Supports AWS S3 and S3-compatible storage services like MinIO, DigitalOcean Spaces, Cloudflare R2, and others.

Features

  • ✅ Express.js server with file upload endpoint
  • ✅ Web-based upload interface with drag & drop
  • ✅ S3-compatible storage integration
  • ✅ File validation and size limits
  • ✅ Upload progress tracking
  • ✅ Unique filename generation
  • ✅ CORS support
  • ✅ Environment-based configuration
  • ✅ Health check endpoint
  • ✅ Error handling

Prerequisites

  • Node.js (v14 or higher)
  • npm or yarn
  • S3-compatible storage account and credentials

Quick Start

  1. Clone and setup

    cd s3-upload-test npm install
  2. Configure environment

    cp .env.example .env # Edit .env with your credentials
  3. Start the server

    npm start # or for development npm run dev
  4. Test the upload

    # Open browser and go to http://localhost:3000 # Or test via curl curl -X POST -F "file=@your-file.jpg" http://localhost:3000/upload

Environment Configuration

Copy .env.example to .env and configure your credentials:

AWS S3 Configuration

AWS_ACCESS_KEY_ID=your_access_key AWS_SECRET_ACCESS_KEY=your_secret_key AWS_REGION=us-east-1 S3_BUCKET=your-bucket-name # Leave S3_ENDPOINT empty for AWS S3 S3_ENDPOINT= S3_FORCE_PATH_STYLE=false

MinIO Configuration

AWS_ACCESS_KEY_ID=minioadmin AWS_SECRET_ACCESS_KEY=minioadmin AWS_REGION=us-east-1 S3_BUCKET=test-bucket S3_ENDPOINT=http://localhost:9000 S3_FORCE_PATH_STYLE=true

DigitalOcean Spaces Configuration

AWS_ACCESS_KEY_ID=your_spaces_key AWS_SECRET_ACCESS_KEY=your_spaces_secret AWS_REGION=nyc3 S3_BUCKET=your-space-name S3_ENDPOINT=https://nyc3.digitaloceanspaces.com S3_FORCE_PATH_STYLE=false

Cloudflare R2 Configuration

AWS_ACCESS_KEY_ID=your_r2_access_key AWS_SECRET_ACCESS_KEY=your_r2_secret_key AWS_REGION=auto S3_BUCKET=your-r2-bucket S3_ENDPOINT=https://your-account-id.r2.cloudflarestorage.com S3_FORCE_PATH_STYLE=false

API Endpoints

GET /

Serves the web upload interface when accessed from a browser, or returns server information as JSON when accessed programmatically.

Browser Response: Returns the HTML upload interface with drag & drop functionality.

API Response:

{ "message": "S3 File Upload Server", "endpoints": { "upload": "POST /upload - Upload a file to S3", "health": "GET /health - Health check", "ui": "GET / - Upload interface" } }

GET /health

Health check endpoint.

Response:

{ "status": "OK", "timestamp": "2023-10-06T12:00:00.000Z" }

POST /upload

Upload a file to S3-compatible storage.

Request:

  • Method: POST
  • Content-Type: multipart/form-data
  • Body: Form data with file field

Example with curl:

curl -X POST -F "file=@example.jpg" http://localhost:3000/upload

Example with JavaScript:

const formData = new FormData(); formData.append('file', fileInput.files[0]); fetch('http://localhost:3000/upload', { method: 'POST', body: formData }) .then(response => response.json()) .then(data => console.log(data));

Success Response:

{ "message": "File uploaded successfully", "file": { "originalName": "example.jpg", "size": 123456, "mimeType": "image/jpeg", "s3Key": "1696608000000_a1b2c3d4_example.jpg", "location": "https://your-bucket.s3.amazonaws.com/1696608000000_a1b2c3d4_example.jpg" } }

Error Response:

{ "error": "No file provided" }

File Handling

  • Size Limit: 1GB (configurable in server.js)
  • Storage: Files are stored in memory temporarily and then uploaded to S3
  • Filename Generation: Automatic unique filename with timestamp and hash
  • Metadata: Original filename, upload date, and file size are stored as S3 metadata

Testing with Different Tools

Using the Web Interface

  1. Start the server: npm start
  2. Open your browser and go to http://localhost:3000
  3. Use the drag & drop interface or click to select a file
  4. Click "Upload File" and watch the progress
  5. View the upload results and file details

Using curl

# Upload a file curl -X POST -F "file=@test.jpg" http://localhost:3000/upload # Health check curl http://localhost:3000/health

Using Postman

  1. Create a new POST request to http://localhost:3000/upload
  2. Go to Body → form-data
  3. Add key file with type File
  4. Select your file and send

Using HTML Form

<!DOCTYPE html> <html> <head> <title>File Upload Test</title> </head> <body> <form action="http://localhost:3000/upload" method="post" enctype="multipart/form-data"> <input type="file" name="file" required> <button type="submit">Upload</button> </form> </body> </html>

Development

Scripts

  • npm start - Start the production server
  • npm run dev - Start with nodemon for development
  • npm test - Run tests (not implemented yet)

Project Structure

s3-upload-test/ ├── public/ │ └── index.html # Web upload interface ├── server.js # Main Express server ├── s3-client.js # S3 client configuration and utilities ├── package.json # Dependencies and scripts ├── .env.example # Environment variables template ├── .gitignore # Git ignore rules └── README.md # This file 

Error Handling

The server handles various error scenarios:

  • No file provided (400)
  • File too large (400)
  • S3 upload failures (500)
  • Invalid endpoints (404)
  • Server errors (500)

Security Considerations

  • Files are validated for size limits
  • CORS is enabled (configure as needed)
  • Environment variables protect sensitive credentials
  • Consider adding file type validation for production use
  • Implement authentication/authorization as needed

Troubleshooting

Common Issues

  1. "Access Denied" errors

    • Check your AWS credentials and permissions
    • Ensure the bucket exists and is accessible
    • Verify the bucket policy allows uploads
  2. Connection timeouts

    • Check your S3_ENDPOINT configuration
    • Verify network connectivity to the S3 service
    • For local MinIO, ensure it's running
  3. "Bucket not found" errors

    • Verify the bucket name in your .env file
    • Ensure the bucket exists in the specified region
  4. CORS errors in browser

    • The server includes CORS headers
    • For production, configure CORS origins appropriately

Debug Mode

Add console logging to troubleshoot issues:

// Add to server.js for debugging app.use((req, res, next) => { console.log(`${req.method} ${req.path}`, req.body); next(); });

Contributing

Feel free to submit issues and enhancement requests!

License

MIT License

About

Simple s3 uploader generated by Claude to test S3 uploading

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published