Skip to content

Using filepath for creating S3 keys #447

@BigMakak

Description

@BigMakak

Description

S3 keys being generated on the download function on s3sync.go have a platform specific prefix that breaks the sync/download process

Steps to Reproduce

  1. Run Sync function to sync a folder on S3 and download files to local disk
  2. the bucket prefix value would be for example arn:aws:ec2:eu-central-1:00000:instance/i-instanceid/eu-central-123/BooksMarks/
  3. Observe S3 keys constructed with ./ prefix like this: ./arn:aws:ec2:eu-central-1:0000:instance/i-instnaceid/eu-central-123/BooksMarks/bookmark.txt
  4. Download fails with 404 error due to the final S3 key having a ./ in the beginning

Findings when testing this problem

Seems that the filepath uses OS specific values when manipulating strings, adding things like \ and .\ to names and paths that will then be used as s3 keys. Using the path package instead of the filepath, would create valid URL strings to then be used as S3 keys and does not add an OS specific prefixes. However, I don't know if this may applicable on all cases

Possible Fix

I did a fork of the project and created a pull request, with a possible fix that is now formatting these strings using the path package: #448

Your Environment

OS: Windows Server
Go Version: 1.25.4
aws sdk go v2: 1.40.1
s3sync Version: 2.0.0-rc.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions