- Notifications
You must be signed in to change notification settings - Fork 249
Storage backend examples
S3Proxy supports IAM roles for service accounts. By setting the environment variables AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE, S3Proxy will automatically use these to assume the role with web identity as the credential provider.
s3proxy.endpoint=http://0.0.0.0:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=aws-s3 jclouds.endpoint=https://s3.xxx.amazonaws.com jclouds.identity= jclouds.credential= This backend uses jclouds and only supports static key authorization.
s3proxy.endpoint=http://0.0.0.0:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.azureblob.auth=azureKey jclouds.provider=azureblob jclouds.identity=xxx jclouds.credential=yyyyyyyyyyyyy jclouds.endpoint=https://xxx.blob.core.windows.net This backend provides similar functionality as the azureblob backend but is implemented using the Microsoft SDK. It supports multiple authentication options, including managed identities.
If both the jclouds.identity and jclouds.credential options are empty strings, the DefaultAzureCredential is used for authentication. Setting these options will disable the attempt to authenticate with managed identities and will authenticate using a static key instead.
s3proxy.endpoint=http://0.0.0.0:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=azureblob-sdk jclouds.endpoint=https://xxx.blob.core.windows.net jclouds.identity= jclouds.credential= s3proxy.endpoint=http://127.0.0.1:8081 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=b2 jclouds.identity=xxxxxxxxxxxx jclouds.credential=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=google-cloud-storage jclouds.identity=999999999999-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@developer.gserviceaccount.com jclouds.credential=/path/to/credentials.key Where /path/to/credentials.key contains:
-----BEGIN RSA PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx -----END RSA PRIVATE KEY----- s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=google-cloud-storage jclouds.identity=999999999999-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@developer.gserviceaccount.com jclouds.credential=jclouds.credential=-----BEGIN RSA PRIVATE KEY-----\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\ -----END RSA PRIVATE KEY----- An example using a user-specified region:
s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=rackspace-cloudfiles-us jclouds.identity=xxxxxxxxxxxxx jclouds.credential=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx jclouds.region=HKG s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=s3 jclouds.identity=xxxxxxxxxxxxxxxxxxxx jclouds.credential=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx jclouds.endpoint=https://objects.dreamhost.com When pointing jclouds to a generic S3 API (jclouds.provider=s3), jclouds uses AWSSIGV2 authentication by default. AWSSIGV2 is deprecated and may not be supported by S3 providers like Cloudflare R2 (e.g. https://github.com/gaul/s3proxy/issues/783).
You may override the the jclouds default to make it use AWSSIGV4 by setting jclouds.s3.signer-version=4 (https://issues.apache.org/jira/browse/JCLOUDS-1594). However, this may not work for PUT requests for certain S3 providers (e.g. https://github.com/gaul/s3proxy/issues/783).
outdated?!
s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=swift jclouds.endpoint=http://IP-OF-SWIFT:PORT/auth/v1.0 jclouds.api=swift jclouds.identity=test:tester jclouds.credential=testing All $OS_ variables can be retrieved from your tenant specific openrc.sh file.
s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=openstack-swift jclouds.endpoint=$OS_AUTH_URL (e.g. https://keystone.example.com/v2.0/) jclouds.region=$OS_REGION_NAME (e.g. paris1) jclouds.identity=$OS_TENANT_NAME:$OS_USERNAME (e.g. production:user1) jclouds.credential=$OS_PASSWORD (e.g. password) All $OS_ variables can be retrieved from your project specific openrc.sh file.
s3proxy.endpoint=http://127.0.0.1:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=openstack-swift jclouds.endpoint=$OS_AUTH_URL (e.g. https://keystone.example.com/v3/) jclouds.regions=$OS_REGION_NAME (e.g. paris1) jclouds.region=$OS_REGION_NAME (e.g. paris1) jclouds.identity=$OS_USER_DOMAIN_NAME:$OS_USERNAME (e.g. domain1:user1) jclouds.keystone.version=3 jclouds.keystone.scope=project:$OS_PROJECT_NAME (e.g. project:project1) jclouds.keystone.project_domain_name=$OS_PROJECT_DOMAIN_NAME (e.g. domain2 - only needed if user and project domains differ) jclouds.credential=$OS_PASSWORD (e.g. password) An example using SoftLayer in the Dallas geo which uses Keystone.
s3proxy.endpoint=http://localhost:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential jclouds.provider=openstack-swift jclouds.endpoint=https://dal05.objectstorage.softlayer.net/auth/v1.0 jclouds.identity=test:tester jclouds.credential=testing jclouds.keystone.credential-type=tempAuthCredentials