@Singleton public class ProfileService { private static final Logger LOG = LoggerFactory.getLogger(ProfileService.class); private final ObjectStorageOperations<?, ?, ?> objectStorage; public ProfileService(ObjectStorageOperations<?, ?, ?> objectStorage) { this.objectStorage = objectStorage; } }
Micronaut Object Storage
Micronaut Object Storage provides a uniform API to create, read and delete objects in the major cloud providers
Version: 2.9.0
1 Introduction
Micronaut Object Storage provides a uniform API to create, read and delete objects in the major cloud providers:
There is also a local storage implementation for testing purposes.
Using this API enables the creation of truly multi-cloud, portable applications.
2 Release History
For this project, you can find a list of releases (with release notes) here:
3 Quick Start
To get started, you need to declare a dependency for the actual cloud provider you are using. See the actual cloud provider documentation for more details:
Then, you can inject in your controllers/services/etc. a bean of type ObjectStorageOperations, the parent interface that allows you to use the API in a generic way for all cloud providers:
@Singleton class ProfileService { final ObjectStorageOperations<?, ?, ?> objectStorage ProfileService(ObjectStorageOperations<?, ?, ?> objectStorage) { this.objectStorage = objectStorage } }@Singleton open class ProfileService(private val objectStorage: ObjectStorageOperations<*, *, *>) { }If your application is not multi-cloud, and/or you need cloud-specific details, you can use a concrete implementation. For example, for AWS S3:
@Controller public class UploadController { private final AwsS3Operations objectStorage; public UploadController(AwsS3Operations objectStorage) { this.objectStorage = objectStorage; } }@Controller class UploadController { final AwsS3Operations objectStorage UploadController(AwsS3Operations objectStorage) { this.objectStorage = objectStorage } }@Controller open class UploadController(private val objectStorage: AwsS3Operations) { }If you have multiple object storages configured, it is possible to select which one to work with via bean qualifiers.
For example, given the following configuration:
src/main/resources/application-ec2.ymlmicronaut.object-storage.aws.pictures.bucket=pictures-bucket micronaut.object-storage.aws.logos.bucket=logos-bucketmicronaut: object-storage: aws: pictures: bucket: pictures-bucket logos: bucket: logos-bucket[micronaut] [micronaut.object-storage] [micronaut.object-storage.aws] [micronaut.object-storage.aws.pictures] bucket="pictures-bucket" [micronaut.object-storage.aws.logos] bucket="logos-bucket"micronaut { objectStorage { aws { pictures { bucket = "pictures-bucket" } logos { bucket = "logos-bucket" } } } }{ micronaut { object-storage { aws { pictures { bucket = "pictures-bucket" } logos { bucket = "logos-bucket" } } } } }{ "micronaut": { "object-storage": { "aws": { "pictures": { "bucket": "pictures-bucket" }, "logos": { "bucket": "logos-bucket" } } } } }You then need to use @Named("pictures") or @Named("logos") to specify which of the object storages you want to use.
Uploading files
public String saveProfilePicture(String userId, Path path) { UploadRequest request = UploadRequest.fromPath(path, userId); // (1) UploadResponse<?> response = objectStorage.upload(request); // (2) return response.getKey(); // (3) }String saveProfilePicture(String userId, Path path) { UploadRequest request = UploadRequest.fromPath(path, userId) // (1) UploadResponse response = objectStorage.upload(request) // (2) response.key // (3) }open fun saveProfilePicture(userId: String, path: Path): String? { val request = UploadRequest.fromPath(path, userId) // (1) val response = objectStorage.upload(request) // (2) return response.key // (3) }| 1 | You can use any of the UploadRequest static methods to build an upload request. |
| 2 | The upload operation returns an UploadResponse, which wraps the cloud-specific SDK response object. |
| 3 | The response object contains some common properties for all cloud vendor, and a getNativeResponse() method that can be used for accessing the vendor-specific response object. |
In case you want to have better control of the upload options used, you can use the method upload(UploadRequest, Consumer) of ObjectStorageOperations, which will give you access to the cloud vendor-specific request class or builder.
For example, for AWS S3:
UploadResponse<PutObjectResponse> response = objectStorage.upload(objectStorageUpload, builder -> { builder.acl(ObjectCannedACL.PUBLIC_READ); });UploadResponse<PutObjectResponse> response = objectStorage.upload(objectStorageUpload, { builder -> builder.acl(ObjectCannedACL.PUBLIC_READ) })val response = objectStorage.upload(objectStorageUpload) { builder: PutObjectRequest.Builder -> builder.acl(ObjectCannedACL.PUBLIC_READ) }Retrieving files
public Optional<Path> retrieveProfilePicture(String userId, String fileName) { Path destination = null; try { String key = userId + "/" + fileName; Optional<InputStream> stream = objectStorage.retrieve(key) // (1) .map(ObjectStorageEntry::getInputStream); if (stream.isPresent()) { destination = File.createTempFile(userId, "temp").toPath(); Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING); return Optional.of(destination); } else { return Optional.empty(); } } catch (IOException e) { LOG.error("Error while trying to save profile picture to the local file [{}]: {}", destination, e.getMessage()); return Optional.empty(); } }Optional<Path> retrieveProfilePicture(String userId, String fileName) { String key = "${userId}/${fileName}" Optional<InputStream> stream = objectStorage.retrieve(key) // (1) .map(ObjectStorageEntry::getInputStream) if (stream.isPresent()) { Path destination = File.createTempFile(userId, "temp").toPath() Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING) return Optional.of(destination) } else { return Optional.empty() } }open fun retrieveProfilePicture(userId: String, fileName: String): Path? { val key = "$userId/$fileName" val stream = objectStorage.retrieve<ObjectStorageEntry<*>>(key) // (1) .map { obj: ObjectStorageEntry<*> -> obj.inputStream } return if (stream.isPresent) { val destination = File.createTempFile(userId, "temp").toPath() Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING) destination } else { null } }| 1 | The retrieve operation returns an ObjectStorageEntry, from which you can get an InputStream. There is also a getNativeEntry() method that gives you access to the cloud vendor-specific response object. |
Deleting files
public void deleteProfilePicture(String userId, String fileName) { String key = userId + "/" + fileName; objectStorage.delete(key); // (1) }void deleteProfilePicture(String userId, String fileName) { String key = "${userId}/${fileName}" objectStorage.delete(key) // (1) }open fun deleteProfilePicture(userId: String, fileName: String) { val key = "$userId/$fileName" objectStorage.delete(key) // (1) }| 1 | The delete operation returns the cloud vendor-specific delete response object in case you need it. |
4 Amazon S3
To use Amazon S3, you need the following dependency:
implementation("io.micronaut.objectstorage:micronaut-object-storage-aws")<dependency> <groupId>io.micronaut.objectstorage</groupId> <artifactId>micronaut-object-storage-aws</artifactId> </dependency>Refer to the Micronaut AWS documentation for more information about credentials and region configuration.
The object storage specific configuration options available are:
| Property | Type | Description |
|---|---|---|
| boolean | Whether to enable or disable this object storage. |
| java.lang.String | The name of the AWS S3 bucket. |
For example:
src/main/resources/application-ec2.ymlmicronaut.object-storage.aws.default.bucket=profile-pictures-bucketmicronaut: object-storage: aws: default: bucket: profile-pictures-bucket[micronaut] [micronaut.object-storage] [micronaut.object-storage.aws] [micronaut.object-storage.aws.default] bucket="profile-pictures-bucket"micronaut { objectStorage { aws { 'default' { bucket = "profile-pictures-bucket" } } } }{ micronaut { object-storage { aws { default { bucket = "profile-pictures-bucket" } } } } }{ "micronaut": { "object-storage": { "aws": { "default": { "bucket": "profile-pictures-bucket" } } } } }The concrete implementation of ObjectStorageOperations is AwsS3Operations.
Advanced configuration
For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:
@Singleton public class S3ClientBuilderCustomizer implements BeanCreatedEventListener<S3ClientBuilder> { @Override public S3ClientBuilder onCreated(@NonNull BeanCreatedEvent<S3ClientBuilder> event) { return event.getBean() .overrideConfiguration(c -> c.apiCallTimeout(Duration.of(60, ChronoUnit.SECONDS))); } }| See the guide for Use the Micronaut Object Storage API to Store Files in Amazon S3 to learn more. |
5 Azure Blob Storage
To use Azure Blob Storage, you need the following dependency:
implementation("io.micronaut.objectstorage:micronaut-object-storage-azure")<dependency> <groupId>io.micronaut.objectstorage</groupId> <artifactId>micronaut-object-storage-azure</artifactId> </dependency>Refer to the Micronaut Azure documentation for more information about authentication options.
The object storage specific configuration options available are:
| Property | Type | Description |
|---|---|---|
| boolean | Whether to enable or disable this object storage. |
| java.lang.String | The blob container name. |
| java.lang.String | The blob service endpoint, in the format of https://{accountName}.blob.core.windows.net. |
For example:
src/main/resources/application-azure.ymlazure.credential.client-secret.client-id=<client-id> azure.credential.client-secret.tenant-id=<tenant-id> azure.credential.client-secret.secret=<secret> micronaut.object-storage.azure.default.container=profile-pictures-container micronaut.object-storage.azure.default.endpoint=https://my-account.blob.core.windows.netazure: credential: client-secret: client-id: <client-id> tenant-id: <tenant-id> secret: <secret> micronaut: object-storage: azure: default: container: profile-pictures-container endpoint: https://my-account.blob.core.windows.net[azure] [azure.credential] [azure.credential.client-secret] client-id="<client-id>" tenant-id="<tenant-id>" secret="<secret>" [micronaut] [micronaut.object-storage] [micronaut.object-storage.azure] [micronaut.object-storage.azure.default] container="profile-pictures-container" endpoint="https://my-account.blob.core.windows.net"azure { credential { clientSecret { clientId = "<client-id>" tenantId = "<tenant-id>" secret = "<secret>" } } } micronaut { objectStorage { azure { 'default' { container = "profile-pictures-container" endpoint = "https://my-account.blob.core.windows.net" } } } }{ azure { credential { client-secret { client-id = "<client-id>" tenant-id = "<tenant-id>" secret = "<secret>" } } } micronaut { object-storage { azure { default { container = "profile-pictures-container" endpoint = "https://my-account.blob.core.windows.net" } } } } }{ "azure": { "credential": { "client-secret": { "client-id": "<client-id>", "tenant-id": "<tenant-id>", "secret": "<secret>" } } }, "micronaut": { "object-storage": { "azure": { "default": { "container": "profile-pictures-container", "endpoint": "https://my-account.blob.core.windows.net" } } } } }The concrete implementation of ObjectStorageOperations is AzureBlobStorageOperations.
Advanced configuration
For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:
@Singleton public class BlobServiceClientBuilderCustomizer implements BeanCreatedEventListener<BlobServiceClientBuilder> { @Override public BlobServiceClientBuilder onCreated(@NonNull BeanCreatedEvent<BlobServiceClientBuilder> event) { HttpPipelinePolicy noOp = (context, next) -> next.process(); return event.getBean().addPolicy(noOp); } }6 Google Cloud Storage
To use Google Cloud Storage, you need the following dependency:
implementation("io.micronaut.objectstorage:micronaut-object-storage-gcp")<dependency> <groupId>io.micronaut.objectstorage</groupId> <artifactId>micronaut-object-storage-gcp</artifactId> </dependency>Refer to the Micronaut GCP documentation for more information about configuring your GCP project.
The object storage specific configuration options available are:
| Property | Type | Description |
|---|---|---|
| boolean | Whether to enable or disable this object storage. |
| java.lang.String | The name of the Google Cloud Storage bucket. |
For example:
src/main/resources/application-gcp.ymlgcp.project-id=my-gcp-project micronaut.object-storage.gcp.default.bucket=profile-pictures-bucketgcp: project-id: my-gcp-project micronaut: object-storage: gcp: default: bucket: profile-pictures-bucket[gcp] project-id="my-gcp-project" [micronaut] [micronaut.object-storage] [micronaut.object-storage.gcp] [micronaut.object-storage.gcp.default] bucket="profile-pictures-bucket"gcp { projectId = "my-gcp-project" } micronaut { objectStorage { gcp { 'default' { bucket = "profile-pictures-bucket" } } } }{ gcp { project-id = "my-gcp-project" } micronaut { object-storage { gcp { default { bucket = "profile-pictures-bucket" } } } } }{ "gcp": { "project-id": "my-gcp-project" }, "micronaut": { "object-storage": { "gcp": { "default": { "bucket": "profile-pictures-bucket" } } } } }The concrete implementation of ObjectStorageOperations is GoogleCloudStorageOperations.
Advanced configuration
For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:
@Singleton public class StorageOptionsBuilderCustomizer implements BeanCreatedEventListener<StorageOptions.Builder> { @Override public StorageOptions.Builder onCreated(@NonNull BeanCreatedEvent<StorageOptions.Builder> event) { return event.getBean() .setTransportOptions(HttpTransportOptions.newBuilder().setConnectTimeout(60_000).build()); } }| See the guide for Use the Micronaut Object Storage API to Store Files in Google Cloud Storage to learn more. |
7 Oracle Cloud Infrastructure (OCI) Object Storage
To use Oracle Cloud Infrastructure (OCI) Object Storage, you need the following dependency:
implementation("io.micronaut.objectstorage:micronaut-object-storage-oracle-cloud")<dependency> <groupId>io.micronaut.objectstorage</groupId> <artifactId>micronaut-object-storage-oracle-cloud</artifactId> </dependency>Refer to the Micronaut Oracle Cloud documentation for more information about authentication options.
The object storage specific configuration options available are:
| Property | Type | Description |
|---|---|---|
| boolean | Whether to enable or disable this object storage. |
| java.lang.String | The name of the OCI Object Storage bucket. |
| java.lang.String | The OCI Object Storage namespace used. |
For example:
src/main/resources/application-oraclecloud.ymloci.config.profile=DEFAULT micronaut.object-storage.oracle-cloud.default.bucket=profile-pictures-bucket micronaut.object-storage.oracle-cloud.default.namespace=MyNamespaceoci: config: profile: DEFAULT micronaut: object-storage: oracle-cloud: default: bucket: profile-pictures-bucket namespace: MyNamespace[oci] [oci.config] profile="DEFAULT" [micronaut] [micronaut.object-storage] [micronaut.object-storage.oracle-cloud] [micronaut.object-storage.oracle-cloud.default] bucket="profile-pictures-bucket" namespace="MyNamespace"oci { config { profile = "DEFAULT" } } micronaut { objectStorage { oracleCloud { 'default' { bucket = "profile-pictures-bucket" namespace = "MyNamespace" } } } }{ oci { config { profile = "DEFAULT" } } micronaut { object-storage { oracle-cloud { default { bucket = "profile-pictures-bucket" namespace = "MyNamespace" } } } } }{ "oci": { "config": { "profile": "DEFAULT" } }, "micronaut": { "object-storage": { "oracle-cloud": { "default": { "bucket": "profile-pictures-bucket", "namespace": "MyNamespace" } } } } }The concrete implementation of ObjectStorageOperations is OracleCloudStorageOperations
Advanced configuration
For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:
//See https://github.com/oracle/oci-java-sdk/blob/master/bmc-examples/src/main/java/ClientConfigurationTimeoutExample.java @Singleton public class ObjectStorageClientBuilderCustomizer implements BeanCreatedEventListener<ObjectStorageClient.Builder> { public static final int CONNECTION_TIMEOUT_IN_MILLISECONDS = 25000; public static final int READ_TIMEOUT_IN_MILLISECONDS = 35000; @Override public ObjectStorageClient.Builder onCreated(@NonNull BeanCreatedEvent<ObjectStorageClient.Builder> event) { ClientConfiguration clientConfiguration = ClientConfiguration.builder() .connectionTimeoutMillis(CONNECTION_TIMEOUT_IN_MILLISECONDS) .readTimeoutMillis(READ_TIMEOUT_IN_MILLISECONDS) .build(); return event.getBean() .configuration(clientConfiguration); } }| See the guide for Use the Micronaut Object Storage API to Store Files in Oracle Cloud Infrastructure (OCI) Object Storage to learn more. |
8 Local Storage
To use the local storage implementation (useful for tests), you need the following dependency:
testImplementation("io.micronaut.objectstorage:micronaut-object-storage-local")<dependency> <groupId>io.micronaut.objectstorage</groupId> <artifactId>micronaut-object-storage-local</artifactId> <scope>test</scope> </dependency>Then, simply define a local storage:
micronaut.object-storage.local.default.enabled=truemicronaut: object-storage: local: default: enabled: true[micronaut] [micronaut.object-storage] [micronaut.object-storage.local] [micronaut.object-storage.local.default] enabled=truemicronaut { objectStorage { local { 'default' { enabled = true } } } }{ micronaut { object-storage { local { default { enabled = true } } } } }{ "micronaut": { "object-storage": { "local": { "default": { "enabled": true } } } } }| When added to the classpath, LocalStorageOperations becomes the primary implementation of ObjectStorageOperations. |
By default, it will create a temporary folder to store the files, but you can configure it to use a specific folder:
| Property | Type | Description |
|---|---|---|
| boolean | Whether to enable or disable this object storage. |
| java.nio.file.Path | The path of the local storage. |
For example:
src/main/resources/application-test.ymlmicronaut.object-storage.local.default.path=/tmp/my-object-storagemicronaut: object-storage: local: default: path: /tmp/my-object-storage[micronaut] [micronaut.object-storage] [micronaut.object-storage.local] [micronaut.object-storage.local.default] path="/tmp/my-object-storage"micronaut { objectStorage { local { 'default' { path = "/tmp/my-object-storage" } } } }{ micronaut { object-storage { local { default { path = "/tmp/my-object-storage" } } } } }{ "micronaut": { "object-storage": { "local": { "default": { "path": "/tmp/my-object-storage" } } } } }The concrete implementation of ObjectStorageOperations is LocalStorageOperations.
9 Guides
See the following list of guides to learn more about working with Object Storage in the Micronaut Framework:
10 Repository
You can find the source code of this project in this repository: