Adding s3 bucket as docker volume

docker s3-volume driver
access s3 from docker container
docker volume driver
docker s3 client
docker volume snapshot
docker shared volume
s3fs docker
docker volume driver minio

I have one spring boot application which is in our internal data center, which process files from a specific folder on the host.

we wanted to deploy this to aws and wanted to use s3 bucket to upload files for processing.

is there any way we can add s3 bucket space as docker volume?


S3 is an object store, not a file system. You should have S3 trigger a message to SQS when new objects are added to the bucket. Then you can code your application running in the Docker container to poll SQS for new messages, and us the S3 location in the message to copy the object from S3 to local storage (using the appropriate AWS SDK) for processing.

Docker volumes are an abstraction provided as part of the engine, providing a In the AWS Console add a new user and make sure to generate a set of Access to see the plugin, as well as your S3 buckets, represented as Docker volumes. With an installed, running service, an S3 bucket, and a correctly configured s3fs, I could create a named volume on my Docker host: docker volume create -d s3-volume --name jason --opt bucket=plugin-experiment And then, use a second container to write data to it: docker run -it -v jason:/s3 busybox sh / # echo "Hello" >> /s3/greetings.txt


No docker volume is for mounting drives on the machine (https://docs.docker.com/storage/volumes/)

You can use the S3 api to manage your bucket from the docker container (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)

An experiment in writing a volume plugin for Docker. My use case boiled down to “How can I expose the contents of an S3 bucket as a volume?”. or a missing option being added, but it did lead to some head scratching. Mount a Bucket S3 as an partition in EC2 of an ECS cluster in order to be used as a volume for persistence data of a container. Tools : Bucket S3; ECS -> EC2; Docker; Plugin rexray/s3fs; Steps


Other answers have correctly pointed out that : AWS S3 is an object store and you can not mount it as volume to docker.

That being said, using S3 with spring application is super easy and there is framework developed called spring-cloud. spring-cloud works excellent with AWS.

Here is sample code :

public void uploadFiles(File file, String s3Url) throws IOException {
   WritableResource resource = (WritableResource) resourceLoader.getResource(s3Url);

   try (OutputStream outputStream = resource.getOutputStream()) {
       Files.copy(file.toPath(), outputStream);
   }
}

You can find detailed blog over here.

No you can't. S3 is an object storage, accessed over HTTP or REST for example. Just as you can't mount an HTTP address as a directory you  Yes, you can mount an S3 bucket as filesystem on AWS ECS container by using plugins such as REX-Ray or Portworx. Install your preferred Docker volume plugin (if needed) and simply specify the volume name, the volume driver, and the parameters when setting up a task definition via the AWS management console, CLI or SDK.


Now, with the support for Docker volumes, you can deploy stateful and storage-​intensive applications on Amazon ECS. You have the flexibility  So I just want to add my s3 bucket from amazon to my docker swarm. I've saw many "possible" solutions on the internet but I can't connect them to add the content of my bucket as volume. So the last


docker-s3-volume. Creates a Docker container that is restored and backed up to a directory on s3. You could use this to run short lived processes that work with  I thought that's exactly what Docker Volume is. A mounted virtual place where files can be shared. At least that is my shallow and simplistic understanding after reading about Docker Volumes. So I do have some questions. Is my assumptions that AWS s3 bucket and Docker volumes provide similar functionality like comparing apples to oranges? If my


Suggestion: Add S3 Bucket as Volume Storage #87. Closed So in this case you could do (1.9) docker volume create --driver=rexray  bucket: The name of your S3 bucket where you wish to store objects. The bucket must exist prior to the driver initialization. The bucket must exist prior to the driver initialization. encrypt : (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified).