Amazon Simple Storage Service (Amazon S3) 是一种对象存储服务,提供行业领先的可扩展性、数据可用性、安全性和性能。Amazon S3具有无限制的存储容量和进出网络带宽,廉价的数据存储费用,同时具有能达到 99.999999999%(11 个 9)的持久性,是对象存储最佳解决方案。
S3FS
S3fs是基于FUSE的文件系统,允许Linux和Mac Os X 挂载S3的存储桶在本地文件系统,S3fs能够保持对象原来的格式。
利用S3fs可以把共享的Amazon S3存储桶直接挂载在用户服务器本地目录下,应用不需要做修改就可以直接使用Amazon S3存储服务,这种方式可以作为临时解决方案将传统应用快速迁移到AWS平台。详细参见:https://github.com/s3fs-fuse/s3fs-fuse
利用S3fs挂载S3存储桶
1.安装s3fs
a.安装必要的软件包
sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel
b.下载,编译并安装s3fs
git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure
make
sudo make install
c.验证安装结果
2.创建IAM用户访问密钥文件
IAM用户访问密钥内容可以写入当前用户默认密钥文件比如"/home/ec2-user/.passwd-s3fs"或者用户自己创建的文件。
命令格式:echo [IAM用户访问密钥ID]:[ IAM用户访问密钥] >[密钥文件名]
命令举例:下面的例子将在当前用户默认路径创建密钥文件
3.手动挂载S3存储桶
命令格式:s3fs [S3存储桶名] [本地目录名] -o passwd_file=[密钥文件名] -o endpoint=[区域名]
如:
s3fs ultrapower-test /home/ec2-user/s3mnt -o passwd_file=/home/ec2-user/.passwd-s3fs -o endpoint=ap-northeast-1
检查挂载结果
挂载操作执行结束后,可以使用Linux "df"命令查看挂载是否成功。出现类似下面256T的s3fs文件系统即表示挂载成功。用户就可以进入本地挂载目录去访问存储在S3存储桶中的对象。
卸载挂载的S3存储桶
如果不再需要通过挂载方式访问S3存储桶,可以使用Linux "umount"命令卸载。
sudo umount /home/ec2-user/s3mnt
设置开机自动挂载S3存储桶
修改/etc/fstab文件,添加后面的自动挂载命令
vi /etc/fstab
总结:
使用S3fs可以方便的把S3存储桶挂载在EC2的本地操作系统中,由于S3fs实际上是使用Amazon S3服务的访问接口,所以不能简单的把S3fs挂载的目录和本地操作系统目录等同使用。用户使用S3f3挂载S3存储桶和直接访问S3服务有类似的使用场景。适用于对不同大小文件对象的一次保存(上传),多次读取(下载),不适用于对已保存文件经常做随机修改的场景。
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Amazon S3 has unlimited storage capacity and network bandwidth in and out, low-cost data storage fees, and can achieve 99.999999999% (11 nines) durability, making it the best solution for object storage.
S3FS
S3fs is a FUSE-based file system that allows Linux and Mac OS X to mount S3 buckets on the local file system. S3fs can maintain the original format of objects.
Using S3fs, you can directly mount a shared Amazon S3 bucket under the user server's local directory. Applications can use Amazon S3 storage service directly without modification. This method can serve as a temporary solution to quickly migrate traditional applications to the AWS platform. For details, see: https://github.com/s3fs-fuse/s3fs-fuse
Mounting S3 Bucket Using S3fs
1. Install s3fs
a. Install necessary packages
sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel
b. Download, compile and install s3fs
git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure
make
sudo make install
c. Verify installation result
2. Create IAM User Access Key File
IAM user access key content can be written to the current user's default key file such as "/home/ec2-user/.passwd-s3fs" or a file created by the user themselves.
Command format: echo [IAM User Access Key ID]:[IAM User Access Key] >[Key File Name]
Command example: The following example will create a key file in the current user's default path
3. Manually Mount S3 Bucket
Command format: s3fs [S3 Bucket Name] [Local Directory Name] -o passwd_file=[Key File Name] -o endpoint=[Region Name]
For example:
s3fs ultrapower-test /home/ec2-user/s3mnt -o passwd_file=/home/ec2-user/.passwd-s3fs -o endpoint=ap-northeast-1
Check Mount Result
After the mount operation is completed, you can use the Linux "df" command to check if the mount was successful. The appearance of a s3fs file system similar to the 256T below indicates successful mounting. Users can then enter the local mount directory to access objects stored in the S3 bucket.
Unmount the S3 Bucket
If you no longer need to access the S3 bucket through mounting, you can use the Linux "umount" command to unmount.
sudo umount /home/ec2-user/s3mnt
Set Up Automatic Mount on Boot
Modify the /etc/fstab file and add the automatic mount command
vi /etc/fstab
Summary:
Using S3fs, you can conveniently mount S3 buckets in EC2's local operating system. Since S3fs actually uses Amazon S3 service's access interface, you cannot simply treat the S3fs-mounted directory the same as a local operating system directory. Using S3fs to mount S3 buckets and directly accessing S3 services have similar usage scenarios. It is suitable for one-time saving (uploading) and multiple reading (downloading) of file objects of different sizes, but not suitable for scenarios where saved files are frequently randomly modified.