In this post, we can find two different ways of mounting the S3 stroage into our machine. I have personally used the Method 2 and it works without any issue. It compiles the s3-fuse source code and installs it. The Method 1 is little bit longer.
Method 1
Step 1: Remove Existing Packages
First check if you have any existing s3fs or fuse package installed on your system. If installed it already remove it to avoid any file conflicts.
CentOS/RHEL Users: # yum remove fuse fuse-s3fs
Ubuntu Users: $ sudo apt-get remove fuse
Step 2: Install Required Packages
After removing above packages. First we will install all dependencies for fuse and s3cmd. Install the required packages to system use following command.
CentOS/RHEL Users: # yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap
Ubuntu Users: $ sudo apt-get install build-essential libcurl4-openssl-dev libxml2-dev mime-support
Step 3: Download and Compile Latest Fuse
Download and compile latest version of fuse source code. For this article we are using fuse version 2.9.3. Following set of command will compile fuse and add fuse module in kernel.
# cd /usr/src/
# wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9. 3/fuse-2.9.3.tar.gz
# tar xzf fuse-2.9.3.tar.gz # cd fuse-2.9.3
# ./configure --prefix=/usr/local
# make && make install
# export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
# ldconfig
# modprobe fuse
Step 4: Download and Compile Latest S3FS
Download and compile latest version of s3fs source code. For this article we are using s3fs version 1.74. After downloading extract the archive and compile source code in system.
# cd /usr/src/
# wget https://s3fs.googlecode.com/files/s3fs-1.74.tar.gz
# tar xzf s3fs-1.74.tar.gz
# cd s3fs-1.74
# ./configure --prefix=/usr/local
# make && make install
Step 5: Setup Access Key
Also In order to configure s3fs we would required Access Key and Secret Key of your S3 Amazon account. Get these security keys from Here.
# echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > ~/.passwd-s3fs
# chmod 600 ~/.passwd-s3fs
Note: Change AWS_ACCESS_KEY_ID and A
Method 2
Ensure you have all the dependencies:
On Ubuntu 14.04:
sudo apt-get install automake autotools-dev g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config
On CentOS 7:
sudo yum install automake fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel
Compile the s3-fuse source code from its git master repo using the following commands:
git clone https://github.com/s3fs-fuse/s3fs-fuse.git cd s3fs-fuse ./autogen.sh ./configure make sudo make install
Examples
Enter your S3 identity and credential in a file
/path/to/passwd:echo MYIDENTITY:MYCREDENTIAL > /path/to/passwd
NOTE: These credentials can be found when you log into your Amazon AWS account!!
In the following example, I am exporting the credentials into a file named passwd-s3fs which will be created on the root directory:
echo AKIAIOARLIUE54Z7QBRA:
Make sure the file has proper permissions (if you get 'permissions' error when mounting)
/path/to/passwd:chmod 600 /path/to/passwd
Run s3fs with an existing bucket (e.g.:
mybucket) and a target mount directory (e.g.: /path/to/mountpoint):s3fs mybucket /path/to/mountpoint -o passwd_file=/path/to/passwd ORsudo mkdir -p /narccaps3/datasudo mkdir -p /narccaps3/workingsudo s3fs testbucketcapstone /narccaps3/data -o use_rrs -o allow_other -o use_cache=/tmp/cache -o passwd_file=~/.passwd-s3fssudo s3fs narccap-capstone-working-dir /narccaps3/working -o use_rrs -o allow_other -o use_cache=/tmp/cache -o passwd_file=~/.passwd-s3fs
If you encounter any errors, enable debug output:
s3fs mybucket /path/to/mountpoint -o passwd_file=/path/to/passwd -d -d -f -o f2 -o curldbg
You can also mount on boot by entering the following line to
/etc/fstab:s3fs#mybucket /path/to/mountpoint fuse _netdev,allow_other 0 0 or mybucket /path/to/mountpoint fuse.s3fs _netdev,allow_other 0 0
No comments:
Post a Comment