AWS: Mount S3 to Ec2 instances

Step1: Install S3FS

a) Install using yum (Centos /Redhat):

# sudo yum install s3fs-fuse

b) Install using apt-get ( Ubuntu):

# sudo apt-get install s3fs

c) Install from source code:

 # sudo yum install automake fuse fuse-devel gcc-c++ libcurl-devel libxml2-devel make openssl-devel
# cd /tmp
# wget
# unzip
# cd  s3fs-fuse-master
# ./ 
# ./configure --prefix=/usr 
# make
# sudo make install

Step2: Create S3 bucket or S3 Folder

You can create S3 bucket in AWS S3, if you want to mount folder you can create folder in existing bucket. for this tutorial, lets say bucket is s3bucket and folder is s3folder.

Step3: mount S3

Create a folder to mount

# mkdir -p /var/s3

We will be adding following entry into /etc/fstab based on our requirement.

a) Mounting as IAM role:

You can change iam_role value from auto to any specific role as required otherwise auto will auto detect role.

s3fs#s3bucket /var/s3 fuse _netdev,nonempty,mp_umask=022,allow_other,use_cache=/tmp,iam_role=auto,rw 0 0
b) Mounting S3 folder:
s3fs#s3bucket:/s3folder /var/s3 fuse _netdev,nonempty,mp_umask=022,allow_other,use_cache=/tmp,iam_role=auto,rw 0 0

In above example s3bucket is post-pended with folder name s3folder

c) mount with access key:

Create access file:

# echo ACCESS_KEY:SECRET_KEY > ~/.passwd-s3fs
# cat ~/ .passwd-s3fs ACCESS_KEY:SECRET_KEY
# chmod 600 .passwd-s3fs

Edit /etc/fstab:

s3fs#s3bucket /var/s3 fuse _netdev,nonempty,mp_umask=022,allow_other,use_cache=/tmp,rw 0 0

If you want to mount folder then edit /etc/fstab with following entry:

s3fs#s3bucket:/s3folder /var/s3 fuse _netdev,nonempty,mp_umask=022,allow_other,use_cache=/tmp,rw 0 0

Step 4: Final steps

Activate the mount:

# mount -a 

Check your mount:

# df

You should able to see all your mounts along with s3 mount.

AWS: CloudFront 502 bad gateway error

Usually 502 error happens when Cloudfront unable to reach origin endpoint. Some of the common reasons are:

Backend is busy: Origin endpoint ( ELB -> EC2) is busy or load is higher than given threshold. If this is happening reguarly, its better to get more instance in auto scaling group or increase existing instance for larger capacity.

WAF rule: Some request might get blocked by WAF rule. So check cloudFront logs to verfiy, if needed modify your application or disable WAF rule ( not recommended).

DNS: If Endpoint has custom DNS, make sure DNS is reachable.

EC2 instances: All EC2 instances in origin endpoint are down or unhealthy.