How to Backup SQL Server Backups to Amazon S3 Storage

It’s been a long time since i posted in my blog. One of the common request i have been asked by my colleagues, friends about “Is there a way to automate backups that are taken on the SQL Server database server running on EC2 instance to Amazon S3 storage?”  Today, I would like to show how we can achieve this.

Requirement: I would like to keep one days of backups on the disk and also maintain remaining backups on Amazon S3 storage for 30 days and later archive them.

I’m assuming that you have a mechanism to take backups to Disk.

  •      Attach a S3 policy to Current Database Server running on EC2 instance. Instance need access to read and write permissions. You may use the below policy.

 

{    "Version": "2012-10-17",
"Statement": [        {
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*", "s3:Put*"
],
"Resource": [ "arn:aws:s3:::Name_of_the_bucket/*"
]
}
]
}
  • Once you attach above the policy to IAM ROLE on EC2 instance, EC2 Instance gets access to the Amazon s3 bucket.
  • Next, Install AWS CLI command line tool on the EC2 instance. You may download here 
  • Now, let’s begin the actual process.  I’m assuming that you already have backups jobs running on database server based on your schedule which takes the backups on disk.  Include the below powershell script in the second step after taking backups to disk.
 
$bucketname = 'S3_Bucket_Name/' # S3 bucket name for copying backups
$backuppath = 'B:\MSSQL\Backups' # Backup location on the disk
$Region = 'us-east-1'
$env:Path += ';C:\Program Files\Amazon\AWSCLI'
 $cmd="aws s3 sync $backuppath s3://$bucketname --region $Region" 
 push-location 'C:\Program Files\Amazon\AWSCLI\'
 Get-Location 
 Invoke-Expression $cmd
 Pop-Location
  • 1
  • The above step will sync the backups to S3, something like this.

2

  • Have we go you have backups sync back to S3. The advantage of this is you can reduce the cost of disk storage and have backups highly available and reliable. You don’t have to worry about disk capacity and maintaining redundancy at disk level. Amazon s3 storage has 99.999999999% durability and 99.99% availability.
  • Now, you can create a life cycle policy to keep the backups on s3 storage for 30 days and archive them to Glacier storage (Much Cheaper storage). You may follow the link here

Hope you enjoyed the post!

Cheers

Ramasankar Molleti

MSDN:LinkedIn:Twitter

About Ramasankar

My name is Ramasankar Molleti, I have been working on various database and cloud systems (SQL Server, Oracle, PostgreSQL, DynamoDB, Redshift, Amazon, Azure) over 12 years, primarily focusing on Architect , Development, Administration and Business intelligence. I hold various Certifications such as Amazon Web services Solution Architect Associate, Microsoft certifications (Microsoft Certified Information Technology Professional, Microsoft Certified Technology Specialist, and Microsoft Certified Solution Associate).
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s