3 Steps to Integrate SharePoint with Amazon S3 to create Enterprise File Storage Solution
Today corporate IT is trying to embrace the influx of cloud storage brought on by the adoption of the cloud. Amazon S3 is the highest deployed cloud storage solution in the world. It is used in many small and large companies throughout the world.
SharePoint developers are aware that SharePoint stores data on their content database and hence SQL Server performance may decrease due to heavy data existence on SQL Server. AWS S3 (Simple Storage Service) provides integration with SharePoint using the AWS SDK.
How to use AWS SDK and SharePoint amidst customized development:
Using customized development on AWS SDK and SharePoint, we can easily store SharePoint documents on AWS S3 bucket instead of SharePoint library and return one unique key. A key is a unique identifier for an object within a bucket. Every object in a bucket has exactly one key.
Once the document is stored on AWS S3 then the key is returned and that key is saved in SharePoint list to retrieve the document from AWS S3. It will save your SharePoint Content Database space and increase your SQL Server performance. In case if SharePoint server is crashed then all stored documents on AWS S3 would remain safe and you can access those using AWS cloudberry tool.
You need Amazon S3 and SharePoint to work together, but the native integration either.
a) Doesn’t exist or
b) Doesn’t do what you want.
Integrating SharePoint with AWS S3:
Before we start integrating SharePoint with AWS let’s have glance on understanding AWS S3 and SharePoint
About Amazon S3:
Amazon Simple Storage Service is a fully redundant data storage system. It makes it easy to store and retrieve any amount of data, anytime, from anywhere.
AWS S3 (Simple storage service) is to provide data storage infrastructure offered by Amazon and is used to store and retrieve any amount of object at any time. Most important unit in AWS S3 is the bucket and is called a logical unit of storage which is used to store objects.
Learn More with DEV IT on AWS S3 here:http://blog.devitpl.com/detailed-insights-to-amazon-web-services-aws/
SharePoint is the business collaboration software that helps simplify content management, search, and business intelligence. It also has a strong component around sharing for intranet and internet sites.
Why AWS S3?
Mainly agenda to introducing AWS S3 is to store your application objects directly on AWS S3 Cloud instead of your application server. It will help to save your application server space for better performance.
3 Steps to Integrate SharePoint with AWS S3
Note: before we start writing the process, you need to have AWS S3 instance, AWS Secrete key, and AWS Access Key Id.
We can do operation on AWS S3 instance in two ways as mentioned below:
- Using AWS cloudberry Application
- Using AWS SDK
Here we will only see the use of AWS SDK in SharePoint Solution as mentioned in the below steps:
Step1: Install AWS SDK from below link and install:
To use the AWS SDK for .NET, you must have the following installed.
- (Required) Microsoft .NET Framework 3.5 or later
- (Required) Microsoft Visual Studio 2010 or later
- (Required) The AWS SDK for .NET
- (Recommended) AWS Toolkit for Visual Studio, a plugin that provides a user interface for managing your AWS resources from Visual Studio, and includes the AWS SDK for .NET. For more information, see Using the AWS Toolkit for Visual Studio.
For Further information, Click Here.
Step 2: Register AWS S3 profile in SharePoint web config file:
Inside “<appSettings>” tag, add following key:
<add key=”AWSProfileName” value=”Amazon” />
<add key=”AWSRegion” value=”us-west-2″ />
In above keys, provide value in “AWSProfileName” and “AWSRegion”.
Step3: Open Visual Studio 2012 OR higher and create new SharePoint project to implement AWS S3 operation:
Note: To use AWS SDK classes in SharePoint, need to add following DLL:
- Add Object in AWS S3 Bucket using SharePoint Solution:
- Add Object in sub directory which is created under AWS bucket:
- Get Object from AWS bucket:
- Delete Object from AWS bucket:
In this blog, we commissioned on how to integrate AWS S3 as an enterprise file storage solution for SharePoint.
Integrating AWS S3 as an enterprise file storage solution is a cloud application scenario that makes your files securely available from any platform. With secure and highly reliable AWS ecosystem your data is very less likely to breach. When you integrate this solution with your current infrastructure you get a better managed cloud, low recurring costs and pay as you go functionalities.
Learn more with us on how to integrate S3 with SharePoint for different versions and APIs.
Latest posts by Mihir Parekh (see all)
- Guide to PowerShell Automation Scripting for Successful DevOps - August 14, 2018
- 3 Steps to Integrate SharePoint with Amazon S3 to create Enterprise File Storage Solution - October 17, 2016
13 thoughts on “3 Steps to Integrate SharePoint with Amazon S3 to create Enterprise File Storage Solution”
Thanks admin for sharing the unique content, you have done a great job
Thanks for the great article. We are also trying to implement this, but I found the article to be missing some important points. I have few queries as given below
1. After creating SharePoint project, what do you do with the Project? Does it need to be deployed in SharePoint? If so, how do you deploy it in SharePoint? Does it need to be converted to WSP & then deploy?
2. Were you successful in moving your DBs to AWS S3 bucket with this approach?
3. What was the performance after you moved the DBS to AWS S3 bucket?
Thanks a million in advance.
Thank you for read my blog. Please find my answer on your query as mentioned below:
1. Yes, you need to install AWS SDK; develop the web part based on your need and deploy WSP on SharePoint server
2. In my scenario, I have kept only SharePoint documents in AWS S3 bucket not fully content data. It helps us to reduce SharePoint content database size.
3. After integrate AWS S3 with SharePoint, we got very nice performance because every year we got 70 GB of documents which directly stored in AWS S3 inserted of SharePoint content database.
Please feel free to connect with me on email@example.com for any further queries. I will glad to assist.
Does this enable the full suite of SharePoint capabilities like indexing and search, collaboration, workflows, tools integration, etc?
Thank you for reading my blog article. I really appreciate that. Also, thank you for sharing your question. It is indeed resourceful. I will get back with the answer to your question very soon.
Hi Dave Wruck,
I am glad to answer your questions. Please find them as under:
Indexing and Search: Yes, You can create indexing on columns to improve the performance of various query operations. However, document content search will not appear in SharePoint out of box search result as documents are located in AWS S3. If you need my advice on how to get AWS S3 documents in the search result, do let me know.
Workflows: You can execute workflows on SharePoint library property, but not on documents which are located on AWS S3
Tools integration: It depends on a tool which you want to integrate with SharePoint and where to use
Collaboration: Social collaboration and communication options do not create conflicts with AWS, except centralized access to documents as documents are stored in AWS S3 instead of SharePoint
Please feel free to reach out to me for any further queries. I will glad to assist.
I have similar requirement where I have to copy the excel files from Share point folder to AWS S3. I am new in AWS so could you suggest me the steps how to achieve this.
Can you move metadata for your documents with this approach?
Can we sync files from S3 to Sharepoint Online? Every new file should get uploaded to sharepoint site. Do you have any solution?
Q. Where do you store and run this solution?
My Scenario: We have some data files which are getting generated on Sharepoint.
I have an s3 bucket where I want to move them. I don’t have any machine in between where I can run any application.
I can run Lamba function which can we run over AWS but the problem is that my SharePoint falls under firewall so I am not sure if my lambda would be able to access SharePoint.
How to achieve the index/crawling over documents in S3 and show the search results on Search Center?
We can leverage AWS Elastic Search service to enable automated indexing that also helps in document search. Lambda function could be leveraged in order to pass the object URL and the index into ES after the index formation is completed.
I need your help to set this up for me and would like to pay for this effort.
Hi looking forward to talk to you, book your convenient time by clicking this link https://calendly.com/rohitfinkraft/15min