How to use Amazon S3 for persistent file storage on AppFog
Right now, we’re working hard on providing persistent file storage for AppFog. This kind of service is nice to have with a variety of languages and frameworks, but for PHP it is essential, and we know that. We’ll make a big announcement as soon as AppFog persistent file storage is both battle-tested and production-ready.
For the time being, we suggest looking at Amazon’s Simple Storage Service (S3) file storage system not only as a temporary but also as a potentially permanent file storage solution (if you wish). S3 is inexpensive, mature, easy to use, and reliable.
Initial setup for the S3 service
If you already have Amazon’s S3 service set up, feel free to skip down to the next section.
If not, then beginning S3 integration with your PHP application will require you to sign up for an account with Amazon Web Services (AWS). From the account/console page, click on the “Amazon Simple Storage Service (S3)” link under “Infrastructure Services” and then on the “Sign Up Now” button on the main S3 page.
At this point, you’ll need to enter your credit card information to continue. But don’t worry: there are no fees up front and you’ll only be charged for the services that you actually use on S3. Once you’ve done that, you’ll be asked to verify your account via email.
Managing buckets in the AWS console
Once your S3 service is set up, head to your AWS console and click on S3 in the “Storage & Content Delivery” section:
In the S3 management console, create a new storage bucket:
I’m going to not-so-creatively name mine
rainyinpdx in honor of the atrocious Portland weather today and store it in Amazon’s Virginia data center (all buckets share the same namespace, so make sure and choose a different name for yours!). Upon creation, you’ll be taken to the console for your bucket:
Here, you can manually upload files, separate them into folders, copy and paste files, and so on, all according to a very simple UI.
The next step is to give my PHP application to the files stored in my
How to access your bucket using PHP
Luckily, the S3 console makes it incredibly easy to manage your file system from a centralized location. Now, it’s time to integrate our application with S3 on the coding side.
A variety of PHP frameworks offer their own ways of managing S3 integration, including Drupal, Zend, CodeIgniter, WordPress, and many others. If you’re comfortable working in one of those frameworks, there are a lot of good resources, plug-ins, and tutorials currently available. This tutorial, however, is for those who’d like to use S3 outside of an established PHP framework or to use S3 without using framework-specific plug-ins.
In order to interact with my
rainyinpdx S3 bucket via PHP, I’ll be using this S3 module. First, we need to download the GitHub repo via Git:
git clone https://github.com/tpyo/amazon-s3-php-class.git
Once we’ve done that, we need to move the
S3.php file into our app’s directory and we’ll be ready to interact with the module.
Now, let’s create a sample
my_s3.php file that will enable us to access our bucket and start by loading up the module and defining our AWS security credentials, which we can always easily access via the “Security Credentials” page in the AWS console:
One way to set these two variables is to simply copy and paste your AWS credentials directly into your application code. But this is not in line with best practices. It is far better to set variables such as this as environment variables for your app. This can be done either in the console, found under the “Env Variables” section of the console for a given app, or in the command line. I’ll show you how to set the
MY_AWS_ACCESS_KEY variable through the command line:
af env-add rainyinpdx MY_AWS_ACCESS_KEY=<insert-key-here>
Now, you need to modify your variable definition code to access environment variables:
Once you’ve done this with both your
MY_AWS_SECRET_KEY variables, you’ll be able to access your S3 account via your application without having to embed that information directly in your application.
Now, let’s create a
$s3 object that will enable us to interact with our S3 bucket:
$s3 = new S3(awsAccessKey, awsSecretKey);
Now, let’s see if our
$s3 object is properly set up. We’ll test this by grabbing a list of our currently available buckets:
print_r command will print the response from the AWS server (in the console) when asked for a list of buckets corresponding to the access credentials I’ve entered. If I run
php my_s3.php in the CLI, here’s what my response looks like:
 => rainyinpdx
So that confirms that we can interact with our S3 account via PHP. Now, let’s try to work with actual files in our
Managing files in our S3 bucket
One way of managing files in S3 is of course to do so manually in the console. But very often, we need to be able to do so dynamically and to embed S3-related functionality into our working PHP apps. Fortunately, the S3 module that I have used thus far provides a very simple API for doing so.
At this point, we no longer need to verify that interaction is working, so let’s remove the
print_r($s3->listBuckets()); line from our
my_s3.php file. Now, let’s add a new file to our bucket. In the same directory where our
my_s3.php file is, let’s create a simple new text file called
hello.txt using a simple command in the CLI:
echo "wake up, sheeple!" >> hello.txt
Now, let’s add a line to our PHP file that will put our
hello.txt file in our
$s3->putObject($s3->inputFile('hello.txt', false), 'rainyinpdx', 'hello.txt', S3::ACL_PUBLIC_READ);
If I navigate to my AWS S3 console and hit “Refresh” in my browser, I’ll see my
hello.txt file there and ready to go:
Now, let’s say that our
hello.txt file has outlived its usefulness and we want to get rid of it. Let’s go back and modify our PHP script to delete it. Let’s start by removing the line we just entered that uploaded the file, and replacing it with this:
If I run the script again, navigate to my S3 console, and click “Refresh” in the browser, I’ll see that my
hello.txt file has indeed been deleted.
Simply uploading and deleting specific files is of course one of a much wider variety of tasks that you can perform via this S3 module. With it, you can also create and delete buckets, find out in which AWS infrastructure a bucket is located, copy objects between buckets, and so on.
I strongly suggest checking out the module’s very readable docs for information on how to use all of these commands, which are all fortunately quite syntactically similar to ones that I have used thus far.
Integrating S3 functionality via HTML forms
Let’s imagine that we want to build an app that enables users to upload files to our S3 bucket. This is actually surprisingly simple once we’ve set up our connection to S3 in our PHP app. This tutorial on NetTuts is the best that I have encountered, and I will rely on it heavily, although with appropriate modifications.
First, let’s set up an HTML form that enables users to post files to our PHP server:
<form action="" method="post" enctype="multipart/form-data">
<input name="ourFile" type="file" />
<input name="Submit" type="submit" value="upload" />
This will enable users to click on a “Browse” button that allows them select a file from their local machine. When they click the “Submit” button, the file will be sent to our server.
Now, we need to set up our server to handle this input. Let’s set up a form handler to handle the HTTP request sent by our file form and then upload it to S3 using the same
putObject function that we used before. Let’s assume that users will only be uploading text files:
$fileName = $_FILES['ourFile']['name'];
$s3->putObject($s3->inputFile($fileName, false), 'rainyinpdx', $fileName, S3::ACL_PUBLIC_READ);
Now, when we click on the “Submit,” our server will save the file directly to our S3 bucket.
All in all, this demonstrates only a small fraction of the functionality available to you using PHP and S3. But I hope that it suffices to provide some basic orientation on how to set up an S3 account, interact with buckets via PHP, and then to integrate some of that functionality into a dynamic PHP application.
Like I said, persistent file storage is coming to AppFog very soon. In the meantime, S3 is a powerful service that we strongly suggest using, be it in the short or long term. If you have any issues connecting to S3, please get in touch with our support team.