Personal Venting Session: Amazon Simple Storage Service

So. A little venting on my part here. Feel free to totally ignore this. I’m writing this more for myself, really. When this project is all said and done, I’d like to be able to look back to this weekend, and remember these particular moments of frustration.

Again, these are less production notes and more just… well, complaining.

So let’s have at it.

It’s been about a week plus since I’ve finished (mostly) the Flash portion of my latest project. Part of the project involves allowing users to upload their own mp3s, and I recently signed up to use Amazon’s Simple Storage Service to handle the storage side of things.

As part of the process in authenticating uploads, you have to do a few things to verify the upload request. After all, since I’ll be paying for every upload and for all bandwidth used, it’s important that Amazon knows definitively that the request is coming from me… or at the very least, authorized by me.

To that end, each request requires what’s called a policy file. On top of that, the policy file needs to be base64-encoded. In addition to the policy file, a signature value is also required. This signature is generated via HMAC SHA-1 with my Amazon Secret Key as the password. That value is also base64-encoded, and included with the upload request.

For now, I’ve been working through the basic HTML POST example. I ultimately want the process to work in Flash, but given all the requirements and complexities, I figured I’d learn to do things in basic HTML first before moving on to the tough stuff. And for the most part, I got it… I understood what needed to happen. The only things I didn’t quite get were how to do the encoding, the base64-encoding and the HMAC SHA-1 stuff.

So I looked around Amazon’s forums, and dug through documentation. And searched and searched and searched. After doing nothing but sifting for a day plus, I finally broke down and posted questions at a few places I frequent.

Over on Ask MetaFilter, I posted my question and got some helpful suggestions regarding using Perl to get the values I needed.

Ok. So then I start digging around and looking into Perl. And I get some basic encoding down, and start trying to figure out how to get the necessary values I need.

Keep in mind here that I’m doing all this (looking up Perl tutorials) to get two values that I need to use in my HTML test… so that I can figure out this process for eventual use in Flash. At each turn, it feels like the more I learn… the farther away from finishing I get.

After an afternoon of searching, I came across Raam Dev’s excellent “Amazon S3 HMAC Signatures without PEAR or PHP5”. And after giving it a few tweaks, I was able to get the values I needed… and successfully got my HTML POST example to work. No Perl needed, I just edited a few PHP functions and got the strings I needed.

I can’t tell you – I felt like crying, I was so happy. I had been banging my head and stuck in place for so long, it felt like I could breathe again… like I could move forward again.

I’ve been obsessed with this project for a few weeks now, and still most days it’s almost all I can think about. At work, in meetings, on the train rides to and from downtown… my mind is focused on this project, and what I need to do to get it launched.

To have been stuck in place for a week or so now has really taken the wind out of my sails. Each day, I find myself getting a little less enthusiastic, a little less motivated. I feel like I’ve sprinting towards a finish line, only to find myself reduced to a slow shuffle because I can’t figure out how to tie my shoes.

So the HTML example. I got the encoded values correct, and got an upload over to my Amazon bucket successfully. Fucking WOOT!

And so where does that leave me? Well, I’ve got one working example, which is good. The next step is to actually get the process working in Flash. As of now, I think I have two choices:

Option #1: Work through Amazon’s S3 POST Example in Flash.

This is based on Actionscript 3, which I don’t really know. I’ve got a good grasp of AS1, but never quite made it over to AS2… and AS3 is something I’ve barely touched at all. I’m not sure if my existing .fla file will actually work exported out as an AS3 .swf, so there’s some questions there.

Also – I never upgraded at home to Flash CS3. I have it at work, but this weekend, I wasn’t able to play with AS3 at all. I thought about dropping the $199 to purchase Flash CS3 just so I could finish the project, but Flash CS4 is just about to be released. I couldn’t get a hold of anyone at Adobe to answer my upgrade questions, and the last thing I wanted was to pay $200 to upgrade to CS3, and then turn around and drop another $200 to upgrade to CS4 some months later.

So – the Flash approach is on hold. If I’m doing any development there, I may have to see about doing it at work, either during my lunch or outside of regular business hours.

Option #2: Use PHP for the upload. I’m currently using a combination of Flash’s FileReference object and a bare bones PHP upload script.

Now I’m not terribly well versed in PHP, but I think I could dabble/tweak enough to figure things out. I found a semi-decent walkthrough that utilized an Amazon S3 PHP class by Geoff Gaudreault… and that’s what I was pursuing for most of this afternoon.

Part of the tutorial (and Gaudreault’s class) requires the installation of the Crypt_HMAC module from PEAR. I’m not entirely sure what that means, so that led me to doing a little digging around regarding PEAR. And then me sending in an email to my host, asking if they could install this module for me… or if it’s something I need to do myself.

So again, I’m at a stopping point. Maybe tomorrow I’ll hear back from my host, and see about progressing on the PHP route. Or maybe I’ll head in to work early, and see how the Actionscript 3 approach looks.


All in all, it’s been a rough week or so with this project. And a brutal past two days. I really want nothing more than to launch this project, and let this thing loose for everyone to play with. And I’m close, really quite close. A few more technical hurdles to get past, and it’s golden.

The “excitement” of being close has been overridden by the “frustration” of being close, and unable to get closer. I’ve blown two days and most of my entire weekend doing nothing but reading and searching. And I’m still finding myself farther away from launching than I’d like.

It’s comparable to having a batch of cookies ready, but being unable to turn on the oven. It’s like wanting to drink a glass of water, but you having to learn how to make your own glass first, before drinking. And when the regular work week kicks in, it’s often difficult to find more than a fleeting hour or two to dig, before something else pulls my attention elsewhere.

I’m complaining, I know. But I should be happy. This weekend, I got the HTML example do
wn… and have one successful, authenticated upload under my belt. It’s not a huge step, but it’s a step. And at this point, I’ll take anything resembling forward motion.

I’m really quite excited by this new project. I’m hoping that bit comes through, despite all the bitching and complaining. I guess it’s BECAUSE I’m so excited that I’m finding these delays so very frustrating.

Ok. I think that got a lot out of my system. I’ve learned a lot of new things this weekend. And I’ve made a little bit of progress. And at this point, any progress is forward progress, is good progress. I would love nothing more than to be writing a blog entry a week from now, next Sunday evening, announcing the launch of this next project.

So it goes. Off to sleep, and hopefully tomorrow will yield what today did not.

This Post Has 1 Comment

  1. Ok. Just moments ago, I was able to get my PHP test page to successfully upload an mp3 over to Amazon.SO happy. SO relieved. I’ve got this exhausted-yet-excited feeling now.I came across a different PHP tutorial (also by Jack Herrington). Unlike the first, this particular tutorial centered around uploading video. But the kicker is that it’s using an updated version of Gaudreault’s class… and the article was written by Herrington some six months after the one I had been using.So hooray! A working PHP version! And what this may possibly mean for me is that I can re-route my existing Flash files to use this particular PHP upload (I think). Rather than mess with the AS3 approach (which may be more elegant, but may take me longer)… I *think* there’s a way for me to hook into this PHP approach with what I’ve got in place already.Exciting stuff. And of course, it’s getting closer and closer to the official start of the day, and I need to start gathering my stuff to head off to work.It’s going to be difficult to focus on anything other than this today. Happy Monday, everyone!

    avoision Reply

Leave A Reply