The S3Fox Organizer for Amazon is another Firefox extension!
After you install it and perform the usual Firefox restart, your Tools menu will contain an entry for "Amazon S3." Select this entry and you will see the S3Fox user interface, which consists of two panes -- local file system and S3 account -- and a number of ancillary controls. Start by getting an S3 account (if you don't have one already) and clicking the button labeled "Manage Accounts." Select a name for your account and enter your Access Key and your Secret Key.
Once you've done this, the right pane will reveal the contents of your S3 account. From there you can select items in the local (left side), and click the right-pointing arrow to copy them over to S3, or vice versa. The transfers are queued up (and shown at the bottom) and performed one at a time. You can also right-click on the items in the S3 account to set the permissions, delete them, copy the URL of the item, and so forth.
The extension is available as a free download, but the author would probably appreciate a donation or two, via the download page.