Amazon Web Services Blog: Amazon Simple Workflow - Cloud-Based Workflow Management
february 2012 by mcroydon
Up next: Simple Singularity Service.
amazon
aws
ec2
workflow
february 2012 by mcroydon
Elastician: Creating an EBS-backed AMI from an S3-backed AMI
october 2011 by mcroydon
Creating new instances based on snapshots of EBS-backed EC2 instances.
amazon
aws
cloud
ec2
ebs
october 2011 by mcroydon
Driving down the cost of Big-Data analytics - All Things Distributed
september 2011 by mcroydon
"The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud."
analytics
aws
bigdata
data
datamining
september 2011 by mcroydon
dl.dropbox.com/u/5586906/slides/Cloud_Scaling.pdf
september 2011 by mcroydon
Performance matters.
amazon
api
aws
cloud
ec2
september 2011 by mcroydon
Architecting for the Cloud: demo and best practices, by Simone Brun...
july 2011 by mcroydon
While not particularly new, this looks like a solid overview of designing for failure in the cloud.
architecture
aws
cloud
design
hosting
july 2011 by mcroydon
AWS Architecture Center
april 2011 by mcroydon
Designing for failure and redundancy.
amazon
architecture
aws
cloud
april 2011 by mcroydon
Summary of the Amazon EC2 and Amazon RDS Service Disruption
april 2011 by mcroydon
The full post-mortem.
amazon
aws
cloud
debugging
failure
april 2011 by mcroydon
The Cloud is not a Silver Bullet – stu.mp
april 2011 by mcroydon
Solid thoughts on how to do it right in the cloud.
cloud
engineering
system
amazon
aws
ec2
ebs
april 2011 by mcroydon
When the Lights Go Out | Eucalyptus
april 2011 by mcroydon
Complex systems are complex. And Fragile.
amazon
aws
outage
april 2011 by mcroydon
How SmugMug survived the Amazonpocalypse « SmugMug's Don MacAskill
april 2011 by mcroydon
How SmugMug survived the AWS outage. Part careful planning part luck, but ultimately due to not using EBS and only suffering a minor outage due to splash damage.
amazon
aws
ebs
ec2
april 2011 by mcroydon
Skynet, EC2, and Zencoder « Zencoder Blog
april 2011 by mcroydon
Zencoder's reaction to the EC2 outage and how they handled it.
amazon
aws
ec2
ebs
april 2011 by mcroydon
Why the AWS Crash is a Good Thing. | SignNow
april 2011 by mcroydon
"The cloud computing world is in crisis and its a good thing for everyone in the long term."
aws
cloud
ec2
ebs
april 2011 by mcroydon
On Cascading Failures and Amazon’s Elastic Block Store « Joyeur
april 2011 by mcroydon
Thoughts from Joyent.
aws
blog
ec2
ebs
april 2011 by mcroydon
Hacker News | Some quotes regarding how Netflix handled this without interruptions:"Netflix sh...
april 2011 by mcroydon
Info from Netflix on how they weathered the outage.
amazon
aws
hosting
cloud
netflix
ec2
ebs
april 2011 by mcroydon
Why Twilio Wasn’t Affected by Today’s AWS Issues | Twilio Engineering Blog
april 2011 by mcroydon
How to architect for failure and redundancy.
amazon
architecture
aws
cloud
distributed
ec2
ebs
april 2011 by mcroydon
Working around the EC2 outage -
april 2011 by mcroydon
More thoughts on the outage.
amazon
aws
cloud
cloudcomputing
ec2
ebs
april 2011 by mcroydon
Post-Mortem of a Rough Day
april 2011 by mcroydon
MongoHQ's take on the EBS outage.
aws
amazon
ebs
outage
architecture
april 2011 by mcroydon
Elastician: S3 MultiPart Upload in boto
march 2011 by mcroydon
This appears to be the only sane way to transfer decently large (>2GB) files in to S3. Knock on wood anyway. Using traditional methods will likely just broken pipe on you after a short time.
aws
python
s3
march 2011 by mcroydon
Hello Singapore (CDN Experiment) - GitHub
march 2011 by mcroydon
Cloudfront makes things fast (usually).
aws
cdn
github
performance
scalability
march 2011 by mcroydon
help/en/howto/cloudfront – Cyberduck
march 2011 by mcroydon
How to set up cloudfront using your own origin server without having to use the API.
ec2
amazon
aws
cloudfront
cdn
march 2011 by mcroydon
Amazon Web Services Blog: AWS CloudFormation - Create Your AWS Stack From a Recipe
february 2011 by mcroydon
I love that the focus of this tool is to make it easier for you to get your job done and consume the underlying AWS services.
aws
amazon
web-service
cloud
february 2011 by mcroydon
Quora’s Technology Examined | Phil Whelan's Blog
february 2011 by mcroydon
I love architecture deep dives like this, regardless of the tech at play.
aws
programming
python
web
february 2011 by mcroydon
Amazon Web Services Blog: Introducing AWS Elastic Beanstalk
january 2011 by mcroydon
I can't wait to see this hit the streets for other platforms like Python.
aws
ec2
scaling
january 2011 by mcroydon
Yelp Engineering Blog: mrjob: Distributed Computing for Everybody
november 2010 by mcroydon
A really nice wrapper around EMR.
algorithms
amazon
aws
cloudcomputing
computing
data
hadoop
framework
distributed
dist
development
datamining
library
map-reduce
map
mapreduce
nosql
opensource
yelp
webservices
search
reduce
python
programming
aa
elasticmapreduce
logs
emr
mrjob
november 2010 by mcroydon
Compute Cycles - Benchmarks for the brand new Compute GPU on Amazon EC2
november 2010 by mcroydon
I think this means GPU computing is a good thing.
amazon
aws
benchmark
benchmarks
cloudcomputing
cloud
performance
hpc
ec2
november 2010 by mcroydon
Rails Development – Interface Design – iPhone/iPad/Android Apps
september 2010 by mcroydon
Web firm that communicates what they do really well.
agency
amazon
agile
business
aws
cloud
company
ec2
development
developer
design
dc
css
cool
consulting
inspiration
rails
ruby
ror
rubyonrails
scalability
social
webdev
webdesign
web2.0
web
vc
intridea
scalr
september 2010 by mcroydon
Amazon CloudFront Adds Invalidation Feature
september 2010 by mcroydon
Amazon solves one of the two truly hard concepts in Computer Science and it's free for the first 1,000 operations!
amazon
aws
cloudfront
september 2010 by mcroydon
7 easy tips to reduce your Amazon ec2 cloud costs
march 2010 by mcroydon
How to save money in the cloud.
amazon
aws
cloud
cost
dns
ec2
isv
optimisation
techniques
tips
server
reduce
provider
projects
price
webservices
march 2010 by mcroydon
Gojko Adzic » Designing applications for cloud deployment
january 2010 by mcroydon
Somewhat obvious but good thoughts on deploying in the cloud.
programming
aws
design
hosting
cloud
ec2
development
deployment
cloudcomputing
webdev
amazon
deploy
qa
server
scalability
performance
architecture
tech
january 2010 by mcroydon
Amazon S3: Versioning Proposal
january 2010 by mcroydon
Potentially quite useful someday.
programming
amazon
storage
aws
s3
documentation
vcs
versioning
faq
january 2010 by mcroydon
Practical Cloud Computing | SimpleDB Performance : 5 Steps to Achieving High Write Throughput
january 2010 by mcroydon
For when I need to load massive amounts of data in to SimpleDB.
howto
scalability
performance
aws
startup
databases
cloud
db
cloudcomputing
migration
nosql
simpledb
load
scalable
january 2010 by mcroydon
Home - Ensuring Inbox Delivery - CritSend
december 2009 by mcroydon
"CritSend ensures your email reaches its appropriate inbox and will not end up in a spam box. We deliver over 2 million automated emails per day: invoices, password notifications, transactional, newsletters, email marketing and more." $1 per 1,000 emails.
amazon
aws
ec2
open
email
mail
service
spam
it
smtp
beta
via:pskomoroch
december 2009 by mcroydon
Crowdsourced document analysis and MP expenses
december 2009 by mcroydon
Simon's experience running two large-scale crowdsourcing efforts.
design
web
django
data
journalism
webdev
article
aws
social
interface
ec2
application
memcached
information
2009
guardian
uk
ia
crowdsourcing
lessons
impressive
light
expenses
iteration
december 2009 by mcroydon
Elastician: Private and Streaming Distributions in CloudFront
december 2009 by mcroydon
Covers some of the new features released by Amazon today and available in boto trunk.
amazon
aws
s3
cloudfront
streaming
video
python
december 2009 by mcroydon
Building a Data Intensive Web Application with Cloudera, Hadoop, Hive, Pig, and EC2 | Cloudera
november 2009 by mcroydon
A nice look at end-to-end data analysis of big datasets using things like Pig and Hive.
programming
web
data
database
tools
webdev
business
toread
howto
tutorial
amazon
dev
scalability
hadoop
architecture
aws
computing
cluster
ec2
pig
trends
hive
cloudera
cloudcomputing
analytics
datamining
mapreduce
cloud
application
november 2009 by mcroydon
AllegroGraph RDFStore Web 3.0's Database
october 2009 by mcroydon
Open source (limited to 50m triples) and commercial RDF triplestore.
web
linux
database
amazon
storage
aws
databases
graph
ec2
db
download
rdf
store
semantic
semanticweb
semweb
ontology
semantic-web
triplestore
datastore
web3.0
sparql
allegrograph
graphdb
allegro
october 2009 by mcroydon
Analyzing Human Genomes with Hadoop » Cloudera Hadoop & Big Data Blog
october 2009 by mcroydon
A fantastic writeup of absurdly fast sequencing software that can analyze a human genome in about 3 hours for less than $100 of AWS resources. Pretty darned impressive.
data
opensource
computer
amazon
algorithms
aws
hadoop
ec2
mapreduce
dna
bioinformatics
cloudera
trend
genetics
genome
foss
genomics
october 2009 by mcroydon
Petabytes on a budget: How to build cheap cloud storage | Backblaze Blog
september 2009 by mcroydon
A pretty fantastic DIY solution to massive storage. I kinda wonder what the failure rate is on a configuration such as this.
design
linux
hardware
database
opensource
technology
amazon
server
diy
computer
storage
sysadmin
cool
s3
aws
debian
open
cloud
infrastructure
custom
cheap
backblaze
hard
drive
harddrives
september 2009 by mcroydon
Tracking Trends with Hadoop and Hive on EC2 » Cloudera Hadoop & Big Data Blog
august 2009 by mcroydon
A detailed run through of data warehousing and creating trending results for wikipedia data.
python
data
database
howto
tutorial
wikipedia
rails
scaling
hadoop
aws
resources
ec2
cloud
mapreduce
datamining
sparklines
rubyonrails
trends
hive
cloudera
bigdata
log
t
trendingtopics
august 2009 by mcroydon
Getting Good IO from Amazon's EBS
august 2009 by mcroydon
How to get good performance out of this mysterious creature.
amazon
storage
performance
startup
aws
testing
ec2
optimization
benchmark
ebs
raid
disk
io
august 2009 by mcroydon
Cloudera's Distribution for Hadoop | Cloudera
june 2009 by mcroydon
Includes lots of feature tickets that are pretty stable but not yet in a Hadoop release. It reminds me a lot of Debian unstable or Ubuntu a month or so before release. Good stuff indeed. Includes RPM and APT package management options.
software
data
linux
google
search
aws
distributed
computing
ec2
hadoop
cloud
mapreduce
cloudcomputing
distribution
clusters
packaging
cloudera
cloud-computing
rpm
apt
june 2009 by mcroydon
aws - simple access to Amazon EC2 and S3
may 2009 by mcroydon
Simple commandline access.
programming
software
web
linux
development
tools
webdev
amazon
storage
web2.0
aws
s3
ec2
management
perl
utilities
backup
cloud
tool
shell
cloudcomputing
admin
cli
access
line
script
commandline
curl
command
command-line
may 2009 by mcroydon
Amazon Web Services Developer Community : Finding Similar Items with Amazon Elastic MapReduce, Python, and Hadoop Streaming
april 2009 by mcroydon
Excellent tutorial on diving in to Hadoop streaming on AWS.
python
howto
code
amazon
aws
streaming
ec2
hadoop
netflix
mapreduce
recommendation
similarity
april 2009 by mcroydon
marajo - Google Code
february 2009 by mcroydon
Interesting. AppEngine compatible framework that runs on EC2.
python
google
amazon
opensource
aws
framework
ec2
infrastructure
project
webservices
simpledb
appengine
gae
license:gpl
license:gplv3
february 2009 by mcroydon
Amazon Web Services Blog: New AWS Public Data Sets - Economics, DBpedia, Freebase, and Wikipedia
february 2009 by mcroydon
Lots of hot new data.
programming
data
blog
amazon
research
wikipedia
statistics
aws
compsci
ec2
resources
transportation
cloud
datamining
nlp
dataset
cloudcomputing
corpus
datasets
freebase
gov
publicdata
scraping
encyclopedia
february 2009 by mcroydon
Chef: Suck on my chocolate salty balls
january 2009 by mcroydon
Like puppet, but with more South Park references.
tools
ruby
server
sysadmin
rails
aws
management
ec2
deployment
couchdb
cloud
capistrano
administration
automation
configuration
merb
rubyonrails
deploy
chef
puppet
ohai
engineyard
ezra
configuration_management
january 2009 by mcroydon
pyaws - Revision 93: /trunk
december 2008 by mcroydon
pyamazon ported to ECS 4.0 with some minor API tweaks. ECS 3.0 is dead, so this is your option. It seems to work for me so far.
python
programming
mashup
web2.0
amazon
aws
ecs
december 2008 by mcroydon
On Demand Video Encoding
december 2008 by mcroydon
Intriguing commercial encoding service.
mobile
video
amazon
api
server
flash
scalability
xml
online
aws
conversion
encoding
flv
convert
encode
ffmpeg
cloud
service
transcoding
webservices
encoder
services
saas
ondemand
demand
codecs
on
december 2008 by mcroydon
Panda - Open source video platform
september 2008 by mcroydon
Video uploading, encoding, and streaming with RESTful API? Yes, please.
programming
web
video
ruby
webdev
amazon
rails
api
flash
s3
webdesign
aws
rest
streaming
ec2
encoding
stream
opensource
flv
encode
ffmpeg
transcoding
upload
clean
panda
rubyonrails
merb
uploading
gem
september 2008 by mcroydon
Persistent Django on Amazon EC2 and EBS - The easy way
august 2008 by mcroydon
Yeah, it's that simple.
python
programming
web
django
development
code
howto
business
webdev
tutorial
amazon
sysadmin
work
web2.0
aws
geek
postgresql
hosting
computers
ec2
postgres
cloud
cloudcomputing
admin
ebs
webservices
persistence
august 2008 by mcroydon
Amazon Web Services Developer Connection : Running MySQL on Amazon EC2 with Elastic Block Store
august 2008 by mcroydon
This applies pretty cleanly to other database engines too.
reference
database
howto
toread
tutorial
amazon
sysadmin
storage
s3
aws
scalability
computing
ec2
databases
mysql
webhosting
cloud
article
tutorials
cloudcomputing
ebs
xfs
webservices
august 2008 by mcroydon
ZFS snapshots to and from S3 - OpenSolaris on Amazon EC2
july 2008 by mcroydon
Wow, that's pretty awesome.
amazon
aws
opensolaris
s3
zfs
july 2008 by mcroydon
Eucalyptus
june 2008 by mcroydon
Roll your own in-house EC2. Nice! It's good to see ROCKS in use still.
amazon
aws
cloud
cluster
clustering
clusters
computing
ec2
firewall
grid
hosting
linux
management
open
open-source
programming
scalability
software
sysadmin
to-read
trend
xen
virtualization
june 2008 by mcroydon
Main Page - RunBlast
june 2008 by mcroydon
I wonder how far off we are from software sequencers where the hardware is "dumb" and just sends info off to an EC2 cluster. Hey, it worked for 56k modems...
bioinformatics
cluster
aws
amazon
ec2
blast
future
june 2008 by mcroydon
related tags
aa ⊕ access ⊕ admin ⊕ administration ⊕ agency ⊕ agile ⊕ ajax ⊕ algorithms ⊕ allegro ⊕ allegrograph ⊕ amazing ⊕ amazon ⊕ analysis ⊕ analytics ⊕ api ⊕ appengine ⊕ apple ⊕ application ⊕ apt ⊕ architecture ⊕ article ⊕ automation ⊕ aws ⊖ backblaze ⊕ backup ⊕ benchmark ⊕ benchmarks ⊕ beowulf ⊕ beta ⊕ bigdata ⊕ bioinformatics ⊕ blast ⊕ block ⊕ blog ⊕ business ⊕ capistrano ⊕ carrier ⊕ cartography ⊕ cassandra ⊕ catapult ⊕ cdn ⊕ cheap ⊕ chef ⊕ clean ⊕ cli ⊕ cloud ⊕ cloud-computing ⊕ cloudcomputing ⊕ cloudera ⊕ cloudfront ⊕ cloud_computing ⊕ cluster ⊕ clustering ⊕ clusters ⊕ code ⊕ codecs ⊕ colo ⊕ command ⊕ command-line ⊕ commandline ⊕ company ⊕ compression ⊕ compsci ⊕ computer ⊕ computers ⊕ computing ⊕ configuration ⊕ configuration_management ⊕ console ⊕ consulting ⊕ conversion ⊕ convert ⊕ cool ⊕ corpus ⊕ cost ⊕ couchdb ⊕ crawl ⊕ crowdsourcing ⊕ css ⊕ cuda ⊕ curl ⊕ custom ⊕ data ⊕ database ⊕ databases ⊕ datamining ⊕ dataset ⊕ datasets ⊕ datastore ⊕ db ⊕ dc ⊕ debian ⊕ debugging ⊕ demand ⊕ deploy ⊕ deployment ⊕ design ⊕ dev ⊕ developer ⊕ development ⊕ disk ⊕ dist ⊕ distributed ⊕ distributed-computing ⊕ distribution ⊕ diy ⊕ django ⊕ dna ⊕ dns ⊕ documentation ⊕ download ⊕ drive ⊕ ebs ⊕ ec2 ⊕ ecs ⊕ elastic ⊕ elasticmapreduce ⊕ email ⊕ emr ⊕ encode ⊕ encoder ⊕ encoding ⊕ encyclopedia ⊕ engineering ⊕ engineyard ⊕ erlang ⊕ etech ⊕ expenses ⊕ export ⊕ ezra ⊕ failure ⊕ faq ⊕ ffmpeg ⊕ file ⊕ filesystem ⊕ firefox ⊕ firewall ⊕ flash ⊕ flv ⊕ foss ⊕ framework ⊕ freebase ⊕ fuse ⊕ future ⊕ gae ⊕ geek ⊕ gem ⊕ genetics ⊕ genome ⊕ genomics ⊕ geo ⊕ geowanking ⊕ gis ⊕ github ⊕ google ⊕ gov ⊕ gpu ⊕ graph ⊕ graphdb ⊕ grid ⊕ guardian ⊕ gui ⊕ hack ⊕ hacks ⊕ hadoop ⊕ hard ⊕ harddrives ⊕ hardware ⊕ health ⊕ hive ⊕ hosting ⊕ how-to ⊕ howto ⊕ hpc ⊕ ia ⊕ image ⊕ images ⊕ import ⊕ impressive ⊕ index ⊕ information ⊕ infrastructure ⊕ inspiration ⊕ interesting ⊕ interface ⊕ internet ⊕ interview ⊕ intridea ⊕ io ⊕ isv ⊕ it ⊕ iteration ⊕ javascript ⊕ jaylinks ⊕ journalism ⊕ large ⊕ learning ⊕ lessons ⊕ library ⊕ license:gpl ⊕ license:gplv3 ⊕ light ⊕ line ⊕ linux ⊕ load ⊕ local ⊕ log ⊕ logs ⊕ lucene ⊕ mac ⊕ macfuse ⊕ mail ⊕ management ⊕ map ⊕ map-reduce ⊕ mapnik ⊕ mapping ⊕ mapreduce ⊕ maps ⊕ mashup ⊕ memcached ⊕ merb ⊕ mercurial ⊕ messages ⊕ micropayments ⊕ migration ⊕ mobile ⊕ mrjob ⊕ my ⊕ mysql ⊕ netflix ⊕ news ⊕ newsletter ⊕ nlp ⊕ nosql ⊕ notification ⊕ nvidia ⊕ ohai ⊕ on ⊕ ondemand ⊕ online ⊕ ontology ⊕ open ⊕ open-source ⊕ open.source ⊕ opensolaris ⊕ opensource ⊕ open_data ⊕ optimisation ⊕ optimization ⊕ osx ⊕ outage ⊕ packaging ⊕ panda ⊕ payment ⊕ performance ⊕ perl ⊕ persistence ⊕ pig ⊕ platform ⊕ plugin ⊕ postgis ⊕ postgres ⊕ postgresql ⊕ price ⊕ pricing ⊕ problem ⊕ programming ⊕ project ⊕ projects ⊕ provider ⊕ publicdata ⊕ pubsub ⊕ puppet ⊕ python ⊕ qa ⊕ queue ⊕ rackspace ⊕ raid ⊕ rails ⊕ rdf ⊕ recommendation ⊕ reduce ⊕ reference ⊕ research ⊕ resources ⊕ rest ⊕ ror ⊕ rpm ⊕ ruby ⊕ rubyonrails ⊕ s3 ⊕ saas ⊕ scalability ⊕ scalable ⊕ scaling ⊕ scalr ⊕ science ⊕ scraping ⊕ script ⊕ scripting ⊕ search ⊕ semantic ⊕ semantic-web ⊕ semanticweb ⊕ semweb ⊕ server ⊕ servers ⊕ service ⊕ services ⊕ sharing ⊕ shell ⊕ shopping ⊕ similarity ⊕ simpledb ⊕ smtp ⊕ smush ⊕ sneakernet ⊕ sns ⊕ soa ⊕ social ⊕ software ⊕ space ⊕ spam ⊕ sparklines ⊕ sparql ⊕ sqs ⊕ stamen ⊕ startup ⊕ statistics ⊕ status ⊕ storage ⊕ store ⊕ strategy ⊕ stream ⊕ streaming ⊕ subscription ⊕ subversion ⊕ svn ⊕ sysadmin ⊕ system ⊕ t ⊕ tech ⊕ techniques ⊕ technology ⊕ testing ⊕ tips ⊕ to-read ⊕ todo ⊕ tool ⊕ tools ⊕ toread ⊕ traffic ⊕ transcoding ⊕ transfer ⊕ transportation ⊕ trend ⊕ trendingtopics ⊕ trends ⊕ triplestore ⊕ tutorial ⊕ tutorials ⊕ ubuntu ⊕ ui ⊕ uk ⊕ upload ⊕ uploading ⊕ utilities ⊕ utility ⊕ vc ⊕ vcs ⊕ versioning ⊕ via:pskomoroch ⊕ video ⊕ virtualization ⊕ web ⊕ web-service ⊕ web-services ⊕ web2.0 ⊕ web3.0 ⊕ webapp ⊕ webdesign ⊕ webdev ⊕ webhosting ⊕ webservice ⊕ webservices ⊕ widgets ⊕ wikipedia ⊕ work ⊕ workflow ⊕ wwdc ⊕ xen ⊕ xfs ⊕ xml ⊕ yahoo ⊕ yelp ⊕ zfs ⊕Copy this bookmark: