Wednesday, July 13, 2011

Are Amazon Spots Struggling?

[Update - heard directly from AWS and they report that the graph issue is a known bug and unrelated to the problem we were seeing getting m2.2xlarge instances. Hopeful we'll get to the bottom of the underlying problem]

[Update #2 - once we shared the info around time and type, AMZN was able to verify that there was a temporary issue involving only the m2.2xlarge, only in our region, and right in the window we happened to want them. Bit unlucky, but we should be all good now :-) ]

This seems pretty coincidental with the AWS announcement yesterday with per-AZ priced spot instances. At yieldbot we do some heavy lifting with clusters of m2.2xlarge servers, and haven't been able to get any spot instances for hours. Luckily we fall back to reserved instances if we have to, we'd just rather not pay the higher price. :-)

Here's a snapshot of the price history taken a few minutes ago for the m1.xlarge instances. Note that us-east-1b and us-east-1d seem to have stopped a while ago. us-east-1a and us-east-1c goes close to current, and our testing verified we could get us-east-1c instances.



Now check out the graph for the m2.2xlarge instances, taken about the same time. Availability seems to have fallen off a cliff hours ago, particularly in us-east-1c which appears to have not had availability for this instance size on the spot market for about 12 hours now.



The AWS health dashboard currently shows all green, which at some level is understandable. Spot instances are, after all, an as-available resource. The interesting thing here though is that the price shows no spikes, no price movement indicating an approaching shortage. The market didn't get expensive, there just stopped being a market.

Maybe AWS suddenly needed to add some instances to the pool of available reserved instances? Happily, reserved instances did come up and with no delays.

But we can't wait to start saving money with the spot instances again.

Anyone out there seeing the same or similar problem?