DailyTech - EBay Improves Performance, Saves Space and Power Moving to SSDs

Submit News

Hardware EBay Improves Performance, Saves Space and Power Moving to SSDs
Shane McGlaun (Blog) - August 3, 2011 1:27 PM

24 comment(s) - last by quiksilvr.. on Aug 4 at 12:17 PM

eBay cuts storage power consumption by 78% with SSD move

Of all the websites online, the auction site eBay likely has some of the highest traffic levels with the huge number of items being bought, sold, and browsed at any given time.

When it came time to improve the performance of its I/O storage system eBay starting looking at its options. ComputerWorld reports that eBay was having issues meeting the I/O storage demands of its virtual machine (VM) deployments and ultimately swapped out some of its HDD-based arrays with SSD-based arrays.

EBay ended up replacing 100TB of HDD-based storage in a single year with the SSD-based storage and saw some impressive gains and improvements from the swap. The reduction in storage rack space between the old solution and the new SSD solution was 50%. The reduction on power use was 78% and in a large data center, the reduction in power use can mean huge savings on electric bills.

While a reduction in the number of storage racks and a big reduction in power consumption are both great things, the performance is what eBay was after. The time to roll out a new VM is now five minutes compared to 45 minutes on the old system according to eBay.

The storage that eBay went to was Nimbus S-class SSD systems with HALO storage operating system. Michael Craft is manager of QA systems at eBay. Craft said, "One rack [of SSD storage] is equal to eight or nine racks of something else."

Right now, the QA division at eBay has 4,000 VMware ESX virtual servers spread across about 200 computers with half of the physical servers attached to the new SSD gear for primary storage. EBay is expecting to boost the use of the SSD storage tech down the road. The Nimbus gear replaced NetApp and HP 3PAR storage systems that used 15,000rpm HDDs. The new SSD storage systems cost about $1,000 per usable terabyte of storage.

Craft said, "This is a pure play in our virtualization stack, but we're looking to expand that as it fits other people's needs. We were having challenges...keeping our space in check by not taking up half the data center with [hard drive] spindles. With the SSD, literally, I've been able to eliminate racks of gear."

Comments

Threshold

Username
Password
remember me

This article is over a month old, voting and posting comments is disabled

"The new SSD storage systems cost about $1,000 per usable terabyte of storage"

By rvd2008 on 8/3/2011 3:42:24 PM , Rating: 2

What? Nimbus website tells quite a different story
Acquisition Cost (10TB) $133,120
Not including support etc.
$13,312 per 1TB

http://www.nimbusdata.com/products/costanalysis.ht...

RE: "The new SSD storage systems cost about $1,000 per usable terabyte of storage"

By SSDMaster on 8/4/2011 8:14:50 AM , Rating: 2

Maybe he meant to put $1000 more? idk...

Parent

RE: "The new SSD storage systems cost about $1,000 per usable terabyte of storage"

By quiksilvr on 8/4/2011 12:17:26 PM , Rating: 2

Actually its a $1000 LESS per TeraBYTE when compared to HDD:

http://www.nimbusdata.com/products/costanalysis.ht...

Parent

Specs

By chmilz on 8/3/2011 2:08:58 PM , Rating: 2

Instead of asking stupid questions, check out the facts on their website. There, in plain sight. I'll let someone else do the math to figure out how they save space and whatnot.

http://www.nimbusdata.com/products/s-class.html

Azure

By Smilin on 8/3/2011 3:30:34 PM , Rating: 2

I bet they saved even more by getting rid of the servers altogether...

http://investor.ebayinc.com/releasedetail.cfm?Rele...

cleaning house

By undummy on 8/3/2011 4:12:05 PM , Rating: 2

They are probably just cleaning house. Time to get rid of all that decade old IBM, EMC, HP, 3par, DataGeneral, Compaq, Sun, HDS, and whatever equipment has been acquired since they were founded. Their tape storage and floppy disks were getting long in tooth.

Nimbus- beware!

By nelyboy27 on 8/4/2011 12:11:32 AM , Rating: 2

Buyer beware...

http://scaledatacom.com/blog/?p=113

I assume they used more hard drives than necessary capacity

By Calin on 8/4/2011 2:37:06 AM , Rating: 2

In order to improve response time, eBay probably used only a part of the usable capacity of the magnetic hard drives. That could mean an SSD could be even price-competitive per GB of used storage (especially considering the only 15,000 rpm drives are high-end, relatively small capacity enterprise drives priced accordingly).

Improved performance ...

By Gondor on 8/4/2011 7:02:37 AM , Rating: 2

They'd improve performance much more if they dumped their lousy page design and reverted back to design they used a decade ago. I was on ISDN (dialup) back then and their site was far more responsive than it is today (on DSL/FTTH). Things like item lists render much more slowly and there is more data to transfer due to the lame page design. Why throw more money - my money - at the problem if there was no need to create the problem in the first place ?

Reduction of space?

By Ralos on 8/3/11, Rating: -1

RE: Reduction of space?

By Flunk on 8/3/2011 1:53:55 PM , Rating: 2

3.5" hard disks vs SSDs on racks, without the metal cases they put the consumer drives into? smaller power supplies as well. I can't see how they couldn't have saved space.

Also, they are likely replacing 10,000 RPM drives which are lower density but higher performance.

Parent

RE: Reduction of space?

By Mech0z on 8/3/2011 1:57:30 PM , Rating: 2

It says in the article its 15k RPM drives and those have a maximum capacity of 600GB atm, might have been much lower when Ebay bought the servers in the first place.

Parent

RE: Reduction of space?

By therealnickdanger on 8/3/2011 2:28:46 PM , Rating: 2

Often times, massive arrays will shortstroke HDDs in order to improve overall performance. This results in reduced storage space. For all we know, they were using 600GB drives shortstroked to 100GB. Either way, I imagine the company replaced much of the other hardware needed to take advantage of the SSD performance. I know when we upgraded our traffic camera storage system with new hardware, capacity and performance increased while the number of full racks decreased. Half of our "state of the art" server room, circa 2004, is now empty.

Parent

RE: Reduction of space?

By GuinnessKMF on 8/3/2011 1:59:50 PM , Rating: 3

They're likely talking the entire storage systems power usage, meaning the machine hosting the drives. As for space savings, they were moving from 15,000 enterprise drives, so yes, most likely they were not 2 or 3TB drives, they couldn't move to 7.2k or 5.4k high capacity consumer drives and even maintained performance. 1 and 2TB SSDs do exist in 3.5" formats (possibly 2.5" but I haven't seen them).

Parent

RE: Reduction of space?

By tastyratz on 8/3/2011 4:02:56 PM , Rating: 2

I would be curious to see what the ROI is based on power usage. 78% storage power usage reduction sounds awesome, but I bet the electric roi shows it a pointless savings if that is the goal.

1tb exists in 2.5 format, but I do not know if it does in enterprise grade.

consumer drives have no business in an array for ebay or other servers of that caliber. Downtime is absolutely critical.

Parent

RE: Reduction of space?

By DanNeely on 8/3/2011 9:06:48 PM , Rating: 2

If you're just looking at costs in terms of wall power it definitely won't; but data centers can be power/cooling limited as easily as rack space limited. Between some data centers including power consumption as a significant fraction of their pricing, and the ups/generator capacity needed to keep them running for a few days or longer without grid power total savings can be fairly large.

A few years ago when myspace started swapping HDD servers for 1st generation SDD servers their overall savings (power + rackspace) were able to make the upgrades pay for themselves in roughly a year. Since SSDs have increased in speed faster than HDDs, for any IO bound system the costs should be recovered even faster now.

Parent

RE: Reduction of space?

By Etsp on 8/3/2011 2:01:50 PM , Rating: 2

As stated in the article, the goal was first and foremost performance. Saving space was a side-benefit. Also keep in mind that many enterprise SSDs are not mounted into 2.5" or 3.5" bays, but are PCB boards.

Parent

RE: Reduction of space?

By Fritzr on 8/3/2011 2:08:32 PM , Rating: 2

They are replacing highspeed HDDs not necessarily high capacity.

1TB SSDs are available today. a 1.6TB SSD has been announced for the near future. Why wouldn't these be available to eBay to replace the 15k drives with lesser capacity per unit?
Even assuming the relatively affordable 512GB-640GB SSD, SSDx1=15kHDDx2 for the sizes that are easily found. The amount of space saved indicates they probably didn't opt for the max capacity now available in SSD :D

Parent

RE: Reduction of space?

By peterlws08 on 8/3/2011 2:10:36 PM , Rating: 2

When you choose storage you have to consider IO's and space.

They might have been able to get sufficient IO's (database performance) with half the racks.

Parent

$1000/TB ?

By name99 on 8/3/11, Rating: -1

RE: $1000/TB ?

By MrTeal on 8/3/2011 6:03:13 PM , Rating: 2

Really. You couldn't even buy 1TB of consumer level SSD storage for $1000.

Parent

RE: $1000/TB ?

By tng on 8/3/2011 6:25:32 PM , Rating: 2

Don't know if this is how they are looking at it or not, but a 58% power reduction in just the drive power could be part of it. Remember for every watt of heat that you have to pull out of the server room, it takes about 2 watts of electricity to run the AC, so their actual savings could be more, but it was not clear if that 58% number included that or not.

If you look at it long term, over the life of a HDD and a SSD that is a lot of money..... Also remember that every dollar that you spend, you have to sell average of $11 of services to earn back.

Parent

RE: $1000/TB ?

By Cypherdude1 on 8/3/2011 7:03:26 PM , Rating: 2

quote:
Right now, the QA division at eBay has 4,000 VMware ESX virtual servers spread across about 200 computers with half of the physical servers attached to the new SSD gear for primary storage.

Are you saying then, I cannot run eBay from my home PC with 6 - 3 TB Seagate hard drives? B^D

Parent