It is reasonably well known that the drives in your Exadata Storage Cells come in 2 possible flavours:
-
High Performance: 600 GB 15K RPM SAS
-
High Capacity: 3 TB 7.2K RPM SAS
So, you might ask yourself, can I take that trade off? That is 5 times the capacity, for only a reduction of 1/2 the RPMs. Storage is often purchased in terms of capacity rather than how much throughput or IOPS that it will actually deliver.
I have already talked about the OneCommand installation procedure, and you’ll note that Step 9 of this procedure is called RunCalibrate. This produces a measurement of the I/O performance of the hard drives you have purchased. The following is the command that is run:
[root@db01] /usr/local/bin/dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root cellcli -e calibrate force
I have some results to compare the running of calibrate on both 600GB High Performance drives and 2TB High Capacity drives, the 3TB being a fairly new innovation. Note the calibrate operation gives results for random read operations.
First up we look at Throughput, in MB per second:
This shows the figures that calibrate has generated on a per disk basis. For the High Capacity drives we can manage just slightly over 100MBPs per drive, while the High Performance come in around the 150MBPs mark. So about 50% increase in using High Performance over the High Capacity drives.
Now lets look at the IOPS results, also produced by the calibrate command:
Again each data point represents a single disk drive in an exadata storage cell. I think this really shows you the biggest difference in performance: You have under 200 IOPS per drive playing over 400 IOPS. Greater than x2 the difference.
Of course, you may argue that your flash assets are designed to handle the random I/O on exadata.
As always, there is a trade off to be made and all that extra capacity does come at a price. It is all too easy to be forced to buy storage based on capacity rather than performance, hopefully these numbers give some people out there the ammunition to tackle the bean counters.
DJ
/ February 7, 2012Hi,
I have a question regarding your test. Are you testing single drive or in a RAID group? Can you clarify?
jarneil
/ February 7, 2012Hello,
It is not in a RAID group. Each data point represents a physical drive, whether High Perf, or High Capacity.
jason.
DJ
/ February 7, 2012I’d be interested to learn more about result in a RAID group, since the RAID controller could become the bottleneck, rendering the single disk performance moot point.
I think you pointed out yourself that Exadata uses hardware RAID controller,
http://jarneil.wordpress.com/2011/10/30/exadata-uses-hardware-raid-you-bet-it-does/
A fair comparison of the two drives (high performance vs high capacity) in Exadata environment would be read/write IOPS at the RAID group level.
Luis Moreno Campos
/ February 15, 2012Jarneil,
There is another element that is not known and public, but HC (High Capacity) disks spend way less power than HP (High Performance) disks.
So again performance comes at an extra price since both types of cells have the same cost. In an multi-rack environment this difference in energy consumption might not justify going HP. And also the flash element plays a part here, depending on the type of use of Exadata (OLTP or DW/OLAP).
Cheers,
LMC
jarneil
/ February 17, 2012Hi Luis,
That is a very interesting point on the energy consumption. And yes, of course the flash may mitigate some of the performance degradation.
jason.