When the Hennessey Venom GT broke the world record as the fastest production car in 2014, it raised the engineering bar in the automobile industry. This world record wasn’t just about how fast a car could go; it was about breaking the barriers on what was possible. John F. Kennedy famously said, “We choose to go to the moon in this decade, then do the other things, not because they are easy, but because they are hard…” As you know going to the moon was not just about being first or doing what some deemed a waste of time and money, it was about the forward progress of knowledge for mankind.
The same can be said for appropriate record-setting endeavors in many differing industries. One appropriate example is the SPC-1 benchmark by the Storage Performance Council. As most often recognized, the SPC-1 benchmark is a vendor neutral endeavor for the storage industry. The Storage Performance Council says it “…. fosters the free and open exchange of ideas and information, and to ensure fair and vigorous competition between vendors as a means of improving the products and services available to the general public.”
Joining the Storage Performance Council (SPC) means that every submitted benchmark is not only audited and verified, but also peer reviewed. The governing authority makes sure there is no participating vendor that is gaming the process in its favor and so will only certify the results once a full audit has been conducted. It’s not until this point until the results and reports are publically shared for comment and review.
Documented, reproducible results and a peer review process are the hallmark of scientific advancement. The SPC methodology basically takes the scientific process and applies it to benefit the storage industry. Part of the peer review process is also to stimulate open debate and learning to advance the state of the art, a good example of the ‘to and fro’ is the recent debate which has ensued on caching and its impact on SPC results, for more on this, read.
DataCore Did What….?
Some may say that DataCore’s latest SPC-1 performance results are too good to be true or the process which DataCore followed the benchmark was not in line with previous vendors’.
As noted above SPC has been around for a number of years with many vendors participating (partial list includes a who’s who in the storage industry: EMC/Dell, HP, Hitachi, NetApp to name a few). It goes without saying that every vendor’s audited result had the opportunity to do better than exhibited. If a vendor hasn't participated in the SPC-1 benchmark, then there is nothing stopping them from participating and achieving the same results or maybe even better. To suggest we did something that others weren’t capable of is a flattering statement noting all the prior SPC-1 results that came from prominent industry leaders.
A vendor showing their own performance numbers without the oversight of a vendor neutral body like SPC doesn’t reflect a fair and balanced exercise. It’s not really about how much memory was used, or even the type hardware that was used in the benchmarking process, its really about your innovation to do better than everyone else, under the same rules of conduct for a standard process of measurement. So, if an audited and peered review process isn’t acceptable then the question remains, how do we all enable open sharing and trust for the advancement of storage innovation for our customers? In this I encourage everyone to join SPC to further develop and innovate, in order to hopefully achieve even better performance. This not only benefits those participating but all those that have participated in the past with measurements that can be shared instead of just talked about.
SPC Mission Statement: “The Storage Performance Council (SPC) is a non-profit corporation founded to define, standardize, and promote storage subsystem benchmarks as well as to disseminate objective, verifiable performance data to the computer industry and its customers.”