Storage Performance Council (SPC) - FAQs

1. What is the SPC?

The Storage Performance Council (SPC) is a non-profit corporation founded to define, standardize, and promote storage subsystem benchmarks as well as to disseminate objective, verifiable performance data to the computer industry and its customers.
See:http://www.storageperformance.org/ and select "Our Mission"

2. Who are the members of the SPC?

There are currently 22 member companies of the SPC, most of whom are suppliers of storage system products.
See: http://www.storageperformance.org/ and select "Membership"

3. Why was the SPC formed?

The SPC was formed out of the confusion that existed in storage performance testing, that made it very difficult for customers to evaluate storage products. There have been common measures used, such as the "Read Cache Hits IOPs" and "Large Block Sequential Bandwidth" tests. However each vendor used their own version of these tests, so the data was not very compatible. Programs, such as IOMeter and IOGEN, have been used, but again the tests have not attempted to represent any form of application workload. They have been used to measure boundary conditions, and some limited single purpose tests, with each vendor using unspecified storage configurations. The SPC was formed by a key group of storage vendors to provide customers with a common measuring stick for the complex operations involved with storage products.

4. Where is the SPC going from the SPC Benchmark-1™?

The SPC has formed a workgroup to address a different class of workload, large block sequential demands, designated SPC-2. This workgroup is in the first phase of gathering traces of representative workloads of this class, to be analyzed The aim is to create another scaleable emulation that can effectively measure storage performance under such a workload.
See: http://www.storageperformance.org/ and browse through the Home page, as well as the "Recent Press" section.

5. What is the SPC Benchmark-1™?

The SPC-1 benchmark is a scaleable suite of tests that emulate the workload seen by a storage system in support of an On-line Application, such as an email server. There are defined relationships between three different areas of storage, and specific workloads applied to each of the areas. It includes a mixed set of read, read/write, and write workloads, at various block sizes.

6. What are the different tests within the SPC Benchmark-1™?

There is a two part Persistence test which confirms that the storage system being tested is non-volatile, and consistent in the storage and retrieval of data.
There is an extended run time Sustain test that measures the performance of the storage system over a three hour period.
There is a Ramp test that measures the performance at various loading levels, ranging from 100% down to 10%.
There is a two part Repeat test which provides confirmation of the performance at both the maximum level (100%) and the low level (10%).

7. What is the measure of performance provided by the SPC Benchmark-1™?

A related family of performance measures are provided by the benchmark, all of which together form the performance level for a storage subsystem.
The Average I/O Transactions per Second (SPC-1 IOPS™) provide a measure of the transaction processing rate the system can support.
The SPC-1 Price-Performance measure provides the cost per SPC-1 IOPS™ indicating the relationship between the performance and the cost of the system.
The Total Storage Space used in the test provides an indication of the size of the system.
The Protection Level provided by the system is included to guide the user in the degree of fault protection included with the supported rates.
The Average Response Time at the low level of demand (SPC-1 LRT™) provides an indication of the responsiveness of the storage to access demands.

8. How is the information made available to users and customers?

The SPC publishes both an Executive Summary and a Full Disclosure Report for every audited test configuration on their web site.
See: http://www.storageperformance.org/ and select "Benchmark Results"

9. How should a reader interpret the published results for a storage system?

The combination of a high SPC-1 IOPS™ rate and a low SPC-1 Price-Performance value, which has Space in the size range the customer is interested in, with the Protection level suitable to the application, should receive a very positive consideration by the customer.
Just having a high SPC-1 IOPS™ value, without consideration of the other factors limits the view by the reader, and it is recommended that all of the factors be considered as a set by all readers.
The workload is scaled by Business Scaling Units (BSUs), with each BSU approximating the workload of about 50 users. Each set of test results includes the BSU level at which the test workload was generated. This can provide the reader with a rough estimate of the users that a given storage system may support, but should not be taken as a hard and fast reference value. It is only a general guideline, for use in comparing the results presented by the different test sponsors.

---