ial size=-1 color=black> Below is a cache of http://www.storageperformance.org/press/cmg2002v2_upd.pdf. It's a snapshot of the page taken as our search engine crawled the Web.
The web site itself may have changed. You can check the current page or check for previous versions at the Internet Archive. Yahoo! is not affiliated with the authors of this page or responsible for its content.
Storage Performance Council (SPC) Page: 1
December 10, 2002
Storage Performance
Council (SPC)
www.storageperformance.org
spcadmin@storageperformance.org Page: 2
December 10, 2002
Presentations and Speakers
The Storage Performance Council (SPC) update
Leah Schoeb (Sun Microsystems, Inc.)
What is SPC-1?
Mel Boksenbaum (Hitachi Data Systems Corporation)
What is an audited SPC-1 result, and how can I use it?
Walter E. Baker (Gradient Systems, Inc.)
How to use SPC-1 results and what they mean to you.
Randy Kerns (Evaluator Group)
Futures direction of the SPC and its benchmarks.
Peter Dreisbach (Adaptec, Inc.) Page: 3
December 10, 2002
The Storage Performance Council
(SPC)
Leah Schoeb Page: 4
December 10, 2002
Storage Performance is Important
Storage cost is 40-80% of total system
cost.
Storage bottlenecks will limit system
performance.
Customers embrace multi-vendor storage
solutions.
SANs allow heterogeneous shared
storage subsystems. Page: 5
December 10, 2002
Council Mission
The SPC is a non-profit corporation
founded in 1998 to
define, standardize, and promote the first
industry standard storage benchmarks
to disseminate objective, verifiable
performance data
to developers, product managers, and
customers of computer systems Page: 6
December 10, 2002
SPC Objectives
Drive performance improvement in the
computer systems industry for storage
Insure that customers can accurately
compare products in a multi-vendor world
Establish a level playing
field for all manufacturers
Publicize benchmark
results
Ensure accuracy and
authenticity of published
results
SPC
Results
Dissemination
Storage Industry
Performance Experts
New Benchmark
Development
Benchmark
Maintenance
Test Tools
Benchmark Kits
Performance
Analysis Tools
Results Validation
Peer Review
Audit
Service and
Support
SPC
Results
Dissemination
Storage Industry
Performance Experts
New Benchmark
Development
Benchmark
Maintenance
Test Tools
Benchmark Kits
Performance
Analysis Tools
Results Validation
Peer Review
Audit
Service and
Support Page: 7
December 10, 2002
SPC Membership
Fidelity Investments
HP
Spirent
StorageTek
Qlogic
Network Appliance
Data Storage Inst.
Fujitsu
Veritas
Yotta Yotta
Unisys
Seagate
Sun
Ideas International
Platypus
LSI Logic
Evaluator Group
NEC
IBM
Preferred Analysts
Intel
Hitachi (HDS)
DataCore
Dell
Cisco
Compaq
Florida Atlantic
Univ
3PARdata
Adaptec
Academic
Associate
Full
The companies in blue are new members for 2002. Page: 8
December 10, 2002
SPC Organizational Structure
Membership
Full Members
Associate
Members
Board of Directors
- Elects Steering Committee
- Approves Standards
- Reviews Compliance
Steering
Committee-
-Management
-Direction
- Administration
Administrator
Benchmark
Subcommittees
Standing Committees
Academic
Members
No Cost Page: 9
December 10, 2002
Organizational Update
Release of SPC-1 benchmark kit
Results publications by the membership started in
May
Public release of the kit AVAILABLE NOW
SPC-2 Workgroup is established
Early stages of defining the workload.
You have the changes to influence its future.
Increase in membership
4 new full members
11 new associate members
1 new academic member Page: 10
December 10, 2002
SPC-1 Benchmark
Mel Boksenbaum Page: 11
December 10, 2002
SPC Benchmark-1 Vision
Application Environment
SPC-1
OLTP
App -
g
App - f
App - e
App -d
Database
Ema
il
SPC-2
App -h
App
- i
SPC-x
Applications are
clustered based on
characteristics of
load presented to
the I/O Subsystem
SPC-x benchmark
will be defined for
each cluster,
starting with SPC-1 Page: 12
December 10, 2002
Figure 1: Response Time/Throughput Curve
0
10
20
30
40
50
60
0
1000
2000
3000
4000
5000
6000
Throughput in I/O Requests per Second
R
esp
o
n
s
e T
i
m
e
i
n
m
i
l
l
i
sec
o
n
d
s
Response Time
Sensitive
Applications
SPC-1 LRT
Throughput
Sensitive
Systems
SPC-1 IOPS
Increasing BSU Load
SPC Benchmark-1 Vision
Important Metrics
SPC-1 IOPS (Maximum Throughput)
SPC-1 LRT (Best Response Time @ lite load)
Price
Capacity
Data Protection Page: 13
December 10, 2002
SPC Benchmark-1 Vision
Metric Test Components
& Repeatability & Data Persistence

Time
I
/
O
R
e
qu
est
Thr
o
ug
hp
ut
Ramp
Up
Sustainability
Test Phase
Measurement
Interval
IOPS Test Phase
Measurement
Interval
Response Time Ramp
Test Phase
95% BSU load level
Measurement
Interval
90% BSU load level
Measurement
Interval
80% BSU load level
Measurement
Interval
50% BSU load level
Measurement
Interval
10% BSU load level
Measurement
Interval
Start
Up
SPC-1 LRT
SPC-1 IOPS Page: 14
December 10, 2002
SPC Benchmark-1 Vision
Repeatability Test
Steady Stat
e
Time
I/O Request
Ramp
Down
Ramp
Up
SPC-1 IOPS Measurement
Interval
Steady State
Ramp
Up
SPC-1 LRT Measurement
Interval
Repeatability Test
Phase #1
Steady State
Ramp
Down
Ramp
Up
SPC-1 IOPS Measurement
Interval
Steady State
SPC-1 LRT Measurement
Interval
Repeatability Test
Phase #2 Page: 15
December 10, 2002
SPC Benchmark-1 Vision
Data Persistence Test
I/O Request
Write Random I/O
Time
Read and verify data
Shut Down
BC Page: 16
December 10, 2002
Structure of SPC-1 I/O
ASU-1
45%
ASU-2
45%
ASU-3
10%
ASU-1 Streams
ASU-2 Streams
ASU-3 Stream
S-2
28%
S-3
S-4
21%
S-1
S-2
S-3
S-1
28%
I/O Stream Generator
I/O Command Generator
I/O Requests
I/O Requests
I/O Requests
I/O Command Completion
Statistics Collection
S-1 Page: 17
December 10, 2002
Running The SPC-1 Benchmark
Tested Storage Configuration (TSC)
Must conform to the Specification
LV
ASU-2
ASU-3
Total Application Storage Unit Capacity
45%
45%
10% Page: 18
December 10, 2002
Running The SPC-1 Benchmark
Tested Storage Configuration (TSC)
Must conform to the Specification Page: 19
December 10, 2002
About The Benchmark Kit
SPC-1 benchmark kit includes:
Benchmark executables & scripts
Solaris
Windows 2000
AIX
HP/UX
Linux
Results summary & FDR Production tools
Documentation Page: 20
December 10, 2002
SPC-1 Results:
Audited & Official, what does that mean
and what can I do with it?
Walter E. Baker Page: 21
December 10, 2002
SPC-1 Results Validation
The SPC-1 Results Validation Process
Successful completion of the three components listed
below produces an official, approved SPC-1
Benchmark Result.
Required Audit
Certifies information for the
FDR is accurate and authentic.
Full Disclosure Report (FDR)
Documents the configuration and
activities that produced the result.
SPC Peer Review
Technical review by
SPC member companies. Page: 22
December 10, 2002
SPC-1 Results Validation
Required Audit
Certifies the information produced for the FDR is
accurate and authentic based on the specification.
Benchmark Configuration (BC)
Benchmark measurements and results production
Pricing and availability
Audits are provided by the SPC Auditing Services
The SPC Administrator Gradient Systems, Inc.
Two type of audits:
On-site
Remote Page: 23
December 10, 2002
SPC-1 Results Validation
Required Audit
On-Site Audit: The Auditor is physically present
during benchmark execution and information
collection.
Required when a Test Sponsor has failed a Remote Audit or
submitted a non-compliant result.
Remote Audit: The Auditor is not physically present.
Results files (digitally signed) and other required materials are
submitted to the SPC Auditing Service.
The SPC Auditing Service ensures completeness, authenticity,
and accurateness. Page: 24
December 10, 2002
SPC-1 Results Validation
Full Disclosure Report (FDR)
Documents the results, procedures, configuration and
equipment used to produce the benchmark result.
The FDR will allow replication of the benchmark
result given appropriate documentation and products.
Also includes:
An audit certification letter issued by the SPC Audit Service
A Letter of Good Faith issued by the Test Sponsor
» An assurance of fidelity and candor even if not explicitly
required by the current benchmark specification. Page: 25
December 10, 2002
SPC-1 Results Validation
SPC Peer Review
A 60-day period