taken as our search engine crawled the Web.
The web site itself may have changed. You can check the current page or check for previous versions at the Internet Archive. Yahoo! is not affiliated with the authors of this page or responsible for its content.
Engineers Comparison Testing Whitepaper 11-07




The Engineers Guide to Comparison Testing












Intertek Testing Services NA, Inc
70 Codman Hill Road, Boxborough, MA 01719

icenter@intertek.com
800-WORLDLAB www.intertek-etlsemko.com




The Engineers Guide to
Comparison Testing




www.intertek-etlsemko.com/performance


1
If the results of comparison testing are to be credible and
useful to engineers, it is essential that the tests are specified,
managed and documented as objectively as possible. Texas
Instruments can attest to the value of this approach.
Introduction

If youve been shopping recently for a new car or a flat-screen TV, theres a good
chance that youve sought out side-by-side comparisons of different makes and
product types in consumer publications or perhaps on e-commerce Web sites. Given
the abundance of data now widely accessible via the Internet, consumers make
more of their buying decisions with hard comparative data rather than always
relying on a handful of loose facts and second-hand anecdotes.
The best product comparisons are those that mix and match generous
lists of product features, performance specifications and price points.
But some of those sites have a characteristic that most consumers
prize even more: their independence from the vendors whose
products they describe.
As an engineer, you place a similar premium on detailed and unbiased
data. You have to do so; these days, you have far less room to make
technology sourcing decisions that could imperil a new products
immediate success. Your company faces increasing pressure from
competitors half a world away; customers expectations keep climbing and
shareholders are clamoring for stronger revenue growth and profitability.
As such, manufacturers have to employ new tactics to stand out from the clutter.
Its not just a question of amassing facts on product features and performance that
the marketing team can use in its new ads or trade-show announcements. Its also
crucial that you know how your offerings compare to industry benchmarks and
competitors offerings. Armed with objective data, you can pilot and rapidly iterate
product design improvements, help minimize warranty issues by resolving problem
areas before the market uncovers them, plan next-generation products more cost-
Its crucial to know
how your offerings
compare to industry
benchmarks and
competitors offerings.

Good, better, best.



The Engineers Guide to
Comparison Testing




www.intertek-etlsemko.com/performance


2
effectively, and improve innovation efficiencies over the long term. Comparison
testing programs can provide you with this critical information.

Overview of Comparison Testing

The fundamental objective of comparison testing is to be objective. It provides an
unbiased, unambiguous means of determining how several similar products perform
related to their intended function. Third-party comparison testing is used in a wide
variety of industries, gauging features and performance of batteries, dishwashers,
refrigerators, electronics and a host of other products.
A comparison test regimen is based on the sponsoring producers unique goals and
objectives and can vary from technical and objective to practical and subjective, or it
could involve a head-to-head comparison to determine ease of use, durability or
safety.
A test program on food freezers, for example, may include:
A review of several freezers features and of their manufacturers
performance claims
Tests to determine the accuracy and stability of temperature at varying
ambient temperatures and loads
Energy consumption tests at varying ambient temperatures and loads
Freezing tests to determine loads that can be safely frozen
Thermal tests to see how long the appliance will maintain cold when power fails
User tests to determine ease of use
Listing and ratings of features which make an appliance easier or more
difficult to use when compared to others.
After completion of the requisite tests, scores can be applied to all features to arrive
at an overall score. This score may be weighed against the products initial cost to
pinpoint a value for the money rating. With such information, sound
recommendations can be made on product enhancements or on the products
conditions for use.



The Engineers Guide to
Comparison Testing




www.intertek-etlsemko.com/performance


3
Many large manufacturing organizations have their own in-house test facilities for
gauging product performance. But a renewed emphasis on sticking to core
competencies means that more and more corporations are outsourcing their
comparison testing activities with positive outcomes.


How Texas Instruments Put Comparison Testing to Work

When Texas Instruments Inc. had pioneered a new display technology in 1987,
manufacturers of display products greeted the innovation with enthusiasm. Video
and graphics produced by the DLP® optical semiconductor circuitry are sharper
because the fully digital DLP chip minimizes the gaps between pixels in an image.
Essentially, a DLP semiconductor is a highly sophisticated light switch: To produce
grayscale images, the chip contains a rectangular array of up to two million hinge-
mounted microscopic mirrors that tilt toward the light source in a DLP projection
system (ON) or away from it (OFF), creating a light or dark pixel on the
projection surface. To add color, the white light in a DLP projection system passes
through a color filter as it travels to the surface of the chip; this filters the light into
red, green and blue, from which a single-chip DLP projector can create at least 16.7
million colors.

TI had gone on to achieve striking success with its DLP products in a range of optical
products from HDTV and digital cinema projectors to small portable digital
projectors. It was in the portables segment where TI had cornered more than half of
the available market worldwide since its DLP data projector product launched in
1998.
To better defend the differentiating advantages of its DLP offering, TI opted to run
independent product tests that would gauge the performance over time of DLP and
LCD technologies. The study was conducted at the Rochester Institute of
Technology's (RIT) renowned Munsell Color Science Laboratory. Texas Instruments
selected and purchased off the shelf the five LCD and two DLP projector models
used in the RIT tests. One sample of each model was evaluated and each sample ran
continuously. In all cases, RIT was testing for the LCD technologys tendency to
produce a phenomenon that TI calls color decay a gradual distortion over time
of the colors rendered by the projector.



The Engineers Guide to
Comparison Testing




www.intertek-etlsemko.com/performance


4
After the study was complete, TI wanted to expand the study to add testing that
included ambient temperature regulation along with a wider variety of cycle time
testing.
So early in 2005, TI commissioned Intertek, an independent testing organization, to
perform a second set of tests. Again, the data projectors containing both display
technologies were picked by TI and purchased on the open market by Intertek. Fifty-
four units were used in this study: three samples of each of the six models were run
on each duty cycle. The tests involved multiple brands of projectors in order to avoid
the possibility of false implications about a particular model or particular brand; as
such, no model numbers were used as identifiers. At the same time, TI did not
dictate the methodology for the tests.
Each of the models chosen was operated on three different duty cycles. The non-
stop duty cycle formed a baseline of data. Then, Intertek developed a 5.5 hours on,
2.5 hours off cycle to mimic the conditions typical in a classroom, along with a 1.5
hours on, 2.5 hours off cycle to reflect how a data projector is used for business
presentations. The projectors were maintained by Intertek and subjected to a
number of color and light measurements including light output, light output
uniformity, full-on/full-off contrast ratio, checkerboard contrast ratio, color
chromaticity, and color uniformity. All of the data for this study was independently
collected by Intertek. Still picture images were also displayed along with the various
test patterns and recorded using a still camera throughout the study.
The tests took about four months to set up and ran for more than a year, with
periodic reports to TI on what the interim results looked like.

The Outcome of the TI Tests

Halfway through the Intertek study, most of the