DCI DATA WAREHOUSE CONFERENCE, ORLANDO, Fla., Feb. 25 /PRNewswire/ --  Red
Brick(R) Systems, Inc. (Nasdaq: REDB), The Data Warehouse Company(TM), today
proposed new real world benchmark criteria for evaluating the performance of
relational databases (RDBMSs) used for data warehousing, including data
marts, data mining and online analytical processing (OLAP).
    Red Brick stated that the most frequently referenced benchmark for
measuring decision support, the Transaction Processing Performance Council
Benchmark D (TPC-D), was irrelevant because it contained transaction
processing and other criteria not suitable to real world data warehousing.
In addition, Red Brick stated that TPC-D had been invalidated not only by
leading industry analysts, but also by the disclaimer in the specification
document's own preamble.
    "We commend the TPC for addressing the issue of performance benchmarking
for decision support," said Steve O'Brien, director, product marketing for Red
Brick.  "For several years the TPC and major RDBMS vendors have established,
refined and battled over benchmarks A, B and C for online transaction
processing (OLTP), and they have been effective in making performance a
critical success factor for OLTP.
    "However, because of their focus on transaction processing, neither the
TPC nor the OLTP RDBMS vendors understand the specialized performance
requirements for data warehousing, and they only confuse the industry by
advocating benchmark D," O'Brien said.
    "Because of our exclusive focus on data warehousing for the past ten
years, Red Brick understands performance is everything for data warehouse
success," said O'Brien.  "With the empirical knowledge gained from our
customers and their successful production data warehouse applications, we are
proposing 'real world' benchmark criteria for evaluating data warehouse RDBMS
performance."

    Highlights of Red Brick's Proposed Data Warehousing Benchmark
    The following table summarizes the performance criteria appropriate for
comparing various database technologies with regards to their suitability for
data warehouse applications:

    LOAD
    Fast batch updates
    Indexing
    Format conversion
    Integrity checking
    Aggregate building
    Metadata update
    Recoverable process

    QUERY
    Response times
     * acceptable, consistent
     * simple to complex
     * small to large data sets
     * few to many concurrent users
    Scalable performance
    Efficient use of hardware
    Simple or complex schemas

    ANALYSIS
    Dimensional analysis
    Summary-to-detail drill
    Business analytics
    Advanced functionality
     * optional
     * fully integrated
     * no negative performance impact on queries
     * high performance

    Red Brick already is working with independent consultants to define a
suite of benchmark tests that reflect these core requirements for data
warehouse success.
    Red Brick will substantiate its performance leadership by contracting for
independent testing of Red Brick Warehouse and publishing the results of the
evaluations beginning in the second quarter of 1997.
    Red Brick's proposed data warehousing evaluation criteria, Performance
Criteria for Data Warehouse Databases, is available on Red Brick's World Wide
Web site -- http://www.redbrick.com -- under the "Performance is Everything"
section, or by calling 800-777-2585.
    Red Brick invites anyone who visits its WWW site to post comments or
suggestions on the proposed benchmark in the "Feedback" section.

    Industry Analysts Conclude TPC-D is Not Meaningful
    Gartner Group reported in a June 21, 1996 Research Note (TV-000-147):
    "...we offer a note of caution regarding TPC-D benchmarks.  These must be
interpreted with great discrimination.  They are laboratory tests that do not
provide any opportunity for meaningful extrapolation to a production
environment, and most of the results so far are based on a single batch
workload stream."
    In an even earlier Research Note (June 28, 1995; TU-530-1639), Gartner
Group reported its assessment of the potential usefulness of the TPC-D
benchmark:  "Bottom Line: The TPC-D transaction benchmark will be eagerly
anticipated but will have minimal impact on comparative vendor positioning
before mid-1996.  The TPC's desire to create wide benchmark coverage is
commendable, but probably only for approximate reference comparisons, rather
than for statistically meaningful comparisons."
    META Group, in a Feb. 10, 1995 META Delta (#392), had this to say about
TPC-D:  "...standard benchmarks will not contribute much to DSS-focused
database evaluations, because each user has radically different query
processing needs.  In particular, the TPC-D benchmark will fail to add
significant value.  The 'D' benchmark does have some interesting queries, but
the TPC's practice of allowing a single 'result' to be published (some
databases will execute some of the queries extremely well and others poorly)
will make TPC-D results irrelevant for most users.  Users will have to base
selections on a detailed understanding of their target database schema and the
queries that will be used to analyze their data."

    Transaction Processing Performance Council Disclaimer about TPC-D
    The preamble to TPC-D, Standard Specification, Revision 1.2 dated Nov. 9,
1996 contains the following disclaimers:
    "Despite the fact that this benchmark offers a rich environment
representative of many decision support systems, this benchmark does not
reflect the entire range of decision support requirements.  In addition, the
extent to which a customer can achieve the results reported by a vendor is
highly dependent on how closely TPC-D approximates the customer application.
The relative performance of systems derived from this benchmark does not
necessarily hold for other workloads or environments.  Extrapolations to any
other environment are not recommended.
    "...TPC-D should not be used as a substitute for a specific customer
application benchmarking when critical capacity planning and/or product
evaluation decisions are contemplated."

    About Red Brick Systems, Inc.
    Red Brick Systems, Inc., based in Los Gatos, Calif., is the industry's
fastest growing database company and the leading provider of high-performance
relational database products for data warehousing.  Its flagship product, Red
Brick Warehouse 5.0, is the world's fastest and most scalable relational
database for data warehousing, including data marts, online analytical
processing (OLAP), and data mining.  Red Brick customers are successful
because the company's high-performance products and service enable more users
to analyze more data and make better decisions faster.
    Information about Red Brick and "Performance is Everything" is available
on the World Wide Web at http://www.redbrick.com.  News releases and corporate
background information may also be retrieved by fax.  For an index of
available materials, please call 800-758-5804, and enter PIN #105756.
    NOTE:  Red Brick and Red Brick and logo are registered trademarks, and The
Data Warehouse Company and Performance is Everything are trademarks, of Red
Brick Systems, Inc.  All other trademarks and registered trademarks are the
property of their respective holders.

Link to this page: