Benchmarks are like statistics – they are only relevant in context.
Take this new TPC-E benchmark from the Transaction Processing Performance Council (TPC) as an example.
If you were to read the gushing Microsoft Press Release you’d be forgiven for thinking that SQL Server 2008 topped out the list of all major database vendors in terms of price/performance. Here’s the quote:
Of course what they don’t mention is that when you look at ALL the results, the only databases being evaluated were SQL Server 2005 and SQL Server 2008 (as of 23 Sep 2008). At least we should be glad that SQL Server 2008 is better than 2005.
(Interesting to see how this has been pumped out by various news outlets.)
You may be interested in the ‘other’ TPC benchmark – the TPC-C – where there’s a different story in the results.
