Datawarehouse Appliance

It's not often you are told about something too good to be true - which then bears up under examination. However, I think I've just met such a product.

What is it? An intelligent data warehousing appliance that integrates storage hardware, database and DBA function for very large databases - half terabyte and up - which are active (i.e., used for near real-time online analyses, with some updates). It's non-invasive - access is through ODBC or JDBC, using standard SQL - you can use Cognos, Business Objects etc without changing the access code. We're looking at 10-50x the performance of a conventional database solution at a fraction of the cost (but we are talking high-end - entry price is about $700k - although third-party ASP "rental" options are available for smaller players). The initial price is a bit of a red herring - IBM and Teradata quickly drop their prices faced with Netezza, apparently - but Netezza's "cost of ownership" advantage is real.

MORE...

Why hasn't this been done already? The model is well known (basically, massively parallel processing for mostly-read queries - it won't work so well for transaction processing, which forces serialisation) and, in fact, similar appliances are current (see Conformative's XML appliance - ADA Trends, Jul/Aug 2004). IBM in particular has the hardware, software and understanding of the parallel processing model to do what Netezza has - but it addresses data warehousing in a DB2/Informix context, by throwing clever indexing and CPU cycles at it (and data volumes are growing faster than Moore's Law says CPU power is). NetApp also has products that seem not dissimilar. So, what is Netezza's key advantage?

Well, according to Jit Saxena, CEO of Netezza, it's the silo mentality in the legacy players. Netezza integrates database design and hardware design, in a way it's competitors don't - but, for example, IBM's hardware designers and database designers are sometimes barely on the same planet, he thinks.

OK, so why do I believe that this "revolutionary product" is real and may well revolutionise data warehousing? Well, for a start, I think that database design has stood still for years (see Blog, passim) - the database companies have become complacent. Then, Netezza's background is reassuring - Jit is one of the founders of Applix, his story about parallel processing makes sense, the database in his appliance is built up from an OpenSource Ingres clone "Postgres" (and Ingres is a good model). Finally, some data warehouse technologies do run out of steam at the terabyte scale - while RFID and compliance initiatives are making real demands on them.

Netezza has some impressive customers in the areas where you'd expect legacy warehousing to hit problems - telco applications at Orange, RFID and stock loss control at a conservative American retail chain, a couple of customers ASPing data warehousing etc. They claim that all but one of their customers has come back for more - and they sell on "proof of concept" anyway.

The possible downsides? Well, Netezza wins or loses on its customer focus and needs to manage growth carefully - but its management team seems to know what it is doing. IBM is almost certainly technically capable of developing a similar appliance - and it fits well with grid computing - but perhaps it's too committed to DB2 (the nearest thing to the Netezza appliance from IBM today is possibly the iSeries, with an integrated database "rather like" DB2). Moreover, Oracle and the others could bad mouth Netezza, which wouldn't affect sales to people that really need its solution but might impact general acceptance. Oh, and database integrity or something might be broken on Netezza, but since it sells through proof of concept, this is unlikely.

Netezza is a young company and managing its growth carefully, so you are unlikely to see a lot of its marketing for a time. It sells largely by word-of-mouth to people that know they have a problem, and its typical sale is about £1 million. Its web site is http://www.netezza.com/