Performance Multipliers for Data Stream Processing

Yes, star crossed in pleasure the stream flows on by
Yes, as we're sated in leisure, we watch it fly.

And time waits for no one, and it won't wait for me
And time waits for no one, and it won't wait for me.

Time can tear down a building or destroy a woman's face
Hours are like diamonds, don't let them waste.

Time waits for no one, no favours has he
Time waits for no one, and he won't wait for me.

- The Rolling Stones, Time Waits for No One (Jagger/Richards),
from the album, "It's Only Rock 'n Roll" (1973)

We've dedicated the last several postings to the Field Programmable Gate Array (FPGA) - a key performance multiplier in the NPS^® system architecture. Last time out I talked about the market growth of FPGAs as a mainstream technology in multiple applications settings outside of data warehousing.

Today, we'll dive in a bit into how FPGAs enable high performance at low cost in the NPS appliance, and what types of applications the technology may enable for the NPS in the future.

OK - So how does Netezza get a performance edge from FPGAs?
A critical element of Netezza's architecture is the implementation of direct-attach storage in a massively parallel array of query processing elements. Called Snippet Processing Units (SPUs), these query processing elements collocate CPU, memory and FPGA with each disk drive. The SPUs are arranged in an array that can be as small as several dozen or as large as nearly a thousand in today's NPS systems.

A critical component of overall data warehouse performance lies in the disk bandwidth that can be applied to a given problem and in turn, the level of processing horsepower that can be applied to that data. In short-hand terms, Netezza refers to its architectural approach as "bringing the query to the data." Rather than moving vast amounts of data across high-speed interconnecting (and sometimes non-blocking) networks as other systems do, the NPS system reduces the data to the information essential to the query as close to the disk source as possible.

The focus of the architecture is to enable streaming processing of the data: eliminating unneeded data as early as possible and processing the rest as rapidly as it can be read from the disk drives. That's where the FPGA comes in. The FPGA in a Netezza SPU has two primary roles.

In the first, it acts as the disk controller, controlling all of the disk read and write activities on the SPU.

In the second, the FPGA efficiently applies low-level database primitives, offloading significant work from other processing elements in the system. As table data streams from the disk on the SPU, the FPGA applies the transaction visibility list (only transactions that were current in the database at the start of the query are visible to it) and then applies the appropriate column projection and row restriction rules. Then only data that satisfies the visibility, projection and restriction rules is sent from the FPGA to the memory and CPU on the SPU for additional processing, if necessary.

Adding to the performance boost provided by the FPGA in general, another important system feature known as "Zone Map" is realized in a software module of the NPS system known as the storage manager. We think of Zone Maps as an anti-Index in Netezza, telling the system what data not to read. For each numerical column, the Zone Map can take advantage of any natural ordering of the data in the table (e.g., date, customer number, order number, etc.) and reduce the number of data blocks read in response to a query to only those required. For example, if a query were looking for information about transactions that took place between the beginning of September and end of October, the Zone Map function of the storage manager would direct the FPGA to read only those data blocks containing records from September or October, thereby eliminating the need to perform a full disk scan for each query.

The FPGA implements the read of the appropriate disk blocks and additionally filters and projects only data relevant to the query. This can improve query-processing rates by two or more orders of magnitude.

FPGA as performance multiplier: an example
As an example, consider the following simple SQL query:

In this example, the storage manager and FPGA would use Zone Maps to first limit the disk read to only those disk extents with dates of birth occurring in the three-month period of January through March 2000, rather than the full table. Then, when the data was read from the disk, the FPGA would further restrict the rows of data returned to those records within the three-month range and a zip code matching the query and finally, the column data projected to the memory and CPU would be limited to only state, gender and age information of each record. If the table in question contained 100 or more columns for each record, this could represent less than 3% of the column data. If one assumes the table in question contained birthdate information for just the last seven years, this would dramatically reduce the row-count of data delivered to memory/CPU as well - specifically by more than 25:1, or 3 months out 84.

Overall, for this example, the combination of Zone Maps with FPGA projecting and filtering of the data would result in just 0.1% of the full table data being sent to the memory and CPU for additional processing.

From this, you can see how the FPGA acts as a Performance Multiplier for query processing. Before a single CPU cycle or RAM memory location has been used, the FPGA has reduced the overall data required for processing by as much as multiple orders of magnitude.

And what does the future hold?
As suggested by Keith Underwood of Sandia National Labs, the price-performance and power efficiency look like they will enjoy an order of magnitude advantage over the 'x86' CPU technology roadmaps by the end of the decade. Using its performance and I/O advantages, FPGA vendors are already able to embed CPU core technology (Xilinx - "Embedded Processing" & DSP-FPGA.com - "FPGAs - Poised to play in embedded applications") directly inside an FPGA.

Projected FPGA Roadmap Capabilities Source: Composite of FPGA Vendors' Historical & Roadmap Data

We at Netezza fully expect the FPGA advantage to increase over time. Based on suppliers' and research technology roadmaps, by the end of the decade we are anticipating 5X enhancements in each of the following areas:

• cost
• available logic
• functionality per unit of power
• speed

Xilinx' Powerful Virtex2Pro FPGA Source: JPL/NASA Tech Brief, p. 12

The result will be extended, differentiated functionality introduced into current and/or future versions of FPGA technology, further increasing the price/performance and capability advantages of the NPS data warehouse appliance. Possibilities for expanded functionality include, but are certainly not limited to, in-line, streaming data compilation or encoding, advanced filtering and analytic logic operations ("Legacy FPGA Designs Can Be Migrated to Achieve Better Performance"); and even much more powerful pre-processing of query data by embedding CPU processing capabilities directly within the device ("FPGA Advances Pave The Way Toward True SoC Solutions"). If, how and when these may be manifest in the Netezza technology roadmap is still to be seen. However on the strength of the FPGA technology roadmap and the technology's significant benefit to the streaming processing needs of data warehousing, it's clear to us that the FPGA will continue to play a major role with Netezza for the foreseeable future.

The technology trends for high-performance systems is clear. In more and more industry domains ("In Praise of FPGAs"), low-power programmable logic devices are going to act as either performance accelerators or even the primary performance engine. By offering high performance, low power requirements and highly-flexible reprogrammability, the use of FPGAs promise to continue as a strong industry trend.

In short, we believe that the advantages that FPGA technology brings to the NPS system have 'legs'. We plan to continue to exploit those advantages for the benefit of our customers and don't intend to hide them "under a bushel" any longer.

Performance Multipliers for Data Stream Processing

“Time is but the stream I go a-fishing in.”  – Henry David Thoreau, (1817-1862) from Walden (1854)

We’re taking a little time to discuss an important facet of the NPS^® system architecture – the Field Programmable Gate Array (FPGA), which is a critical performance multiplier in the context of the NPS appliance.  Last time out I spent some time discussing what an FPGA is and what its general role is in stream-based processing.

|</center>

Today, I’d like to look at other mainstream applications of the FPGA and its growing role in the market.

Recent News About FPGA Adoption

The applications space for FPGA technology is expanding rapidly.  Nearly concurrently with my last posting, William Fellows of The 451 Group (subscription required) published an article discussing a segment of the FPGA applications market very different from data warehousing that virtually makes the case for this series postings.  Here are a few snippets of Mr. Fellows’ 28^th November article (emphasis mine):

|</center>

FPGAs in the Mainstream

It’s important to note that this is no small, specialized technology market, but one that is very much in the high-performance and consumer mainstream.  Recently Bryan Lewis, a vice president and chief analyst at Gartner Dataquest projected (“Gartner: FPGA/PLD market to grow 14 percent in '06”) that the market for FPGAs and PLDs will grow from $3.2B in 2005 to $6.7B by 2010 – including 14% growth in 2006 and another 18% in 2007.  Overall they project approximately a 16% compound annual growth rate through the end of the decade.

|</center>

Some of the Mainstream/Practical Uses for FPGAs

The FPGA has blossomed in recent years to take on a key role in driving cost-effective high performance in a broad sweep of applications (Altera, Lattice Semiconductor & Xilinx).  Here are just a few:

• consumer products such as DVD players, HDTV and plasma displays;

• mission-critical telecommunications and networking systems;

• medical x-ray and ultrasound imaging systems;

• real-time audio & video processing gear;

• high-speed event processing; and

• high-performance computing (HPC)And with the NPS data warehouse appliance from Netezza, FPGAs are also playing an important role in delivering high performance in low-power, cost-effective data warehouse systems.

Common Traits and Common Benefits

What the Netezza implementation and the great majority of the uses mentioned earlier share is that they deal with data streams; filtering or performing functions on data as it streams through the device at high speed.  The FPGA is able execute its functionality on the data, without interrupting the flow of data through them – in effect, adding an in-line performance boost.  In this way, they can greatly accelerate performance in these applications over “brute force” CPU-based processing.

Conversely, these systems tend not to rely on the FPGA for recursive algorithms or processing requiring access of data from cache, memory or disk in non-sequential modes that might be more well-suited to CPU technologies.

Another trait many of these FPGA implementations have in common is that the very “field programmability” of the FPGA device gives the implementations themselves a high degree of design agility – allowing for easy and fast reprogramming of the functionality of the devices.

In addition, all of these applications lean on FPGA technology for performance at low power – making possible highly scalable high-performance solutions without breaking power and cooling budgets in the process.  This was evident in the commentary by Chris Swan of Credit Suisse in the above 451 Group story quotes and in the discussion of the 1000-node MPP RAMP project in my previous posting.

Particularly in networking, HPC and data warehousing, the FPGA provides speed and low power consumption combined with design agility – essentially supporting reprogramming of the functionality of the FPGA at start-up time.

FPGAs and Their High Performance Punch

But one of the most important uses of the FPGA technology is as an application-specific performance enhancer.  In HPC, FPGA technology is typically used to provide a performance boost.  SGI^® touts the use of reconfigurable performance of its RASC™ module as a performance accelerator (emphasis mine).

|</center>

Here’s a report on the effectiveness of FPGAs for supercomputing from an April 2005 FPGA Journal article discussing the new Cray XD1:

|</center>

And there are numerous others, even including Advanced Micro Devices and its use of accelerating co-processors based on FPGA technology with the current AMD64 line of processor boards.  Here are a few snippets of a 19^th June Electronic Engineering Times article on the topic ("Programmable chips rev critical algorithms"):

|</center>

Next Time: FPGAs Move Into Data Warehousing and the NPS Data Warehouse Appliance

The uses for the FPGA have expanded greatly in recent years to take on a key role in driving cost-effective high-performance in a broad sweep of applications.  I’ve also made some suggestions in this and my previous posting about how the same sort of technology is a key performance multiplier in the NPS data warehouse appliance.  In coming days, I’ll be posting the 3rd and final installment of this series on FPGAs, discussing the benefits that accrue to the Netezza system and what additional benefits of the FPGA might be possible as we progress the product forward in the future.

"Innovation is the specific instrument of entrepreneurship... the act that endows resources with a new capacity to create wealth."  – Peter Drucker (1909-2005), from Innovation and Entrepreneurship, 1985

Accelerating Innovation – Notes from the 2006 Netezza User Conference

This has been a big week for Netezza and the Netezza marketing team.  We've just completed the 2^nd Annual Netezza International User Conference at the Boston Seaport Hotel with 100% attendance growth over last year's conference.  The company also had ten customer and/or partner press releases in the same period, spanning virtually all of our markets.  Wednesday morning, Netezza CFO Pat Scannell told the audience at the closing session that the attendees included over 360 people, including customers, partners, analysts and prospects.

These attendees did, quite literally, represent a global audience for the event - coming in to participate in the event from Asia, Australia, Europe, Africa, and both North and South America (only 6-of-7 continents covered – there were no attendees from Antarctica that I know of), including 55 from Europe and Asia, alone.

The incredibly strong attendance demonstrated Netezza's dominant leadership in the Data Warehouse Appliance market, as I daresay no other vendor in or aspiring to this market category could have mounted even 1/3^rd the level of participation.  As Netezza President and COO, Jim Baum put things in his closing remarks of the User Conference Wednesday morning:bq. "We have always been focused on the Netezza community, but it's clear from what we've seen over the past year that there is a real change... something of a tipping point is happening here... There is something that is bigger than just Netezza that is changing the course of BI and Data Warehousing for the future... and you, our customers & partners, are leading the industry in that change."

In a "show of force" of their own, many of the UK participants (employees, customers, partners and prospects alike) had fun sporting customised "Netezza in the UK" tee shirts on opening day (Yeah, baby!).  Jolly good.  And the international dinner on Monday was, well, quite International - it was great to share the dinner table(s) in Boston's North End with people from Australia, Brasil, France, Italy, Japan, Mexico, UK, Spain and elsewhere.

As for the sessions themselves – in my admittedly-subjective view, the User Conference provided a mix of detailed technology & product information alongside numerous customer and partner business case studies.    If anything, there was a request to provide more opportunities to view multiple sessions that spanned the four tracks of the conference.  The 1-hour sessions included more than ten customer case studies, four partner case study sessions and seven technical sessions about the Netezza product and roadmap plans.  And after the conference, through Friday, there were a number of follow-on training courses for customers, partners and employees.

A Few Great Quotes From the Plaza Ballroom Sessions
Monday Afternoon "Keynote"
"The target market for these portable pocket radios was teenagers, the rebar of humanity" – Professor Clayton Christensen, Harvard Business School
Tuesday Morning "Customer Panel"
"Is there anything that you can't do with Netezza today that you used to do?" – Mark Beyer, Research Director, Gartner
"We don't need to send big checks to 3^rd party vendors anymore for DW consulting services." – Jagpal Jheeta, Business Systems Manager, Debenhams
Wednesday Morning "Partner Perspective"
"Data Mart versus Data Warehouse?  That question is simply irrelevant" – Donald MacCormick, Vice President of Product Marketing, Business Objects

Opening Keynotes

The Opening Keynote addresses were delivered on Monday afternoon by Motorola Chairman and CEO Ed Zander as well as author & Harvard Business School Professor Clayton Christensen (author of The Innovator's Dilemma,  The Innovator's Solution, and Seeing What's Next

• Understand the job that the innovation will be hired to do* Allow competitors to flee, not fight by choosing an innovation entry path that may disrupt incumbents from below, but not directly attack them and* Identify who the right customers are by competing in a new market whose members' alternate choice might be non-consumption.Professor Christensen closed by saying that from what he understood of Netezza and its growing presence in the market, the company and our Data Warehouse Appliance could very-well be characterized as a "New Market Disruption".{size}

Tuesday Keynote & Panel

On Tuesday, Netezza CEO Jit Saxena led off the morning with our vision of how "Netezza appliances position companies for the demands of tomorrow’s BI" and ushered in the morning's panel discussion before the swelling crowd in the large Plaza Ballroom.  The panel was led by Gartner Research Director, Mark Beyer and included Kelly Carrigan, Senior Director of Database Architecture with Catalina Marketing, Jagpal Jheeta, Business Systems Manager of Debenhams, and Emory Heisler, Vice President of Information Technology Services, Healthcare Analytics Group with Wolters-Kluwer Health.

Jit's presentation went back to Netezza's founding principles and why those are still very much what matter today with the dynamically growing uses of data warehouse systems and especially  data warehouse appliances being pushed by megatrends.  "Netezza" he said, "has re-energized the data warehouse market and is the clear leader in the data warehouse appliance segment."  Jit also stressed the importance of Community to Netezza – in terms of its 75  customers as well as partners, resellers and employees.  And he gave a broadened vision for the future of data warehouse appliances that will deepen their impact, broaden their use cases and move appliances to the forefront as a competitive weapon for our customers to use.

Note: Watch for much more on these topics in this space in the weeks to come.

Mark opened this segment with a brief presentation on Gartner's perspective of Data Warehouse Trends - specifically noting the trends toward balanced disk-I/O-memory-processor systems, larger and more complex workloads of users and growing data volumes. He also discussed the importance of "high-value users" in increasing the value of the data warehouse and the dimensions of use & users that most data warehouses now experience.  Then he turned to the panel members for their perspectives.  Responding to questions from Mark and several more from the floor, Kelly, Jagpal and Emory recounted how large a difference the transition to the NPS data warehouse in their data warehouse operations had been – in terms of much higher productivity, simpler operations & maintenance, reduced latency and bottom-line results.

Next the conference moved headlong into the series of four parallel track sessions for the remainder of the day – a veritable sprint through 24 one-hour sessions with the occasional break for refreshments and for colleagues to share their learnings from the various presentations that they had seen.  The Netezza presentations included everything from how (and why) the architecture of the NPS appliance is what it is "from the inside out", to achieving optimal performance with the NPS system to the product roadmap plans over the next 15-months, with several other items in-between.  Customers' case study presentations (thirteen of them, in all) ranged the gamut of experiences in the decision-making to purchase a company's first NPS appliance, to detailed performance benefits and applications, to full-up business return-on-investment resulting directly from having deployed the NPS system.

|

And Then It Was Done

And finally, Wednesday morning arrived with some reflective views about the conference from Pat Scannell, on the state of BI from Donald MacCormack of Business Objects, and on the Netezza roadmap for technology direction by Bill Blake, Netezza's SVP of Development.

Donald's sharp wit, understanding of the industry and rapid-fire delivery was just the jolt of adrenaline everyone needed to start the morning after the previous night's "frivolities".  He summarized the effect that Netezza's data warehouse appliance is having in the industry when teamed with partners like Business Objects as, "leading the second revolution in Business Intelligence and Data Warehousing".

Bill Blake's presentation started from the basis of the accepted role of the Netezza Data Warehouse Appliance in the enterprise data center: customers no longer have to "settle" for weak performance, high expense and/or high degrees of complexity for their data warehouses.  His presentation outlined the technology directions that will extend the capabilities of the NPS system forward and deliver unprecedented price-performance leadership building off key elements of the Netezza AMPP^TM architecture for accelerating analytic database applications.  Included in Bill's presentation was our roadmap to enable the Data Warehouse to "graduate from algebraic set operations on tables to the calculus of predictive analytics."  Following Bill, Jim Baum's brief closing remarks (quoted above) put the whole event in perspective.

Note: Just as with Jit's comments, watch for much more discussion in this space in the weeks to come.

Before we knew it, the conference was over and people were preparing for their travels home and/or elsewhere.  There were many warm handshakes and good-byes all around, along with 5-10 minute synopses of the things we had shared, learned and done over the past 72-hours.  And the marketing team drew a collective sigh, thanked our luck for the fabulous Boston weather and clinked glasses all around before we (gasp!) again begin planning for next year's Conference.

We in Marketing had a great time putting this together and having it come off as well as it did and we hope that all of our guests enjoyed themselves and took a substantial amount of new innovative ideas away to help accelerate their businesses with Netezza.  We hope to see you all (and then some!) at the 2007 International User Conference...  ...and we'll see you back at this blog and elsewhere in the Netezza Community!