Thinking Out Cloud

GridGain-GigaSpaces Integration

Our friends at GridGain just released the GA of their version 1.5, including integration to GigaSpaces as a data grid.

GridGain makes very simple, robust open source software for computational grids. It's ideal for performing parallelizable tasks, such as MapReduce (i.e, split the work, calculate, aggregate the results).
Check out technical info on the integration here, Nikita Ivanov's blog, and their integration page.

From an integration point-of-view what's interesting about this is that they used GigaSpaces' new OpenSpaces Framework (our open-source declarative API using Spring).

GridGain is the latest in our ongoing integration efforts with open source projects and cooperation with the companies behind them, including Mule ESB (MuleSource), Spring Framework (Interface21) and Hibernate.

Expect to hear more from us on this very soon.

SIFMA & the GigaSpaces-Microsoft Joint Solution

Microsoft and us announced today our joint solution of GigaSpaces integration with Excel and Compute Cluster Server.

I've written before about the problem this joint solution solves in Grid Meets the Middle Office.

Here's an excerpt:

    The solution addresses two fundamental challenges grid users in capital markets are facing today. First, it allows organizations to move large volumes of data to compute nodes with low-latency performance, and, second, it eliminates the disconnect between the front office and the data grid. In addition, GigaSpaces' platform provides a highly scalable application architecture that enables organizations to keep pace with rapid growth.

    "In the financial services industry, large and fast-growing applications based on Microsoft technologies need to process high volumes of transaction data very fast," said Stevan Vidich, U.S. capital markets industry technology strategist at Microsoft Corp. "Our work with GigaSpaces supports those requirements by offering end users a way to process high volumes of low-latency data using Excel-based applications. Smooth interoperability between Java and .Net is an added bonus."

Congrats to my business development team at GigaSpaces, and specifically Amnon Raviv and Dekel Tankel, for making this happen.

Come check it the solution at our SIFMA booth (#1419) this week:

    GigaSpaces and Microsoft will preview the combined solution, providing live product demonstrations,  June 19-21 in New York City at SIFMA's Technology Management Conference. Additionally, a white paper published by the two companies will be available on Microsoft Developers Network (MSDN) Web site in late June.

Hope to see you there.

Excuse My French

Marc Fleury is funny. Must be fun to have FU money.

Tower of Babel

Our industry has a problem, and it has to do with words. We use the same words to mean different things, and different words to mean the same thing. In particular, there is a problem with naming categories of technology or products. Consider the terms grid computing, utility computing, on-demand computing, high-performance computing. Some people see them all as referring to the same thing. Others believe there are nuances among them, and yet others feel they refer to completely different things. Now, also add to the mix the terms fabric, virtualization, distributed computing -- all of which are also frequently used to refer to similar -- if not the same -- things, and you've got a big mess.

I was thinking about this because it seems that GigaSpaces, and the tech category it belongs to, is now at a point where it is crossing the proverbial chasm from early adopters to mainstream customers. And an important part of maturing is having a  clear and consistent name for the product category.

This Tower of Babel phenomenon occurs partly by coincidence -- different people come up with different words to describe what they or others are doing more or less at the same time. But partly, this is happening by design.

I see three main reasons for this (in no particular order):

Reason 1: The prevailing Silicon Valley common wisdom of marketing technology is based on Geoffrey Moore's seminal Crossing the Chasm. In it, Moore writes that if you are introducing a new technology, invent a new category for it and position yourself as the leader.
So you've now got thousands of vendors -- start-ups or established -- inventing new categories and crowning themselves supreme leader of the category. For example, the terms Business Service Management (BSM) emerged to describe the suite of application development, testing and monitoring tools provided by vendors such as BMC Software. To establish itself as a leader in its category, Mercury Interactive (now part of HP) went ahead and created the competing term Business Technology Optimization (BTO). You now have two terms referring to what is essentially the same thing.

Reason 2: Competing analysts need to make their mark on the world. The Forresters and Gartners of the world want to be able to say "I called it." So they describe an emerging trend -- and name it. For professional prestige reasons, the competing analyst firm cannot use the other firm's terminology, so they describe the same phenomenon in a slightly different way and give it a different name. Luckily, the big analyst firms have consolidated and there are only a few of them now, but on the other hand, there is a growing number of boutique firms, Wall Street analysts, the press, the blogosphere -- all throwing their own phrases into the mix.
So as an example from my own experience with GigaSpaces: Gartner categorizes us under the general area of Extreme Transaction Processing (XTP). Within it, they place us in two sub-categories: Grid-Based Application Platforms (where they name GigaSpaces as the leader and include vendors such as Appistry) and Distributed Caching Platforms (Where they name GigaSpaces as the leader together with Tangosol/Oracle). Forrester categorizes us under the umbrella of the Information Fabric (and more recently Information-as-a-Service (IaaS), and within it in the sub-category "Data Grid." The various vendors in the spaces use any and all of these terms, as well as things like "Data Fabric" and "Clustered JVM."

Reason 3: "The king is dead; long live the king!" What do you do if you provide hosted application services, commonly known as an Application Service Provider (ASP), but no one wants to touch that with a ten-foot pole, because of the billions of VC investments lost in that category during the dot-com bust? You invent a new name for the category: you call it Software-as-a-Service (SaaS). Presto! Out with the old and un-sexy; in with the new and trendy.

The reason this is an industry-wide problem that we should all care about is because the confusion caused by this creates an impediment to the adoption of new technologies.

Moore states this concisely when he writes:

Potential customers cannot buy what they cannot name, nor can they seek out the product unless they know what category to look under.

Our industry is already dealing fairly competently with a similar issue: technology standards. Why not apply the same approaches to "marketing standards"? Sure, reaching technology standards is often a  lengthy process, fraught with  political maneuvering and compromise, but at the end of the day it works. And the reason it works is because vendors realize that lack of standards are a barrier to overall adoption of their products. It's an issue of increasing the size of the overall pie, and not just fighting for a slightly bigger slice of a small pie.

So borrowing from the tech standards approach, here are a few possible ideas to consider when it comes to technology category naming conventions:

• A Standards Body -- made of analyst firms, vendors and other stakeholders
• A Community Process -- similar to the Java Community Process (JCP)
• And my favorite: A Wikipedia-like model (perhaps with some modifications making it a hybrid with a standards body). So for example, under the entry Object Spaces in Wikipedia, it says: "It has been suggested that this article or section be merged into Tuple space. (Discuss)". Besides a discussion process, one can also imagine a polling function and actual voting.

Some of this will always remain a problem, because the different products in a ctegory don't usually overlap 100% and therefore it does make some sense to categorize them differently. It will also remain a problem because vendors, and perhaps rightly so, will always want to claim their turf and have the game played by their rules. This also happens with technology standards. But still, seems to me that there is a lot of room for improvement.

It's the architecture, stupid!

I haven't posted in a while due to extensive travel during the past three weeks: San Diego for the CMP Exchange Solution Provider show, London for QCon and last week in Las Vegas for TheServerSide Java Symposium. More on some of these in future posts, but the Oracle-Tangosol acquisition news that came out on Friday (and had been anticipated by us for some time -- it's a small world...) is the big thing everyone's talking about.

Nati posted an excellent analysis on this on the GigaSpaces blog. His post seems to have resonated well with others, such as Patrick Logan and John Powers of Digipede.

So to re-emphasize some of Nati's points in my own words:

• The Oracle acquisition of Tangosol is a strong validation of a new emerging category of middleware software by a major vendor
• It was certainly an excellent move for Tangosol (as a non-VC backed company, a lot of people there, and especially Cameron, are to get a big fat check). Furthermore, because the market is heating up and getting more competitive with well-funded companies such as GigaSpaces, this was the right time for Tangosol to do this
• It was a pretty good move for Oracle, and shows that they have a fair grasp of where the world is going to, but it leaves much to be desired. Nati discusses this in detail in the post referenced above, as well as touches on it in When You Need More Than Just a Data Grid.

The Tangosol approach all along has been that in order to solve performance and scalability problems, you need to solve the data problem -- i.e., move from a centralized, remote, disk-based database to a distributed, local, in-memory cache (aka Data Grid). That's fine.

The GigaSpaces approach has been all along that by only addressing the data bottleneck, you are merely taking an aspirin, not fundamentally curing your chronic migraines. In other words, the crux of the issue lies in the architecture -- n-tier architecture to be exact. Without a complete paradigm shift, you will not find the ultimate solution to the needs of the fast-growing category of what Gartner calls Extreme Transaction Processing (XTP), of real-time analytics, of high-performance SOA and of massive web applications of all sorts.

Besides the many GigaSpaces customers who are proof that this approach is being accepted. Look at the architectures of Google, Amazon, eBay, MySpace, LiveJournal and other Web stalwarts. They have all come to the same conclusion - with different nuances. They have all realized that the level of scalability, reliability and performance they need -- while keeping cost and complexity down -- will not come from a J2EE app server + database + messaging. It will not come from an n-tier architecture. Instead, they moved to a scale-out architecture, which aims for a shared-nothing approach.

So what is the GigaSpaces approach?

We call it Space-Based Architecture (SBA). I will not go into it in great detail here, because it has been explained extensively in our various blogs and white papers, but it follows the following principles:

First:

• Collapse the tiers into a single process
• Co-locate the services in a single process
• Manage state and other in-flight data in memory

You have essentially created a self-sufficient process ("Processing Unit" in GigaSpaces parlance). No more network hops. No more database calls.

Now:

• Scale-out these self-sufficient processes across your hardware infrastructure (cheap, standard hardware, mind you)
• Have the middleware partition and load-balance incoming requests across the many processes
• Dynamically manage the environment from a single-point-of-access (but not a single-point-of-failure) with SLAs for response times and reliability

The resulting architecture is:

• Linearly scalable -- because there is no dependency among the Processing Units, the law of diminishing returns does not apply. Each additional unit added provides the same throughput as the one before it
• Low latency -- because network hops between the tiers and the services that make up the application have been eliminated, and because data and events are accessed locally and in memory
• Simple -- because you have a single clustering model to manage high-availability, load-balancing and partitioning across your entire environment

Until the Oracles of the world acknowledge that the architecture their products assume is not viable for this class of applications, the headaches will keep coming back to plague them and their customers.