Thinking Out Cloud

Is Cloud Computing on the Horizon for Enterprise Customers?

There is an ongoing debate in the industry, in the blogosphere, and frankly, internally at GigaSpaces, on whether or not cloud computing is anywhere near becoming a viable business offering for enterprise customers.

First, a general comment on these types of paradigm shifting technological developments (or for that matter social and other developments). We tend to either grossly under-estimate how quickly they will be adopted, or grossly over-estimate it. For example, as an industry, I believe the rapid adoption of the Internet was grossly under-estimated (I'm talking '95-'96). During the bubble period, we grossly over-estimated how rapidly things such as online B2B Exchanges will be adopted (I shudder even saying these words now: 'b2b exchanges'. ugh...).

Not in my lifetime

Another example from the social/political world is what we're seeing in the U.S. presidential race. I believe that if you asked Americans as recently as 2-3 years ago about a woman president or an African-American president, the great majority would have responded with "Not in my lifetime" or "America isn't ready". Today, you'd get a very different response, of course. And even if this election does not produce a woman or African-American president, the barrier has been crossed. The psychological barrier of the voters, and the psychological barrier of many other potential candidates who might run in future elections (including Asians, Hispanics, Jews and other minorities). It is clearly a viable option.

Sometimes change comes fast & furious

Cloud computing is one of those things that many people are under-estimating. Change is coming fast. It is inevitable and closer than many think. The reason: it just makes too much sense to ignore. The economics are so compelling, that all you need is one player in an industry to adopt it, and the rest will have no other choice but do the same, if they value their profits and their existence.

Objects in the mirror may be closer than they appear

A few days ago Erick Schonfeld blogged a post on TechCrunch entitled: Who Are The Biggest Users of Amazon Web Services? It's Not Start-Ups. I highly recommend reading the full post and the ensuing comments. A couple of fascinating things come out of it. The first was that Amazon made $131 million in revenues in the last quarter from AWS. Tiny compared to the $5.7 billion in total revenues, but unbelievable for a business so young (meaning the AWS business). Imagine it was a start-up company with such quarterly revenues so soon. They'd be the next Google.

The second thing was this:

So who are using these services? A high-ranking Amazon executive told me there are 60,000 different customers across the various Amazon Web Services, and most of them are not the startups that are normally associated with on-demand computing. Rather the biggest customers in both number and amount of computing resources consumed are divisions of banks, pharmaceuticals companies and other large corporations who try AWS once for a temporary project, and then get hooked.

Like I said, the economics (including simplicity, time-to-value, etc.) are just to compelling. once the psychological barrier is crossed, it'll catch on like wildfire -- and you can't go back.

The skepticism we are seeing now from some people about security and other issues is reminiscent of the skepticism that Salesforce.com and other SaaS players faced. People said that companies will never let a "web site" keep enterprises' most valuable lead, customer and deal information for access through the public Internet. So much for that.

Personal Experience

Just from GigaSpaces' own activity in this area I can sense the tsunami fast-approaching. We started "playing around" with the idea of offering GigaSpaces -- middleware software, mind you --  as a service on Amazon EC2. It makes a lot of sense because GigaSpaces XAP is essentially application infrastructure software born for the cloud -- as I describe on this GigaOm post. We put up a GigaSpaces AMI (Amazon Machine Image), maybe wrote a couple of blog posts about it here and on the GigaSpaces Blog, and that's pretty much it. Within days, we had significant amounts of traffic leading to both our AMi and the GigaSpaces web site in Amazon EC2 related searches and inquiries. Pretty soon we started getting commercial interest. Today, only a few months after this quiet non-launch, we have approximately 16 companies in various stages of working towards running their application on GigaSpaces as the platform on the Amazon EC2 cloud. In addition, we're seeing a lot of partner interest with a couple of very concrete things to be announced shortly. Wait till we actually launch this thing!

Barriers remain

But I don't mean to be delusional. There are certainly barriers left to overcome before we see  mainstream adoption of cloud computing. However, I do expect them to be overcome quicker than expected because there is so much vested interest for a lot of people to do so.

I'll write about the remaining barriers and how they are being addressed in a future post.

Cloud Computing overtaking the term Grid Computing

With the term "cloud computing" rapidly being hyped everywhere, I did this little exercise on Google Trends to see how it fares against its predecessor "grid computing". Here's the result -- cloud is just about to overtake grid:

If we zoom in on cloud computing it looks like this:

We are at the very beginning of the hype-cycle, and not yet at the "peak of deflated expectations". If it were a stock, I'd say BUY for now.

Excel That Scales: The Movie

Back in June of last year I wrote about our partnership with Microsoft and our plans to work together on a solutions for scaling out computations on Microsoft Excel spreadsheets. Since then Microsoft and us both released joint material (see here on MSDN) and held joint events promoting the solution. The most up-to-date white paper on the solution can be found here.

But now, Owen Taylor produced a screencast that describes the Microsoft-GigaSpaces joint "Excel That Scales" solution, in which he walks you through the problem and the solution.

Listen to the presentation.

Synopsis:
In many organizations -- for example in capital markets and oil & gas exploration -- Excel is used widely for complex computations and analytics. Excel is a flexible tool that many people are familiar with, so over time huge investments have been made in creating complex analytical models in Excel. However, it was never designed to be an enterprise-grade analytical tool. As data volumes are growing, the need to have real-time information is intensifying and the number of users who wish to share the same computational logic and data is increasing, desktop-based Excel spreadsheets could no longer handle the loads. Also, the functions they perform are becoming mission-critical and valuable time and information could be lost in case of failure.

Enter the GigaSpaces solution. It combines the best of both worlds: Excel as the front-end and the power of your data center -- through GigaSpaces as the scale-out, highly-reliable application server - -at the back-end. In other words, the logic and the data are handled server-side with enterprise-grade reliability and performance.

Owen says it better and shows a demo.

GigaSpaces Interviews

Recently, a few interviews with me were published.

You might want to check them out:

• GigaSpaces XAP 6.0: Simplified, Spring-based API for Space-Based Architecture
• Interview with GigaSpaces
• Balancing Performance and Simplicity for Wall Street - this excellent piece by Derrick Harris also includes an interview with Sebastian Titakis of Commerzbank, a GigaSpaces customer

Before you read these, let me just quote Yogi Berra and say: "I never really said everything I said."

Grid-Enabling Resource-Intensive Applications

Timothy Hoehn and Bob Zeidman from Zeidman Consulting published on Dr. Dobbs a really interesting analysis of what's the optimal way to distribute an application across multiple machines.

They examined several methods and reached the conclusion that the optimal approach is distributed objects and the master-worker paradigm.

Sounds familiar? :-)

Grid in Financial Services

Earlier this week, GridToday published an excellent piece by Marc Jacobs of Lab49 (a GigaSpaces partner) entitled Grid in Financial Services: Past, Present and Future. What an eloquent, well-written piece.

Each trading day is a perfect storm. Every month, every quarter, the volume of data increases, the sophistication of algorithms and business processes grows, and the competitive pressure to get things done as quickly and efficiently as possible mounts.

Marc reviews the state of the union on distributed computing in financial markets Including the motivation (we can no longer throw expensive hardware at the problem, we need a new approach) and the current state of affairs (distributed computing is real not academic, with commercial vendors and real implementations).

Our appetite for computing power isn’t satisfied with lone, uncoordinated machines. For financial services, distributed computing isn’t a luxury: it puts food on the table.

One of the challenges Marc observes is the fact that until recently (and still going on in some places) developers have been dealing with a lot of the "plumbing" issues of distributed computing and have been doing so in an inconsistent way. Part of the solution he sees is a new generation of vendors and products that address this:

The range of stable, usable distributed computing platforms -- such as those from Platform Computing, GigaSpaces and Digipede Technologies... Thus, it is becoming much rarer to find software development teams in financial services working on this type of plumbing.

However, one of the obstacles that Marc points out is the fact that various vendors are only dealing with one aspect of the issue or another, and rarely take and end-to-end approach. He has this to say, which is especially nice for GigaSpaces (my emphasis):

For example, while it is positive that there is a wave of vendor products that solve different parts of the distributed computing puzzle, few of them treat distributed application development as a holistic endeavor that encompasses many problems (i.e., job scheduling, event processing, data distribution and caching, security, deployment, APIs, IDEs, etc.) at once. Except for GigaSpaces, most distributed computing architectures require the assembly of infrastructure from several different vendors. While this does permit architectures built from best-of-breed solutions, it can be challenging to stitch the various pieces together into a coherent developer framework.

There are many other interesting topics, such as the need for both IT management of distributed systems as well as developer-friendliness. Again, he had something nice to say about GigaSpaces (and our friedns at Digipede):

Unfortunately, few vendors have been able to make progress on both fronts. Some products, such as Digipede and GigaSpaces, are clearly more developer-friendly than others.

Highly recommended read. BTW, Marc has a great blog, which I read regularly: Serial to Parallel to Distributed.

As an aside, the Lab49 guys are a very sharp, well-spoken, experienced group that's worth paying attention to (see their group blog). I recently had the pleasure of doing a web seminar with Daniel Chait (founder and managing director of Lab49) for CMP. You can see it here.

Update: Here's Tom Groenfeldt's take on Marc's piece.

GridGain-GigaSpaces Integration

Our friends at GridGain just released the GA of their version 1.5, including integration to GigaSpaces as a data grid.

GridGain makes very simple, robust open source software for computational grids. It's ideal for performing parallelizable tasks, such as MapReduce (i.e, split the work, calculate, aggregate the results).
Check out technical info on the integration here, Nikita Ivanov's blog, and their integration page.

From an integration point-of-view what's interesting about this is that they used GigaSpaces' new OpenSpaces Framework (our open-source declarative API using Spring).

GridGain is the latest in our ongoing integration efforts with open source projects and cooperation with the companies behind them, including Mule ESB (MuleSource), Spring Framework (Interface21) and Hibernate.

Expect to hear more from us on this very soon.

Dealing With Technology

I'm participating on a panel at the Dealing With Technology (DWT) event in London this week on Thursday (May 3).

It's an interesting group and interesting topic. The rest of the event seems pretty cool too. Come check it out, if you can.

11.00 – 12.00 Panel discussion: Tackling grid management

• Managing compute grid resources
• Implementing application servers within a grid environment
• Addressing the file-management bottleneck
• Improving SOA performance in the grid environment

Moderator: Barry Childe, Director, Head of High Performance Computing, BARCLAYS CAPITAL
    Carey Jack, Grid Architect, Investment Banking IT, CREDIT SUISSE
    Pramod Srivatsa, Product Manager, Server Virtualisation Business Unit, CISCO
    Geva Perry, Chief Marketing Officer, GIGASPACES

Tower of Babel

Our industry has a problem, and it has to do with words. We use the same words to mean different things, and different words to mean the same thing. In particular, there is a problem with naming categories of technology or products. Consider the terms grid computing, utility computing, on-demand computing, high-performance computing. Some people see them all as referring to the same thing. Others believe there are nuances among them, and yet others feel they refer to completely different things. Now, also add to the mix the terms fabric, virtualization, distributed computing -- all of which are also frequently used to refer to similar -- if not the same -- things, and you've got a big mess.

I was thinking about this because it seems that GigaSpaces, and the tech category it belongs to, is now at a point where it is crossing the proverbial chasm from early adopters to mainstream customers. And an important part of maturing is having a  clear and consistent name for the product category.

This Tower of Babel phenomenon occurs partly by coincidence -- different people come up with different words to describe what they or others are doing more or less at the same time. But partly, this is happening by design.

I see three main reasons for this (in no particular order):

Reason 1: The prevailing Silicon Valley common wisdom of marketing technology is based on Geoffrey Moore's seminal Crossing the Chasm. In it, Moore writes that if you are introducing a new technology, invent a new category for it and position yourself as the leader.
So you've now got thousands of vendors -- start-ups or established -- inventing new categories and crowning themselves supreme leader of the category. For example, the terms Business Service Management (BSM) emerged to describe the suite of application development, testing and monitoring tools provided by vendors such as BMC Software. To establish itself as a leader in its category, Mercury Interactive (now part of HP) went ahead and created the competing term Business Technology Optimization (BTO). You now have two terms referring to what is essentially the same thing.

Reason 2: Competing analysts need to make their mark on the world. The Forresters and Gartners of the world want to be able to say "I called it." So they describe an emerging trend -- and name it. For professional prestige reasons, the competing analyst firm cannot use the other firm's terminology, so they describe the same phenomenon in a slightly different way and give it a different name. Luckily, the big analyst firms have consolidated and there are only a few of them now, but on the other hand, there is a growing number of boutique firms, Wall Street analysts, the press, the blogosphere -- all throwing their own phrases into the mix.
So as an example from my own experience with GigaSpaces: Gartner categorizes us under the general area of Extreme Transaction Processing (XTP). Within it, they place us in two sub-categories: Grid-Based Application Platforms (where they name GigaSpaces as the leader and include vendors such as Appistry) and Distributed Caching Platforms (Where they name GigaSpaces as the leader together with Tangosol/Oracle). Forrester categorizes us under the umbrella of the Information Fabric (and more recently Information-as-a-Service (IaaS), and within it in the sub-category "Data Grid." The various vendors in the spaces use any and all of these terms, as well as things like "Data Fabric" and "Clustered JVM."

Reason 3: "The king is dead; long live the king!" What do you do if you provide hosted application services, commonly known as an Application Service Provider (ASP), but no one wants to touch that with a ten-foot pole, because of the billions of VC investments lost in that category during the dot-com bust? You invent a new name for the category: you call it Software-as-a-Service (SaaS). Presto! Out with the old and un-sexy; in with the new and trendy.

The reason this is an industry-wide problem that we should all care about is because the confusion caused by this creates an impediment to the adoption of new technologies.

Moore states this concisely when he writes:

Potential customers cannot buy what they cannot name, nor can they seek out the product unless they know what category to look under.

Our industry is already dealing fairly competently with a similar issue: technology standards. Why not apply the same approaches to "marketing standards"? Sure, reaching technology standards is often a  lengthy process, fraught with  political maneuvering and compromise, but at the end of the day it works. And the reason it works is because vendors realize that lack of standards are a barrier to overall adoption of their products. It's an issue of increasing the size of the overall pie, and not just fighting for a slightly bigger slice of a small pie.

So borrowing from the tech standards approach, here are a few possible ideas to consider when it comes to technology category naming conventions:

• A Standards Body -- made of analyst firms, vendors and other stakeholders
• A Community Process -- similar to the Java Community Process (JCP)
• And my favorite: A Wikipedia-like model (perhaps with some modifications making it a hybrid with a standards body). So for example, under the entry Object Spaces in Wikipedia, it says: "It has been suggested that this article or section be merged into Tuple space. (Discuss)". Besides a discussion process, one can also imagine a polling function and actual voting.

Some of this will always remain a problem, because the different products in a ctegory don't usually overlap 100% and therefore it does make some sense to categorize them differently. It will also remain a problem because vendors, and perhaps rightly so, will always want to claim their turf and have the game played by their rules. This also happens with technology standards. But still, seems to me that there is a lot of room for improvement.

Not Your Father's Grid

I haven't posted in a while as I've been given the added responsibility (and privilege) of running marketing at GigaSpaces. It means a lot more work for me, but it's also fun. One fun aspect of the job is telling the GigaSpaces story to different audiences in ways that are relevant to their world. And, yeah, getting some press.

So a few months back, Derrick Harris, now the former editor of GRIDtoday, posted this column, asking himself why the "grid vendors" don't have transactional grids on their radar. I wrote Derrick, telling him that the so-called "traditional grid vendors" that he's been talking to are not positioned to solve the particular issues related to transaction processing on the grid, namely managing data, workflow and achieving low-latency (and a couple of other issues). But GigaSpaces actually is, and if he'd like, I'd tell him all about it. Which I did.

Yesterday, GRIDtoday published the resulting story -- a very well-written piece on transactional grids. Greg Nawrocki also compliments them in his own well-written and insightful blog, Grid Meter, and says: "This is truly a glimpse of what the future holds for grid." I love the support, but of course, from where I am sitting, the future is now.

The GRIDtoday piece is especially great because the writers did not just count on what the vendors say, but talked to analysts and customers as well. And Bec Wilson from Sempra Energy Trading said some wonderful things about GigaSpaces that even surprised me (100X performance improvement!). Gartner's Massimo Pezzini also does a good job of giving context to the whole phenomenon. He's one of the few analysts out there who are actually trying to size, give names and categorizes these emerging technologies.

We've always been a bit ambivalent about the term "grid" at GigaSpaces. Partly because the term (not the technology) has been widely hyped, misused and misunderstood. But to me it has quite a simple definition. I arrived at it when I read this blog from Lee Gomes at the Wall Street Journal who defined virtualization as "using software to make one computer act like many." Grid, therefore, is reverse-virtualization: using software to make many computers act like one.

That's fine when you are just dealing with compute-intensive tasks. Transactional systems are a different story. They are made of multiple tiers: presentation, messaging, business logic and data. You cannot parallelize and distribute transaction processing unless you fundamentally solve this issue and collapse the tiers into one unit -- virtualize the tiers, if you will. Otherwise, the resulting system is broken up both vertically (into tiers) as well as horizontally (across machines) -- and you end up with a spaghetti-architecture that doesn't scale and is too slow. And that is the crux of what GigaSpaces solves and how it grid-enables transactional applications.

Nati, Owen and Guy have all posted great detailed explanations on how this works.