Thinking Out Cloud

Cloud Computing. Literally.

Last week we made a very exciting announcement about Miwok Airways selecting GigaSpaces as the application server for running their reservation and pricing engine which will run on EC2. This is a great case study for cloud computing.

For one thing, you have to love the fact that it is cloud computing used for a business that literally runs in the clouds (the actual meteorological kind). Second, it is an on-demand compute infrastructure for a business that has an on-demand business model in the real world. A perfect fit.

There is a great piece in the LA Times that describes Miwok, but let me give you a brief description from the software application angle. 

The idea is that for so-called ultra-short flights (typically, less than 250 miles), as a traveler you have a terrible dilemma: use commercial airlines or drive your car. I don't need to tell you the hassle and costs involved in both options these days.

Miwok overcomes the hassles of these options by providing you with an on-demand "air taxi" service. You book your flight when you need it. So, say, you want to fly from Santa Monica to Orange County or Palm Springs. You go to the Miwok web site and say when and where, you get pricing and you can book the flight on the spot. The flight you are booking is for a private Cirrus SR22. You can park 100 feet from the airplane itself (at a local airport, not just the major ones) and you don't need to go through security (imagine that!). All of this at the same cost of a commercial flight.

But here's the part I really like:You can connect to other people via Miwok's own social network, or through a Facebook app (and others to come). As the Cirrus can seat 3 passengers, you can split the costs with other passengers who need to make the same trip. So the flight could end up significantly cheaper than a commercial airline.

Think about it: This is the exact opposite pricing model of big airliners, where the more people go on a flight, the price goes up. From a marketing point of view, this has tremendous viral potential.

One of the biggest technology (technology as in software, not aviation) challenges Miwok was facing was developing an extremely sophisticated real-time pricing engine. It needs to take many parameters into account to offer you a price on the spot, including location, path, season, date, time of booking, number of passengers and several other criteria. It needs to be able to grow and shrink on-demand, especially because of the social networking and viral effect.

The architecture Miwok selected uses MySQL and Hibernate for the persistence layer, but the database is not used as the system of record for calculation and reservations. Instead they use GigaSpaces' in-memory data grid, which gives you in-memory speeds and can also grow and shrink dynamically in the EC2 environment. The benefit for Miwok is that having very little advance knowledge on the traffic they will get, and expecting extreme peaks and troughs in activity, they don't need to pre-plan and invest upfront in the infrastructure. They use GigaSpaces and EC2 and will only pay for hardware and software on a per-use basis -- when and if they actually need it.

They also use GigaSpaces XAP (which includes the in-memory data grid) as the container for the business logic, written in Java, and as a bus for integrating the various underlying services involved in generating pricing and booking reservations.

In short, on-demand application scalability for an on-demand air travel service.

Check out Miwok's web site.

Sign up for the GigaSpaces pay-per-use license for Amazon EC2.

GigaSpaces on the Joyent Cloud

Kent Lagley writes on the Joyent blog about the work he has done with Dekel and Owen running GugaSpaces on the Joyet Accelerators. It's a well-written detailed blog and in a post entitled Cloud Computing with Java, Kent adds some more info on his personal blog, ProductionScale. 

My favorite quote from the Joyent blog:

Some other exciting things about this exercise in the lab was that it was, in general, easy. We added and removed nodes, scaled linearly, pulled nodes out, re-deployed the application in seconds to minutes, we processed large amounts of data, and we did all this in the cloud with really minimal efforts.

Some formal announcements about our partnership with Joyent coming soon.

Large-Scale Web Sites and Java

Nati Shalom's post, Why most large-scale web sites not written in Java, created a bit of a stir, to say the least, with a raging debate in the comments to the blog and one on TheServerSide and Artima.

And it also got referenced and commented on here, here, and here  - and several others.

Some people interpreted Nati's question as Java-bashing. And that created a silly PHP/Ruby Vs. Java flame war. As I had something to do with this post, as Nati mentioned, I thought that a couple of clarifications are in order.

As anybody who knows Nati and GigaSpaces (and as Guy Nirpaz pointed out), we're Java people at heart. Our product is written in Java. So there was no intention to bad-mouth it in any way.

I agree that the way the title of the blog was worded was a bit provocative, implying that most large scale web apps are not written in Java, but that was merely done to make things more interesting and based on an arbitrary analysis done by Pingdom, based on information they simply aggregated from High-Scalability. So obviously, no, it's not a statistically significant sample, and the word "most" is problematic.

Then, of course, some people rightly asked what is the definition of "large-scale". Again, it's a legitimate question.

But I want to explain what was the motivation for this post. We had an internal discussion on the relevance of GigaSpaces to the world of Web applications. At GigaSpaces (and yes, some of our competitors too - no need to shout) we've been pounding the scale-out drums, the partitioned data drums, the shared-nothing drums (and many other kinds of percussion instruments) for quite some time. In the last year or two it seems that a lot of people -- especially from the Web world -- are joining that funky beat. Google, eBay, Amazon, MySpace, Flickr, YouTube, Wikipedia and many others have published their architectures (and yes, I know some of them use Java) - and spoke of such architectures.

In other words, in the architectures that these folks used to make their Web apps highly scalable, performant and reliable, they are using very similar principles to the ones GigaSpaces has been advocating.

That said, we noticed that many of these folks are going with LAMP stack. The question was whether this is the trend, and consequently, what does it mean for us. If the trend is that Web 2.0 is moving to LAMP, but the enterprise market is still going to be a Java world, we need to make some decisions. If the whole world is moving to LAMP, we need to make some other decisions. And if Java is still dominant (or at least major) in the Web-world, it means yet other decisions and implications.

Although we don't have clear statistics, my intuition is that the trend for Web apps that are coming from start-ups or large pure web players (as opposed to web apps from airlines, banks, etc.) is definitely towards the LAMP stack. However, Java will still remain strong for a while and  especially for Web apps that have to deal with more complex processes in the back-end.

[BTW, I have a theory that 2004 was a watershed year in which many trends changed. It is the year that Spring Framework came out, that the term Web 2.0 was officially coined, that Ruby on Rails was released to the public, and several other interesting events. It would be interesting to see an analysis that compares what Web companies that are pre-2004 and post-2004 use for their infrastructure.]

Another interesting aspect, which Nati raises in his latest post, is that there is a possible convergence between Java and LAMP/RoR in various shapes and forms -- so it is not necessarily a zero-sum game.