Earlier this week, GridToday published an excellent piece by Marc Jacobs of Lab49 (a GigaSpaces partner) entitled Grid in Financial Services: Past, Present and Future. What an eloquent, well-written piece.
Each trading day is a perfect storm. Every month, every quarter, the volume of data increases, the sophistication of algorithms and business processes grows, and the competitive pressure to get things done as quickly and efficiently as possible mounts.
Marc reviews the state of the union on distributed computing in financial markets Including the motivation (we can no longer throw expensive hardware at the problem, we need a new approach) and the current state of affairs (distributed computing is real not academic, with commercial vendors and real implementations).
Our appetite for computing power isn’t satisfied with lone, uncoordinated machines. For financial services, distributed computing isn’t a luxury: it puts food on the table.
One of the challenges Marc observes is the fact that until recently (and still going on in some places) developers have been dealing with a lot of the "plumbing" issues of distributed computing and have been doing so in an inconsistent way. Part of the solution he sees is a new generation of vendors and products that address this:
The range of stable, usable distributed computing platforms -- such as those from Platform Computing, GigaSpaces and Digipede Technologies... Thus, it is becoming much rarer to find software development teams in financial services working on this type of plumbing.
However, one of the obstacles that Marc points out is the fact that various vendors are only dealing with one aspect of the issue or another, and rarely take and end-to-end approach. He has this to say, which is especially nice for GigaSpaces (my emphasis):
For example, while it is positive that there is a wave of vendor products that solve different parts of the distributed computing puzzle, few of them treat distributed application development as a holistic endeavor that encompasses many problems (i.e., job scheduling, event processing, data distribution and caching, security, deployment, APIs, IDEs, etc.) at once. Except for GigaSpaces, most distributed computing architectures require the assembly of infrastructure from several different vendors. While this does permit architectures built from best-of-breed solutions, it can be challenging to stitch the various pieces together into a coherent developer framework.
There are many other interesting topics, such as the need for both IT management of distributed systems as well as developer-friendliness. Again, he had something nice to say about GigaSpaces (and our friedns at Digipede):
Unfortunately, few vendors have been able to make progress on both fronts. Some products, such as Digipede and GigaSpaces, are clearly more developer-friendly than others.
Highly recommended read. BTW, Marc has a great blog, which I read regularly: Serial to Parallel to Distributed.
As an aside, the Lab49 guys are a very sharp, well-spoken, experienced group that's worth paying attention to (see their group blog). I recently had the pleasure of doing a web seminar with Daniel Chait (founder and managing director of Lab49) for CMP. You can see it here.
Update: Here's Tom Groenfeldt's take on Marc's piece.
