How Nimble is your OSGi runtime?

Hands up all of you managing OSGi dependencies via an editable list of bundles. Easy isn't it! It just works right!?

Well actually - it 'just works' for a single application running in a small number of containers. From an enterprise perspective you are unintentionally contributing to an impending complexity meltdown; an explosion of dependency and configuration management issues. And if you are unfortunate enough to end up supporting your own composite creations, you may well end up envying the fate of Prometheus and rueing the day you learnt to code.

Possible harsh? But I'm not alone voicing this concern!

In his recent article "Reuse: Is the Dream Dead?", Kirk Knoernschild continues his efforts to educate the industry on the tensions between code 're-use' and 'simplicity of use'. Kirk argues that as you increase potential re-use via lightweight fine-grained components, the complexity of dependencies and necessary environmental configurations corresponding increase, so making these same components harder to use.

A simple concept, yet if unaddressed, an issue that will make your life as an enterprise developer increasing uncomfortable and help edge OSGi closer to that seemingly inevitable 'trough of disillusionment'.

Yet, from a development perspective the issue of dependency management is well understood.

Whilst initially found wanting, a number of projects now exist to address this; including the SIGIL eclipse plug-in which Paremus recently contributed to the Apache Felix project, (SIGIL leveraging Peter Krien's BND tool).

In contrast, the issue of dependency management in Production is less immediately obvious, its impact more profound and generally ignored.

* Will aspects of the runtime environment affect the runtime dependencies within the application?

* Will applications be isolated from each other, or might they run within the same JVM?

* How are the released artifacts subsequently managed in the production environment with respect to ongoing bundle dependency and version management?

Echoing Kirk's concerns, Robert Dunne started his presentation at OSGi DevCon Europe with the observation that; 'whilst modularity was good, its benefits are often undermined by dependency and configuration complexity'. The subject of Robert's presentation? The Paremus Nimble Resolver, which is our response to the concerns posed by Kirk.

Nimble is a high performance runtime dependency resolver. To deploy a composite application to a Nimble enabled runtime (i.e. the Paremus Service Fabric) one specifies:

* The root component of the artifact.

* And a set of associated policies and constraints.

Nimble then does the rest.

Presented with the 'root', Nimble dynamical constructs the target composite; ensuring that the structural dependencies are resolved in a manner consistent with both organizational policies and the runtime environment within which it finds itself.

Nimble's OSGi capabilities include:

* Fragment attachment policies.

* Optional import policies.

* Import version range narrowing.

* The ability to resolve dependencies on extender bundles (DS, 'classic' Spring, Spring DM, iPOJO).

With Nimble policies allowing:

* The configuration of selected extensions.

* Flexible constraint requirement -> capability matching.

* The ability to configure optional dependency resolution behaviors.

Not just OSGi, Nimble is a generic artifact resolver with a plug-able architecture. Any artifact type may be supported, with support currently available for:

* OSGi Bundles

* POJO's, 'classic' Spring & Spring DM

* WAR

* Configurations.

A Nimble enable runtime quite literally dynamically assembles all required runtime application and infrastructure service dependencies around the deployed business components! Specify a WAR artifact and Nimble will instantiate the appropriate Servlet engine dictated by runtime policy attached to the WAR; i.e. Tomcat or Jetty Sir? Specify a 'Configuration', and Nimble responds by installing the target of the configuration, and of-course its dependencies.

Nimble not only directly addresses Kirk's concerns, but goes on to radically transforms our understanding of the responsibilities and capabilities of next generation composite aware Service Platforms. But most importantly, Nimble was created to enable effect re-use whilst making life simpler for you and the organizations you work for.

Whilst recently writing up a white paper, I idly spent sometime looking through my usual archive - the Internet (anything to avoid writing). :-/

When did we (Paremus) first announce distributed OSGi again? Answer, not 2009 as one believe if you listened to all the IT vendor noise about RFC119 - but in December 16th 2005.

OK - we were a little early :)

This press release even had a quote from Jon Bostrom. Jon, six years early in 1998 actually visited Salomon Brothers UK to provide a Jini train course too, what turned out to be, a proto-Paremus team.

This morning I was alerted to a blog concerning Jini and OSGi which I dually half-read, then responded. Then realized that the blogger had actually reference a short 5 minute talk I gave at the Brussels JCM 10 Jini event September 2006. As the message from this presentation had been ignored by the community since that point - I has somewhat surprised / pleased to see it referenced.

My message at the time was simple and quite unpopular...

To survive and flourish Jini must embrace OSGi

The other thing that sprang to mind was Jim Waldo's presentation at the same conference. Unlike mine, this widely report with great enthusiasm; I really don't mind Jim:)

The interesting thing was - at least to my mind - one of Jim's most profound comments seemed to be missed by most.

Program v.s. Deploy - we'll put the management in later

This struck particular resonance with the Paremus engineering team - as our dynamic target state provisioning sub-system for Infiniflow had been released earlier that very year. This leveraging those very ideas!

Its now 2009 - we have the industry has defined the relevant required standards for distributed OSGi based frameworks. Now the industry is wondering how to develop, deploy and manage runtimes that consist of 1000's of dynamical deployed bundles running on a Cloud of Compute resource. No problem! Paremus have been doing that for half a decade ;)

Conclusions? Nothing profound. Perhaps the slow pace of the IT industry? But isn't the Internet a great communal memory!

Forget Cloud - OSGi is the new Cool Thing!

Or so an Industry Analyst recently informed me. 

Yet the flurry of Twittering & Blogging concerning the distributed OSGi section of the new OSGi 4.2 specification is certainly interesting. Is OSGi approaching some sort of enterprise adoption tipping point? These along with other commercial indications imply this is likely.

This is good news. OSGi deserves to be wildly successful, OSGi is one of the key enablers for the next generation of enterprise. 

Yet a danger lurks in the shadows. 

The use of OSGi does not in itself guarantee any sort of coherent architecture, nor is capable of addressing the current complexity crisis with the enterprise. OSGi is simply a tool - and in the wrong hands OSGi runtime systems will seem orders of magnitude more complex than the systems they replaced.  Meanwhile, the distributed OSGi section of the 4.2 specification is simply an acknowledgment that "things" exist outside the local JVM - no more - no less. 

Distributed OSGi has little to say about how to address Deutsch's 8 Fallacies ( actually if you follow the link you'll notice that Wikipedia now have a 9th :)  ). How these distributed entities discover each other, interact with each other, and which protocols are used is left as an exercise to the software vendor. This is not a criticism of the standard - this is a good thing. OSGi doesn't constrain distributed architectures. 

Yet this allows business as usual for the Software Vendors. And so we see the same old tired SOA rhetoric.

"ESB's & WS-*, would you like OSGi with that sir?"

But joking aside - the real danger is that OSGi's fate may become hopelessly entangled with the current disillusionment surrounding the web of vendor SOA Market-ectures. 

Paremus have always argued that OSGi deserves to be complemented by a network SOA framework that is as adaptable and dynamic as OSGi is locally within the JVM. A Self-Similar Architecture!

It was for this reason that Paremus fused OSGi (the new Cool technology) with Jini (was Jini ever Cool?) within the Newton project in 2006. A solution, in its commercial Infiniflow guise, which has been in customer production for over 2 years. 

As for Cloud Computing - that story has only just started ;-)

We live in exciting times!

Java EE 6 is announced. The Interface21 folks think its finally "right", and the daggers are drawn as the old JBoss boys feel the need to defend their position as popular open source JEE appserver vendor (see theserverside).

Extensibility and Profiling are a couple of key features in Java EE 6.

Mmmm. So I can take my very bloated Java EE infrastructure and reduce it to merely bloated.

I'm almost sold on the idea ;-)

But hang on? What about OSGi and SCA. Can I not already dynamically build very sophisticated distributed composite applications that adapt and evolve to their resource landscapes? Such distributed application services only running loading and running what is required at each specific point in time. These solutions self-managing, self-configuring and self healing?

Well actually, yes I can - and Java EE - in any form - doesn't figure!

On a finishing note - a nice article (concerning Web Services) whose underlying message is, I'd suggest, as equally applicable to the monolith Java EE v.s. composite OSGi / SCA debate.

An new white paper concerning the synergies between OSGi, SCA and Spring can be found on the OSOA site, well worth reading for those you want and introduction to this field of activity.

The white paper concludes that "SCA, OSGi and Spring together provide powerful capabilities for building service implementations from simple sets of simple Java Beans using a few simple API's".

The one interesting omission is any real discussion about the challenges of building distributed OSGi, SCA, Spring based distributed systems. Whilst the white paper explains the virtues of dynamic dependency resolution, this is only within the context of a static resource landscape, and so fails to acknowledge the additional changes presented by Peter Deutsch's 8 Fallacies of Distributed Computing.

For those interest in dynamic distributed systems based on OSGi and SCA that support Spring, check out the Newton project. Newton is itself built from the ground up to be a robust distributed runtime environment using OSGi, SCA and Jini as foundation technologies, providing a "robust" - in the true non-marketing sense of the word - enterprise runtime platform for Java Pojo based applications including Spring.

A final thought - there still seems to be a real lack of understanding within the industry w.r.t. the fundamental relationship between agility, distributed systems, complexity and OPEX (operational expenditure). A subject I suspect I'll post more on, once I've caught up with my day job.

cognito ergo sum?

Unfortunately, 'I blog, therefore I am' seems to have greater resonance these days.

So, dragged kicking and screaming by my Paremus colleagues, I've agreed to start, what feels to be the somewhat unnatural behavior, that of 'blogging'.

The Usual Disclaimer:

Being the founder and CEO for a enterprise software company based in the UK (yes - I did say the UK!), my views hopefully influence my colleagues, and company direction ;) That said, my views are my own, and do not formally represent those of my colleagues and company.

In summary, my career initially started as an Astrophysicist, but a family and mortgage finally convinced me that I need to earn a living, and so I became an economic migrant, moving into an IT career with a major Investment Bank with offices based in London. After 7 very interesting years, I made the difficult decision to leave the Bank, and set-up Paremus in 2001.

"Great timing!", I hear from those of you whom started ventures around the same time ;)

I've always been more interested in the underlying principles, concepts and fundamental "truths", rather than the specifics of a system implementation. With the creation of Paremus, I found these interests translated into a deep curiosity concerning Complex Adaptive Systems (CAS), Recovery Oriented Computing ROC and how these concepts might finally address some of the fundamental issues faced by modern distributed enterprise systems.

Whilst these interests have ongoing influence in our internal research and product development programs, they were also a key driver for the formation of the codeCauldron community. The Cauldron community, founded by Paremus in 2006, has the simple intention of fostering the development of next generation of distributed autonomic system. For our part, Paremus engineers, have successfully leverage some of the CAS / RoC design principles to which I refer to create Newton; a distributed OSGi / Jini / SCA based service framework, again hosted by the codeCauldron open-source community.

So, if you find my subsequent ramblings of interest, you may find a visit to Cauldron worthwhile.