Dependency Manager 4

ApacheFelixDM

The Apache Felix Dependency Manager has a long history! It started out as an open source project over ten years ago, when Herko and me were working on a free tool for simracers called VROC 3. That was a lobby and matchmaking service aimed at uniting simracers using different simulations and racing them on-line. To make that a modular application, we used OSGi and in those days there were no frameworks to help manage dependencies, which is why we started working on Dependency Manager.

Some time later, I was invited to present a paper about it at the OSGi DevCon in Paris in 2005 and eventually, when the Oscar OSGi framework moved to Apache to become Felix, so did the DM project, and it became what it is today: one of the many sub projects of Apache Felix. Over the years, the codebase evolved substantially, always guided by real-world requirements that naturally evolved as people started building more and bigger applications with OSGi.

About a year ago, Pierre, Xander and me sat down to look at the codebase and discussed some of the new requirements we had regarding concurrency and scalability. We quickly realised that in order for us to really take advantage of multi-core systems, we had to re-write the codebase from scratch. Initially we started out with an abstract model of the components and dependencies that we manage, and focussed on creating test cases that demonstrated the new use cases we wanted to implement. We really took our time at that point to get things right, and went through a few design iterations. Eventually we settled on a model where a lot of the code related to a single component always runs on one thread, guarding concurrency “at the gate” which meant that internally we needed no synchronisation or other arbitration at all anymore and at the same time the model could be parallelised easily by using a thread pool to concurrently start many components. The results of this looked very promising and turned out to show huge performance gains on big, multi-core (think 64 core) servers!

After completing our prototype and tests and feeling confident that we “got it right” we now went on to port all existing features into the new codebase. As we already decided at that point we wanted to break a few existing APIs to make them easier to use, we bumped the version number to 4. We did keep in mind that migration from version 3 should be as painless as possible and I think we succeeded in that as well.

Another big change is that we moved from using Apache Maven as our build system to a combination of Bndtools (a popular Eclipse plugin, also used extensively in Amdatu and Apache ACE) and Gradle. That streamlined the development workflow considerably and also made our tests run a lot quicker.

The shell is another area where we moved forward, having a lot more powerful options to diagnose systems using DM. We’ve added extensive ways of querying for specific information, as well as a command that does “root cause analysis” to see what missing services are causing a chain effect of components not being available. Such commands will help you diagnose your systems in development and production.

We also decided to move to Java 7. In fact even support by Oracle for that version ends soon, so at some point we probably will move to Java 8 and take full advantage of the new APIs that it provides. Still, the move to a more recent JVM allowed us to leverage some of its newer features during the redesign.

Finally we took the opportunity to update our documentation, adding tutorials, guides and an extensive reference section as well as providing a much requested feature: JavaDoc!

Version 4 of the Dependency Manager just passed the vote, got announced at Apache Felix and is available for download on the Apache Felix download page. You can download the binaries, which you can add to your repository, or the sources and dependencies to compile everything yourself. Have fun with it and let us know what you think!

Posted in Article, News Tagged with: , , , , ,