Extending OTP

This question came in an Elixir discussion but it is definitely not Elixir specific so I thought it might interest people.

Many people seem to think that OTP is set in stone, encased inside an armoured concrete block and practically impossible to do anything with without launching a major attack. So if it doesn't have just the feature you are looking for then there is not much you can do except grit your teeth and accept it or avoid OTP completely and roll your own instead. This is a fundamental faulty assumption.

First, however, what is OTP? It is many things: it is a large collection of libraries for doing things; a set of design principles and patterns for building systems and tools; and code, the behaviours, for implementing and supporting these principles and patterns. Most of these principles and patterns link together. So a system is a release, which consists of applications, which consist of code and processes at runtime organised into supervision tree. And at each level you can define how it is to manage the underlying systems. Many of the libraries are independent of the design patterns, you can do all the wonderful stuff with TCP without being in a behaviour.

OTP is actually quite an open system which is very amenable to development and extension. This is actually not surprising as it all implemented in Erlang and the Erlang ecosystem is a very egalitarian system: all modules are equal, all processes are equal, and all applications are equal. The only difference is who wrote them. There are of course some properties you can't change, for example how releases, applications and supervision trees interact, but within those limitations you can do a lot.

For example it is easy to create a new behaviour. So if you feel that gen_server does not give you the functionality you need or want then the easiest solution to create a new behaviour which has the right functionality. One way to this is to build your new behaviour on top of an existing behaviour. For example implement your behaviour on top of a gen_server using the gen_server callbacks to implement a base to call the callbacks in your new behaviour. There is nothing strange about this, it is done in many places. For example the supervisor is implemented as a gen_server behaviour. Doing it this way allows you to inherit a lot of functionality for free, the new behaviour will automatically follow all the rules.

Another way is of course to write the behaviour yourself. For example if the functionality you want does not fit nicely into an existing behaviour. This is also very easy, in some ways even easier than building on top of something else. All the tools, modules, which are used inside OTP are available. By using them and following a simple set of rules it easy to create a process which fits smoothly into a supervision and behaves exactly as it should to make OTP happy. Everything is also documented, look under the proc_lib and sys modules for starters.

Sometimes you don't need or want another behaviours then using these tools it of course easy to a create a singleton type of process. Again all the tools for doing this is there.

My point is that it is easy to extend OTP to provide the functionality you need without having to do any serious hacking of the system. Unfortunately too few people seem to know this. It is sometimes amazing to see the lengths people go to squeeze something into a gen_server when it would have been much easier to roll their own.

OTP was never meant to be the answer to (life, the universe and) everything, it "just" provides the basic building blocks and all the tools necessary to extend it. View the existing behaviours as a starter kit. So if you feel that OTP is lacking something then don't complain but fix it.

Robert