Cloud-related solutions were a jungle… or the leading path to Roboconf

I have been working on Roboconf for a little more than 2 years now.
And it is true I did not really talked about this project on my blogs. Sure, I mentioned it in articles related to OSGi or technical issues. But I never took time to introduce the project.

This post will allow me to describe the genesis of Roboconf.

About 3 years ago, my company was willing to develop a web-based enterprise solution to ease and improve communication and collaboration within organizations. Beyond all the usage things (tools, ergonomy, devices…) were also hidden technical « low-level » challenges, mainly scalability and adaptation.

It was clear there were moments where the platform would have to support heavier traffic while some others would imply a minimal activity. This is adaptation. Another challenge was about scaling: what was the maximum limit (e.g. number of simultaneous users, data storage…) the platform could support? Since my company works with important organizations (each one including many thousands of persons, geographically distributed), that was an important topic.

Having a Software stack that allows such possibilities was a first issue. There was a dedicated team for this. Having something that makes this stack effectively adaptative was another thing. It was my job and those of my team mates. Hopefully, this part of the project was totally in coherence with the trending topic back then: cloud computing.

Our first job was to study existing solutions.
We looked at IaaS parts (mainly Amazon Web Services, Openstack and Open Nebula). We looked at Software solutions, including Bash, Puppet, Software appliance generation, etc. We looked at PaaS solutions, like Heroku, Openshift… At the time, nothing was really matching what we were looking for. Something fully open source. Something that could manage infrastructure parts but also providing deployment and reconfiguration features, that could adapt the topology of a deployed application to a new, better topology. Something that could give our developers the full freedom of their choices and their architecture.

Clearly, IaaS were too low-level options. They were infrastructures. Bash and Puppet were system solutions, but required additional integration with cloud infrastructures. Software appliances were interesting but were too big and too complex to manage in multi-IaaS environments (even with formats like qcow2). Some solution providers had online services to host virtual images and push them to cloud providers. But having such a business dependency was not a valid option for my company. Heroku had several advantages for developers. But the solution to develop was too important, in terms of code and business matter. Openshift, on its side, was not as mature as it is now. Even solutions like Cloud Foundry and so on required a very high-level of expertise. Solutions like Right Scale and others were moved apart because they were not open source (or not fully open source).

No solution was fully satisfying.
And we were not entirely compliant with the general cloud layers. We needed IaaS. But what we wanted to build above was a mix between a PaaS and a SaaS. Our PaaS was made up of a management (what we called the « technical PaaS » : deployment, reconfiguration, administration, monitoring) and our Software stack (which databases, which application servers, which load balancers, which mail servers, etc). And on the other hand, we had mainly one SaaS application, with very tight links with the Software stack below.

With hindsight, we might have built our own PaaS from an existing one.
Now, all the PaaS that target development teams have defined buildpacks. This notion comes from Heroku and was adopted by Apache Stratos, Openshift, Cloud Foundry, etc. If we had to do it today, maybe would we investigate in this direction. But even with that, I am not sure we would be able to manage reconfiguration correctly.

The direction we took, three years ago, was another one.
We needed something to build our own PaaS. A minimal PaaS. A custom one. We needed a « PaaS framework ». We had contacts with Joseph Fourier University, in Grenoble (France). And they had a short demo that illustrated reconfiguration of a distributed application in a cloud context (Amazon Web Services). It was really a prototype, but there were very powerful ideas. This demo showed the premices of what could be a generic framework for cloud-based applications… and potentially for any distributed application that needed to adapt to its environment.

Roboconf is a bet Linagora started more than two years ago to address these issues. It was and is still developped in collaboration with Joseph Fourier University. This joint work took the shape of a co-localized team, with the main objective of providing a production-ready solution for elastic deployments. Roboconf has been used in various R&D projects. One of them is called Open PaaS and it is not so far from going in production. There will be new milestones before we can consider Roboconf as complete, but this will be part of future posts.