CumuloNimbo is a European project that has been funded by the European Commission under the 7th Programme Framework (FP7). The project has run from 1st Oct. 2009 to 30th Sept. 2013. The project has attained outstanding results. In particular, it has solved one of the most demanding problems for cloud databases, the scalability of transactional and SQL processing.
The project results include:
- A distributed transactional management system that is able to manage very high rates of update transactions in the order of a million transactions per second and as many read-only transactions as required.
- Three APIs integrated with the transactional processing:
- A fully ACID transactional version of HBase. This empowers HBase with arbitrary multi-item transactions. Now an HBase application can fully benefit of full ACID transactions.
- A full SQL and fully ACID database. That is, an ultra-scalable database providing standard SQL with full ACID support. Transactions can access any arbitrary set of data. Now regular database applications can get seamless scalability without any change.
- A full Java EE stack that enables to scale automatically Java EE applications. We have integrated JBoss application server on top of our ultra-scalable datatabase to provide a fully scalable cloudified version of JBoss. With this you can run any arbitrary Java EE application and scale it automatically without changing a single comma.
- The creation of a startup to commercialize the core innovations from this project.
These results enable to attain high scalability (100s to 1000s of nodes able to serve millions of clients) but without trading off consistency as it is the norm in today's clouds. In this way CumuloNimbo results provide ease of programming, consistency and scalability at the same time.
The targeted PaaS will be a multi-tier software stack for cloud computing providing the same functionality to current, software multi-tier stacks, such as Java EE, SAP Netweaver or .NET. Providing scalability without trading off consistency is a major breakthrough that will enable European stakeholders in service platforms, such as SAP, to position in the cloud computing market with a competitive advantage.
The CumuloNimbo PaaS will compete with the most recent developments in the PaaS arena that scale at the cost of weakening data consistency and/or constraining the programming model of applications.
CumuloNimbo approach lies in deconstructing transactional processing at fine granularity components and scaling each of these components in an independent but composable manner. The architecture consists of several tiers:
- Application server: This tier contains both a component container such as an Java EE container and a persistent manager such as a JPA manager. The project has exercised JBoss application server and Hibernate.
- Object cache: This is a distributed object cache (in fact, it is a generic scalable and elastic data grid with multi-version concurrency control) to improve the performance of the application server.
- Query engine: This tier consists of a number of query engines able to process SQL. The query engine has been extracted from the Apache Derby database.
- NoSQL data store: This tier caches acts as API to the storage. Apache HBase has been used for this tier.
- Distributed file system: This is the file system where tables are stored. Apache Hadoop File System (HDFS) has been used for this purpose.
- Storage: The storage subsystem.
- Communication: The communication subsystem.
- Transaction management: Based on an innovative transactional processing protocol orchestrates transactional management at holistic level in a highly scalable way.
- Platform Management Framework: Enables to make deployments and monitor them. It takes care of reconfiguration actions such as elasticity, load balancing and fault-tolerance.
This project has been partially funded by the European Commission under the 7th European Framework contract number FP7-257993.