C++ to Microservices: Back-end Migration to Continuous Integration

What is Lyra?

Lyra is a state-of-the-art solution for managing payments and liquidity. It receives all payment operations (money moving from one bank to another) and communicates with European settlement systems, while maintaining an up-to-date view of the bank’s liquidity.

Technically speaking, Lyra is an application based on a LIST proprietary technology, FastTrade.

FastTrade is a software cluster that provides high reliability and load balancing of services on a set of heterogeneous nodes. Focusing on mission critical applications, its key feature avoids the incidence of single points of failure, by means of a distributed and redundant system.

Lyra’s features are divided into multiple integrated and scalable modules, each dedicated to various inter-connected settlement systems, in order to allow better monitoring and efficient management of operations.

What problem does Lyra solve?

The EU is working on a centralized liquidity management to be released in 2022. The technical name for this solution is “T2-T2S Consolidation”.

Lyra is a modern solution, based on a strong stack, designed to solve all issues. This software was created with the experience of a previous software solution, written in C++ and perfectly suited to the former banking environment. 

Let’s look at the current description of the problem, together with the requirements for the new solution, in more detail.

A single entry point to the European Banking System

According to the official documentation from Banca d’Italia, the T2-T2S Consolidation platform is a strategy to develop market infrastructures. This project was presented by the Eurosystem as part of Vision 2020, and involves the ‘consolidation’ of TARGET2 (T2) and TARGET2-Securities (T2S), in terms of both technical and functional aspects.

The 4CB (Four Central Banks – the Deutsche Bundesbank, Banque de France, Banco de España and Banca d’Italia) were entrusted with developing the consolidated platform. This work began in December 2017, following the European Central Bank Governing Council’s formal approval of the project. The project is currently scheduled to go-live in 2022. 

The completion of T2-T2S Consolidation will mean savings both for market players, who will be provided with new functions, and for the Eurosystem, since operating costs will decrease.

With T2-T2S Consolidation, the Eurosystem seeks to optimize the services currently provided by T2 and T2S, integrating the securities and cash settlement services. The system will also offer new functions involving liquidity management and support tools for reporting in order to meet current market needs. 

From a technical point of view, the most important innovations will address the implementation of the ISO 20022 standard for communication in T2 and the use of the advanced technology currently employed for T2S. Some components will also be shared, delivering savings and improved overall efficiency.

The new platform will provide technical and organizational solutions to make it more resilient, especially in terms of cyber security, in response to the escalation and evolution of cyber threats and in line with the Guidance on cyber resilience for financial market infrastructures published by international industry monitors CPMI-IOSCO.

Starting from C++ and a Desktop App

The previous back-end solution developed by LIST was based on C++ coding and a relational database. An old way to approach problem-solving, C++ software is normally thought of as a legacy system. Nonetheless, this approach already offered the possibility of writing good modular code.

The solution previously in use was a services architecture, with multiple processes communicating over the network, so migration to modern architecture was not as difficult as it’s often the case in a standard C++ migration.

The previous frontend solution involved a desktop app communicating with the back-end through a real-time proprietary protocol. The system could perform all required tasks in real time, and had a push feature linking the server to the desktop GUI. 

However, this solution needed to evolve towards a more modern, easy-to-manage web-based solution, based on queueing and websockets, that could be integrated with LIST’s code when required.

The new Lyra: attractive, performing, and CI-based 

Lyra needed to be fully updated to become a version that would be attractive to new potential employees, junior (young) developers and engineers. This step is fundamental to building a sustainable business today: any software editor needs to demonstrate that it’s a place where innovation is welcome, and sometimes even originates.

LIST’s main driver was a regulatory change: the ECB began a process of changing settlement systems in a large-scale project called Vision 2020, so the time was right to implement a new back-end to interface with this settlement house.

The new system includes many functional changes to the overall process, and an opportunity arose to redesign the back-end of the application wholesale, rather than simply adapting the existing back-end.

LIST created a new version of Lyra (liquidity management) switching from C++ apps server-side and UI for desktop to really up-to-date software architecture. All functions were redefined as microservices with SpringBoot, Java, and Kotlin, while ActiveMQ was selected to provide message brokering. 

The WebApp was redesigned with Angular and communicates through WebSockets.

One very important aspect of the application is its performance, especially in regards to throughput, as hundreds of thousands of messages have to be processed within a few hours.

The continuous integration (CI) paradigm was the first approach to be located correctly inside the new production flux. As a result, automated performance tests set to run nightly were included in the CI environment – the ideal solution for the new Lyra project.

Training on the new technological stack for the development staff occurred once this was in place. 

Building on an ACID relational database

The data management system is the solution’s central technology. Lyra is based on a relational Database Management System (DBMS); the solution manages payments, so transactionality is a must and anything less than full ACID (which stands for Atomicity, Consistency, Isolation, and Durability) could not be tolerated. The Oracle relational database is a reference of choice, and Oracle is fully ACID compliant.

This characterization for database properties is a de-facto standard and not a true compliance rule. Standing for Atomicity, Consistency, Isolation, and Durability, ACID describes a set of properties of database transactions that are intended to guarantee data validity despite errors, power failures, and other problems that could eventually arise. 

A transaction is a single logical operation on the data that is performed not atomically, but as a sequence of operations in the database that satisfies the ACID properties. For example, a transfer of funds from one bank account to another, even involving multiple changes such as debiting one account and crediting another, is a single transaction. 

This is the true foundation of the Lyra project.

Challenges and solutions

LIST’s software architects encountered many issues and had to solve them. In light of their prior experience, the new processes were designed around a well-defined, robust, attractive technological stack. It was clear that a move to a microservices architecture, to help with scalability, was the way forward.

The old C++ back-end was also quite modular and had been developed as a series of processes, so it was already a ‘halfway to microservices’ platform, and a good overall paradigm.

The goal was a large and complex system, so something reliable was required. LIST has a lot of experience building systems in Java, finding that the Java Virtual Machine offers very good performance, reliability and lots of parameters with which to fine-tune performances.

A fundamental requirement was that the new solution must support both ‘classic’ deployments for specific clients, on application servers such as JBoss, and more modern deployments, such as using containers with Docker on Kubernetes for internal testing environments, for other clients. Spring Boot allows this to be done easily.

Red Hat’s JBoss Web Server combines the world’s most deployed web server (Apache) and a performing servlet engine (Tomcat) with Red Hat support in middleware.
This allows for a secure, more stable environment with enterprise-level features, able to deliver large-scale websites and lightweight WebApps.

WebSockets were introduced to push the real-time position of the back-end onto the web application. The old UI – a desktop application that communicated with the server using a proprietary protocol – allowed real-time push of data. Users therefore also expected this kind of feature in the WebApp. 

The choice of tool to achieve this goal fell on ActiveMQ, in its Artemis version, as the messaging broker for both inter-process communication and pushing updates to the UI. ActiveMQ makes writing from a Java application and receiving the data from the browser easy, using WebSockets.

Why Spring Boot rather than Quarkus or Micronaut?

Spring Boot is an open-source, microservice-based Java web framework. The Spring Boot framework creates a fully production-ready environment that is completely configurable, using prebuilt code within its codebase. 

The microservice architecture provides developers with a fully enclosed application, including embedded application servers.

Spring Boot is an improvement on the extant Spring framework. Extensions include pre-configurable web starter kits that facilitate the responsibilities of an application server required for other Spring projects and improve the application’s use within the developer ecosystem. 

LIST has a lot of experience with Spring Boot and uses it for many applications.

Spring Boot is a great framework; the modern releases are excellent and have been streamlined from the old XML-heavy versions of ten years ago. The framework now allows developers to focus on applicative code, without getting in the way too much. 

The new Spring Boot is user-friendly and has a lot of useful features built-in, such as easy configuration via environment variables or properties files.

Quarkus and Micronaut are currently under evaluation, as their features are compared with those that motivated LIST’s choice of Spring Boot. Improvement is a way of coding at LIST.

Kotlin, a modern Java by another name

Java is a very good technology for developing software systems. Java is familiar and appreciated at LIST, but it is a bit ‘verbose’ when compared to modern languages. 

Kotlin is a great language: it really looks like a 2020 version of Java. It’s very expressive, without being verbose or too cryptic. Kotlin was designed inside the industry and not in academia. Although Kotlin is not the only route to modernised Java, it’s much more user friendly than other similar languages such as Scala or Groovy. 

Chosen as the official Android language in the Android environment, Kotlin’s reach has been increased and its robustness confirmed. One particularly strong point is its high level of interoperability with Java itself.

Another feature to check in modern languages is their nullability. No data types in Kotlin include the null value. To use a null value, the type must be explicitly marked as nullable with a question mark.

LIST believes that the way Kotlin manages nullability is extremely well designed.  The process really helps to prevent bugs: an experiment on this subject started a couple of years ago – so far the feedback from developers has been very good.

Conclusions

Most legacy software is written in old languages such as C++. Porting old systems to a modern architecture based on microservices can look like a straightforward process, but it requires a great deal of knowledge to solve all the issues of each real solution. 

The technological stack is simultaneously the toolset for building and the open door for skilled and ambitious developers, DevOps, and the like.

It may be that the most frequent choice of database, or language, is the one that will allow quick delivery and long-term hassle-free maintenance. 

Lyra is a solution based on a relational database and the Kotlin language. The solution solves the problems faced by new banking systems that need to be up-to-date and ready for the future.