So you’ve decided to migrate from monolith to microservices, what next? Such a redesign to service-oriented architecture (SOA) is a long, arduous journey that benefits from an incremental, iterative approach. Yet, such a microservices migration often must be done while still shipping new features, accelerating developer velocity, and growing the team in addition to ensuring there are no performance regressions.
Jessica Tai is a software engineer at Airbnb and works as an an infrastructure engineer on the Core Services team. She leads the user service, which integrates with all business verticals as one of Airbnb’s highest trafficked services. In addition to driving best practices for service development, she focuses on Leadership & Development as the Vice President of the Airbnb women in tech community.
At Codemotion Milan, she shared the story of Airbnb‘s journey focusing on how Airbnb is building, operating, and scaling its expanding network of services and changes to its organization to support the migration. We sharing some of the main themes and points here, but you can gain the full story by watching her presentation below.
The early days at Airbnb
Jessica shares,
“When I joined Airbnb in 2014, the engineering team was much smaller, less than 100 people. We had a new hire tradition of welcoming employees to the company by having them run underneath what we call a human tunnel. At the end of this tunnel was a large beanbag for engineers to jump onto while we all clapped and cheered. After our new hires completed their human tunnel run, we gave them an onboarding task and our monolithic code base. At the time, almost all of our engineers were coding and our monolith was daily.
Now in 2019 Airbnb has over 2000 engineers across the world so the human tunnel is no more.
Why you need a microservices migration?
Jessica details:
“At Airbnb, our monolithic application was a Ruby on Rails app. We refer to this as monorail. A monolith is a single tear unit responsible for both client and server-side functionality. This means that the model view and controller layers are together in a single application.
And this was a really easy place for me to get started. It was super convenient with the presentation layer being in the same place as the business logic and the data accessing. Airbnb had a really large database that had All types of data stores shared in a single place.”
In terms of the booking flow, Jessica detailed the work that preceded the microservices migration: “My onboarding task was to create a feature requiring a guest to send a message to their host. This required some new text in the view, accessing the messaging data model and some business validation logic. Together my new hire changes shipped in one up MonoRail so if MonoRail could handle everything, why did we bother taking the time to move away from it?”
The short answer is for the people. Airbnb‘s engineering team was growing rapidly, often doubling year over year. The need of the business was also evolving quickly. Jessica explains, “So we needed an architecture that could move quickly and grow with our team. In MonoRail, we were seeing difficulties with more production incidents. Engineers had to spend more time firefighting instead of developing features. Our codebase was tightly coupled and difficult to navigate and debug. It also took a long time to deploy our code. When I first joined, I could deploy a code change on MonoRail in just a few minutes. Now it took several hours just to get a single change out.
This made engineers very frustrated and productivity was at an all-time low. So we need to figure out a way to improve our architecture so we could continue growing our business.” Enter microservices.
Migrating to Microservices from a Service Orientated Architecture
Airbnb opted for service-oriented architecture (SOA) – a network of loosely coupled services, where some client makes a request flows through an API gateway, and that fans out to various other services, which can call services or other data stores. According to Jessica, “the benefits that we saw for SOA is that each service is built independently and can be scaled separately as well. Smaller teams of engineers could work on a service so it’s much easier to manage the changes. We hoped this would help solve our developer velocity problems, as well as being able to introduce new verticals.”
Using an SOA lens we could now break a checkout page into different components. An example is the reservation service pricing service. And now there are beginning to look like a lot of services on this one page that used to be handled by a single application. How can we ensure that we’re not actually making a bigger mess and entanglement for ourselves?
Principles of Service Design applied to Microservices Migration
- Read/Write: according to Jessica, “The service should own both the reads and writes to its own data. This means if another service is interested in particular services data, they must go through the owning services API. This helps with consistency and encapsulation of our data.
- Services should also address a specific concern. “We wanted to avoid taking our monolith and switching to really tiny granular microservices.”
- Avoid duplicate functionality: “We wanted our services to be larger and focus on business functionality, but not to duplicate this functionality amongst multiple services.”
- Data mutation should propagate via standard events. “So what this means is if there’s a change to a data store and another service is interested in, it should be able to find out about this change in an asynchronous way. “
Jessica shared:
“At Airbnb, we’ve developed a Change Data Capture service called Spinal Tap. Spinal Tap listens to various data sources and with low latency emit to a standard event. In our case, we use Kafka. No other services can then consume. So an example of this is if I were to book a home on Airbnb, we would create a row in our reservation data store. Our calendar availability service would want to know that this new reservation so it could mark those days as busy so it would listen to the standard events that our reservation data emitted.”
Good microservices means building for production: we wanted to avoid the temptation of cutting corners for services that might be prototypes, or for internal admins only. Instead, we need to have a production checklist for resiliency, robustness, alerts observability, that all services should be adhering to, regardless if they’re in the critical path or for something more internal.
