This article, developed together with Be Digitech Solutions, analyzes the gRPC architecture created by Google, the special advantages it offers compared to REST architecture, and how we can migrate from REST to gRPC in terms of microservices too.
Introduction: what is an API?
APIs are essential to the world of web and software programming. In fact, we hear about APIs whenever we talk about (web) development. Before analysing the differences between the REST API and the gRPC API, it is necessary to define what API means.
We can say that an API is a kind of software intermediary that allows two components to communicate with each other.
This article focuses on “Web APIs”, i.e. the exchange of information between a client and a server to request, receive, and process certain data that are useful for our application. Let’s take a look at two big names in the Web API context: REST and gRPC.
API REST HTTP vs. API GRPC HTTP2
When talking about API construction, there are two distinct methodologies: REST and RPC. While there are also alternatives, such as SOAPand GraphQL , these will not be discussed in this article.
Currently, the REST architecture is one of the most commonly used in generating web APIs.
REST (which is an acronym for Representational State Transfer) is a type of software architecture that provides simple and uniform interfaces which can be implemented to make data and digital assets available through a web URL. REST describes a set of guidelines for creating stateless, reliable web APIs. A web API that implements REST constraints is informally described as ‘RESTful’. In a typical RESTful Web service, requests are sent to a resource’s URI and produce a response that may be formatted in HTML, XML or JSON, although JSON format is the most commonly used. In REST architecture, the most common protocol for requests and responses is HTTP, which provides operations (commonly called HTTP methods) such as GET, POST, PUT, and DELETE.
- GET: retrieves data matching the resource URL
- POST: sends data to the server to create a new entry
- PATCH: makes a partial update to an existing resource
- PUT: updates the entire existing resource
- DELETE: deletes the resource specified by the resource URL
REST APIs abide by the following design constraints:
- statelessness: a request to a server needs to contain all the information necessary to process it, as servers do not store client request data
- client-server decoupling: client and server should be two separate and independent entities
- cacheability: the response states whether or not it is cacheable. This prevents future superfluous and useless requests to the server, when data is already cached
- layered: client and server don’t know if they are communicating directly with each other, or if there are other layers of intermediaries between them
- uniformity: data is transferred in a standardised form
The RPC (acronym for Remote Procedure Call) model is the classic – and the oldest – API style currently in use. RPC wields procedure calls to request a service from a remote server via direct actions to the server, in the same way you would request from a local system. In distributed systems, a remote procedural call involves a client program calling a function for which implementation is in a remote server. RPC is an efficient way to build APIs, using lightweight messages with straightforward interactions.
A common RPC architecture works like this:
- the client calls a stub that has the same signature as the function in the server
- the stub encodes the function arguments in a message that is passed to the server through the network – this process is called marshalling
- the server decodes the message and calls the function, passing the given arguments to it. This process is called unmarshalling
- the function returns the value which is then passed back to the client as a response
In Web services, RPC-style APIs focus on actions. Each endpoint represents an action the client can perform on the server. Although REST was built to overcome some of the limitations of RCP, providing an improved format for interacting with many systems and reducing tight coupling with the underlying systems, as the number of microservices in the infrastructure increases, the service graph becomes more complex, and as a result, there pain points and limitations arise in using REST for communications. For example:
– when a client makes a call to a service, the service could in turn make calls to other services to provide a response to the client
– for every REST call between the services, a new connection is established and there is the overhead of an SSL handshake. This would cause an increase in overall latency
– the client has to be implemented in every language required for each service, and also has to be updated whenever there is a change in the API definition
– JSON payloads are simple messages that have relatively slower serialisation and deserialisation performance
Moreover, there are many other HTTP verbs (OPTIONS, HEAD, TRACE, CONNECT) to complete the common REST set of verbs, and the list of HTTP status codes on which REST is based is vast. To make things worse, some old browsers do not support all the HTTP verbs and some company networks actually block PUT or PATCH requests, forcing users to rely on POST instead.
These were the main reasons preventing REST from replacing simple and lightweight RPC, and among the main reasons for the later emergence of a new, modern approach: gRPC from Google.
gRPC (Google Remote Procedure Call) is an open-source extension of the RCP architecture. This technology follows the implementation of an RPC API that uses the HTTP 2.0 protocol.
Designed by Google to achieve high-speed communication between microservices, gRPC allows developers to integrate services programmed in different languages. gRPC uses the Protobuf (protocol buffers) messaging format, which is a highly-packed, highly-efficient messaging format for serialising structured data. For a specific set of use-cases, a gRPC API can serve as a more efficient alternative to a REST API.
From REST to GRPC: why migrate?
The use of gRPC technology allows the introduction of new, specific advantages over the REST architecture. The main success of gRPC essentially derives from two factors: the use of the HTTP / 2 protocol instead of HTTP / 1.1 and the implementation of protocol buffers (Protobuf) as an alternative to traditional XML and JSON technologies. Let’s see what essential advantages these two features offer, and consider the resulting advantages.
HTTP/2
REST APIs are usually built on HTTP 1.1, which uses a request-response model of communication. This means that when a microservice receives multiple requests from more than one client, it has to serve them one at a time, slowing the entire system, gRPC upgrades the overall quality of the old RPC structure by using HTTP/2. This specification was published in 2015 and improved on the 20-year old HTTP/1.1 design. Here are some of the advantages offered by this advanced protocol:
– binary framing layer: HTTP/2 communication is divided into smaller messages and framed in binary format. In HTTP/2 elements such as headers, methods, and verbs are encoded in the binary format during transmission. Each message consists of several frames. Unlike text-based HTTP/1.1, sending and receiving messages is compact and efficient
– multiple parallel requests: HTTP/1.1 allows the app to process just one request at a time, while HTTP/2 supports multiple calls via the same channel. Beyond that, communication is bidirectional since in HTTP/2 a single connection can send requests and responses at the same time
– real-time streaming: HTTP/2 not only enables real-time communication, but – thanks to binary framing – enhances this with high performance. Each stream is divided into frames that can be prioritised and run via a single TCP connection. This improves overall performance by cutting down network utilisation and processing load. gRPC makes three types of streaming available: server-side, client-side and bidirectional
Protobuf (Protocol Buffers)
Protocol buffers are a popular technology for structuring the messages developed and used in nearly all inter-machine communication at Google. In gRPC, protocol buffers (protobufs) are used instead of XML or JSON in REST. Protobufs offer some significant advantages over both JSON and XML:
– parsing with high performance: since data is represented in a binary format, the size of encoded messages is minimised; protocol buffers offer much less CPU-intensive parsing. The communication exchange happens faster, even in slower CPU devices like IoT or mobile devices, since data is transmitted as quickly and compactly as possible, with a focus on data serialisation and deserialisation making gRPC very valuable when working with microservices
– scheme obligation: in gRPC, the data structure is described in a schema. This schema is saved in a .proto text file that uses numbers instead of field names for storing data. Using a schema, users can be assured that the message has arrived correctly at its destination and that its components remain unchanged in different services
The implementation of the HTTP / 2 protocol and the protocol buffers provide several derived advantages:
- lightweight messages: gRPC-specific messages can be up to 30 percent smaller in size than JSON messages.
- high performance: gRPC may perform five, seven, or even eight times faster than REST+JSON communication
- more connection options: as described earlier, while REST focuses on request-response architecture, gRPC provides support for data streaming with event-driven architectures allowing server-side streaming, client-side streaming, and bidirectional streaming
Some of the important considerations when migrating services to gRPC include:
– a minimal amount of rework is required, and no change to the current deployment pipeline
– enable REST first together with gRPC and then gradually migrate all communications to gRPC. This allows existing clients to continue to talk to servers via REST, until they are migrated
– compliance with the same base platform for gRPC as for REST microservices
Advantages of a gRPC structure in Microservices
gRPC was created by Google as a variant of the RPC architecture in order to speed up data transmission between microservices and other systems that need to interact with each other. gRPC is mostly commonly used for communication between microservices because it is available in multiple languages and offers high performance.
The advantages of using gRPC with microservices include:
– increased speed.
As previously stated, gRPC is much faster compared to traditional JSON-based REST APIs. A gRPC architecture is based on a binary protocol-buffer format which is very lightweight compared to text-based formats such as JSON. For example, the payload below takes almost three times as long with JSON encoding when compared with the same payload in the protocol buffer:
{
"name": "Riccardo",
"age": 100,
"skills": ["front-end", "back-end"]
}
– strong typing
The gRPC architecture introduces a well-built service interface and strong typing, which in turn allows the prevention of run-time errors. For the client application, it’s as good as calling the local method.
– polyglot capability
gRPC supports multiple programming languages, enabling users to write microservices in both the programming language and their preferred stack without worrying about interoperability issues
– top-quality features
gRPC has in-built support for resiliency, authentication, encryption, compression, load-balancing, and error handling. Furthermore, many modern frameworks provide native support for gRPC
Conclusions
Although all technologies have advantages and disadvantages, the gRPC architecture is a winning choice that offers major advantages over a REST architecture when certain conditions are met. These advantages include:
- high-performance communication
- support for multi-language environments
- real-time communication services
- dealing with streaming calls
- avoiding forcing technology choices on clients
- provision of a more modern alternative to previous RPC architecture
gRPC is simply the best choice for microservices communication. Microservices allow the use of different technologies for each service, and these services should coordinate with the API to exchange data, data format, error patterns, load balancing, and so on. gRPC allows service contracts to be described in a binary format, creating a standard specification method for those contracts that is independent of any language. High performance and its polyglot nature make gRPC a top choice for microservices infrastructure.
All of these conditions are certainly enclosed within microservices, leading several big companies (such as Dropbox, Netflix, Lyft, WePay) to migrate to gRPC.
