Inter-Service Communication for Microservices

In a monolithic application, all parts of the app access a shared database. Each part can easily invoke the functionality of another part. In a microservices architecture, an app is composed of many microservices, each potentially managing its own database. What happens if one service requires data or processing from another service? This is not as trivial or efficient as in a monolithic application.

Inter-service communication (ISC) is an important consideration when designing a microservices-based application. A badly designed app can result in a lot of communication among the different services, resulting in a chatty app or chatty I/O. Communication can be reduced by having fewer microservices but this replaces the monolith with smaller monoliths. The goal is therefore to achieve a balance by following good design principles and patterns.

Discussion

  • Why do microservices need to communicate with one another?
    A taxi booking app is made of many communicating microservices. Source: Richardson 2015.
    A taxi booking app is made of many communicating microservices. Source: Richardson 2015.

    In the traditional approach to building monolithic applications, lot of communication was internal to the app. Such communication was often local, fast and easily manageable. When designing a microservices architecture, we break up the monolithic into independent parts, each of which has a well-defined role. While each microservice can be deployed and scaled independently, none of them deliver the full value of the application. A microservice will often require data managed by another, or require the services of another.

    For example, consider a taxi booking application. Trip management and passenger management are separate microservices but a trip cannot be initiated without some knowledge or authentication of the passenger. Hence these two independent microservices, each doing its specific roles, will still need to communicate.

    While microservices architecture has brought benefits to build large-scale applications, it has also exposed the communication across microservices. Complexity that was previously hidden is now visible. Dozens or even hundreds of microservices that make up an app must be "wired together" properly to make the whole thing work.
  • What are the different types of inter-service communication for microservices?
    Illustrating the call flows of sync and async communication. Source: Walking Tree Technologies 2018.
    Illustrating the call flows of sync and async communication. Source: Walking Tree Technologies 2018.

    Broadly, there are two types:

    • Synchronous: Client sends a request and waits for a response. Client code execution itself may prefer to receive the response via a callback (thread is not blocked) or wait (thread is blocked). Either way, the communication to the external world is synchronous.
    • Asynchronous: Client sends a request and doesn't wait for a response.

    With synchronous communication protocols, the receiver has to be available to send back the response. From application perspective, synchronous implies a less responsive user experience since we have to wait for the response. If one of the services in a chain of synchronous requests delays its response, the entire call flow gets delayed.

    With asynchronous communication protocols, the request (often called message) is typically sent to a message queue. Even if the receiver is not available, the message will remain in the queue and can be processed at a later time. Even if a service fails to respond, the original asynchronous call is not affected since it's not waiting for a response.

  • What protocols and data formats are suitable for inter-service communication for microservices?
    Different interaction styles among microservices. Source: Richardson 2015.
    Different interaction styles among microservices. Source: Richardson 2015.

    HTTP/HTTPS is a synchronous protocol. Often service APIs are exposed as REST endpoints. AMQP and MQTT are examples of asynchronous protocols. To manage the queue, we can use RabbitMQ message broker. Instead of a message queue, we can also use an event bus for updating data asynchronously.

    Synchronous protocols are usually limited to one-to-one interactions. Asynchronous protocols have more options: one-to-one (notifications), one-to-many (publish/subscribe), or even allow for responses coming back asynchronously. For example, a user sends a tweet on her Twitter account that has many followers. This is an example of one-to-many publish/subscribe model.

    The data that these protocols carry must be formatted in a manner understood by all. Text-based formats include JSON and XML. XML in particular is very verbose. Therefore, some implementations may prefer binary formats: MessagePack, Thrift, ProtoBuf, Avro. Note that these are well-known and popular formats and using them enables easier integration of microservices. It's also possible (but not preferred) to use a proprietary non-standard format internal to an application.
  • What design patterns are available for inter-service communication for microservices?

    Here are a few patterns to note:

    • Saga Pattern: A sequence of transactions, each one local to its database. A microservice triggers another via an event or message. If something fails, reverse operations undo the changes.
    • API Composition: Since table joins are not possible across databases, a dedicated service (or API gateway) coordinates the "joins", which are now at application layer rather than within the database.
    • Command Query Responsibility Segregation (CQRS): Services keep materialized views based on data from multiple services. These views are updated via subscribed events. CQRS separates writes (commands) from the reads (queries).
    • Event Sourcing: Rather than store state, we store events. State may be computed from these events as desired. This is often used with CQRS: write events but derive states by replaying the events.
    • Orchestration: A central controller or orchestrator coordinates interactions across microservices. API Composition is a form of orchestration.
    • Choreography: Each microservice knows what to do when an event occurs, which are posted on an event stream/bus.
    • Service Mesh: Push application networking functions down to the infrastructure and not mix them with business logic.
  • Could you compare orchestration and choreography?
    Comparing orchestration and choreography for microservices. Source: Adapted from Bonham 2017.
    Comparing orchestration and choreography for microservices. Source: Adapted from Bonham 2017.

    Orchestration is a centralized approach. Calls are often synchronous: orchestrator calls service A, waits for response, then calls service B, and so on. This is good if service B depends on data from service A. However, if service A is down, service B can't be called. By coupling B with A, we've created a dependency. The orchestrator also becomes a single point of failure.

    Choreography enables peer-to-peer interactions without a centralized controller. It's more flexible and scalable than the orchestration approach. It's event-driven architecture applied to microservices. The logic of handling an event is built into the microservice. Choreography is asynchronous and non-blocking. The patterns CQRS and Event Sourcing are applicable to choreography.

    There are also hybrid approaches where a service orchestrates a few services while it interacts with others via an event stream. In another approach, an orchestrator emits events for other services and consumes response events asynchronously from the event stream for further processing.

    To conclude,

    Orchestration is about control whereas choreography is about interactions.
  • How do we handle service API calls that fail?
    Circuit breaker can isolate failures and help in recovery. Source: Gopal 2015.
    Circuit breaker can isolate failures and help in recovery. Source: Gopal 2015.

    The simplest solution is to retry after a specified timeout. A maximum number of retries can be attempted. However, if the operation is not idempotent (that is, it changes application state), then retry is not a safe recovery method.

    The other approach is to use a circuit breaker. Many failed requests can result in a bottleneck. There's no point sending further requests. This is where we "open the circuit" to prevent further requests to a service that's not responding.

    We can also proactively reduce chances of failure by load balancing requests. A request must be processed by a service instance and we can select an instance that has less load. Container orchestrators (Kubernetes) or service meshes (Istio) enable this.

  • Are there any best practices for defining a service API?

    Microservices must be designed to be independent of one another. One approach is to use Domain-Driven Design. This talks about understanding the problem space, using design patterns, and refactoring continuously. API should model the domain. It shouldn't leak internal implementations.

    APIs must have well-defined semantics and versioning schemes. A microservice can support multiple versions of an API or you could have a service for each version. Public APIs are usually REST over HTTP. Internal APIs can adopt RPC-style, where remote calls can look like local calls. However, they should be designed right to avoid chattiness. Consider the trade-off between making many I/O calls and retrieving too much data.

    Since application state is now distributed across microservices, design for and manage Eventual Consistency.

    While REST calls may use JSON, RPC calls can be more efficient with binary formats enabled by RPC frameworks such as gRPC, Apache Avro and Apache Thrift. To simplify API design and development, use an Interface Definition Language (IDL). This will generate client code, serialization code and API documentation.

  • What are some anti-patterns to avoid when microservices need to communicate?
    Canonical data model is an anti-pattern. Source: Harsanyi 2017.
    Canonical data model is an anti-pattern. Source: Harsanyi 2017.

    Sharing a database across many microservices is an anti-pattern since this introduces tighter coupling. A single data model for all microservices is another. Using synchronous protocols across many microservices increases latencies and makes your app brittle to failures. If microservices are not properly defined, this may result in chatty I/O that affects performance and responsiveness.

    An application may depend on hundreds of shared libraries. In the spirit of code reuse, all microservices may be relying on these libraries. This results in another anti-pattern called distributed monolith. Reusing code within a domain or service boundary is fine but anything beyond that is coupling. This form of coupling is worse than code duplication. Shared libraries can be considered but not made mandatory in some areas: logging, tracing, routing.

    It's not required that every event should contain full data, particularly when consumers are going to use only some of it. Consider sending essential data and URLs pointing to additional data. To communicate, consider REST plus its alternatives such as messaging and event sourcing.

  • What tools can I use to implement inter-service communication for microservices?

    Among the message brokers are RabbitMQ, Kafka, ActiveMQ, and Kestrel. Cloud providers offer their own messaging systems such as Amazon SQS, Google Cloud Pub/Sub, and Firebase Cloud Messaging (FCM). Microsoft Azure offers Event Grid, Event Hubs and Service Bus for messaging. NATS, a CNCF project, is an open source messaging system.

    Istio enables service mesh technology. Azure Service Fabric Mesh is an alternative. These rely on Envoy as the networking proxy. Similar proxies include Linkerd and Cilium. Conduit is a service mesh designed for Kubernetes.

    Netflix's Conductor can help with orchestration. For logging and monitoring, we have Retrace, Logstash, Graylog, and Jaeger. OpenTracing is an API that enables distributed tracing. Circuit breaker pattern can be implemented with Netflix's Hystrix.

    To define service APIs, Swagger can be used. In Java, REST-based microservices can be created with Spring Boot.

Milestones

2004

Eric Evans publishes Domain-Driven Design: Tackling Complexity in the Heart of Software. The book relates directly to object-oriented programming. Only later, it's relevance to microservices would be understood.

2007

Michael Nygard explains the circuit breaker pattern in his book Release It!: Design and Deploy Production-Ready Software. This helps us build fault tolerant systems. In 2011, Netflix invents the Hystrix framework that includes the circuit breaker pattern.

2009

Netflix embraces API-driven architecture that affects both development and operations. This is today seen as the birth of microservices.

2010

At the start of this decade, the three-tier model (web tier, app tier, database tier) is found to break under heavy load. Microservices come to the rescue but they also bring problems relating to inter-service communication. Companies introduce libraries that are built into microservices to handle the networking aspects: Google's Stubby, Netflix's Hystrix, Twitter's Finagle. This however introduces coupling. A few years later these evolve to networking proxy and service mesh.

Jan
2016

Version 0.0.7 of Linkerd is open sourced on GitHub. It's based on Twitter's Finagle and Netty. This is one of the early beginnings of a service mesh. Likewise, Istio version 0.1.0 is released as an alpha version in May 2017.

References

  1. Bhowmick, Payel. 2018. "4 Message Brokers You Need to Know." SpringPeople, March 15. Accessed 2019-02-15.
  2. Bonham, Andrew. 2017. "Microservices — When to React Vs. Orchestrate." Capital One Tech, on Medium, January 23. Accessed 2019-02-13.
  3. Daniel, Chacko. 2018. "Exploring Azure Service Fabric Mesh: A Platform for Building Mission Critical Microservices." InfoQ, December 01. Accessed 2019-02-15.
  4. Fowler, Martin. 2014. "CircuitBreaker." March 06. Accessed 2019-02-15.
  5. Fowler, Martin. 2015. "Microservice Trade-Offs." July 01. Accessed 2019-02-15.
  6. Gopal, Senthilkumar. 2015. "Application Resiliency Using Netflix Hystrix." Blog, eBay, September 08. Accessed 2019-02-15.
  7. Harsanyi, Teivah. 2017. "Why is a Canonical Data Model an Anti Pattern." Medium, September 05. Accessed 2019-02-13.
  8. Harsanyi, Teivah. 2018. "1 Year of Event Sourcing and CQRS." Hacker Noon, June 11. Accessed 2019-02-13.
  9. Hystrix Wiki. 2017. "Wiki Homepage." Hystrix Wiki, on GitHub, July 04. Accessed 2019-02-15.
  10. Indrasiri, Kasun. 2016. "Microservices in Practice - Key Architectural Concepts of an MSA." WSO2, August. Accessed 2019-02-13.
  11. Janssen, Thorben. 2017. "Communication Between Microservices: How to Avoid Common Problems." Stackify, September 21. Accessed 2019-02-13.
  12. Joliveau, Matteo. 2018. "Microservices communications. Why you should switch to message queues." Dev.to, February 23. Accessed 2019-02-13.
  13. Khan, Aslam and Obi Oberoi. 2016. "Domain-Driven Design." DZone Refcard #076, February 12. Accessed 2019-02-13.
  14. Kröhling, Juraci Paixão. 2018. "Distributed tracing in a microservices world." Opensource.com, September 20. Accessed 2019-02-15.
  15. Microsoft Docs. 2017. "Chatty I/O antipattern." Microsoft Azure, June 05. Accessed 2019-02-13.
  16. Microsoft Docs. 2018a. "Communication in a microservice architecture." .NET Guide, Microsoft, September 20. Accessed 2019-02-13.
  17. Microsoft Docs. 2018b. "Designing microservices: Interservice communication." Microsoft Azure, October 23. Accessed 2019-02-13.
  18. Microsoft Docs. 2018c. "Designing microservices: API design." Microsoft Azure, October 23. Accessed 2019-02-13.
  19. Microsoft Docs. 2018d. "Challenges and solutions for distributed data management." .NET Application Architecture Guide, September 20. Accessed 2020-07-23.
  20. Microsoft Docs. 2019. "Choose between Azure messaging services - Event Grid, Event Hubs, and Service Bus." Microsoft Azure, January 30. Accessed 2019-02-15.
  21. Morgan, William. 2018. "The History of the Service Mesh." The New Stack, February 13. Accessed 2019-02-15.
  22. Osowski, Rick. 2018. "Introduction to microservices." IBM Developer, November 10. Accessed 2019-02-15.
  23. Posta, Christian. 2017. "Application Network Functions With ESBs, API Management, and Now.. Service Mesh?" Software Blog, August 04. Accessed 2019-02-13.
  24. Richardson, Chris. 2015. "Building Microservices: Inter-Process Communication in a Microservices Architecture." NGINX Blog, July 24. Accessed 2019-02-13.
  25. Richardson, Chris. 2017a. "Pattern: Database per service." Microservices.io. Accessed 2018-10-07.
  26. Richardson, Chris. 2017b. "Pattern: Saga." Microservices.io. Accessed 2018-10-07.
  27. Richardson, Chris. 2017c. "Pattern: Shared database." Microservices.io. Accessed 2019-02-13.
  28. Stenberg, Jan. 2016. "Microservices Ending up as a Distributed Monolith." InfoQ, February 24. Accessed 2019-02-13.
  29. Stenberg, Jan. 2018. "Strategies for Microservices Communication." InfoQ, August 16. Accessed 2019-02-13.
  30. Thorpe, Stefan. 2018. "29 Top Tools for Building Microservices on All Levels." DZone, August 27. Accessed 2019-02-13.
  31. Vengalasetty, Krishna. 2017. "Orchestration Vs Choreography." Technology Zone, on WordPress, May 12. Accessed 2019-02-13.
  32. Walking Tree Technologies. 2018. "Inter-service communication in Microservices." Medium, August 16. Accessed 2019-02-13.
  33. istio GitHub. 2019. "istio/istio." istio, GitHub, February 15. Accessed 2019-02-15.
  34. linkerd GitHub. 2019. "linkerd/linkerd." linkerd, GitHub, February 02. Accessed 2019-02-15.

Further Reading

  1. Posta, Christian. 2017. "Application Network Functions With ESBs, API Management, and Now.. Service Mesh?" Software Blog, August 04. Accessed 2019-02-13.
  2. Indrasiri, Kasun. 2016. "Microservices in Practice - Key Architectural Concepts of an MSA." WSO2, August. Accessed 2019-02-13.
  3. Plöd, Michael. 2018. "Reducing Chattiness: Communication Strategies for Microservices." GeeCON Conference, on YouTube, July 23. Accessed 2019-02-13.
  4. Mińkowski, Piotr. 2017. "Communicating Between Microservices." DZone, December 19. Accessed 2019-02-13.
  5. Marini, Thiago. 2017. "Doing CQRS + Event Sourcing without building a spaceship." Software Engineering, on Medium, September 04. Accessed 2019-02-13.
  6. Gupta, Priyank. 2018. "Patterns for Microservices — Sync vs. Async." DZone, June 05. Accessed 2019-02-13.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
Avatar of user arvindpdmn arvindpdmn
6
0
3002
Avatar of user devbot5S devbot5S
1
1
20
DevCoins due to articles, chats, their likes and article hits are included.
1937
Words
2
Authors
7
Edits
1
Chats
14
Likes
27K
Hits

Cite As

Devopedia. 2022. "Inter-Service Communication for Microservices." Version 7, February 15. Accessed 2023-11-13. https://devopedia.org/inter-service-communication-for-microservices
Contributed by
2 authors

Last updated on
2022-02-15 11:51:57
networking system distributed computing data serialization

See Also

  • Microservices
  • Migrating from Monolithic to Microservices Architecture
  • Microservices Workflows
  • Service Mesh
  • Service Discovery for Microservices
  • Service-Oriented Architecture