This is a guest posting by Bob Reselman
Microservices are a new style of architecture for distributed systems that is gaining wide acceptance among companies operating at web-scale. Companies such as Netflix and Amazon have embraced microservices in order to release software at increasingly faster rates, as older, monolithic systems cannot scale to meet modern demands.
However, the benefits that a microservices architecture offers are only as good as the test practices in place to support them. Designing tests for microservices with a near-instantaneous release cycle requires a new way of thinking — at both the micro and macro levels.
Let’s explore the three types of microservices-oriented applications, some of the challenges to accommodate in terms of testing them, and how you can create tests to best realize the benefits of microservices.
for QA and Development Teams
Microservices 101
Before delving into some of the finer aspects of microservices test design, we should get into what a microservice is. A microservice is a fine-grain software component that has a distinct semantic definition and carries its own data independent of any other data structures and data sources. A microservice has its own deployment cycle, making it so that revisions to the microservice can be released without disrupting the operation of any other microservice in the application domain it’s operating within.
Advantages Over Monolithic Applications
Let’s also take a look at what a microservice is not. Figure 1 below illustrates an example of a typical monolithic application.
Figure 1: Monolith application architectures tend to be tightly coupled
The Customers, Products, Orders and Comments components can be realized as a set of classes in an object-oriented language such as Java or C#, where customers is an array of customer objects, products is an array of product objects, etc. An order object might use a customer object as well as a product object. Or, due to the fact that all components in the monolithic application use the same database, an order object might just go directly to the database to get the product and order information it needs to do its tasks. Not only is direct database access possible — as much as it violates the spirit of ORM-based object-oriented programming — it happens a lot, particularly in businesses that have a high demand to get features out the door no matter what.
The result is a tightly coupled, sometimes brittle system in which releasing a new version of the application requires a high degree of coordination among all the development teams involved. The impact of tight coupling is that the release cycle can never go any faster than the slowest dependency’s revision activity. In other words, if the application is due for a revision that includes a new feature in products and a new feature in customers, the release cannot happen until both products and customers are ready to go. If it takes one day to make the upgrade in products but three weeks to make the upgrade in customers, the release will happen in three weeks. And, to make matters a bit more difficult, not only will the release process need to coordinate new code in products and customs, the database might have schema changes that will need to be part of the release management process.
Releasing database changes is a very delicate undertaking. Altering the structure of a database runs the risk of incurring unintended side effects. Remember, if any component can access the database, any component will access the database, even to the point of playing with data that is not in the area of concern of a given component. Such “playing around” outside an area of concern can cause breakage in other components that can go undetected until calamity occurs. Sometimes the potential hazard can go undetected until the code is released into production. Sad to say, this happens all the time.
Some companies can tolerate slow release cycles that go with monolithic applications. But, if you are a company that supports hundreds of thousands of users and has thousands of components in play, going no faster than the slowest component’s release is unacceptable.
Microservices-Oriented Applications
In a microservices-oriented application (MOA), each component is decomposed into the finest grain of functionality possible (within reason). Each area of functionality is then represented as a microservice. Every company has its own technical legacy and culture to deal with, so there is no definitive playbook by which to do the decomposition.
When it comes to decomposing an existing monolithic application into an MOA, the trick is to not let the perfect be the enemy of the good. Make sure that each microservice is structured in terms of its semantic definition and carries its own data as well as its own release cycle. The depth of implementation depends on what the company can do given the time, expertise and resources available. Some microservices might have a single function, and others might contain many functions.
There are three types of MOAs: synchronous, asynchronous and hybrid.
Synchronous Microservices-Oriented Applications
Figure 2 below illustrates a synchronous MOA. Interservice communication is done using a request-response pattern common to HTTP interactions on the web.
Figure 2: A microservices-oriented application is based on synchronous interservice communication
Each microservice is segmented behind an HTTP server that provides access to the microservice’s logic. The given microservice knows about its own area of concern only; it might be aware of the interface for another microservice, but as far as the internal logic of that other service goes, there is no visibility. Also, the only data the given microservice knows about is its own. The datastores of other microservices are unknown and inaccessible. “Hijacking” another microservice’s data using direct access to the datastore is not possible. The only way to get data to and from a microservice is to interact with the microservice via its public interface.
Synchronous MOAs have the benefit of independence. For example, the Customers microservice shown above can upgrade itself any time it wants. There are no external dependencies to accommodate. As long as the microservice does not alter its public interface and the structure of the data it intends to consume from a request and return as a response, breakage risk to the overall MOA is minimal.
The very nature of an MOA architecture is self-reinforcing in terms of structural integrity. Because the microservice carries its own data, other services’ datastores will not be affected. And, because the microservice is represented as a set of HTTP URLs and associated request-response data structures, interface boundaries are well defined.
Synchronous MOAs are becoming quite popular. The interfaces of many synchronous MOA are based on REST, a style that’s been in play since 2000. Yet, for all the popularity, synchronous MOAs have a drawback: speed. Consumers of a synchronous microservice will never go any faster than the request and response time of the given microservice. This can be an impediment, particularly for microservices that have processes that take a long time to execute. An example is a complex analytic service that consumes and processes terabytes of data. Few consumers want to sit around for minutes waiting for analysis to complete. They’d rather tell the microservice to do the work and then be notified when the results are ready.
In situations such as this, an asynchronous approach to microservice design is better suited to meet the need at hand.
Asynchronous Microservices-Oriented Applications
Figure 3 below shows an asynchronous implementation of the MOA, where interservice communication is facilitated as an exchange of messages between interested parties. A common term used to describe the interaction is “fire and forget.”
Figure 3: An asynchronous MOA architecture is message-driven
In an asynchronous MOA, messages can be generated arbitrarily or in response to a given event. For example, when an order is created in the Orders microservice in the figure above, that service might publish an orders_created message to an associated message queue that another microservice, billing (not pictured), is listening to for incoming messages. The billing microservice picks up the message containing information about the order and processes it in a way that is relevant to billing’s area of concern.
The benefit of taking an asynchronous approach to microservices-oriented application design is that such systems avoid bottlenecks and thus are highly efficient. The downside is that they are very complex to create and to manage.
It’s not unusual for large-scale asynchronous systems such as Uber to process hundreds of thousands of messages a second. Debugging such a system is difficult, given that there is no direct path to a process’s workflow. It’s not a question of do this, then do that; logic executes according to a message received, which means that anything can happen at any time.
This is something to remember when designing testing strategies. For example, in an asynchronous system, relying on request-response time to measure performance is impractical because there is no one-to-one request-response activity in play at all.
Hybrid Microservices-Oriented Applications
A balanced way to implement a microservices-oriented application is to take a hybrid approach. Services are both synchronous, in that they support direct request-response communication between services, and also asynchronous, in that messages are generated within or as a result of a synchronous exchange.
Figure 4 illustrates the communication patterns that take place when using a hybrid approach to microservices-oriented application design. The Customers microservice is represented as both a URL bound to an HTTP server and a queue in a message broker.
Figure 4: A hybrid approach to microservices-oriented application design uses both synchronous and asynchronous communication between services and consumers
It’s possible to add a customer to the microservice using a standard HTTP request-response interaction. The request is received and processed, and then a response is generated. However, before the response is generated, a message containing the new customer information is published to an associated message queue to be consumed by other interested services. The information sent to the message queue might be the same as that generated in the HTTP response, or the microservice might send information to the message queue that differs from the information contained in the HTTP response. It all depends on how the microservice is designed and the service-level agreements the microservice needs to support.
GraphQL is an API technology that supports both synchronous and asynchronous communication between consumer and service. For more on this technique, read up on GraphQL subscriptions here.
The important thing about taking the hybrid approach is that you get the best of both worlds. But you also get the drawbacks, particularly around potential bottlenecks in performance execution and in the added complexity of supporting two distinctly different types of interfaces to and from the microservice.
The Challenges in Microservice Test Design
In terms of testing such applications, the first thing to remember is that it is rare for a single microservice to be shared among a variety of MOAs. Usually, a microservice is one of many in a particular application domain. The benefit of taking a microservices-oriented approach to application architecture design is the increased speed by which code can get into the hands of demanding consumers (Netflix does over 4,000 deployments a day.) A release velocity such as this just isn’t possible under traditional monolithic applications.
Another thing to remember is that testing an MOA needs to happen at the micro and macro levels.
Testing the Micro Level
At the micro level, each service needs to be thoroughly tested within the boundaries of its area of concern. In some circles, a microservice’s boundary is considered to be a function.
Go Beyond Typical Unit Testing
Focusing on the function within a microservice makes sense given the rise of the serverless movement, which advocates that a microservice be represented only as a single function. Yet the inclination among many test practitioners is to confine such testing to unit tests. While a unit test is fine for a single function running under very limited test conditions, a microservice is intended to run at web scale. This requires extreme testing conditions.
For example, a good micro-level test can involve running a hundred thousand instances of the microservice at once and observing their behavior at this degree of scale. Running a single unit test on one function at one time is insufficient. You need to run that unit tests on a thousand instances of the function running simultaneously according to the prescribed hosting environment.
The takeaway here is that in an MOA, a single function might very well be running on a thousand instances. Plan accordingly.
Test the Deployment Unit
In addition to testing the functionality of the microservice, you need to consider also testing the deployment unit in which the microservice is released. Microservices are typically deployed as containers that are part of some orchestration technology, such as Kubernetes or Docker Swarm. An important aspect of container orchestration is ensuring microservice persistence.
Microservices are expected to fail for a variety of reasons: Hosts go down; microservices malfunction. It’s not unusual when running thousands of containers simultaneously to have ongoing failures. The relevance to testing is that a microservice’s “exit” behavior is just as important as the way it operates. Micro-level tests need to make sure a microservice comes to life gracefully and dies gracefully, all at scale.
Make Sure Comprehensive Logging Is in Force
Tests also need to verify that all critical events within the microservice are logged in a meaningful manner — and, more importantly, that these log entries make sense. In the world of microservices, log entries are critical, particularly in asynchronous MOAs where behavior execution is not sequential. Log data is often the only thing you have to make sense of what’s going on in an application.
Testing the Macro Level
Whether the MOA is synchronous, asynchronous or hybrid, a thorough test regime must be administered for the microservice. When testing microservices at the macro level, you must ensure that two aspects are working as expected: interservice communication, and deployment processes.
Ensuring Sound Interservice Communication
Microservices are, by definition, independent of each other. The way they know what to do is according to the information they get, so the accuracy of interservice communication is critical to the integrity of a microservices-oriented application’s operation.
Ensuring interservice communication means making sure that the correct information goes to and fro as expected. Tests must observe how messages are exchanged and processed. This is true for HTTP request-response communication and for asynchronous messages distributed using a message broker. Testing needs to ensure “happy path” message formats are supported and that improperly formatted messages are rejected in a sensical manner (with more than just a “bad message” error).
Testing the CI/CD Process
A healthy process for continuous integration and continuous delivery (CI/CD) is important in any software development paradigm, but when it comes to microservices-oriented applications, an efficient, accurate, fast CI/CD process is essential. MOAs can experience revision cycles on the order of a thousand updates a day, so one slow-building microservice can be a bottleneck that hinders the entire release process.
The best precaution is to give the testing of the CI/CD pipeline the same priority as that bestowed on any other high-level testing regime. Identifying and addressing issues such as slow builds of microservices code artifacts, slow provisioning of the runtime environments that host the microservices, and slow spin-up time of microservices once deployed is critical to ensuring that the CI/CD pipeline is healthy. A microservice with rocket-science engineering is of little use if you can’t get the deployment units out and operational quickly.
As much effort must be given to testing the release process as to testing the items that are being released. In short, assign the same priority to testing the CI/CD pipeline as to testing the application your CI/CD is deploying.
Putting It All Together
Microservices are a game-changer. Microservices-oriented applications provide the flexibility and speed necessary to bring new features online at near-instantaneous speed. Big enterprises that support millions of users understand this already, but more companies are adopting this architectural style every day as their applications move to web-scale.
Although companies will earnestly try to embrace the spirit of microservices-oriented applications in their development efforts, many times they will test these MOAs using practices that are traditionally applied to monolithic applications. This is a shortsighted approach.
Instead, companies need to incorporate modern testing techniques geared to the independence of microservices and the dynamics of the application in which they’re used. Once development and testing teams are in sync around the principles that drive microservices-oriented application design, companies will be better positioned to enjoy the benefits that microservices have to offer.
Cross-Technology | Cross-Device | Cross-Platform
Article by Bob Reselman; nationally-known software developer, system architect, industry analyst, and technical writer/journalist. Bob has written many books on computer programming and dozens of articles about topics related to software development technologies and techniques, as well as the culture of software development. Bob is a former Principal Consultant for Cap Gemini and Platform Architect for the computer manufacturer, Gateway. Bob lives in Los Angeles. In addition to his software development and testing activities, Bob is in the process of writing a book about the impact of automation on human employment. He lives in Los Angeles and can be reached on LinkedIn at www.linkedin.com/in/bobreselman.
