The Microservices Architecture has gained popularity for building complex, scalable software systems.
This approach organizes an application into a collection of loosely coupled, independently deployable services, with each microservice focused on a specific business capability. These services can be developed, deployed, and scaled independently, promoting flexibility and agility.
While microservices offer significant advantages—such as enhanced scalability, flexibility, and faster time to market—they also introduce challenges, particularly in data management.
A core principle of microservices is that each service should own and manage its data. This is often summarized as "don't share databases between services," emphasizing the importance of loose coupling and service autonomy to allow independent evolution.
However, it's essential to differentiate between sharing a data source and sharing data. Although sharing a database across services is discouraged, data sharing between services—through APIs or events—is often necessary and encouraged to ensure effective communication.
In this post, we’ll look at Microservices Data Design and Challenges Solution to Sharing data between microservices and the various advantages and disadvantages of specific approaches.
Microservices Data Design:
At it's core, microservices are built as Autonomous Entities and should have control over the data layer that they operate on. This essentially means that microservices cannot depend on a data layer that is owned by another entity. Thus to build autonomous services, it's required to have an Isolated Persistent Layer for Each microservice separately.
This section attempts to show patterns/practices for transforming centralized or shared database-based enterprise applications to microservices that are based on decentralized databases.
In Monolithic applications, usually has single centralized database (or a few) is shared among multiple applications and services. Below shows an example of online retail application - Where all services of the retail system share a centralized database (Retail DB).
In centralized database it's quite trivial to model a complex transactional scenario that involves multiple tables. Most RDBMS support such capabilities out of the box.
Despite such advantages, it has serious drawbacks, which does not allow to build autonomous and independent microservices.
- Single point of failure,
- Potential performance bottleneck due to heavy application traffic directed into a single database,
- Tight dependencies between applications, as they share same database tables.
- Easily modify the database schema without worrying about the external consumers of the database.
- No external applications can access the database directly.
- Allows freedom to select the technology to be used as the persistent layer of microservices.
- Different microservices can use different persistent store technologies, such as RDBMS, NoSQL or other cloud services.
B). Implementing Foreign Key Concepts (Soln: Synchron Lookup or Asynchronous Events)
C). Sharing Static Data (Soln: Shared Libraries)
D). Data Composition (Soln: Composite Services, Client Side Mash-ups)
E). Transactions - Major Problem! (Soln, 2 PC / 3PC Protocols)
- Identify the shared table and identify the business capability of data stored in that shared table.
- Move the shared tabled to a New dedicated database and on top of that database,
- Create a New service (with business capability) identified in the previous step.
- Remove all direct database references and only allow them to access the data via the services
- Using Synchronous Lookups
- Using Asynchronous Events
1). Using Synchronous Lookups: Used to access the data owned by other services. This technique is quite trivial to understand and at the implementation level, you need to write extra logic to do an external service call. We need to keep in mind that, unlike databases, we no longer have the referential integrity of a foreign key constraint. This means that the service developers have to take care of the consistency of the data that they put into the table. For example, when you create an order you need make sure (possibly by calling the product service) that the products that are referred from that order actually exist in the Product Table.
2). Using Asynchronous Events
Challenge_C). Shared Data: This is like Country, State etc data. Two approaches:
- Add another microservices with the static data would solve this problem, but it is overkill to have a service just to get some static information that does not change over time.
- Hence, sharing static data is often done with shared libraries for example, if a given service wants to use the static metadata, it has to import the shared library into the service code.
Challenge_D). Data Composition: Composing data from multiple tables/entities and creating different views is a very common requirement in data management. With monolithic databases (RDBMS), it's very easily to build the composition of multiple tables using joins and SQL statements.
However, in the microservices per DB approach, building data compositions becomes very complex, as we no longer can use the built in constructs such as joins to compose data as data is distributed among multiple databases owned by different services.
Therefore, with a microservices architecture we need to think better solution:
- Composite Services Or
- Client Side Mash-Ups
Hope this helps.
No comments:
Post a Comment