I have discussed about the driving factor in choosing Microservices in the first part of my blog series about Microservices Architecture through Enterprise Architecture framework. In this second part of the series, I am discussing about Data Architecture which is one of the important architecture under Enterprise Architecture umbrella especially in Microservices.
The obvious question is why we need EA Framework to develop Microservices Architecture and answer is based on following requirements:
Better Traceability and Alignment of Models
Easy and effective Change Management through relationship (direct or indirect)
Easy to maintain Complexity Management
Models are easier to maintain and manage compare to Documents or other configuration files.
Models can be maintained as live architectural elements
In this part of blog I am discussing Data Architecture of Microservices Architecture.
Microservices Architecture is a new Architectural Style rooted in Service Oriented Architecture (SOA) and inherits building services (identification, specification and design), governance (design time and run time) and granularity management between process and service.
Why Data Architecture is important in Microservices
There are many design patterns that are available to build Microservices architecture based application and the most popular design pattern is Domain Driven Design (DDD) pattern. So basically Microservices is a service oriented architecture but it starts from database and most of the operation is related to data only, meaning that it emphasises on CRUD operation. As per my opinion, Microservices are not for complex processes as these processes are cross functional and require cross domain data. DDD is a perfectly fine design pattern where domain overlapping does not exist, and even if it does, there is minimal overlapping with low data complexity in terms of relationship between data. Whether RDBMS should be used or not that is different call.
So basically DDD drives componentisation through Domain and the services are grouped accordingly. There are two additional patterns, Domain Entity Pattern and Aggregate Root Pattern which help to implement DDD. There is a set of best practices for doing DDD based architecture like keeping Microservices size as small as Business Entity’s scope (this is similar to Service Data Object, SDO, of SOA landscape. So granularity of Microservices is similar to atomic services or fine grain services.
SOA Services are independent from each other perhaps Deploy together (Separation of Concerns: Microservices should be distinct features with zero overlap.) while Microservices are deployed based on Domain boundary.
When we break the whole data design into pieces based domains then we achieve Decentralised Data Management and Data Governance. This is another point of discussion; it is common notion that complexity is reduced by decentralisation but rather, in my opinion, it is actually increased, as only a part of it is decreases while in whole (at enterprise level), complexity increases and in totality more time is required to manage. So in this way business domain is vital for starting the Microservices architecture
.Following diagram depicts how to architect data for Microservices.
When we talk about domain, some data is spread across the domains, and is equally important in each domain. This data is called master data. So Master Data Management (MDM) is very important for Microservices architecture. MDM is one of the most important, complicated and independent service component which manages Master Data (including inter Domain).
MDM (Master Data Management) should have the following capabilities for the management and maintenance of master entity:
Managing Master Entity
Capture and perform CRUD event on Master Entity
Archive Master Entity
Synch Master Entity (if replica table exist in any domain other than MDM schema/schemas)
Profile Master Entity
Cleansing Master Entity
Validate Master Entity
Verify Completeness of Master Entity
Resolve Duplicate Master Entity
Resolve Master Entity Relationship (if MDM contains multiple schema strategy)
Avoid this type of use cases where Business Use Case might have to update several database for a single transaction. In this case databases of other Microservices should be updated through its service API only but it is complex and not easy to maintain transaction. However, if it is required, then IMO Microservices is not the right architecture, and choose something else to build Enterprise IT Architecture.
The following diagram (Figure 4) depicts how perspective wise data model looks like and what are the attributes that should be captured in the corresponding model. The diagram also depicts how DDD drives data model for Microservices. The enterprise framework is required to manage and show holistic view of enterprise data model, otherwise it is very difficult to understand data dependencies, data flow, and relationship across domain etc.
Once data models are created, then the next thing is database, for which there are three main strategies: first is separate database per Microservices application or domain; second is shared database amongst domains; and third depends on the data size and complexity. Decentralized Data Management gives liberty to choose different databases for different domains to meet NFR for the Domain.
DDD: Data Model Building-Blocks
Entity: an Object with individual Identity
It should start from Contextual or Conceptual perspective
Logical Model should have all the key attributes
Physical Model should have all attributes aligned with Technology
At least 2 Physical (Entity & Table) Models should be created
Value Objects: (VO) an Object without individual Identity
Should start from Designer perspective (Logical Model)
Rest of the features are same as Entity
Aggregates: composite domain objects; features same as Value Objects
Data model for Microservices should contain these artifacts. All these artifacts are not applicable for all the six perspectives, only relevant artifacts should be created in any perspective. Attributes, characteristics and relationships also vary perspective wise.
There is no thumb rule for selecting the Design Patterns or Anti-Patterns; choose best suitable design pattern based on the requirement, best practices within your organisation, etc.
Database/schema strategy also depends on the requirement, best practices in the organization, etc.
Requirement of MDM depends on the organization’s needs.
Microservices is not a silver bullet.
Database centric services are the most suitable use case for Microservices but transaction should not be complex.
My next part of the Microservices series will be based on “Service Design” for Microservices.