Microservices - Designing Concepts

By Book - Steve Jobs - Design is not something what looks or feels like, rather it's about how it works. While designing Microservice, one should focus on its inner and outer architecture. Inner defines how you design a microservice/how it works within itself  and Outer talks about how it communicates with other microservices.   

This post attempts to discuss Microservices Design Principles & Data Design.

Design Principles:    

At it's core key elements in microservices design are -Time to Production (Automation), Scalability (Layered, Separation of Concerns, High Cohesion, Loose Coupling), Resiliency (Timely Timeouts, Retry, Circuit Breaker, Load Shedding, Fallback etc) and Complexity Localization. 

• High Cohesion & Loose Coupling
• Resilience
• Observability
• Automation

1). High Cohesion & Loose Coupling

• These two concepts are related of a system design. A highly cohesive system is naturally loosely coupled.
• High Cohesive system means, group all related or interdependent functionalities together, into one system.
• A microservice architecture must be highly cohesive and loosely coupled.

2). Resilience 

Resilience is the capability of the system or individual components in a system to quickly recover from a failure. 

The most common way to encounter attack failures in a distributed systems is via a 'Redundancy'.  

Well to still meet zero downtime, building application with recovery-oriented mindset is equally important. Below are the resiliency patterns.

• Timely Timeouts - 
• In Microservices, anything over network is fallible, hence we should not wait indefinitely expecting response. I.e. Do not indefinitely wait to timeout, if no response, else it will degrade system.
• Prefer to have timeouts, neither long nor short
• Well if timeout is reached, should you return error - No, rather we should log and return some meaningful meaning message .

• Circuit Breakers - 
• Stop making requests after a certain threshold of failures reached.
• It's like electric system, where if flow increases then it drops the connection to avoid further damage.
• I.e. If our microservice keeps timing out at a given endpoint then there is no point in keep trying, rather break the system in a nicely fashion.

• Bulkheads - 
• Bulkheads are used in ships to build water compartments, so that if one is full, passengers can use other. 
• Same Bulkhead patterns is applied here - Do not have single thread pool for all outbound endpoints, rather plan one thread per endpoint.

• Steady State - 
• This pattern adhere to designs which allows your system to run in a steady state for long time. Could be thru Automated Deployments, Clearing Log Files to avoid growing them indefinitely, Clear Cache before growing them enormous etc.

• Fail Fast - 
• Make decision early to fail, if you know the request is going to fail/rejected. E.g. If load balance knows there is a  failed node, then there is no point in sending request again and again.
• Even Circuit Breakers can also be used to implement Fail Fast Strategy.

• Let it Crash - 
• It's like doctor decide to cut the leg, before it damages entire body.
• This strategy believes to abandon a broken sub-system, to preserve the overall stability of the system. E.g. Remove Failed Node, Remove Failed Endpoint etc

• Load Shedding - 
• Application should shed load when the system starts performing behind SLA.
• Load shedding drops some proportion of load by dropping traffic as the server approaches overload conditions. 
• Load shedding can be like Reduce Queue size, Introduce Caching, Notifying Load Balance that application is not ready to accept more requests etc.

• Fallback - 
• Sometimes a request is going to fail no matter how many times you retry. The Fallback policy lets you return some default or perform an action  - Like paging an admin, scaling a system or restarting a service. 

3). Observability

Observability means, having right set of data collected to infer internal state of a system from knowledge of their external outputs. E.g. Having employees right check-in/check-out data can help to predict productivity of your employees.

Thus Observability is one of the most important aspect to be cooked in any microservice design. I.e.

• Keep track of throughput of each microservice
• Number of Success/Failed requests
• Utilization of CPU, Memory etc.
• Business related metrics (e.g SLA)

4). Automation

Key rationale behind a microservice is - Less time to Production and Shorter Feedback Cycles.

Automation helps to meet this goal. E.g. CI-CD.

Hope this helps.

Arun Manglick