Microservices - Security

"Strength of your system is as strong as strength of the weakest link" 😖😓

Microservices expands the attack surface as now - Multiple microservices communicating with each other remotely - Thus instead of one to two endpoints we have 100s of endpoints to worry about.

Thus, with a microservices architecture we need to think better security solution, from multiple perspectives - 

• Application Level Security
• Authenticate and Access Control Users
• Secure Communication Channel between Microservices
• Edge Level Security:  Using API Gateway Pattern
• Secure Development Life Cycle & Test Automation - 
• Speed to production is the key behind Microservices - To ensure engg be able to introduce a change to a microservice, test it and instantly push to production - Proper secure development life cycle is required to not introduce vulnerabilities at code level (Use Static Code Analysis & Vulnerability Testing)
• Proper Test Automation is required to ensure all possible use cases are tested to ensure no regression is injected.
• DevOps Level Security - 
• DevOps Security need to concerned about Container Level Security.
• Isolation among Containers
• Isolation between Containers and Host OS.
• Secure communication between Containers and Pods (Kubernetes as Container Orchestration Platform introduced another level of isolation, in the form of 'Pod'.

A). Application Level Security - In Monolithic application, each service need not to authenticate the user independently as all services are deployed in the same application server and thus interactions happens over the local calls. Rather authentication happens centrally at an interceptor which intercept service call. Once authentication is complete, it keeps passing the login context between services or components. 

Well in Microservices world this does not work, as microservices are scoped and deployed in separate containers in a distributed setup. I.e. Interactions are no more local, but remote over Http. 

Thus, with a microservices architecture we need to think better security solution.

• JWT (Json Web Token) Based
• TLS - Transport Layer Security

1). JWT: -JWT contains three parts - Header, Payload/Claim Set and Signature. Signature is generated using first two parts.

JWT assert one's identity given that the recipient of the JWT trust the asserting party (token issuer).Issuer of a JWT can sign the payload with a private key to which recipient can validate using the public key.

Note: When issuing a JWT token from a token issues it has to be issued to a given audience. (E.g. Microservice Foo calls Microservice Bar, then the token issuer is Foo and token audience is Bar). The 'aud' parameter in the JWT claim set, represents audience. There can be one or more audience. The value of 'aud' is pre-agreed one between issuer and audience(s).Audience can be generalized using regex like *.facebook.com

In JWT Payload/Claim Set, carries a parameter called 'sub', which represents the subject or the user who owns the JWT. The audience uses this parameter to know the caller.

2). TLS:  In Microservices TLS Mutual Authentication is used between microservices to authenticate each other.

B). Edge Security: Using API Gateway:

With the API Gateway pattern, Microservices which are exposed outside, should have a corresponding API in API Gateway. Now the end user's access to the microservice (via corresponding API) must be validated at the edge - or the API Gateway.

Most recommended approach is using OAuth 2.0

OAuth 2.0 - Framework for access delegation, works using four characters - Client, Authorization Server, Resource Server and Resource Owner.

Here are simple steps:

• Web/Mobile Application user get authenticate using Identity Provider via OpenIDConnect (Or can be SAML 2.0 too)
• Web/Mob app gets an 'access_token', 'refresh_token' and 'id_token'. 'id_token' will identify enduser to Web/Mob app.
• Now Web/Mob app makes an API call - Passing 'access_token' 
• API_Gateway reads 'access_token', connect to STS, which will validate 'access_token' and Issue a JWT to the API Gateway. While validating 'access_token' STS will talk to corresponding 'OAuth' Authorization server via an API. This JWT will also carry user context.
• Now the API Gateway will pass thru this JWT while request to downstream microservices.
• Each Microservice will validate the JWT received and then for further downstream calls, it can create a new JWT token (As we discussed in above JWT section) and sends it with request to another microservice. 

Note: Responsibility of Authorization Server is:

• Issue tokens ('access_token', 'refresh_token' and 'id_token') to it's clients. 
• Also respond to the validation request from downstream services. 
• At times plays role of STS too.

There are many open source Authorization Server exists - E.g. WSO2 Identity Server, Keycloak, Gluu etc. Full List of 77 OAuth Providers (Authorization Server) - https://en.wikipedia.org/wiki/List_of_OAuth_providers

Hope this helps... 

Arun Manglick

Microservices Frameworks

 <<Under Construction>>

This post attempts to cover the most popular existing Microservices Frameworks. Well, there are few characteristics you need to think over while choosing MS frameworks:

• Support for RESTful services
• Support for Container-Based
• Support for Better Security
• Support for Better Observability - You can check here - https://arunmanglick-ms.blogspot.com/2021/01/microservices-designing-concepts.html

Few Frameworks:

1. Netflix OSS (Open S/w Solution)
2. Spring Boot
3. Istio
4. Dropwizard
5. Vert.x
6. Lagom

1). Netflix OSS 

This is a framework + library that is used to build microservices architecture.

Below are key essentials to architect microservices:

This framework provides all the essentials required to build microservices.

#	Requirements	Component	Comments
1	Service Registration & Discovery	Eureka Server & Client	Each microservice should register itself by providing - Host, NodeName, Service Metadata etc.
2	Latency & Fault Tolerance Library	Hystrix	Hystrix behaves like a Circuit Breaker. For e.g. if A ->B ->C->D and assume D breaks then C should not cascade failure to B.  By having Hstrix at 'C 'microservice, failures are not cascaded.
3	Gateway API, Routing, Filtering	Zuul	Supports Dynamic Routing, Resiliency, Security and Monitoring
4	Monitoring	Hystrix Dashboard
5	Client-Side Load Balancer	Ribbon	Ribbon - IPC (Inter Process Communication) Library. Uses RoundRobinRule, AvailabilityFiteringRule, WeightedResponseTimeRule Algorithm. Ribbon - Also helps to determine if servers are up and running.
6	Declarative REST Client	Feign	This helps to create a stub to call microservice without writing full set of code. Annotate the interface with  @FeignClient
7	NA	NA	NA
8	NA	NA	NA
9	NA	NA	NA

Best Explained here - Youtube

Note: Almost every Microservice, has Ribbon, Hystrix, and Feign.

2). Spring Boot & Spring Cloud:

Most popular Microservices development framework for Java Developers. Spring Cloud came after Boot and both together provide a great development environment.

Importantly, Spring Cloud provides Netflix OSS integrations for Spring Boot apps - Includes Discovery with Eureka, Circuit Breaker with Hystrix, Routing and API Gateway with Zuul and Client-Side Loadbalancing with Ribbon.

Best explained here - Youtube

Rest Frameworks - Not covered here. :) 

Hope this helps.

Arun Manglick

Microservices - Why Not Monolithic or SOA/ESB

This post attempts to explore evolution of enterprise architectures from Monolithic Applications to Microservices.

1). Monolithic Applications:

In Monolithic style, all business functionality is clubbed in a single unit. 

In a real use case (Online Retail Application) - Entire application is a collecting of several components as shown. Adding or modifying to a component is extremely complex, due a single unit, any change in one breaks another. Also communication between these components uses proprietary protocol/standards. 

Monolithic style posses below features:

• Design, Developed and Deployed as a Single Unit
• Redeployment requires to change any part of it.
• One stable service brings whole application down.
• All functionalities are built on homogenous technologies/framework, so tough to adopt any new technologies.
• As application grows, due Monolithic nature, application starts taking more time to start up.

As a solution to Monolith drawbacks, SOA & ESB emerged.

2). SOA and ESB

SOA tries to fill up Monolith challenges, by separating functionalities of  Monolith applications into reusable, loosely coupled entities called 'Services', which are accessible over the network.

SOA/ESB style posses below features:

• Service is a well-contained implementation of a well-defined business functionality and loosly coupled.
• Services offers well-defined interfaces that are implementation independent. I.e. Separation from what from it's how part.
• Consumers are only concerned about service interface and do not care about implementation
• Services can be dynamically discovered using service metadata.
• Composite Service can be built from aggregate of other services.

Role of ESB - With the SOA paradigm, services are deployed within an 'Application Server'.

While consumption, you may need to integrate/plumb multiple such services (To create 'Composite Service') and other systems. ESB is used to integrate those services and form Composite Service.  Consumers uses the 'Composite Service' exposed from the Service Layer.

Thus ESB becomes centralized bus that connects all services and systems.    

I.e. At ESB layer, we integrate business capabilities and create composite business capabilities which are then consumed by consumer. 

ESB not just contains a significant portion of the business logic of the entire application - Other cross-cutting concerns such as 'Security', 'Monitoring', 'Analytics' etc.

API Management Layer over SOA/ESB:

Due to the complexity of Web Service-related technologies such as SOAP message format, WS-Security, WSDL etc - Most organizations put a new API Management /API Gateway on top of the existing SOA implementation.

This layer is called as 'API Facade', hides all internal complexity of ESB - All the business functionalities are now being exposed as managed API. This API layer is also responsible for Security, Throttling and Caching.

3). Microservices

With increasing demand for complex business capabilities, it's better to further get rid of the central ESB and break functionalities into each service. Microservice recommends individual services to take care of composition logic and inter-service communication. 

With SOA/ESB, most of the business functionalities are implemented at ESB level thru integration or plumbing of the underlying services and system. 

Microservices architecture introduces a new style of service integration - SMART Endpoints and Dumb Pipes, instead of ESB. With SMART Endpoints and Dumb Pipes, all the business logic resides at microservice level.

The foundation of microservice pattern is about an application as a suite of small and independent services, that are running in their own processes, developed and deployed independently. 

Below depicts microservices design of ESB based design shown above. Here MS got rid of ESB by breaking business functionality into each service, so that each microservice takes care of inter-service communication and business logic. 

    

Key concept of MS architecture is that service has to be designed meeting business functionalities, serving a specific business purpose - Focus on doing one thing and doing it well/complete - I.e. Autonomous Service.

Microservice do not share same execution runtime, rather they are deployed as isolated runtimes using 'Containers'.

This autonomous nature ensure 'Resiliency' as system will have isolated failures along with Service Isolation.

Also these loosely coupled services communicates thru messaging via inter-service communication, using various communication styles (Synchronous/Asynchronous) over the network. 

Hope this helps!!'

Arun Manglick

Microservices - Designing Concepts

By Book - Steve Jobs - Design is not something what looks or feels like, rather it's about how it works. While designing Microservice, one should focus on its inner and outer architecture. Inner defines how you design a microservice/how it works within itself  and Outer talks about how it communicates with other microservices.   

This post attempts to discuss Microservices Design Principles & Data Design.

Design Principles:    

At it's core key elements in microservices design are -Time to Production (Automation), Scalability (Layered, Separation of Concerns, High Cohesion, Loose Coupling), Resiliency (Timely Timeouts, Retry, Circuit Breaker, Load Shedding, Fallback etc) and Complexity Localization. 

• High Cohesion & Loose Coupling
• Resilience
• Observability
• Automation

1). High Cohesion & Loose Coupling

• These two concepts are related of a system design. A highly cohesive system is naturally loosely coupled.
• High Cohesive system means, group all related or interdependent functionalities together, into one system.
• A microservice architecture must be highly cohesive and loosely coupled.

2). Resilience 

Resilience is the capability of the system or individual components in a system to quickly recover from a failure. 

The most common way to encounter attack failures in a distributed systems is via a 'Redundancy'.  

Well to still meet zero downtime, building application with recovery-oriented mindset is equally important. Below are the resiliency patterns.

• Timely Timeouts - 
• In Microservices, anything over network is fallible, hence we should not wait indefinitely expecting response. I.e. Do not indefinitely wait to timeout, if no response, else it will degrade system.
• Prefer to have timeouts, neither long nor short
• Well if timeout is reached, should you return error - No, rather we should log and return some meaningful meaning message .

• Circuit Breakers - 
• Stop making requests after a certain threshold of failures reached.
• It's like electric system, where if flow increases then it drops the connection to avoid further damage.
• I.e. If our microservice keeps timing out at a given endpoint then there is no point in keep trying, rather break the system in a nicely fashion.

• Bulkheads - 
• Bulkheads are used in ships to build water compartments, so that if one is full, passengers can use other. 
• Same Bulkhead patterns is applied here - Do not have single thread pool for all outbound endpoints, rather plan one thread per endpoint.

• Steady State - 
• This pattern adhere to designs which allows your system to run in a steady state for long time. Could be thru Automated Deployments, Clearing Log Files to avoid growing them indefinitely, Clear Cache before growing them enormous etc.

• Fail Fast - 
• Make decision early to fail, if you know the request is going to fail/rejected. E.g. If load balance knows there is a  failed node, then there is no point in sending request again and again.
• Even Circuit Breakers can also be used to implement Fail Fast Strategy.

• Let it Crash - 
• It's like doctor decide to cut the leg, before it damages entire body.
• This strategy believes to abandon a broken sub-system, to preserve the overall stability of the system. E.g. Remove Failed Node, Remove Failed Endpoint etc

• Load Shedding - 
• Application should shed load when the system starts performing behind SLA.
• Load shedding drops some proportion of load by dropping traffic as the server approaches overload conditions. 
• Load shedding can be like Reduce Queue size, Introduce Caching, Notifying Load Balance that application is not ready to accept more requests etc.

• Fallback - 
• Sometimes a request is going to fail no matter how many times you retry. The Fallback policy lets you return some default or perform an action  - Like paging an admin, scaling a system or restarting a service. 

3). Observability

Observability means, having right set of data collected to infer internal state of a system from knowledge of their external outputs. E.g. Having employees right check-in/check-out data can help to predict productivity of your employees.

Thus Observability is one of the most important aspect to be cooked in any microservice design. I.e.

• Keep track of throughput of each microservice
• Number of Success/Failed requests
• Utilization of CPU, Memory etc.
• Business related metrics (e.g SLA)

4). Automation

Key rationale behind a microservice is - Less time to Production and Shorter Feedback Cycles.

Automation helps to meet this goal. E.g. CI-CD.

Hope this helps.

Arun Manglick

Microservice Fundamentals - Communication Styles & Standard Protocols

This post attempts to discuss MS 'Communication Styles' & 'Standard Protocols' used for MS communication.

Predominantly Communication Styles are - Synchronous & Asynchronous

Synchronous -

1. REST
2. gRPC
3. GraphQL
4. WebSockets
5. Thrift

Asynchronous -

1. Single Receiver
2. Multiple Receiver

Note: Synchronous communication does not mean, it's I/O Blocking. We can have Non-IO Blocking implementation by using 'Callback' function.

1). REST

• REST is an architectural style, uses navigational scheme to represents 'Resources' (Objects and Services) over the network. A client then access these resources uses unique URI (In case of HTTP, this interface consists of standard HTTP ops e.g. GET, PUT, POST, DELETE)
• Though REST does not depend of any of the implementation protocols, well most used is HTTP.

More details - http://arun-ts.blogspot.com/2017/04/rest-vs-soap_2.html

2). gRPC:

• In past, RPC was a popular inter-process communication, well multiple drawbacks
• gRPC - uses Protocol Buffers, and HTTP2 as the Transport Protocol, a key reason for gRPC wide adaption.

To get better understanding of gRPC, please check this - https://arunmanglick-ms.blogspot.com/2020/10/grpc.html

To find real use-case (Order Service), assume there is a 'Return Service' (Java Impl) and 'Product Management Service' (Go Lang Impl) to initiate return and update product inventory with returned item.

Here 

• PM Service, uses gRPC and exposes it's contract via ProductMgmt.proto file using Go Language.
• Return Service (Consumer), will also use ProductMgmt.proto file to generate client-side Stub (In Java) and invoke PM Service

Here, under the hood, 

• When a client invokes Retrun Service, the client-side gRPC library uses the Protobuf and marshals the remote function call, 
• This call is then sent over HTTP2
• At the server side, the request is un-marshalled and the respective function is executed using Proto Buf.

Here you can also use Client Streaming, Server Streaming or Bi-directional Streaming RPCs - Details in https://arunmanglick-ms.blogspot.com/2020/10/grpc.html

3. GraphQL:

With REST, wherever client needs data from multiple resources at the same time, it results in multiple service calls. 

GraphQL addresses such concern in a conventional REST based service by providing a Query Language and a Runtime for fulfilling those queries with your existing data.

• Developed in 2012 by Facebook, publicly released in 2015
• In it's simplistic form - It's a Query Language for your API
• GraphQL gives client power to request what exactly they need and nothing more or less. E.g. Only First Name, makes API evolve over time.
• Allows multiple resources to be requested in a single request. Where as with REST, you need to make two request to two endpoints - /users, /books

GraphQL Server exposes a schema describing APIs  - Organized into Types and not Endpoints. Each Type has one or more fields, which each takes zero or more arguments and return a specific type.

Note: Operation Type is Query, Mutation or Subscription, describing what type of operation you are intending to do.

4) WebSockets

• In simple words, WebSockets is TCP but over the Web.
• WebSocket is an architecture consist of a socket that is opened between client and server, for full-duplex (Bi-directional) communication. This anytime your MS need duplex communication, WebSocket is right choice.
• Uses a single TCP connection for traffic in both directions and uses HTTP as the initial handshaking protocol. (As below - Showing interaction between client and server using WebSockets)

Note: WebSockets has few similarities like HTTP2, such as Single Connection, Bi-directional Messaging etc. However difference is WebSockets allows you to build any Message Protocol on top of WebSockets For instances MQTT or AMPQ over WebSockets).

MQTT - MQTT - MQ Telemetry Transport-

Residing on top of the TCP/IP network stack, MQTT is a extremely simple and lightweight publish/subscribe messaging protocol designed for low-bandwidth,limited devices,networks with high latency, low bandwidth or unreliable networks. MQTT's features make it an excellent option for sending high volumes of sensor messages to analytics platforms and cloud solutions.

AMPQ - Advanced Message Queue Protocol

Is a specification for how messaging clients and brokers can interoperate. An open standard for passing business messages between applications or organizations.  MQP is a specification of a wire-level protocol for client to message broker communication.

5). Thrift

This is an alternative to gRPC, but Thrift-based communication requires a lot of work from developer side, as it expose low-level network details to the user. Also being TCP based, makes it less suitable for modern API or Mobile devices. Well on the other hand, being TCP based, performance is high.

Well conceptually Thrift allows you to define data-types and service interfaces in a Simple Definition File. Similar to gRPC, using interface definition compiler generates code to be used to build RPC clients and server communicate across languages.

Asynchronous -

Here client does not wait for a response in a timely manner. Possibly client may not in fact receive a response, or receive response asynchronously via  a different channel.

Asynchronous messaging between microservices is implemented using using Dumb Message Broker (Having No-Business Logic).

Both Single/Multiple can be read here in detail - https://arunmanglick-ms.blogspot.com/2020/12/make-microservices-communicate.html 

1). Single Receiver Mode - 

• Here a given message is reliably delivered from a produce to exactly one consumer.
• AMPQ messaging protocol is most commonly used with Single Receiver based communication.
• AMPQMessage Broker (Such as RabbitMQ, ApacheMQ) can be used - Where producer microservice will produce a message to a specific queue in the broker.
• Consumer microservice can subscribe to the message broker to receive messages.

2). Multiple Receiver Mode - 

• Used when message produced by one producer go to multiple consumers. (Pub-Sub Model)
• Here the services interested will subscribe to the topic.
• Most of the AMPQ based brokers support pub-sub. Well Kafka is mostly used. (Amazon SNS & Event Bridge is also widely used).

Source Used: Link

Hope this helps.

Arun Manglick