Log tracing with OpenTelemetry and Jaeger: Code Example

Here, I am going to discuss about the Distributed Log Tracing in case of Microservices architecture. I have also recorded an Youtube video. Please go through the below Youtube video to get detailed information.

Problem statement: 

Suppose we have 3 services in our E-commerce application like Order, Payment and User service. Now, if user is calling Order service to create an order, then the order service is triggering the Payment service and Payment service is triggering the User service.

Client -> Order service -> Payment Service -> User service

Now, for each request, each of this 3 services is creating some logs. Suppose if some error comes and we need to debug then how can we identify which all logs are created from different microservices for a particular request. There is nothing in the logs by which we can group together the logs of a request

Solution:

This problem can be solved by adding an unique ID for each request in all logs for that request. So, if request 1 is creating 10 logs and each of these 10 logs are having the same unique ID then we can put these 10 logs in a single group based on this unique ID and this group of logs is related to request 1. Similarly each request will generate a separate unique ID in logs for grouping the logs and associate these groups with corresponding requests. 

This unique ID is basically termed as Trace ID. This trace ID can be created manually in each request and can be made propagated to each intervening microservices. But, this implementation is not Industry standard. There are few Industry recognized, well tested tool to achieve it. 

One of these tools is OpenTelementry. OpenTelemetry generates Trace ID for each request and instrument into each request. It gathers some metrics of each request life span through out each intervening microservices. These metrics are span ID, trace ID, span duration, request flow etc. These metrics are not stored in OpenTelemetry. OpenTelemetry just gathers these information by instrumenting the requests and the export these data to some backend services like Jaeger, ELK stack (Elastic Search, Loadash, Kibana), Grafana etc.

Here, I have implemented the open telemetry in my 3 microservices in Node.js. I am giving here the link of my GitHub repository. The technology stack I have used here are: 

Node.js

Microservices

OpenTelemetry

Jaeger

Please go through the above Youtube video and GitHub link to get complete idea.

GitHub repository: https://github.com/jksnu/opentelemetry_poc.git