Navigating microservices with OpenTelemetry

The way your software organization remains relevant is through building new features for ever-savvier and more advanced customers. Microservices support this, allowing teams to build, deploy, and scale their code faster and easier than ever before. The agility afforded by microservices has become critical for teams embracing the DevOps culture, and is undoubtedly a critical component of the modern software organization.

The challenge of microservices

Microservices present a unique challenge, however. As a distributed system, they put up fences and boundaries around your services and teams that stymie traditional monitoring systems. When a microservice is broken, slow, or causing errors it can be extremely frustrating to diagnose the problem. A given microservice may have a myriad of upstream producers or downstream consumers who may be owned by far-flung teams with limited coordination or ability to control their dependencies. This problem, among others, has led to the explosion in popularity of distributed tracing and other observabilityobservability technologies.

Until recently, distributed tracing was often deemed “too confusing”, “too difficult”, or “too expensive” to implement in a microservice architecture. Competing standards and tools, the lack of automatic instrumentation, and competing proprietary analysis tools all contributed to these opinions. At Lightstep, we’ve always believed in Open Source Observability — you shouldn’t be locked into a given vendor, platform, or framework as a consequence of adopting distributed tracing. 

The solution of microservices

To this end, we founded the OpenTelemetry project along with Google, Microsoft, and others. Our aim as a founding member of the project is to resolve the confusion of multiple competing standards, create easy and seamless automatic instrumentation of code for observability, and do it all without requiring vendor lock-in.

Rather than having to choose between competing standards, such as OpenTracing and OpenCensus, developers can have the best of both worlds. OpenTelemetry represents the next version of both of these projects, adopting design principles and practices from each. In addition, we’ve committed to backwards compatibility for existing users – you shouldn’t have to throw away the work you’ve already done in order to get the latest and greatest! OpenTelemetry offers a seamless, drop-in solution for generating telemetry data from hundreds of popular frameworks and libraries thanks to this backwards compatibility.

In addition, OpenTelemetry supports a variety of popular observability platforms for the analysis and processing of your microservice’s telemetry data. The OpenTelemetry exporter model allows for your trace and metric data to be transmitted to a variety of backend systems – both Open Source and SaaS – depending on your needs and requirements. You have the freedom to choose the solution that makes the most sense, without worrying about potentially expensive vendor costs.

The result of microservices

OpenTelemetry moves towards a world where high-quality telemetry data is built in from “day zero” rather than being an add-on later down the line. Conventional monitoring and APM suffer from gaps in their knowledge — they can’t follow end-to-end requests through your entire system, from client to database and back. OpenTelemetry, through its integration into popular frameworks and libraries (such as ASP.NET, Spring, and others) can provide these insights seamlessly.

As OpenTelemetry matures and is embedded more deeply into the tools you build your services with, these deep insights can extend into dependencies you don’t even control. You’ll be able to follow a request from your iOS application, through your load balancer, into your business logic, and back into managed databases and object stores from your cloud provider. Ultimately, this easy data collection will provide valuable insights and analytics to developers, freeing them to spend more time building critical features rather than fighting fires and fixing bugs.

Interested in joining our team? See our open positions herehere.