OpenTelemetry: Emerging standard for all DevOps solutions - cloud services

Modern software development in a large organization is siloed: tools, processes and technology vary by team. This trend is accelerating with microservice and cloud-based architectures, where individual teams choose the best available solutions to build and operate their services— 25-plus different tools on average, according to a recent report by William Blair. When teams need to coordinate to fix a problem, this creates slow DevOps workflows, fragmented data, and complex to non-existent integrations across dozens of different SaaS, homegrown, and open-source solutions.

Extending OpenTelemetry to new workflows

OpenTelemetry is a unified standard that helps companies combine different types of telemetry across applications and services in a vendor-neutral way. Its benefits also extend to any DevOps or cloud tool. OpenTelemetry, at its core, defines a data model that can describe what’s happening throughout the development lifecycle: from metrics related to the performance of code or infrastructure, relationships between services, or detailed technical metadata.

The OpenTelemetry standard creates a foundation for best of breed technology to connect solutions. Even a simple breadcrumb of information - whether it’s about the current state of feature flag, the deployment version, or even injected debugging information - connects one vendor’s product value to the entire development pipeline. With OpenTelemetry, this is done using automatic instrumentation of a customer’s code. Now other products or even custom tooling can use this data as part of productive developer workflow.

With over 100 different integrations100 different integrations (and counting), OpenTelemetry also dramatically increases the number of potential workflows between solutions without requiring any new development, custom instrumentation, or API connections.

Getting started with Lightstep

Lightstep wants to help customers and our DevOps partnersDevOps partners adopt OpenTelemetry. As a founding memberfounding member and core contributor of the standard, we have the expertise, tools and templates to help vendors easily adopt it in their solutions.

We are launching a series of OpenTelemetry-based tutorials and example instrumentation for different DevOps solutions. We’ll show how to connect these solutions to observability data within Lightstep, and show how that’s a better user experience for running experiments, operating cloud services, or investigating errors.

In this post we’re showing how you can extend instrumentation to API requests to AWS cloud services.

Observable cloud services

For software teams, the reality of cloud computing is adopting some of the hundreds of different services offered from Amazon, Microsoft or Google—over 200 of which are available on Amazon Web Services alone as of this writing. Cloud services, from storage solutions like Amazon S3 to bleeding-edge serverless and Kubernetes-based compute platforms, are a critical dependency that influence performance and reliability. Determining the optimal technical architecture also has enormous financial implications for any business.

Diagnosing cloud performance issues

A frequent pain point Lightstep hears from customers is related to diagnosing performance issues that involve both a customer’s code and the cloud provider APIs it interacts with. If performance is slow or errors increase: who or what is to blame? This problem is especially magnified when one team is unfamiliar with the details of an underlying infrastructure or cloud service that they might not be responsible for.

Telemetry based on standards links problem and root cause

AWS has already announced the AWS Distro for OpenTelemetryAWS Distro for OpenTelemetry, which builds on OpenTelemetry auto-instrumentation to add AWS-specific metadata from cloud resources and services.

As an open standard, anyone can write and share OpenTelemetry instrumentation. For Node.js applications, Aspecto recently wrote instrumentationinstrumentation that captures all requests from the AWS JavaScript SDK. With a single line of code that enables the instrumentation, we can see telemetry related to AWS APIs calls. With the plugin, it’s possible to link the performance of customer-facing requests with the performance of supporting cloud services in distributed traces. This helps answer previously difficult performance questions related to cloud services.

We’ve released a working demo of this instrumentation in our demo applicationdemo application in GitHub and have written a new learning pathlearning path for any Amazon Web Services customer that wants to adopt or extend how they use OpenTelemetry.

Next steps

Check our Lightstep Developer Toolkit:

• Our demo appdemo app demonstrates OpenTelemetry-aware Cloud Services with Lightstep using an experimental feature flag plugin.

• A Lightstep learning pathLightstep learning path to learn how to enable this in your services or see a demo app on GitHub.

• Our OTel DocsOTel Docs for software teams considering adopting OpenTelemetry

Contact usContact us if you’d like to learn more or know what’s planned for future integrations or know more about OpenTelemetry.