OpenTelemetry: Emerging standard for all DevOps solutions - cloud services
by Clay Smith
Modern software development in a large organization is siloed: tools, processes and technology vary by team. This trend is accelerating with microservice and cloud-based architectures, where individual teams choose the best available solutions to build and operate their services— 25-plus different tools on average, according to a recent report by William Blair. When teams need to coordinate to fix a problem, this creates slow DevOps workflows, fragmented data, and complex to non-existent integrations across dozens of different SaaS, homegrown, and open-source solutions.
OpenTelemetry is a unified standard that helps companies combine different types of telemetry across applications and services in a vendor-neutral way. Its benefits also extend to any DevOps or cloud tool. OpenTelemetry, at its core, defines a data model that can describe what’s happening throughout the development lifecycle: from metrics related to the performance of code or infrastructure, relationships between services, or detailed technical metadata.
The OpenTelemetry standard creates a foundation for best of breed technology to connect solutions. Even a simple breadcrumb of information - whether it’s about the current state of feature flag, the deployment version, or even injected debugging information - connects one vendor’s product value to the entire development pipeline. With OpenTelemetry, this is done using automatic instrumentation of a customer’s code. Now other products or even custom tooling can use this data as part of productive developer workflow.
With over 100 different integrations (and counting), OpenTelemetry also dramatically increases the number of potential workflows between solutions without requiring any new development, custom instrumentation, or API connections.
Lightstep wants to help customers and our DevOps partners adopt OpenTelemetry. As a founding member and core contributor of the standard, we have the expertise, tools and templates to help vendors easily adopt it in their solutions.
We are launching a series of OpenTelemetry-based tutorials and example instrumentation for different DevOps solutions. We’ll show how to connect these solutions to observability data within Lightstep, and show how that’s a better user experience for running experiments, operating cloud services, or investigating errors.
In this post we’re showing how you can extend instrumentation to API requests to AWS cloud services.
For software teams, the reality of cloud computing is adopting some of the hundreds of different services offered from Amazon, Microsoft or Google—over 200 of which are available on Amazon Web Services alone as of this writing. Cloud services, from storage solutions like Amazon S3 to bleeding-edge serverless and Kubernetes-based compute platforms, are a critical dependency that influence performance and reliability. Determining the optimal technical architecture also has enormous financial implications for any business.
A frequent pain point Lightstep hears from customers is related to diagnosing performance issues that involve both a customer’s code and the cloud provider APIs it interacts with. If performance is slow or errors increase: who or what is to blame? This problem is especially magnified when one team is unfamiliar with the details of an underlying infrastructure or cloud service that they might not be responsible for.
AWS has already announced the AWS Distro for OpenTelemetry, which builds on OpenTelemetry auto-instrumentation to add AWS-specific metadata from cloud resources and services.
We’ve released a working demo of this instrumentation in our demo application in GitHub and have written a new learning path for any Amazon Web Services customer that wants to adopt or extend how they use OpenTelemetry.
Check our Lightstep Developer Toolkit:
- Our demo app demonstrates OpenTelemetry-aware Cloud Services with Lightstep using an experimental feature flag plugin.
- A Lightstep learning path to learn how to enable this in your services or see a demo app on GitHub.
- Our OTel Docs for software teams considering adopting OpenTelemetry
Contact us if you’d like to learn more or know what’s planned for future integrations or know more about OpenTelemetry.