Why do I need an o11y platform?

In The Beginning

Welcome to the beginning of our observability journey. It can be challenging to keep up with new philosophies and vocabulary when our day jobs so rarely incorporate them. Then suddenly, someone in your organization hears that observability (or, O11y, if you will) is important and we have to figure it out — fast. So why do we really need an observability platform?

A Very Brief History of observability

Essentially, observabilityobservability is the answer to a question: what is going on in my system? In the past, we’ve relied on traditional application monitoring to answer that question. But as our systems become more complex, get split into microservices or deployed on thousands of containers, the question of what is going on in our system is increasingly hard to answer. This led us into the age of instrumentation, where all worthwhile things are monitored and all monitored things are worthwhile. It’s easy enough to add telemetry to our applications, but it’s hard to turn that data into information that solves problems. 

The Problem

When we monitor so many distinct components of our systems, we can end up with a lot of data gathered by many different tools. Hopefully, all of the logs and metrics and traces can be used to answer the questions we have about our system — but what if they don’t? Personally, I have spent too much time staring at dashboards that mean nothing to me, but are supposed to represent my system. How well we can answer the question of what is happening in our system is determined in large part by the accessibility and context of that data. For developers’ sakes, we need a way to answer our questions quickly, so that we can get back to the rivers and lakes that we’re used to (i.e., our actual jobs and areas of expertise). 

Enter the O11y Platform

In the world of observability, we care about being able to ask questions about our system and find answers (or at least clues). When the tools at our disposal fail to help us answer our questions, an observability platform makes sense. Remember all of that instrumentation we had to think about? An observability platform can turn the data we’re already collecting into useful information by combining streams, performing analysis, and detailing the full life of a request. 

Consider the amount of time we spend investigating errors or long response times. While I do enjoy a good mystery, the sheer volume of monitoring data can be impossible to navigate without help. I like watching dashboards update as much as anyone else, but without any kind of tool to organize our investigation, we (okay, I) recruit other engineers to help search for the solution.

Sure, we could follow this diagram or we could stand around a single monitor while everyone tells a single engineer what to try next.

It’s true that we can do all of this without adding an outside tool, and some folks do; generally, the very big tech forces have the time and money to either build a solution internally or acquire some company with the technology. For most of us, connecting the many observability tools that run across services and teams takes time that we just don’t have.

The Answer: an observability platform

The answer to whether we need an observability platform is, of course: it depends. Can an observability platform improve our ability to ask questions of our system and find answers? Can an observability platform save us time so that we don’t have to spend late nights playing detective in the deep, dark recesses of our system? 

Much like my love for glitter, our ability to observe our system should be limitless. If we need to track requests across all parts of the system regardless of process boundaries, we should be able to find a solution. If we want to analyze our data for patterns, we should be able to find a solution. And it would make my life as a developer a lot easier if every problem didn’t require pulling out a bunch of code or re-instrumenting the same components with a new tool. These are the types of problems that observability platforms excel at solving. A better question to ask is not why we need an observability platform, but what problems an observability platform can solve and what questions it can answer.

Observability in Action

If you want to explore the types of problems observability can solve, try Lightstep's interactive sandboxLightstep's interactive sandbox.

Interested in joining our team? See our open positions herehere.