Identify outliers faster with Heatmaps

The human brain is exceptionally good at identifying patterns. It is thought that humans’ ability to identify patterns (notably faces) was both a function of primitive survival, and community. As an adult, I can cite Wikipedia:

This robust and subtle capability is hypothesized to be the result of eons of natural selection favoring people most able to quickly identify the mental state, for example, of threatening people, thus providing the individual an opportunity to flee or attack pre-emptively.[11] This ability, though highly specialized for the processing and recognition of human emotions, also functions to determine the demeanor of wildlife

The term that is used to describe this ability is pareidoliapareidolia which is defined as “the tendency to perceive a specific, often meaningful image in a random or ambiguous visual pattern.” This is the human ability to see shapes or make pictures out of randomness.

Does this also apply to our ability to spot trends or outliers when it comes to our observability solutions? The answer is kind of.

Take a look at these two (seemingly identical) charts:

Chart 1 - shows a standard line chart with normal deviations / baselines

Chart 2 - shows a seemingly standard line chart with normal deviations / baselines, but with a slightly elevated point along the line chart

Looking at the above charts it’s hard to tell where to dig in. There are peaks and valleys along the line chart and both services appear to be functioning normally. The problem is - you have an email from the VP, that one of your customers is having an issue and you need to fix it. Not only did your VP not include which customer (or when the issue occurred), but you’re not seeing anything with these line charts.

Let’s quickly change our chart type up:

Chart 1 - shows a standard heatmap with normal deviations / baselines

Chart 2 - shows a seemingly normal heatmap with deviations / baselines, but with a grouping of data points significantly above the norm

This is a pretty accurate scenario where heatmaps change the game - the ability to isolate outliers quickly with contextual information to help answer the questions of when, how many times, and what value (duration in the above image) those outliers occured. You can now easily focus on the deviation, see an example trace, perform a bit of analysis, and report back to the VP the reason the customer was experiencing an issue. You can even save and share that Notebook with the relevant team to proactively optimize where the issue was occurring so it doesn’t happen again.

Lightstep is known for its ability to handle seriously large volumes of data at scale. With so much data, it requires an ability to sift through that data firehouse quickly. Without the ability to spot trends and outliers quickly and easily, the amount of data being digested becomes less important. Using our latest Heatmaps visualization type, you can significantly increase the speed at which you can identify “interesting moments” and identify outliers and edge cases to speed up investigation into problems impacting more customers.

Heatmaps are also fully integrated with Change Intelligence and other investigative workflows within NotebooksNotebooks and dashboards. In the above example, you’d be able to run Change Intelligence on the slice of time in question by selecting the “Analyze deviation” button, and selecting the outliers. Lightstep can also include span samples by clicking on the "show span samples" toggle allowing you to click on an example trace from the chart itself, or you can start doing group-by's on your query if you are trying to isolate a specific customer.

Lightstep’s visualizations are designed to make it easier to understand cross-system performance and dependency relationships. This enables developers to analyze millions of requests together to create custom charts displaying latency, throughput, and error rate over time, or generate custom service diagrams and heatmaps.

Lightstep allows organizations to monitor the experience of their most-valued customers and prioritize investments that would deliver tangible improvements for their users. Providing your developers and engineers an improved ability to seamlessly, effectively, and confidently monitor the health of their services and operations. Schedule a demoSchedule a demo today to see heatmaps in action, or try it yourself in our sandboxour sandbox.