Announcing Query Builder: Powerful Searches on Unsampled Data

As systems become increasingly distributed and data rich, sifting through live data becomes more complex. We built Query Builder to search through such data efficiently — whether in a single service or across thousands of services — and to amplify the effects of Lightstep’s architecture. 

While investigating a live regression, you need access to the full picture: the composure of every request flowing through your system to pinpoint the root cause. Lightstep performs no sampling at the source, allowing developers to minimize MTTR by viewing 100% of spans within the product. The Query Builder provides a way to effectively search through massive amounts of data.

Leave No Trace Behind

Using the Builder, you can query across any service or operationquery across any service or operation, including or excluding multiple values using a combination of NOT and IN operators. 

For example:

service IN (“auth”, “web”) will return all spans from traces that pass through the ‘auth’ or ‘web’ services.

service IN (“router”) AND operation NOT IN (“health_check”) will return all spans from traces that pass through the ‘router’ service, but will exclude all spans that are from the operation ‘health_check’.

The Builder also allows you to search across tags, without any cardinality limitations. You can query for tags that are user defined, as well as those generated by Lightstep or your telemetry agent. 

For example:

To surface regional error alerts, you could query for ”error” IN (“true”) AND ”aws.region” IN (“us-east”). This will return spans in which the auto-defined ‘error’ tag results as ‘true’, and spans in which the custom-defined ‘aws.region’ tag is set to ‘us-east’.

You can then dig deeper by analyzing the query results using Snapshot AnalyzerSnapshot Analyzer.

If you prefer to type free form queries instead of using the dropdown UI, you can use the Manual mode. We’ve included a syntax cheat sheet to help you with your query creation.

Analyze Historical Data

Looking to capture historical data for a query? Want to know if you are meeting your SLAs? Use the query builder to create a predefined query, also known as a Stream. A StreamStream persists statistical time series data and example traces based on your query. This means you can look at sample traces from a week, a month, or a year ago, as well as get real-time monitoring on the latency, rate, and error percentage for your predefined query.

What’s Next?

As your systems get deeper, you will be able to continue searching effectively through high cardinality data. Look out for more query types in the future! 

If you’re interested in trying Lightstep’s Query Builder, you can sign upsign up.

Thanks to the team that built the Query Builder: Alice Fuller, Dan Hedgecock, Jonah Moses, Valjean Clark, and Paul DeVay.

Interested in joining our team? See our open positions herehere.