This content originally appeared on DEV Community and was authored by Amanda Guan
The Thanos Query UI provides robust querying capabilities across multiple Prometheus instances, offering a comprehensive analysis of your metrics. Using the Thanos Interactive example dataset, I experimented with various PromQL queries to explore metrics. Here’s a summary based on frequent use cases:
Available Metrics in the Metrics Explorer:
- continuous_app_metric0
- continuous_app_metric1
- continuous_app_metric2
- continuous_app_metric3
- continuous_app_metric4
PromQL Query Examples:
-
Sum of Metrics:
sum(continuous_app_metric0)
- Sum across all clusters and replicas. -
Sum by Cluster:
sum(continuous_app_metric0) by (cluster)
- Grouped by cluster. -
Sum by Replica:
sum(continuous_app_metric0) by (replica)
- Grouped by replica. -
Average by Cluster:
avg(continuous_app_metric0) by (cluster)
- Average grouped by cluster. -
Maximum Value by Replica:
max(continuous_app_metric0) by (replica)
- Max value grouped by replica. -
Rate of Change:
rate(continuous_app_metric0[5m])
- Rate of change over 5 minutes. -
Increase Over Time:
increase(continuous_app_metric0[1h])
- Increase over the past hour. -
Histogram Quantile:
histogram_quantile(0.95, sum(rate(continuous_app_metric0_bucket[5m])) by (le))
- 95th percentile if part of a histogram. -
Count of Instances:
count(continuous_app_metric0)
- Count of reporting instances. -
Specific Cluster and Replica:
continuous_app_metric0{cluster="eu-1", replica="0"}
- Current value for a specific cluster and replica. -
Comparison Between Clusters:
continuous_app_metric0{cluster="eu-1"} / continuous_app_metric0{cluster="us-1"}
- Metric comparison between clusters. -
Alerting:
continuous_app_metric0 > 100
- Create an alert if the metric exceeds a threshold.
Understanding Empty Query Results:
-
Histogram Quantile Query: May return empty if
_bucket
suffix is missing, required forhistogram_quantile
. - Comparison Between Clusters: Returns empty if data points are missing or unmatched in timestamps.
PromQL Info Messages:
-
Rate and Increase Queries: Indicate that
continuous_app_metric0
might not be a counter as it does not end in_total
,_sum
,_count
, or_bucket
.
Metric Types Explained:
- Counters: Metrics that increase or reset to zero. Used for tracking quantities like server requests or encountered errors.
- Gauges: Metrics that can both increase and decrease. Used for measuring fluctuating values like memory usage or temperature.
- Non-Counter Metrics: Includes gauges and histograms, useful for tracking non-monotonically increasing values.
This content originally appeared on DEV Community and was authored by Amanda Guan

Amanda Guan | Sciencx (2024-06-19T18:36:55+00:00) Exploring Metrics with Thanos Query UI and PromQL. Retrieved from https://www.scien.cx/2024/06/19/exploring-metrics-with-thanos-query-ui-and-promql/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.