Mastering Grafana Variables

Dynamic dashboards are key to smooth DevOps. You need real-time insights, and Grafana is a go-to tool for many. But what if your dashboards are static, showing only one view? That’s where Grafana variables come in. They’re the secret weapon for building dashboards that respond to your needs, not the other way around. This article will show you how to unlock the power of variables to create truly adaptable dashboards. You’ll learn to move past rigid setups and towards flexible, data-driven visualizations.

Understanding Grafana Variables

Grafana variables are a way to add dynamic elements to your dashboards. They let you swap out data sources, metrics, and even time ranges without having to change each panel. Instead of hard-coding values, you use variables, which act as placeholders. This makes your dashboards much more flexible and easy to manage.

Think of a variable as a shortcut. Instead of selecting a specific server in a query, you can use a variable like $server. Then, you create a list of servers, and choose which one to display. This makes it easy to switch views, and see data from multiple sources.

Variables are not just for simple lists. They can pull values from data sources, use regular expressions for filtering, or even depend on each other, creating complex, interactive dashboards. They are a core part of what makes Grafana powerful.

Types of Grafana Variables

Grafana provides a range of variable types to handle different situations:

• Query Variables: The most common type, query variables fetch values from your data source. They are perfect for listing available databases, tables, metrics, or any other data that exists in your system.
• Constant Variables: These are simple, static variables. They’re useful for things that rarely change, like a specific URL or a fixed error code.
• Custom Variables: This type lets you define your own list of values. You can then select them in your dashboards. Use them for things like grouping servers, or defining environment names.
• Text Box Variables: These create a simple text input box. Users can type in values to filter data on your dashboard. They’re ideal for searching and quickly filtering results.
• Data source Variables: This lets you switch between different data sources. You can have one dashboard that shows data from multiple systems.
• Interval Variables: Define time intervals. You can use them for quick time range changes.

Each of these variable types serves a specific purpose. Understanding which one to use will help you create more efficient dashboards. Let’s dive into how to use each type.

Setting Up Grafana Variables

Now that you have a basic understanding of variables, let’s walk through the steps to set them up. It’s quite simple, and you’ll get a grasp of the concepts quickly.

Creating a Variable

First, you need to open your Grafana dashboard. Then, go to the settings menu, and select “Variables.” This is where you create and manage all the variables for your dashboard. You will see a “New variable” button. Click this button to create your first one.

Give your variable a descriptive name like server_name, database, or environment. This helps other people understand how it’s used. The name also becomes the variable identifier you use later.

Select the variable’s type. Depending on what you wish to accomplish, you can choose from any of the types explained above.

Fill in the relevant settings. For a query variable, you will select a data source, and add a query. For custom variables, you’ll add the options you want. And finally save the variable.

After saving the variable, it should appear in the variable list. You can now use it in the queries in your dashboard panels.

Using Variables in Queries

To make use of a variable, you simply reference it in your query using the syntax $variable_name. For example, if you have a variable named server_name, you can use it in a query like this:

SELECT metric FROM data WHERE host = '$server_name'

When Grafana loads the panel, it will replace $server_name with the selected value from the variable. You can have multiple variables in the same query. This will allow for very flexible dashboards.

Applying Variables Across Dashboards

Variables, once set up, can be used in multiple panels of the same dashboard. Also, dashboard templates let you use the same variables for multiple dashboards. This makes it very easy to have a consistent look across all your dashboards.

To create a dashboard template, you first create a dashboard with variables. Once that’s finished, save it as a template. When you create a new dashboard using the template, you’ll be able to select the values for the variables in it.

Diving Deeper into Variable Types

Let’s take a closer look at each of the variable types. We will start with the most commonly used, query variables, and move through the rest.

Query Variables: Dynamic Data Selection

Query variables fetch their values from a data source. This means the options in your dropdown will automatically update.

Use Cases:

• Listing Databases or Tables: If you’re working with SQL, you can use a query variable to list all databases, or tables in a database. The query may look like SHOW DATABASES or SHOW TABLES.
• Selecting Metrics: If you are using a time-series database such as Prometheus, you can list all metrics in the system. For instance, use a query like label_values(metric). Then, select the metric you wish to see in a panel.
• Filtering by Host: In many setups, you track performance per server. With query variables, you can list servers and select one to see its metrics.
• Environment Selection: Use query variables to list different environments (like “development,” “staging,” or “production”). The query can fetch the names from the database, or a tagging system.

Configuration Options:

• Data Source: Select the source from which you want to retrieve data.
• Query: The query is what gets the data. It depends on the source, but should always return one column.
• Refresh: You can choose to refresh the options of the variable on dashboard load, or time range change.
• Multi-select: Lets users choose multiple options. You then use a IN clause in your queries.
• Include All option: Lets users select all options. You then use a IN clause in your queries.
• Regex: Use a regular expression to filter or transform the results of the query. For example, you can extract part of a string for the variable.

Example Scenario:

Let’s look at a specific example. Say you want to display the CPU usage of different servers. You have Prometheus as your data source. You could use a query variable to select which server to display.

1. Name: Set the name as server_name.
2. Type: Choose “Query”.
3. Data Source: Select your Prometheus data source.
4. Query: Enter the PromQL query label_values(instance). This query gets the names of all the instances that have metrics.
5. Refresh: Select “On Dashboard Load” to refresh the list each time you open the dashboard.
6. Multi-select: If you want to choose multiple servers at the same time, enable this option.
7. Include All option: If you want to have an option to select all servers, enable this.
8. Regex: You might need to extract the hostname if the values are IP:port. You can do that with a regex, such as ([^:]+).

Then, you’d use the variable in a PromQL query in the dashboard panel: cpu_usage{instance='$server_name'}. Now you can see CPU usage for different servers.

Constant Variables: Fixed Values

Constant variables are for fixed values that don’t change. Unlike query variables, they do not pull data from a data source.

Use Cases:

• API Keys: If your data source needs an API key, it is good practice to store it in a variable. You can then reference the variable in the source configuration.
• Error Codes: Use them to display graphs of specific error codes across a system.
• URLs: For dashboards that use API calls, store the API endpoint in a variable.
• Static labels: Define a static label to add to all the panels in a dashboard.

Configuration Options:

• Name: A descriptive name for the constant variable.
• Value: The fixed value.

Example Scenario:

Suppose that your data source needs an API key to fetch data. To set this up as a constant variable:

1. Name: Set the name as api_key.
2. Type: Choose “Constant”.
3. Value: Input your API key.

In your query, you can then use the constant: SELECT metric FROM api WHERE key = '$api_key'. This keeps your dashboards secure and easy to manage.

Custom Variables: User-Defined Options

Custom variables allow you to define a list of options. These options can be anything you need for your dashboard.

Use Cases:

• Environment Groups: You might want to group servers by environment. For example, “web servers” or “database servers”.
• Severity Levels: Use it to select metrics for “critical,” “warning,” or “info” level events.
• Application Names: If you have a multi-application setup, use this variable to switch between the apps.
• Custom Data Sets: You can use this to choose data ranges that are not time based.

Configuration Options:

• Name: Give the variable a descriptive name.
• Values: Define all available options separated by commas.

Example Scenario:

You need to select metrics from different application groups. To set up a custom variable:

1. Name: Set the name as application_group.
2. Type: Select “Custom”.
3. Values: Input “web,database,messaging”.

You can then use the variable in your query: SELECT metric FROM data WHERE app = '$application_group'. This lets you switch views with only one selection.

Text Box Variables: User-Defined Input

Text box variables allow users to input any value. They are great for flexible filtering of data.

Use Cases:

• Filtering by Log Message: Allow a user to enter a keyword to filter a log message.
• Searching for a User: Use this variable to search for a specific user in your system.
• Filtering by Host: It is useful for searching for a server by its name.
• Arbitrary Queries: Let the user specify a specific value to query.

Configuration Options:

• Name: A unique name for the variable.
• Label: The label that users will see in the text box input.
• Default value: You can pre-fill the box for users.

Example Scenario:

You want to let users search for specific log entries. To set up this variable:

1. Name: Set the name as log_search.
2. Type: Choose “Text Box”.
3. Label: Set the label as “Log Search”.

In a panel you’ll use the variable this way: SELECT log FROM logs WHERE message LIKE '%$log_search%'. This allows users to perform ad-hoc searches in dashboards.

Data source Variables: Switching Data Sources

Data source variables let you switch between different data sources. This makes a dashboard very versatile.

Use Cases:

• Multi-environment Monitoring: You can use this for monitoring the same metrics across different environments.
• Comparing data from different systems: Use it to compare metrics between different databases, or tools.
• Vendor Selection: If you are using different products for similar tasks, this variable helps compare them in a single dashboard.

Configuration Options:

• Name: Name your variable.
• Data Source Type: Specify what types of data sources this variable will list.

Example Scenario:

Suppose you use Prometheus and InfluxDB for monitoring. To set up this variable:

1. Name: Set the name as data_source.
2. Type: Select “Data source”.
3. Data Source Type: Choose “Prometheus” and “InfluxDB”.

Now, in your panel’s settings, you can choose $data_source to switch between them. Your queries will adapt to the new source.

Interval Variables: Time-Based Selections

Interval variables are for making time-based selections. They are especially useful when you need to quickly switch the data you are viewing based on different time ranges.

Use Cases:

• Quick Time Range Switching: The user can quickly switch from 5 minute to 1 hour or 1 day ranges.
• Dynamic Aggregations: The time range can control the level of detail shown in the graphs.
• Adapting to Different Analysis Needs: When you need to see high detail for troubleshooting or long term analysis trends.

Configuration Options:

• Name: The name of the variable.
• Values: The allowed values as time intervals, like 5m, 1h, 1d.
• Auto option: Create an auto option for adaptive intervals.

Example Scenario:

You want to quickly switch the time range of a panel. To set up this interval variable:

1. Name: Set the name as time_interval.
2. Type: Select “Interval”.
3. Values: Input “5m,15m,1h,1d”.
4. Auto option: Enable this if you want Grafana to pick the interval automatically.

Now, when you create a time range query, use time_bucket('$time_interval', timestamp) to automatically group the metrics on the panel.

Advanced Variable Techniques

After getting the hang of basic variables, you can explore more advanced ways to use them. This will open up even more ways to use your dashboards.

Regular Expressions in Variables

Regular expressions (regex) can be used with variables to filter and transform values. This is very powerful when you have complex lists of values.

Use Cases:

• Extracting Hostnames from IP Addresses: You can extract hostnames by using regex that filters out the port number and other parts of a string.
• Filtering Metrics: If you have metrics that follow a pattern, use regex to filter for only the metrics that match the pattern.
• Transforming Variable Names: Use regex to extract information for the variable name from complex values.

Configuration Options:

• Regex field: In a query variable, use the regex to transform the query results.

Example Scenario:

You have a list of instance values in the form of IP:Port. You want to extract only the IP address. To set this up:

1. Query: Create a variable that queries label_values(instance).
2. Regex: In the regex field, add the following regex: ([^:]+). This regex captures all the text before the colon :.

Now the variable will only show the IPs.

Dependent Variables

Dependent variables depend on the values of other variables. They create dashboards that are very interactive.

Use Cases:

• Cascading Filters: You can select an environment, then a server that is in that environment, then metrics from the selected server.
• Context-Aware Metrics: The metrics that are shown change based on the other variables that are selected.
• Dynamic Queries: The queries will adjust to the options that the user selected.

Configuration Options:

• Query field: The query can use other variables in its definition.

Example Scenario:

You want to select an environment first, then servers within that environment. To set this up:

1. Environment Variable: Create a variable named environment with values for each one of the environments. For example, “dev”, “test”, “prod”.
2. Server Variable: Create a query variable named server. In the query field enter label_values(instance, environment='$environment'). This will fetch the servers with the selected environment value.
3. Panel Query: In a panel you can now write queries like this: metric{instance='$server', environment='$environment'}.

Now, when you select an environment, the server list will update automatically. The selected server will affect the final panel.

Using Variables in Text Panels

Variables can also be used in text panels. You can use them to create dynamic titles, descriptions, and alerts.

Use Cases:

• Dynamic Titles: The title of your panel can include the name of a server or metric, based on your selected variable.
• Descriptive Text: Create descriptions in your text panels using selected values, adding contextual information.
• Alert Messages: When there is a problem, you can include the name of the affected component in the alert.

Configuration Options:

• Use the $ symbol to access a variable within a text panel.

Example Scenario:

You want a panel to display the currently selected server name. To set it up:

1. Create a Text Panel: Create a new panel using the text visualization.
2. Enter the Variable: Input the following text in the panel: “Showing metrics for server: $server_name“.

The text panel will update the value when the variable changes.

Using the __interval Variable

Grafana provides an internal variable named __interval. This variable provides a suggested time interval, based on the time range in the dashboard. Using this in a panel query, ensures the data shown is well aggregated for the time window selected.

Use Cases:

• Automatic Data Aggregation: Grafana will dynamically choose the right time interval for the panel.
• Optimized Queries: When you use __interval, your queries are efficient even with a different time range.
• Flexible Dashboards: The same dashboard can work for different time ranges with a good level of aggregation.

Configuration Options:

• Use the variable in your query in a function like time_bucket(__interval, timestamp).

Example Scenario:

You want to show the average CPU usage in a time-series chart for the selected time range.

1. Query: In a time series panel use the following query avg(cpu_usage) by time_bucket(__interval, timestamp).
2. Aggregation: Grafana will automatically adapt to the time range selected.

This ensures the data shown is properly aggregated.

Best Practices for Grafana Variables

To make your Grafana dashboards efficient and easy to use, you should follow a few best practices when dealing with variables.

Keep Variable Names Clear

Variable names should be descriptive and easy to understand. Use consistent naming across dashboards and data sources. This prevents confusion and makes it easier for others to use and modify the dashboards.

Limit the Number of Variables

Too many variables can make a dashboard complex. It can be hard for the user to get value out of it. Use variables only when they are needed. Use dashboard templates when you want to reuse the same variables and layout across multiple dashboards.

Use Descriptions for Variables

Add descriptions to your variables to explain how they should be used. This helps other users understand the purpose of the variables. It also avoids confusion.

Validate Input

If you’re using text box variables, you should validate the user inputs. This will prevent errors. Use regular expressions to format the values to the query. For example, use regex to convert a text to lowercase before being added to the query.

Refresh Variables on a Schedule

When using query variables, make sure they’re set to refresh correctly. This ensures that the list of values is always up to date. If the list of values changes frequently, use “On Time Range Change”, or “On Dashboard Load” options.

Choose the Right Type of Variable

Select the correct type of variable for the job. Use query variables when you need to dynamically fetch data. Use constant variables for static values. Use custom variables to create a dropdown menu of options.

Test Your Dashboards

After creating a dashboard with variables, test it with different combinations. This ensures the dashboards work correctly under different conditions. Pay special attention to filters that might not apply to the other panels.

Troubleshooting Grafana Variables

Sometimes, things don’t work as expected. Here are some common problems and how to fix them:

Variable Not Displaying Values

If the dropdown menu for a variable is empty, first verify that the query is correct, and it’s returning data. Also, check that the data source is correct. Sometimes the data source does not match the query that is being used.

Query Errors

If the query errors out, ensure the variable syntax is correct. Double check that the variable name matches the name you specified. Also check if the values are correctly included in your query. You must use single quotes for text fields, and double quotes for numbers.

Performance Issues

Dashboards that use complex query variables might run slow. Optimize the queries to run faster. Also, try to fetch only the needed columns. Make sure you are using the correct aggregation in your query. Avoid using aggregations in panels where they are not needed.

Unexpected Values

If a variable contains unexpected values, double check the regex and settings of the variable. Verify that the query matches what you intend to fetch. Make sure that the filters you added to the variable are correct. Also check if other dependent variables are working as you expect.

Variables not Updating Correctly

If the variables are not updating when you expect them to, ensure the refresh setting is set to “On Dashboard Load” or “On Time Range Change”. You should also double check that all the dependencies of your variables work as intended.

Real-World Examples of Grafana Variables

Let’s explore some real-world scenarios where Grafana variables are very useful.

Cloud Infrastructure Monitoring

In a cloud environment, you may monitor multiple instances. Use a query variable to select the instance or region you are monitoring. Use text box variables to filter by tag, or other information. Use dependent variables to cascade options. This gives you an easy way to switch views, and monitor exactly what you need.

Application Performance Monitoring

When monitoring an application, you can use variables to select the application or the micro-service that is being monitored. You can use custom variables to filter by log level, or severity. Using a data source variable will help you see metrics from different sources in the same dashboard.

Database Performance Monitoring

If you have multiple databases, use a query variable to switch databases or tables. This will help you see the performance of individual components. The same can apply to database schemas, and you can use custom variables to show metrics based on database role.

Log Analysis

Use text box variables to search for logs that contain a keyword, or user identifier. You can use query variables to select log levels and other log metadata. You can combine text box and query variables to get very fine grained searches.

Business Metrics Monitoring

You can use Grafana to monitor business metrics. Use query variables to switch between different products, regions, or segments. Use custom variables to display sales or revenue by category. Then you can track the performance of all different business segments in a single place.

Implementing Grafana Variables: Step-by-Step Guide

To implement Grafana variables, let’s go through the following steps:

1. Plan your Dashboard: Before setting up the variables, you must decide what metrics you will show. Plan the layout, and all the filters that the user might need. This helps you know what variables you’ll need, and how to organize them.
2. Set Up Data Sources: Make sure Grafana is connected to all your data sources.
3. Create the Variables: Start setting up the variables. Use the correct variable type for each one. Define the values, and how they are configured.
4. Use Variables in Queries: Update the queries in the panels to use the variables. Then verify that the data is shown correctly when the variables are updated.
5. Test and Refine: Test the dashboard to make sure everything works as expected. Make adjustments to the dashboard as needed. Validate the input of variables, and double check the queries to ensure no errors.
6. Save and Share: Once you are happy with the result, save it as a dashboard, or as a template. Then share it with your team. Add clear instructions on how to use the dashboard, and how to modify it if needed.

By following these steps, you’ll create efficient dashboards that use Grafana variables. This will make it easier to manage them.

Making The Most of Grafana Variables

Grafana variables are a key tool for creating flexible, dynamic dashboards. By learning all the different types of variables, and how to use them, you can create dashboards that work for a range of different use cases. Always remember to start with a plan, to test your work, and to share your dashboards with clear instructions. If you do that you will be in the right path to mastering Grafana variables. With time and practice, you can create dashboards that provide valuable insights into your data.