Fly.io offers a managed Grafana dashboard, an invaluable tool for monitoring your applications. While Fly's out-of-the-box offering is undeniably handy, there's limitations to be made aware of:
- Fly.io deploys the Open Source version of Grafana, which unfortunately means missing out on some advanced features, such as alerting.
- Customization is somewhat restricted; for instance, modifications to the
grafana.inifile aren't possible.
In this article I'll guide you through the steps to deploy Grafana Enterprise as a Fly app, so you can leverage the full power of Grafana Enterprise. Once we have Grafana Enterprise up and running, I'll show you how to get Fly metrics that we have on fly-metrics.net into our custom Grafana instance. Lastly, I'll show you how to copy the dashboard from fly-metrics.net to our Grafana Enterprise instance.
Creating the app
Let's start with the basic configuration for running Grafana enterprise on Fly. Create a file called
fly.toml in the
root of your project and add a file called
fly.toml with the following contents:
internal_port = 3000
force_https = true
auto_stop_machines = false
auto_start_machines = false
The image we're using is specified in the
[build] section. In this case we are using the Ubuntu version of Grafana
Enterprise. There's also an Alpine version, but I chose to use the Ubuntu version because that's what I am most familiar
with. As of writing,
10.0.3 is the latest version. Grafana stores its data in
/var/lib/grafana, so to be able to
persist the data across restarts we mount a volume called
grafana_data on that directory. Lastly, in
[http_service] section we configure the port that Grafana
listens to, which is port 3000.
You can now deploy using the following command:
fly launch --copy-config --auto-confirm --ha=false --now
This creates a new app based off the existing
fly.toml file. A 1GB volume will be created to store the Grafana data
lastly, the app will be deployed to a single instance. Run
fly open to check out your Grafana Enterprise installation.
You can log in with username
admin and password
The easiest way to configure Grafana is by defining environment variables. E.g. to change the default color mode to
'light', we can define the following environment variable in our
Similarly, we can use Fly secrets to configure sensitive information that we might not want in our codebase:
fly secrets set GF_SMTP_PASSWORD='secr3t!'
For detailed information about configuring Grafana, see the Grafana documentation.
Getting Fly metrics into Grafana
Now we're getting to the fun part. Fly exposes an API endpoint from which we can fetch Prometheus metrics. We can add this as a data source to our new Grafana instance. On your Grafana Enterprise instance, press "Connections", search for "Prometheus" and click "Create a Prometheus data source". There's a few important input fields:
- The name should be
prometheus_on_fly. This name is important. I will get to that later.
- Prometheus server URL:
https://api.fly.io/prometheus/<org-slug>. The organization slug you can find by running
fly orgs list.
- Custom HTTP Headers: Add a customer header called
Bearer <api-token>. You can get the API token by running
fly auth token.
Save & test. If everything went well, you should see a green message saying "Successfully queried the Prometheus
API". Now we have access to all the metrics that we have on fly-metrics.net!
Copying the dashboard from fly-metrics.net
Although we have access to the data, we don't have the cool dashboard yet that we see on fly-metrics.net. Luckily, it's quite easy to transfer this to our new instance.
Open fly-metrics.net and navigate to the dashboard that you want to copy. Next, press the 'Dashboard settings' button in the top-right corner. By default, you should see a message along of the lines of "Dashboard not editable". Press "Make editable" and save. After refreshing, you will see a sidebar menu with a bunch of options. Press "JSON model" and copy the complete JSON object. Now go back to your Grafana Enterprise instance, go to Dashboards and press "New" -> "Import". Paste the JSON object into the "Import via panel json" field and press "Load". Voilà!
In the previous section I mentioned that the name of the Prometheus data source is important. This is because if you
look carefully at the JSON model you will see references to a data source called
prometheus_on_fly. If you had chosen
different name for your data source, you would have gotten errors. If you want to change the name of the data source,
that's fine, but you will have to change all references of
prometheus_on_fly to the name of your data source.
In this article I showed you how to deploy Grafana Enterprise on Fly. We configured it in such a way that everything from fly-metrics.net can be found on our Grafana Enterprise instance. You can now start leveraging all the cool features that Grafana Enterprise has to offer and configure it to your liking. You can for example add plugins that you like or connect other data sources such as your Postgres or MySQL database.
The full source code used in this article can be found on GitHub.
If you have feedback or questions, please reach out to me on Twitter: @dejagersh.