database - Ingest data into QuestDB via Telegraf with autthentication - Stack Overflow

admin管理员组

文章数量:1396802

I have a telegraf setup reading data from mqtt and I want to ingest into QuestDB OSS. With no authentication and this config, it works

[outputs.influxdb_v2]]
  urls = ["http://localhost:9000"]
  bucket = "questdb-metrics"
  content_encoding = "identity"  

But I wanted to add authentication, so created a JWK token and passed it as

[outputs.influxdb_v2]]
  token = "my_token"
  urls = ["http://localhost:9000"]
  bucket = "questdb-metrics"
  content_encoding = "identity"  

But that is also failing with 401. The token is working, as I can use it from a python script to send data into QuestDB.

conf = (
        f"tcp::addr={questdb['host']}:{questdb['ilp_port']};"
        f"username={questdb['ilp_json_web_key_kid']};"
        f"token={questdb['ilp_json_web_key_d']};"
        f"token_x={questdb['ilp_json_web_key_x']};"
        f"token_y={questdb['ilp_json_web_key_y']};"
    )
    with Sender.from_conf(conf) as sender:
        sender.row(
        'table',
        symbols={'id': 'test1'},
        columns={'value1': 20.0, 'value2': 0.5},
        at=TimestampNanos.now())
        sender.flush()

How can I configure this to send data from Telegraf with authentication?

I have a telegraf setup reading data from mqtt and I want to ingest into QuestDB OSS. With no authentication and this config, it works

[outputs.influxdb_v2]]
  urls = ["http://localhost:9000"]
  bucket = "questdb-metrics"
  content_encoding = "identity"  

But I wanted to add authentication, so created a JWK token and passed it as

[outputs.influxdb_v2]]
  token = "my_token"
  urls = ["http://localhost:9000"]
  bucket = "questdb-metrics"
  content_encoding = "identity"  

But that is also failing with 401. The token is working, as I can use it from a python script to send data into QuestDB.

conf = (
        f"tcp::addr={questdb['host']}:{questdb['ilp_port']};"
        f"username={questdb['ilp_json_web_key_kid']};"
        f"token={questdb['ilp_json_web_key_d']};"
        f"token_x={questdb['ilp_json_web_key_x']};"
        f"token_y={questdb['ilp_json_web_key_y']};"
    )
    with Sender.from_conf(conf) as sender:
        sender.row(
        'table',
        symbols={'id': 'test1'},
        columns={'value1': 20.0, 'value2': 0.5},
        at=TimestampNanos.now())
        sender.flush()

How can I configure this to send data from Telegraf with authentication?

• database
• time-series
• questdb

Share Improve this question Follow asked Mar 26 at 17:30 Javier RamirezJavier Ramirez 4,08511 gold badge2727 silver badges3636 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

QuestDB can speak ILP using either the TCP or the HTTP transport. Autnentication is different depending on which transport you choose https://questdb/docs/reference/api/ilp/overview/#authentication

We recommend typically using HTTP when sending data to questdb, as it gives you greater control about auto flushing parameters, retries on resumable errors, and gives you error information for non-recoverable errors.

to configure basic auth, you need to enable these two variables, via env variable or server.conf

export QDB_HTTP_USER=your_user
export QDB_HTTP_PASSWORD=your_password

or in server.conf

http.user=your_user
http.password=your_password

Now in telegraf.conf you can do

[[outputs.influxdb_v2]]
  urls = ["http://host.docker.internal:9000"]
  bucket = "questdb-metrics"
  http_headers = {"Authorization"="Basic eW91cl91c2VyOnlvdXJfcGFzc3dvcmQ="}

  content_encoding = "identity"  # Important to ensuring no gzip encoding

to generate the value of the header you need to base64 encode the user and password separated by a semicolon. In my case I based encoded my user (influx) and password (my_password) as in your_user:your_password, which is eW91cl91c2VyOnlvdXJfcGFzc3dvcmQ=.

Note I am using http://host.docker.internal:9000 as I am running telegraf on docker, change appropriately to your QuestDB instance making sure you use the HTTP port, not the TCP one (HTTP is the same where the web console and the REST API are listening and defaults to 9000)

本文标签： databaseIngest data into QuestDB via Telegraf with autthenticationStack Overflow