database - How to ingest ILP over HTTP without external dependencies? - Stack Overflow

admin管理员组

文章数量:1332687

I am using QuestDB and I can ingest data with ILP over HTTP using the client libraries, so there is an available HTTP endpoint in QuestDB for ingestion.

I would like to be able to ingest with no dependencies, just using curl or any other HTTP client, but when I try doing this:

curl -X PUT http://localhost:9000 --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\n'

I get Method not supported

I am using QuestDB and I can ingest data with ILP over HTTP using the client libraries, so there is an available HTTP endpoint in QuestDB for ingestion.

I would like to be able to ingest with no dependencies, just using curl or any other HTTP client, but when I try doing this:

curl -X PUT http://localhost:9000 --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\n'

I get Method not supported

• database
• time-series
• questdb

Share Improve this question Follow asked Nov 20, 2024 at 17:11 Javier RamirezJavier Ramirez 4,03211 gold badge2727 silver badges3636 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

The url and port for ingestion are indeed (by default) http://localhost:9000, but the endpoint for ingestion is /write, so we have to send the data like this:

 curl -X POST http://localhost:9000/write --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\n'

It is worth noting that when using the client libraries, they take care of things like buffering rows (by number of rows or by frequency) or reconnecting in case of any transient failures, but when using a standalone HTTP client we won't be getting any of that.

At the very least, it is a good idea to write the data in batches, unless we have very low ingestion volume. Otherwise, the server will experience a lot of overhead due to small transactions. For reference, the official libraries have a default of 75K rows per batch (or 1000 milliseconds, whatever comes first).

Sending multiple rows in one go is as easy as separating them with newlines, as in:

 curl -X POST http://localhost:9000/write --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\ntrades,symbol=BTC-USD,side=sell price=39269.98,amount=0.001 1646762637710419000'
``

本文标签： databaseHow to ingest ILP over HTTP without external dependenciesStack Overflow