admin管理员组

文章数量:1391929

I have a log file I am ingesting to Grafana via Loki.

Below is the log structure and a sample from the logs.

2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=Request STATUS=START EPOCH=1741922545322
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=getLoginInformation STATUS=START EPOCH=1741922545327
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=getLoginInformation STATUS=END EPOCH=1741922545335
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=getPurchaseInformation STATUS=START EPOCH=1741922545343
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=doPurchaseValidation STATUS=START EPOCH=1741922545343
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=doPurchaseValidation STATUS=END EPOCH=1741922545346
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=doBankValidation STATUS=START EPOCH=1741922545347
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=doBankValidation STATUS=END EPOCH=1741922545347
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=doPurchaseValidation STATUS=START EPOCH=1741922545348
2025-03-14 11:22:25 THREADID=52 KEYID=8009189999003224 TASK=doPurchaseValidation STATUS=END EPOCH=1741922545350
2025-03-14 11:22:26 THREADID=52 KEYID=8009189999003224 TASK=Request STATUS=END EPOCH=1741922546643

I am trying to calculate the time taken for each of the TASK values. TASK values can be associated with the THREADID, KEYID, TASK combination.

So far what I have done is

  1. Create two PromQL queries using Pattern to identify START logs and END logs. Pattern used = {filename="/var/log/test.log} | pattern `<time> THREADID=<threaded> KEYID=<keyID> TASK=<task> STATUS=<status> EPOCH=<epoch>`
  2. Applied Extract fields, Filter data by values to filter erroneous data on both queries, Filter field by name to get the desired table format.

Now I have two tables for end and start each containing columns epoch, status, task, threadId, time and keyID.

How can I merge these two tables based on task, threadId and keyId to get the time taken for each task.

Example:

2025-03-14 11:22:25 THREADID=52 TOKID=8009189999003224 TASK=Request STATUS=START EPOCH=1741922545322

2025-03-14 11:22:26 THREADID=52 TOKID=8009189999003224 TASK=Request STATUS=END EPOCH=1741922546643

So, Graph should show time taken to Request at time 2025-03-14 11:22:25 is 1321ms (1741922546643 - 1741922545322)

I trying to show all the time executions on the same panel or multiple panels so I can know which task takes most time.

本文标签: How to combine two queries in Grafana to get the time differenceStack Overflow