admin管理员组

文章数量:1406312

We recently built a data processing pipeline that uses Clickhouse as our big data database.

In the pipeline, data is continually ingested into a huge table (let's call it table A) with ~60 columns. It now has billions of rows of data.

Table A is the data source for multiple dependent target tables that perform aggregation on the data. The target tables are connected to Table A via the Incremental type of Materialised views (MV).

We recently need to connect additional Incremental MVs + target tables to Table A.

However, since the MVs are incremental, the new target tables are not populated.

To solve this problem, we've considered:

  1. Renaming Table A to Table B
  2. Recreate an empty Table A, and then
  3. Re-insert the data from Table B back to Table A batch by batch

However, this seems excessively troublesome and time-consuming.

What would be the best approach to handle this scenario?

本文标签：

版权声明：本文标题：database - How to trigger data population when connecting new Incremental type materialised views to a massive table with existi 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://www.betaflare.com/web/1745000616a2636952.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。