Apache Flink Application Mode: Resume from Savepoints - Stack Overflow

admin管理员组

文章数量:1127656

I run a Flink cluster in Application Mode. My main() method successfully submits multiple jobs to the cluster. The cluster is started using this command: docker-entrypoint.sh standalone-job --python /opt/flink/usrlib/streams/main.py --pyFiles file:///opt/flink/usrlib. The Flink version is 1.20. All jobs are implemented in Pyflink using the Table API.

Now, I want to implement resuming jobs from savepoints.

Using the CLI, I can list jobs and create savepoints as expected:

> bin/flink list

------------------ Running/Restarting Jobs -------------------
09.01.2025 13:35:24 : cad9ac0bacf68f6d2deca61c47af7063 : job1 (RUNNING)
09.01.2025 13:35:32 : 654cbfa24ccdaed17e2df9a4ad6a3eb7 : job2 (RUNNING)
--------------------------------------------------------------

> bin/flink savepoint cad9ac0bacf68f6d2deca61c47af7063

Triggering savepoint for job cad9ac0bacf68f6d2deca61c47af7063.
Waiting for response...
Savepoint completed. Path: wasbs://mystorage.blob.core.windows/savepoints/savepoint-cad9ac-055182d6fdb8
You can resume your program from this savepoint with the run command.

After creating savepoints for all jobs, I want to be able to shut down the cluster and restart it later, while resuming from these checkpoints. However, it is unclear how this should be done. The docs only explain how to do this for single jobs.

Is it possible to resume an Application Mode cluster running multiple jobs from savepoints? What is the intended way to do this for jobs implemented in python (Table API)?

本文标签： Apache Flink Application Mode Resume from SavepointsStack Overflow