How to submit job with python language to Apache Flink? - Stack Overflow

admin管理员组

文章数量:1391955

I run the code with python3 and it works fine. However, when submitting the job to apache flink using the command flink run -py {path_code}, I get an error java.

Error When Submit Job

Is there anything that needs to be done before submitting a python job? if it is a file with a .jar extension, it will run fine.

I have installed apache-flink, python version 3.10.

I run the code with python3 and it works fine. However, when submitting the job to apache flink using the command flink run -py {path_code}, I get an error java.

Error When Submit Job

Is there anything that needs to be done before submitting a python job? if it is a file with a .jar extension, it will run fine.

I have installed apache-flink, python version 3.10.

• python
• apache-flink

Share Improve this question Follow edited Mar 17 at 16:05 Lewis 1,36011 gold badge1212 silver badges3737 bronze badges asked Mar 14 at 7:03 Alvindra RenaldoAlvindra Renaldo 1322 bronze badges 1

• Your image of text isn't very helpful. It can't be read aloud or copied into an editor, and it doesn't index very well, meaning that other users with the same problem are less likely to find the answer here. Please edit your post to incorporate the relevant text directly (preferably using copy+paste to avoid transcription errors). – Toby Speight Commented Mar 14 at 14:50

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

PyFlink relies on a Java Virtual Machine to execute the Flink core and a Python interpreter to run your Python code. This means Flink needs to know where to find your Python executable.
Your Python version is 3.10 so it should be

flink run -pyexec /usr/bin/python3.10 -py {path_code}

Use the command which python3.10 in your terminal to find the location of your Python 3.10 installation. If the result is different from /usr/bin/python3.10, use the path you obtained instead.

You can configure the Python executable path within Flink's configuration files, which avoids having to specify it on the command line every time you submit a job.

• Open flink-conf.yaml in a text editor.
• Add the following line: python.executable: /path/to/your/python3.x . Replacing /path/to/your/python3.x with the actual path to your Python executable.
• Save the file and restart your Flink cluster for the changes to take effect.

本文标签： How to submit job with python language to Apache FlinkStack Overflow