Dynamic libraries not found when launching a python script using Slurm - Stack Overflow

admin管理员组

文章数量:1400201

I have difficulties setting my environment variables when launching a python script using Slurm.

The following python script import petsc4py without error when launched from a shell session :

from petsc4py import PETSc
print("petsc4py imported successfully")

However when I try to run it using srun with the following sbatch script (first setting the same environment by sourcing the same python env and loading the same modules) :

#!/bin/bash
#SBATCH --job-name=petsc4py_import_test      
#SBATCH --ntasks=2                 
#SBATCH --cpus-per-task=8        
#SBATCH --time=00:30:00            
#SBATCH --output=output.log        
#SBATCH --error=error.log          
#SBATCH --nodes=1                  

source $HOME/.bash_profile
module purge
module load python/3.10.0 openmpi gcc openblas/latest slurm
source $HOME/my_env/bin/activate

srun --export=ALL --mpi=pmix_v3 python3 petsc4py_test.py

It fails with :

ImportError: liblapack.so.3: cannot open shared object file: No such file or directory
srun: error: node001: tasks 0-1: Exited with exit code 1

I have difficulties setting my environment variables when launching a python script using Slurm.

The following python script import petsc4py without error when launched from a shell session :

from petsc4py import PETSc
print("petsc4py imported successfully")

However when I try to run it using srun with the following sbatch script (first setting the same environment by sourcing the same python env and loading the same modules) :

#!/bin/bash
#SBATCH --job-name=petsc4py_import_test      
#SBATCH --ntasks=2                 
#SBATCH --cpus-per-task=8        
#SBATCH --time=00:30:00            
#SBATCH --output=output.log        
#SBATCH --error=error.log          
#SBATCH --nodes=1                  

source $HOME/.bash_profile
module purge
module load python/3.10.0 openmpi gcc openblas/latest slurm
source $HOME/my_env/bin/activate

srun --export=ALL --mpi=pmix_v3 python3 petsc4py_test.py

It fails with :

ImportError: liblapack.so.3: cannot open shared object file: No such file or directory
srun: error: node001: tasks 0-1: Exited with exit code 1

• python
• slurm

Share Improve this question Follow edited Mar 25 at 18:25 sbrisson asked Mar 24 at 14:47 sbrissonsbrisson 2344 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

This could be because liblapack is available as a operating system package on the node where the shell session is run and not available on the node on which the job runs. You might have to load the lapack module if one exists

module load python/3.10.0 openmpi gcc openblas/latest lapack/latest slurm

本文标签： Dynamic libraries not found when launching a python script using SlurmStack Overflow