docker - I need help in building Pytorch (CPU) from Source in a Dockerfile - Stack Overflow

admin管理员组

文章数量:1193157

I received the task to build Pytorch (Pytorch) from Source to see if it makes our processing faster. I never did this, and i am not a expert using Docker so i managed to piece together a Dockerfile to build from source:

# Base image with Python 3.11 slim
FROM python:3.11-slim

# Set environment variables
ENV GUNICORN_WORKERS=1
# Install miniconda
# conda at end of path in order to use OS python in container
ENV PATH="${PATH}:/root/miniconda3/bin"
ARG PATH="${PATH}:/root/miniconda3/bin"


# Install system dependencies required for building PyTorch
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    # cmake \
    # ninja-build \
    curl \
    libopenblas-dev \
    liblapack-dev \
    libx11-dev \
    wget \
    git \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*


RUN wget .sh \
    && mkdir /root/.conda \
    && bash Miniconda3-latest-Linux-x86_64.sh -b \
    && rm -f Miniconda3-latest-Linux-x86_64.sh \
    && conda update conda \
    && conda install cmake ninja setuptools

ENV CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}

# Install additional dependencies for Python
# RUN pip install --no-cache-dir --upgrade pip setuptools wheel
RUN pip install --no-cache-dir --upgrade pip wheel

# Set the working directory
WORKDIR /usr/src/app

# # Create and activate a virtual environment (optional)
# RUN python -m venv env \
#     && . env/bin/activate

# Clone PyTorch source code
RUN git clone --recursive .git \
    && cd pytorch \ 
    && make triton
# COPY ./pytorch ./pytorch
# RUN cd pytorch

# Install Python dependencies for PyTorch
RUN pip install -r pytorch/requirements.txt

# Install additional libraries like MKL and MAGMA (adjust based on your system)
# RUN pip install mkl-static mkl-include

# Build PyTorch
WORKDIR /usr/src/app/pytorch
RUN python setup.py develop

When i it gets to the part to python setup.py develop it breaks with the following error:

269.3 ninja: build stopped: subcommand failed.
269.4 Building wheel torch-2.7.0a0+git2efa98d
269.4 -- Building version 2.7.0a0+git2efa98d
269.4 cmake -GNinja -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/src/app/pytorch/torch -DCMAKE_PREFIX_PATH=/usr/local/lib/python3.11/site-packages;/usr/src/app/env: -DPython_EXECUTABLE=/usr/local/bin/python -DTORCH_BUILD_VERSION=2.7.0a0+git2efa98d -DUSE_NUMPY=True /usr/src/app/pytorch
269.4 cmake --build . --target install --config Release
------

 1 warning found (use docker --debug to expand):
 - UndefinedVar: Usage of undefined variable '$CMAKE_PREFIX_PATH' (line 6)
Dockerfile:46
--------------------
  44 |     # Build PyTorch
  45 |     WORKDIR /usr/src/app/pytorch
  46 | >>> RUN python setup.py develop
  47 |
  48 |     # Optional: Clean up unnecessary files to reduce image size
--------------------
ERROR: failed to solve: process "/bin/sh -c python setup.py develop" did not complete successfully: exit code: 1'

本文标签： dockerI need help in building Pytorch (CPU) from Source in a DockerfileStack Overflow