Skip to content

[XPU] update kunlun Dockerfile.develop.xre #73669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 27, 2025
Merged

[XPU] update kunlun Dockerfile.develop.xre #73669

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f1577e5
update kunlun Dockerfile.develop.xre
Luckycheng222 Jun 27, 2025
2cb81b1
Update kunlun Dockerfile.develop.xre
Luckycheng222 Jun 27, 2025
48e4c49
Update kunlun Dockerfile.develop.xre
Luckycheng222 Jun 27, 2025
6376a46
Update kunlun Dockerfile.develop.xre
Luckycheng222 Jun 27, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 28 additions & 35 deletions tools/dockerfile/Dockerfile.develop.xre
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,39 +30,29 @@ ENV PATH=/opt/cmake-3.27.7/bin:${PATH}

# default python version
ARG PY_VERSION=3.10
RUN apt-get install -y python3-distutils python3.8 python3.8-dev python3.9 python3.9-dev python3.10 python3.10-dev
RUN apt-get update && apt-get install -y python3.10-distutils python3.10 python3.10-dev

# install pip
RUN curl -s -q https://bootstrap.pypa.io/get-pip.py | /usr/bin/python3.8 && \
curl -s -q https://bootstrap.pypa.io/get-pip.py | /usr/bin/python3.9 && \
curl -s -q https://bootstrap.pypa.io/get-pip.py | /usr/bin/python3.10
RUN curl -s -q https://bootstrap.pypa.io/get-pip.py | /usr/bin/python3.10

# set default python
RUN rm -rf /usr/bin/python3 && ln -s /usr/bin/python${PY_VERSION} /usr/bin/python3 && \
rm -rf /usr/bin/python && ln -s /usr/bin/python${PY_VERSION} /usr/bin/python

# install pylint and pre-commit
RUN pip3.8 install pre-commit==2.17.0 pylint pytest astroid isort coverage qtconsole distro && \
pip3.9 install pre-commit==2.17.0 pylint pytest astroid isort coverage qtconsole distro && \
pip3.10 install pre-commit==2.17.0 pylint pytest astroid isort coverage qtconsole distro
RUN pip3.8 install attrs pyyaml pathlib2 scipy requests psutil Cython clang-format==13.0.0 && \
pip3.9 install attrs pyyaml pathlib2 scipy requests psutil Cython clang-format==13.0.0 && \
pip3.10 install attrs pyyaml pathlib2 scipy requests psutil Cython clang-format==13.0.0
RUN pip3.10 install pre-commit==2.17.0 pylint pytest astroid isort coverage qtconsole distro
RUN pip3.10 install attrs pyyaml pathlib2 scipy requests psutil Cython clang-format==13.0.0

# add more libs
RUN apt-get update && apt-get install libprotobuf-dev protobuf-compiler libprotoc-dev lsof libgeos-dev \
pkg-config libhdf5-103 libhdf5-dev lrzsz libsndfile1 tree ninja-build -y

# install Paddle requirement
RUN wget --no-check-certificate https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/python/requirements.txt -O requirements.txt && \
pip3.8 install -r requirements.txt -i https://pip.baidu-int.com/simple --trusted-host pip.baidu-int.com && \
pip3.9 install -r requirements.txt -i https://pip.baidu-int.com/simple --trusted-host pip.baidu-int.com && \
pip3.10 install -r requirements.txt -i https://pip.baidu-int.com/simple --trusted-host pip.baidu-int.com && rm -rf requirements.txt
pip3.10 install -r requirements.txt -i https://pypi.org/simple && rm -rf requirements.txt

RUN wget --no-check-certificate https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/python/unittest_py/requirements.txt -O requirements.txt && \
pip3.8 install -r requirements.txt -i https://pip.baidu-int.com/simple --trusted-host pip.baidu-int.com && \
pip3.9 install -r requirements.txt -i https://pip.baidu-int.com/simple --trusted-host pip.baidu-int.com && \
pip3.10 install -r requirements.txt -i https://pip.baidu-int.com/simple --trusted-host pip.baidu-int.com && rm -rf requirements.txt
pip3.10 install -r requirements.txt -i https://pypi.org/simple && rm -rf requirements.txt

# git credential to skip password typing
RUN git config --global credential.helper store
Expand Down Expand Up @@ -90,40 +80,43 @@ ENV CCACHE_MAXSIZE=80G \
CCACHE_LIMIT_MULTIPLE=0.8 \
CCACHE_SLOPPINESS=clang_index_store,time_macros,include_file_mtime

# Install XRE 4.31.0
ARG XRE_VERSION=4.31.0
# Install XRE 5.0.21.21
ARG XRE_VERSION=5.0.21.21
ARG XRE_INSTALL=/usr/local/xpu-${XRE_VERSION}
RUN wget -q https://klx-sdk-release-public.su.bcebos.com/xre/release/${XRE_VERSION}.1/xre-ubuntu_2004_x86_64.tar.gz && \
tar -zxf xre-ubuntu_2004_x86_64.tar.gz && \
RUN wget -q https://klx-sdk-release-public.su.bcebos.com/xre/kl3-release/${XRE_VERSION}/xre-ubuntu_2004-x86_64-${XRE_VERSION}.tar.gz && \
tar -zxf xre-ubuntu_2004-x86_64-${XRE_VERSION}.tar.gz && \
mkdir -p ${XRE_INSTALL} && \
cp -af /opt/xre-ubuntu_2004_x86_64/bin/ ${XRE_INSTALL}/ && \
cp -af /opt/xre-ubuntu_2004_x86_64/include/ ${XRE_INSTALL}/ && \
cp -af /opt/xre-ubuntu_2004_x86_64/tools/ ${XRE_INSTALL}/ && \
cp -af /opt/xre-ubuntu_2004_x86_64/version.txt ${XRE_INSTALL}/ && \
mkdir -p ${XRE_INSTALL}/lib64 && \
cp -af /opt/xre-ubuntu_2004_x86_64/lib/* ${XRE_INSTALL}/lib64/ && \
cp -af /opt/xre-ubuntu_2004_x86_64/so/* ${XRE_INSTALL}/lib64/ && \
mv -f /opt/xre-ubuntu_2004-x86_64-${XRE_VERSION}/* ${XRE_INSTALL}/ && \
ln -sf ${XRE_INSTALL} /usr/local/xpu && \
ln -sf ${XRE_INSTALL}/bin/xpu_smi /usr/local/bin/xpu_smi && \
rm -rf xre-ubuntu_2004_x86_64.tar.gz && rm -rf xre-ubuntu_2004_x86_64/
rm -rf xre-ubuntu_2004-x86_64-${XRE_VERSION}.tar.gz
ENV PATH=${XRE_INSTALL}/bin:$PATH

# Update RDMA
RUN wget "https://su.bcebos.com/v1/klx-sdk-release-public/xccl/resource/MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu20.04-x86_64.tgz?authorization=bce-auth-v1%2FALTAKlxQapmxlH5xQFcp7rEkCr%2F2025-05-28T02%3A24%3A09Z%2F-1%2Fhost%2Fbaad25d036a6eb868dad8ab19468884e5016507fdd6879fe1259db4bbef694e6" \
-O MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu20.04-x86_64.tgz && \
tar -zxf MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu20.04-x86_64.tgz && \
cd MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu20.04-x86_64 && \
./mlnxofedinstall --user-space-only --skip-distro-check --without-fw-update --force && \
rm -rf MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu20.04-x86_64.tgz && \
rm -rf MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu20.04-x86_64


# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service
RUN mkdir /var/run/sshd && echo 'root:root' | chpasswd && \
sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config && \
sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config

# Set language environment in .bashrc
RUN echo "export LANG=en_US.UTF-8" >> /root/.bashrc && \
echo "export LANGUAGE=en_US.UTF-8" >> /root/.bashrc && \
echo "export LC_ALL=en_US.UTF-8" >> /root/.bashrc

CMD source ~/.bashrc

# /proc/sys/kernel/core_pattern
RUN mkdir -p /var/core

Copy link
Contributor

@dynamicheart dynamicheart Jun 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

最后可以加上，使得中文字符可以显示

RUN echo "export LANG=en_US.UTF-8" >> /root/.bashrc \ && echo "export LANGUAGE=en_US.UTF-8" >> /root/.bashrc \ && echo "export LC_ALL=en_US.UTF-8" >> /root/.bashrc
# Update rc
RUN wget -q https://paddle-ascend.bj.bcebos.com/cvmodel/ascend-materials/Ascend-cann-kernels-910b_8.0.RC2_linux.run && \
wget -q https://paddle-ascend.bj.bcebos.com/cvmodel/ascend-materials/Ascend-cann-toolkit_8.0.RC2_linux-x86_64.run && \
wget -q https://paddle-ascend.bj.bcebos.com/cvmodel/ascend-materials/Ascend-cann-nnal_8.0.RC2_linux-x86_64.run && \
chmod +x Ascend-cann-* && ./Ascend-cann-toolkit_8.0.RC2_linux-x86_64.run --full --install-for-all --quiet && \
./Ascend-cann-nnal_8.0.RC2_linux-x86_64.run --quiet --install && ./Ascend-cann-kernels-910b_8.0.RC2_linux.run --quiet --install && \
rm -rf ./Ascend-cann-nnal_8.0.RC2_linux-x86_64.run ./Ascend-cann-kernels-910b_8.0.RC2_linux.run ./Ascend-cann-toolkit_8.0.RC2_linux-x86_64.run

# Clean
RUN apt-get clean -y
Expand Down
Loading