Add docker build doc #3676

wangkuiyi · 2017-08-25T04:47:21Z

luotao1

中文部分LGTM。英文部分请美研的同学再看看。

三个直接放链接的部分，能否改成[标题](链接)的格式呢？

luotao1 · 2017-08-25T05:01:48Z

doc/howto/dev/build_cn.md

+
+- 学习 Docker 有多难？
+
+ 理解 Docker 并不难，大概花十分钟看一遍 https://zhuanlan.zhihu.com/p/19902938 即可。这可以帮您省掉花一小时安装和配置各种开发工具，以及切换机器时需要新安装的辛苦。别忘了 PaddlePaddle 更新可能导致需要新的开发工具。更别提简化问题复现带来的好处了。


分布式机器学习的故事：Docker改变世界

luotao1 · 2017-08-25T05:02:09Z

doc/howto/dev/build_cn.md

+
+- 可以并行编译吗？
+
+ 是的。我们的 Docker image 运行一个 Bash 脚本 https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh 。这个脚本调用 `make -j$(nproc)` 来启动和 CPU 核一样多的进程来并行编译。


Bash 脚本

luotao1 · 2017-08-25T05:02:49Z

doc/howto/dev/build_cn.md

+
+- Docker on Windows/MacOS？
+
+ Docker 在 Windows 和 MacOS 都可以运行。不过实际上是运行在一个 Linux 虚拟机上。可能需要注意给这个虚拟机多分配一些 CPU 和内存，以保证编译高效。具体做法请参考 https://github.com/PaddlePaddle/Paddle/issues/627 。


putcn · 2017-08-25T17:32:52Z

doc/howto/dev/build_en.md

+ git clone https://github.com/paddlepaddle/paddle
+ ```
+
+2. Install build tools.


would this be clear if we call this step as: Create docker image for building Paddlepaddle binary

putcn · 2017-08-25T17:40:58Z

doc/howto/dev/build_en.md

+1. A computer -- Linux, BSD, Windows, MacOS, and
+1. Docker.
+
+Nothing else. Not even Python and GCC, because you can install all build tools into a Docker image.


do we need to add bit more explanation: think the docker container as a standalone factory, it takes paddlepaddle source code and produce paddlepaddle binary. Developer plays the role of a regular consumer who doesn't need to know any detail of the factory, and just use/test the product of the factory.

I am afraid that it is intractable to explain Docker clearly enough in this document, so I have a hyperlink later pointing to an official Docker document. Also, I added

We run all the tools by running this image.

putcn · 2017-08-25T17:47:03Z

doc/howto/dev/build_en.md

+ docker run -v $PWD:/paddle paddle:dev
+ ```
+
+ This builds a CUDA-enabled version and writes all binary outputs to directory `./build` of the local computer, other than the Docker container. If we want to build only the CPU part, we can type


how about '{source}/build'

The fact is that all output binary files are in the build directory under the source code root directory.

I changed to

to the sub-directory build in the source root directory on the local computer,

putcn · 2017-08-25T17:51:21Z

doc/howto/dev/build_en.md

+
+- Docker or virtual machine?
+
+ Some people compare Docker with VMs, but Docker doesn't virtualize any hardware, and it doesn't run a guest OS.


which means there is no compromise on performance.

Great point! Done.

putcn · 2017-08-25T17:54:30Z

doc/howto/dev/build_en.md

+
+- Why Docker?
+
+ Using a Docker image of build tools standardize the building environment, and easier for others to reproduce your problem, if there is any, and help.


Using a Docker image of build tools standardize the building environment, which makes it easier for others to reproduce your problem and easier to help.

Thanks! Done.

putcn · 2017-08-25T17:56:51Z

doc/howto/dev/build_en.md

+
+ Also, some build tools don't run on Windows or Mac or BSD, but Docker runs almost everywhere, so developers can use whatever computer they want.
+
+- Can I don't use Docker?


Can I choose not to use Docker?

Thanks! Done.

putcn · 2017-08-25T17:58:36Z

doc/howto/dev/build_en.md

+
+- Can I don't use Docker?
+
+ Sure, you don't have to install build tools into a Docker image; instead, you can install them onto your local computer. This document exists because Docker would make the development way easier.


you can install them into your local computer.

putcn · 2017-08-25T18:04:11Z

doc/howto/dev/build_en.md

+
+- How difficult is it to learn Docker?
+
+ It takes you ten minutes to read https://docs.docker.com/get-started/ and saves you more than one hour to install all required build tools, configure them, and upgrade them when new versions of PaddlePaddle require some new tools.


It takes you ten minutes to read https://docs.docker.com/get-started/ and saves you more than one hour to install all required build tools, configure them, especially when new versions of PaddlePaddle require some new tools. Not even to mention the time saved when other people trying to reproduce the issue you have.

Great rephrase! Done.

putcn · 2017-08-25T18:06:53Z

doc/howto/dev/build_en.md

+
+- Docker requires sudo
+
+ An owner of a computer has the administrative privilege, a.k.a., sudo. If you use a shared computer for development, please ask the administrator to install and configure Docker. We will do our best to support rkt, another container technology that doesn't require sudo.


An owner of a computer has the administrative privilege, a.k.a., sudo, and Docker requires this privilege to work properly.

putcn · 2017-08-25T18:10:25Z

doc/howto/dev/build_en.md

+ If we used `WITH_GPU=OFF` at build time, it generates only CPU-based unit tests, and we don't need nvidia-docker to run them. We can just run
+
+ ```bash
+ docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"


looks this document is majorly for Docker muggles, do we need to add some explanations to all the commands and parameters used above?

Good point! Done.

putcn · 2017-08-25T18:13:47Z

doc/howto/dev/build_en.md

+
+- Does Docker do parallel building?
+
+ Our building Docker image runs a Bash script https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh, which calls `make -j$(nproc)` to starts as many processes as the number of your processors.


to starts as many processes as the number of your CPU cores.

putcn · 2017-08-25T18:17:52Z

doc/howto/dev/build_en.md

+
+ Our building Docker image runs a Bash script https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh, which calls `make -j$(nproc)` to starts as many processes as the number of your processors.
+
+- Docker on Windows/MacOS?


We might want to group "Docker requires sudo" section and "Docker on Windows/Mac" in to "Some Gotchas" or "trouble shooting" section?

Great point! Done.

putcn · 2017-08-25T18:54:16Z

just followed the instructions to have a trial run on the CPU version, looks all command works as expected, except the process stuck at the memory unit test, I might need to adjust Docker's memory setting.

putcn · 2017-08-25T23:40:56Z

LGTM

putcn

LGTM

putcn · 2017-08-25T23:53:38Z

#3690

dzhwinter

LGTM++

Yi Wang added 3 commits August 24, 2017 19:44

Add a document on building using Docker

f22ece9

Update unit test running and CUDA

9fdf397

Update index and add Chinese version

1e61d91

wangkuiyi requested review from QiJune, gangliao, luotao1, putcn and typhoonzero August 25, 2017 04:47

This was referenced Aug 25, 2017

cross compile on mac #3368

Closed

Paddle needs better quick start documents #2780

Closed

luotao1 reviewed Aug 25, 2017

View reviewed changes

putcn reviewed Aug 25, 2017

View reviewed changes

Yi Wang added 2 commits August 25, 2017 11:24

In response to comments from Luo Tao

c8d0c9a

In response to comments from Chen Xi

f71f393

putcn mentioned this pull request Aug 25, 2017

unit test stuck at memory test step #3687

Closed

Yi Wang and others added 2 commits August 25, 2017 14:31

Update build.sh

4b0235c

Run a specific test

ab53bea

putcn closed this Aug 25, 2017

putcn reopened this Aug 25, 2017

putcn previously approved these changes Aug 25, 2017

View reviewed changes

Add clean build section

852f341

wangkuiyi dismissed putcn’s stale review via 852f341 August 25, 2017 23:49

Remove stopped containers and dangling images

ec5e20c

dzhwinter approved these changes Aug 26, 2017

View reviewed changes

wangkuiyi merged commit 111812f into PaddlePaddle:develop Aug 26, 2017

luotao1 mentioned this pull request Aug 29, 2017

http://doc.paddlepaddle.org/develop/ don't updated now #3745

Closed

heavengate pushed a commit to heavengate/Paddle that referenced this pull request Aug 16, 2021

fix output in cpp infer (PaddlePaddle#3676)

37572ab


		- 学习 Docker 有多难？

		理解 Docker 并不难，大概花十分钟看一遍 https://zhuanlan.zhihu.com/p/19902938 即可。这可以帮您省掉花一小时安装和配置各种开发工具，以及切换机器时需要新安装的辛苦。别忘了 PaddlePaddle 更新可能导致需要新的开发工具。更别提简化问题复现带来的好处了。


		- 可以并行编译吗？

		是的。我们的 Docker image 运行一个 Bash 脚本 https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh 。这个脚本调用 `make -j$(nproc)` 来启动和 CPU 核一样多的进程来并行编译。


		- Docker on Windows/MacOS？

		Docker 在 Windows 和 MacOS 都可以运行。不过实际上是运行在一个 Linux 虚拟机上。可能需要注意给这个虚拟机多分配一些 CPU 和内存，以保证编译高效。具体做法请参考 https://github.com/PaddlePaddle/Paddle/issues/627 。


		- Docker or virtual machine?

		Some people compare Docker with VMs, but Docker doesn't virtualize any hardware, and it doesn't run a guest OS.


		- Why Docker?

		Using a Docker image of build tools standardize the building environment, and easier for others to reproduce your problem, if there is any, and help.


		Also, some build tools don't run on Windows or Mac or BSD, but Docker runs almost everywhere, so developers can use whatever computer they want.

		- Can I don't use Docker?


		- Can I don't use Docker?

		Sure, you don't have to install build tools into a Docker image; instead, you can install them onto your local computer. This document exists because Docker would make the development way easier.


		- How difficult is it to learn Docker?

		It takes you ten minutes to read https://docs.docker.com/get-started/ and saves you more than one hour to install all required build tools, configure them, and upgrade them when new versions of PaddlePaddle require some new tools.


		- Docker requires sudo

		An owner of a computer has the administrative privilege, a.k.a., sudo. If you use a shared computer for development, please ask the administrator to install and configure Docker. We will do our best to support rkt, another container technology that doesn't require sudo.


		- Does Docker do parallel building?

		Our building Docker image runs a Bash script https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh, which calls `make -j$(nproc)` to starts as many processes as the number of your processors.


		Our building Docker image runs a Bash script https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh, which calls `make -j$(nproc)` to starts as many processes as the number of your processors.

		- Docker on Windows/MacOS?

Add docker build doc #3676

Add docker build doc #3676

Conversation

wangkuiyi commented Aug 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

luotao1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangkuiyi Aug 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

putcn Aug 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

putcn Aug 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangkuiyi Aug 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

putcn commented Aug 25, 2017

putcn commented Aug 25, 2017

putcn left a comment

Choose a reason for hiding this comment

putcn commented Aug 25, 2017

dzhwinter left a comment

Choose a reason for hiding this comment

Labels

4 participants

wangkuiyi commented Aug 25, 2017 •

edited

Loading

wangkuiyi Aug 25, 2017 •

edited

Loading

putcn Aug 25, 2017 •

edited

Loading

putcn Aug 25, 2017 •

edited

Loading

wangkuiyi Aug 25, 2017 •

edited

Loading