Demo: https://youtu.be/OOPVA-eqBTY

This project leverage the power of multiple GPUs with the target is to reduce the training time of complex models by data parallelism method with 2 approaches:

1. Multi-worker Training using 2 PCs with GeForce RTX GPU as Workers via:
   • Local area network (LAN).
   • VPN tunnel using OpenVPN (not included in the demo).
2. Parameter Server Training using 5 machines in LAN:
   • 2 Laptops as Parameter Server connected via 5GHz Wi-Fi.
   • 2 PCs with GeForce RTX GPU as Workers.
   • 1 PC just with CPU as a Coordinator.

We used our self-built 30VNFoods dataset which includes collected and labeled images of 30 famous Vietnamese dishes. This dataset is divided into:

• 17,581 images for training.
• 2,515 images for validation.
• 5,040 images for testing.

In addition, we also used a small TensorFlow flowers dataset with about 3700 images of flowers, which includes 5 folders corresponding to 5 types of flowers (daisy, dandelion, roses, sunflowers, tulips).

The iperf3 tool is used to measure the bandwidth of machines in network.

⇒ For more information, see Report.pdf.

• Distributed training with Keras
• A friendly introduction to distributed training (ML Tech Talks)
• Distributed TensorFlow training (Google I/O '18)
• Inside TensorFlow: Parameter server training
• Performance issue for Distributed TF
• When is TensorFlow's ParameterServerStrategy preferable to its MultiWorkerMirroredStrategy?