Download to read offline
Uploaded byMartin Abbott
Azure Batch and MPI
Martin Abbott discusses using MPI (Message Passing Interface) for parallel computing on Azure Batch. He explains that MPI allows applications to communicate across multiple VMs and is supported on Linux and Windows VMs. Examples of MPI applications include computational fluid dynamics (OpenFOAM) and fire simulation (FDS). The process involves preparing input files, copying files to storage, creating a pool and job, mounting files, running the parallel application using mpirun, and downloading results. Automation is possible using PowerShell and Azure Functions to trigger jobs from a service bus queue.
Azure Batch and MPI
- 1. Azure Batch andMPI Martin Abbott @martinabbott
- 2. What is MPI? •Message Passing Interface • Communication protocol for parallel computing • Scalable and portable • Good for HPC • On Azure, two main implementations • Open MPI - Linux • Intel MPI – Linux and Windows • A8, A9, H16r, H16mr VMs • RDMA networking
- 3. Some scenarios • ComputationalFluid Dynamics • OpenFOAM • Free on Linux • Also available on Windows • Fire Simulation • FDS (Fire Dynamics Software) • Free on both Windows and Linux
- 4. Approach • Create inputfiles in a preprocessor • Copy scripts and applications to storage • Create pool • Enable internode communication • Set startup task to install pre-requisites if required • Create Job • Create Task • Mount blob container • Unzip code, libraries and input files • Run code in parallel mode (mpirun) • Download results for post processing
- 5.
- 6. Automation possibilities • Creatingtask can be achieved through PowerShell • Azure Functions supports PowerShell • Potential Approach • Upload input files and push message to Service Bus Queue • Use QueueTrigger to execute a PowerShell command • Use BlobTrigger to execute a Function command to zip results
- 7. More information • Scenariosand run throughs (NOTE: Lots of errors in the script so mostly trial and error) https://blogs.technet.microsoft.com/windowshpc/2016/07/20/introducing-mpi- support-for-linux-on-azure-batch/ • GitHub repository for samples https://github.com/Azure/azure-batch-samples • OpenFOAM http://www.openfoam.com/ • ParaView http://www.paraview.org/ • Fire Dynamics Simulator https://www.nist.gov/services-resources/software/fds-and-smokeview
- 8.