Downloaded 386 times
Uploaded byHamza Zahid
message passing vs shared memory
This document compares message passing and shared memory architectures for parallel computing. It defines message passing as processors communicating through sending and receiving messages without a global memory, while shared memory allows processors to communicate through a shared virtual address space. The key difference is that message passing uses explicit communication through messages, while shared memory uses implicit communication through memory operations. It also discusses how the programming model and hardware architecture can be separated, with message passing able to support shared memory and vice versa.
message passing vs shared memory
- 2. Group Members Hamza Zahid(131391) Fahad Nadeem khan Abdual Hannan AIR UNIVERSITY MULTAN CAMPUS
- 3. Shared memory VSMessage passing
- 4. Topics • Message passing •Shared memory • Difference b/w message passing and shared memory
- 5.
- 6. INTRODUCTION • The architectureis used to communicate data among a set of processors without the need for a global memory • Each PE has its own local memory and communicates with other PE using message
- 8. MP network Two importantfactors must be considered; Link bandwidth –the number of bits that can be transmitted per unit of times(bits/s) Message transfer through the network
- 9. Process communication Processes runningon a given processor use what is called internal channels to exchange messages among themselves Processes running on different processors use the external channesls to exchange messages
- 10. Data exchanged Data exchangedamong processors cannot be shared; it is rather copied (using send/ receive messages) An important advantage of this form of data exchange is the elimination of the need for synchronization constructs , such as semaphores , which results in performance improvement
- 11. Message-Passing Interface –MPI Standardization MPI is the only message passing library which can be considered a standard. It is supported on virtually all HPC platforms. Practically, it has replaced all previous message passing libraries. Portability There is no need to modify your source code when you port your application to a different platform that supports the MPI standard
- 12. Message-Passing Interface –MPI Performance Opportunities Vendor implementations should be able to exploit native hardware features to optimize performance. Functionality Over 115 routines are defined. Availability A variety of implementations are available, both vendor and public domain.
- 13. MPI basics Start Processes SendMessages Receive Messages Synchronize With these four capabilities, you can construct any program. MPI offers over 125 functions.
- 14.
- 15. Introduction Processors communicatewith shared address space Easy on small-scale machines Shared memory allows multiple processes to share virtual memory space. This is the fastest but not necessarily the easiest (synchronization- wise) way for processes to communicate with one another. In general, one process creates or allocates the shared memory segment. The size and access permissions for the segment are set when it is created.
- 17. Uniform Memory Access(UMA) Most commonly represented today by Symmetric Multiprocessor (SMP) machines Identical processors Equal access and access times to memory Sometimes called CC-UMA - Cache Coherent UMA. Cache coherent means if one processor updates a location in shared memory, all the other processors know about the update. Cache coherency is accomplished at the hardware level.
- 18.
- 19. Non-Uniform Memory Access(NUMA) Often made by physically linking two or more SMPs One SMP can directly access memory of another SMP Not all processors have equal access time to all memories Memory access across link is slower If cache coherency is maintained, then may also be called CC- NUMA - Cache Coherent NUMA
- 20.
- 21. Advantages Global addressspace provides a user-friendly programming perspective to memory Model of choice for uniprocessors, small-scale MPs Ease of programming Lower latency Easier to use hardware controlled caching Data sharing between tasks is both fast and uniform due to the proximity of memory to CPUs
- 22. Disadvantages Primary disadvantageis the lack of scalability between memory and CPUs. Adding more CPUs can geometrically increases traffic on the shared memory-CPU path, and for cache coherent systems, geometrically increase traffic associated with cache/memory management. Programmer responsibility for synchronization constructs that ensure "correct" access of global memory. Expense: it becomes increasingly difficult and expensive to design and produce shared memory machines with ever increasing numbers of processors.
- 23.
- 24. Message Passing vs.Shared Memory • Difference: how communication is achieved between tasks Message passing programming model – Explicit communication via messages – Loose coupling of program components – Analogy: telephone call or letter, no shared location accessible to all Shared memory programming model – Implicit communication via memory operations (load/store) – Tight coupling of program components – Analogy: bulletin board, post information at a shared space • Suitability of the programming model depends on the problem to be solved. Issues affected by the model include: Overhead, scalability, ease of programming
- 25. Message Passing vs.Shared Memory Hardware • Difference: how task communication is supported in hardware Shared memory hardware (or machine model) – All processors see a global shared address space • Ability to access all memory from each processor – A write to a location is visible to the reads of other processors Message passing hardware (machine model) – No global shared address space – Send and receive variants are the only method of communication between processors (much like networks of workstations today, i.e. clusters) • Suitability of the hardware depends on the problem to be solved as well as the programming model.
- 26. Programming Model vs.Architecture • Machine Programming Model – Join at network, so program with message passing model – Join at memory, so program with shared memory model – Join at processor, so program with SIMD or data parallel • Programming Model Machine – Message-passing programs on message-passing machine – Shared-memory programs on shared-memory machine – SIMD/data-parallel programs on SIMD/data-parallel machine
- 27. Separation of Modeland Architecture Shared Memory – Single shared address space – Communicate, synchronize using load / store – Can support message passing Message Passing – Send / Receive – Communication + synchronization – Can support shared memory