Downloaded 47 times
Parallel and Distributed Computing chapter 1
- 1.
- 2. OUTLINE Computing Parallel computing Distributed computing Needof parallel and distributed computing Scalability
- 3. COMPUTING Computing is theprocess to complete a given goal oriented task by computer technology Computing may include the design and development of software and hardware systems for a broad range of purposes, often consist of structuring, processing and managing any kind of information
- 4. Parallel vs. serialcomputing Serial Computing: problem is broken into stream of instructions that are executed sequentially one after other on a single processor One instructions executes at a time. Parallel Computing: problem is broken into parts that can be solved concurrently, and executed simultaneously on different processors.
- 5. Parallel vs. serialcomputing
- 6. NEED OF PARALLELAND DISTRIBUTED COMPUTING We assess evolutionary changes in machine architecture, OS, Network connectivity & application workload Instead of using centralized computer to solve computational problems, parallel and distributed computing systems are uses to solve large scale problems. It uses multiple computers to solve computational problems Therefor enhanced the quality of life and information services.
- 7. NEED OF PARALLELAND DISTRIBUTED COMPUTING Billions of people use internet every day. As a result large data center must provide high performance computing services to huge numbers of internet users concurrently.
- 8. Where is parallelcomputing used Artificial intelligence Machine learning Online transaction processing Real time system and control applications Disaster prevention
- 9. HPC Vs. HTC Supercomputersand large data centers must provide High Performance Computing services to deal huge numbers of internet users concurrently But HPC is no longer optimal for measuring performance, as the speed of HPC systems has increased from Gflops (1990) to Pflops(2010) It also demands High Throughput Computing built with parallel and distributed computing technologies as internet searches and web services by millions + users simultaneously. Thus performance goal shift to measure high throughput or number of tasks completed per unit time.
- 10. PARALLEL COMPUTING SYSTEM Parallelcomputing systems are the simultaneous execution of the single task on multiple processors in order to obtain results faster The idea is based on the fact that the process of solving a problem usually can be divided into smaller tasks(divide and conquer), which may be carried out simultaneously with some coordination.
- 11. The terms parallelcomputing architecture sometimes used for computer with more than one processor (few to thousands), available for processing The recent multicore processors (chips with more than one processor core) are some commercial examples which bring parallel computing to the desktop. Communication is done through Bus in parallel computing PARALLEL COMPUTING SYSTEM
- 12. Types of ParallelComputing System Bit Level Parallelism: Uses larger “words” which is fixed sized piece of data Instruction Level Parallelism: allow processors to execute more than one instruction per clock cycle Task-Level Parallelism: run computer code across multiple processors to run multiple tasks at the same time on same data.
- 13. Parallel Computing (Advantages) Savetime and money: Solve larger/ complex problems: Provide concurrency
- 14.
- 15. Distributed Systems We definea distributed system as one in which hardware or software components located at networked computer communicate and coordinate their actions only by passing messages. The simple definition covers the entire range of systems in which networked computers can usefully be deployed. A distributed system is a collection of independent computers that appears to its users as a single coherent system. Communication is done by message passing scheme. Every computer in distributed system has its own Instruction manager/CU
- 16. Distributed Systems There aretwo types of architecture in distributed systems 1. General Purpose Including PC, Laptops etc. 2. Special purpose Mainframe computers or super computers
- 17. Differences (Parallel Vs.Distributed) 1. Multiple operations are performed at a time 2. Single computer is required 3. Multiple processors perform multiple operations 1. System components are located at different locations 2. Uses multiple computers 3. Multiple computers perform multiple operations
- 18. 4. May haveshared or distributed memory 5. Communication is done through bus 6. Improve system performance 4. Only have distributed memory 5. Communication is done through message passing 6. Improves scalability, fault tolerance and resource sharing capabilities. Differences (Parallel Vs. Distributed) Cont.
- 19. Scalability Property of asystem to handle a growing amount of work by adding resources to the system A system is described as scalable if it will remain effective when there is a significant increase in the number of resources and the number of users.
- 20. Scalability 1. Physical scalability/ load scalability: system can be scalable w.r.t its size, meaning that we can easily add/remove more users and resources to the system 2. Administrative scalability: increasing number of organization or users to access a system. 3. Functional scalability: enhance system by adding new functionality without disturbing exciting.
- 21. Scalability 1. Geographic scalability:maintain effectiveness during expansion from local area to larger region 2. Generation scalability: ability to scale by adopting new generations of components 3. Heterogeneous scalability: ability to adopt components from different vendors
- 22. Horizontal (scale out)& vertical (scale up) scaling Scaling Horizontally (out/in): adding/ removing nodes from a system Examples: scaling out (to increase) from one web server to three. This requires improvements in maintenance and management.
- 23. Scaling Vertically (up/down):means adding/removing resources to single i.e. addition of CPUs, memory to single computer These needs management, more sophisticated programming to allocate resources and handles issues Horizontal (scale out) & vertical (scale up) scaling
- 25. Scalability: Issues/Challenges Scaling Vertically(up/down): means adding/removing resources to single i.e. addition of CPUs, memory to single computer These needs management, more sophisticated programming to allocate resources and handles issues
- 26. Amdahl’s law It isnamed after a computer scientist Gene Amdahl Formula which gives speedup in latency of execution of task at fixed workload that can be expected of a system whose resources are improved. Often used in parallel computing to predict the theoretical speedup when using multiple processors.
- 27. Amdahl’s law Speedup isthe ratio of the time it takes to execute a program serially to the time it takes to execute in parallel and it is denoted by S(n) S(n) = single processing time ‘n’ processing time
- 28. Amdahl’s law Amdahl’s lawshows that a program’s serial parts limit the potential speedup from parallelizing code. When program executes on single core: Fs + Fp = T1 When program executes on single core: Fs + Fp/N= T1 Hence according to Amdahl’s law
- 29. Amdahl’s law Numerical Serial RunTime =40% Parallel Run Time =60% Number of processors=5