Lecture01 algorithm analysis

Lecture01 algorithm analysis

• 1.
  1 Algorithms Analysis andDesign Lecture 1 Prepared by ZARA
• 2.
  2 Major objective ofthis course is: ● Design and analysis of modern algorithms ● Different variants ● Accuracy ● Efficiency ● Comparing efficiencies ● Motivation thinking new algorithms ● Advanced designing techniques ● Real world problems will be taken as examples ● To create feelings about usefulness of this course Objective of This Course
• 3.
  3 On successful completion,students will be able to ● Argue and prove correctness of algorithms ● Derive and solve mathematical models of problems ● Reasoning when an algorithm calls certain approach ● Analyze average and worst-case running times ● Integrating approaches in dynamic and greedy algos. ● Use of graph theory in problems solving ● Advanced topics such as • Computational geometry, number theory etc. ● Several other algorithms such as • String matching, NP completeness, approximate algorithms etc. Expected Results
• 4.
  4 Lecture No 1 Introduction (What,Why and Where Algorithms . . .)
• 5.
  5 In this lecturewe will cover the following ● What is Algorithm? ● Designing Techniques ● Model of Computation ● Algorithms as a technology ● Algorithms and other technologies ● Importance of algorithms ● Difference in Users and Developers ● Kinds of problems solved by algorithms ● Conclusion Today Covered
• 6.
  6 ● A computeralgorithm is a detailed step-by-step method for solving a problem by using a computer. ● An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. ● An Algorithm is well defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output. ● More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output. Algorithm Input output What is Algorithm?
• 7.
  7 ● Most basicand popular algorithms are ■ Sorting algorithms ■ Searching algorithms Which algorithm is best? ● Mainly, it depends upon various factors, for example in case of sorting ■ The number of items to be sorted ■ The extent to which the items are already sorted ■ Possible restrictions on the item values ■ The kind of storage device to be used etc. Popular Algorithms, Factors of Dependence
• 8.
  8 Problem ● The statementof the problem specifies, in general terms, the desired input/output relationship. Algorithm ● The algorithm describes a specific computational procedure for achieving input/output relationship. Example ● One might need to sort a sequence of numbers into non- decreasing order. Algorithms ● Various algorithms e.g. merge sort, quick sort, heap sorts etc. One Problem, Many Algorithms
• 9.
  9 ● Brute Force ■Straightforward, naive approach ■ Mostly expensive ● Divide-and-Conquer ■ Divide into smaller sub-problems ● Iterative Improvement ■ Improve one change at a time ● Decrease-and-Conquer ■ Decrease instance size ● Transform-and-Conquer ■ Modify problem first and then solve it ● Space and Time Tradeoffs ■ Use more space now to save time later Important Designing Techniques
• 10.
  10 ● Greedy Approach ■Locally optimal decisions, can not change once made. ■ Efficient ■ Easy to implement ■ The solution is expected to be optimal ■ Every problem may not have greedy solution ● Dynamic programming ■ Decompose into sub-problems like divide and conquer ■ Sub-problems are dependant ■ Record results of smaller sub-problems ■ Re-use it for further occurrence ■ Mostly reduces complexity exponential to polynomial Some of the Important Designing Techniques
• 11.
  11 ● Analysis ■ Howdoes system work? ■ Breaking a system down to known components ■ How components (processes) relate to each other ■ Breaking a process down to known functions ● Synthesis ■ Building tools ■ Building functions with supporting tools ■ Composing functions to form a process ■ How components should be put together? ■ Final solution Problem Solving Phases
• 12.
  12 ● Problem ● Strategy ●Algorithm ■ Input ■ Output ■ Steps ● Analysis ■ Correctness ■ Time & Space ■ Optimality ● Implementation ● Verification Problem Solving Process
• 13.
  13 ● Design assumption ■Level of abstraction which meets our requirements ■ Neither more nor less e.g. [0, 1] infinite continuous interval ● Analysis independent of the variations in ■ Machine ■ Operating system ■ Programming languages ■ Compiler etc. ● Low-level details will not be considered ● Our model will be an abstraction of a standard generic single-processor machine, called a random access machine or RAM. Model of Computation (Assumptions)
• 14.
  14 ● A RAMis assumed to be an idealized machine ■ Infinitely large random-access memory ■ Instructions execute sequentially ● Every instruction is in fact a basic operation on two values in the machines memory which takes unit time. ● These might be characters or integers. ● Example of basic operations include ■ Assigning a value to a variable ■ Arithmetic operation (+, - , × , /) on integers ■ Performing any comparison e.g. a < b ■ Boolean operations ■ Accessing an element of an array. Model of Computation (Assumptions)
• 15.
  15 ● In theoreticalanalysis, computational complexity ■ Estimated in asymptotic sense, i.e. ■ Estimating for large inputs ● Big O, Omega, Theta etc. notations are used to compute the complexity ● Asymptotic notations are used because different implementations of algorithm may differ in efficiency ● Efficiencies of two given algorithm are related ■ By a constant multiplicative factor ■ Called hidden constant. Model of Computation (Assumptions)
• 16.
  16 First poor assumption ●We assumed that each basic operation takes constant time, i.e. model allows ■ Adding ■ Multiplying ■ Comparing etc. two numbers of any length in constant time ● Addition of two numbers takes a unit time! ■ Not good because numbers may be arbitrarily ● Addition and multiplication both take unit time! ■ Again very bad assumption Drawbacks in Model of Computation
• 17.
  17 Finally what aboutOur Model? ● But with all these weaknesses, our model is not so bad because we have to give the ■ Comparison not the absolute analysis of any algorithm. ■ We have to deal with large inputs not with the small size ● Model seems to work well describing computational power of modern nonparallel machines Can we do Exact Measure of Efficiency ? ● Exact, not asymptotic, measure of efficiency can be sometimes computed but it usually requires certain assumptions concerning implementation Model of Computation not so Bad
• 18.
  18 ● Analysis willbe performed with respect to this computational model for comparison of algorithms ● We will give asymptotic analysis not detailed comparison i.e. for large inputs ● We will use generic uniprocessor random-access machine (RAM) in analysis ■ All memory equally expensive to access ■ No concurrent operations ■ All reasonable instructions take unit time, except, of course, function calls Summary : Computational Model
• 19.
  19 Algorithm As aTechnology ● Efficiency: ● – Different algorithms solve the same problem often differ noticeably in their efficiency ● – These differences can be much more significant than difference due to hardware and software ● For example, insertion sort takes time roughly equal to c1n 2 (c1 is constant) to sort n items. But, merge sort takes time roughly equal to c2nlg n (c2 is constant)
• 20.
  20 Algorithm As aTechnology ● Algorithms as a technology • For example, assume a faster computer A (1010 instructions/sec) running insertion sort against a slower computer B (107 instructions/sec) running merge sort. • Suppose that c1=2， c2=50 and n = 107 . • the execution time of computer A is 2(107 ) 2 / 1010 instructions/sec = 20,000 seconds • the execution time of computer B is 50 · 107 lg 107 / 107 instructions/sec = 1,163 seconds
• 21.
  21 ● What, Whyand Where Algorithms? ● Designing Techniques ● Problem solving Phases and Procedure ● Model of computations ■ Major assumptions at design and analysis level ■ Merits and demerits, justification of assumptions taken ● We proved that algorithm is a technology ● Compared algorithmic technology with others ● Discussed importance of algorithms ■ In almost all areas of computer science and engineering ■ Algorithms make difference in users and developers Conclusion