Computer Programming II Algorithm Analysis and Sorting Techniques - - PowerPoint PPT Presentation
Computer Programming II Algorithm Analysis and Sorting Techniques Algorithm A sequence of well-defined instructions, which with a finite number of steps solves a problem or calculation. Components Sequence: a succession of instructions
A sequence of well-defined instructions, which with a finite number of steps solves a problem or calculation. Components
This week you have been working on constructing algorithms using recursion
There are many algorithms that can solve the same problem A common problem to solve in CS is sorting of lists and there are many different ways to go about it
Some more efficient (and effective) than others
def bogo_sort(a_list): while (is_sorted(a_list)== False): shuffle(a_list)
https://youtu.be/aSH-7EICngo
def bubble_sort(a_list): n = len(a_list) for i in range(n - 1): for j in range(0, n - i - 1): if a_list[j] > a_list[j + 1]: a_list[j], a_list[j + 1] = a_list[j + 1], a_list[j]
https://youtu.be/R-WHcRdxaEk
Your first assignment contains two common sorting solutions
These can also be algorithmically implemented in different ways since they are ideas of how to solve a particular problem
1. Iterate from a_list[1] to a_list[n] 2. Compare the current element (key) to its predecessor. 3. If the key element is smaller than its predecessor, compare it to the elements
swapped element.
def insertion_sort(a_list): length = len(a_list) for i in range(1, length): value = a_list[i] while a_list[i - 1] > value and i > 0: a_list[i], a_list[i - 1] = a_list[i - 1], a_list[i] i -= 1
def insertion_sort_rec(a_list, n): if n <= 1: return insertion_sort_rec(a_list, n - 1) last = a_list[n - 1] i = n - 2 while i >= 0 and a_list[i] > last: a_list[i + 1] = a_list[i] i = i - 1 a_list[i + 1] = last
https://youtu.be/sl1PeLOzxQI
1. Find the middle point to divide the array into two halves 2. Do a recursive call for the first half 3. Do a recursive call for the second half 4. Merge the two halves sorted in step 2 and 3
https://youtu.be/fuCYYoRejc0
Visually, we can see how differently these algorithms work How do we choose which approach to use for a particular problem? We can of course analyse them by running them a large number of times, take the average time and compare them
However, the time might vary a lot depending on
So how do we compare if there are so many variables? We get an approximation of their efficiency by considering the change in performance as the number of operations increases as a function of the number of elements we provide the algorithm That is, how does the algorithm perform with a larger input size compared to a smaller input size, given that everything else is constant? Studying the time complexity of an algorithm is defined as asymptotic analysis
def search(x, a_list): for i in a_list: if i == x: return True return False
l = [7, 2, 4, 1, 5, 3, 9] search(7, l)
The search ends immediately: Best case scenario
l = [7, 2, 4, 1, 5, 3, 9] search(9, l)
The search has to iteration through all indices: Worst case scenario
l = [7, 2, 4, 1, 5, 3, 9] search(6, l)
The search has to iteration through all indices: Worst case scenario
O Ordo or Big O (upper bound) Signifies the worst case scenario Ω Omega (lower bound) Signifies the best case scenario Θ Theta (upper and lower bound) Signifies the average case
O(1) Constant time O(n) Linear time O(n2) Quadratic time
Elements Operations
l = [7, 2, 4, 1, 5, 3, 9] search(7, l)
The search ends immediately: Ω(1)
l = [7, 2, 4, 1, 5, 3, 9] search(9, l)
The search has to iteration through all indices: O(n)
l = [7, 2, 4, 1, 5, 3, 9] search(6, l)
The search has to iteration through all indices: O(n) Average: Θ(n)
def bubble_sort(a_list): n = len(a_list) for i in range(n - 1): for j in range(0, n - i - 1): if a_list[j] > a_list[j + 1]: a_list[j], a_list[j + 1] = a_list[j + 1], a_list[j]
In bubble sort, when the input array is already sorted, the time taken by the algorithm is linear i.e. the best case: Ω(n) But, when the input array is in reverse condition, the algorithm takes the maximum time (quadratic) to sort the elements i.e. the worst case: O(n2) When the input array is neither sorted nor in reverse order, then it takes average time: Θ(n2)
Algorithm Time Complexity Best Average Worst Selection Sort Ω(n2) θ(n2) O(n2) Bubble Sort Ω(n) θ(n2) O(n2) Insertion Sort Ω(n) θ(n2) O(n2) Heap Sort Ω(n log n) θ(n log n) O(n log n) Quick Sort Ω(n log n) θ(n log n) O(n2) Merge Sort Ω(n log n) θ(n log n) O(n log n)
Will two algorithms with a time complexity of O(n2) always perform the same? When would you use a more inefficient algorithm over an efficient one? Is asymptotic analysis really necessary when CPUs are so fast as they are today?
Will two algorithms with a time complexity of O(n2) always perform the same? No, the notation only expresses how the particular algorithm performs given an increase in its input
Will two algorithms with a time complexity of O(n2) always perform the same? No, the notation only expresses how the particular algorithm performs given an increase in its input When would you use a more inefficient algorithm over an efficient one? When the gain in efficiency is no worth the extra complexity
More important to run a program correctly than fast Make no sacrifices for minimal gains in efficiency
If your code runs slow: Find a better algorithm!
Will two algorithms with a time complexity of O(n2) always perform the same? No, the notation only expresses how the particular algorithm performs given an increase in its input When would you use a more inefficient algorithm over an efficient one? When the gain in efficiency is no worth the extra complexity Is asymptotic analysis really necessary when computer are so fast as they are today?
With increased computational power, we tend to compute even more data For example
For those of you interested more in how to analyse time complexity, I recommend this video explaining how to analyse your code: https://www.youtube.com/watch?v=D6xkbGLQesk&list=PLBZBJbE_rGRV8D7XZ0 8LK6z-4zPoWzu5H&index=7 I’ve also included a few more slides showing visualisations of sorting algorithms here for those who enjoy watching them :)
https://youtu.be/NfQQGVN9fI4
https://youtu.be/EgWnJrpjpF4
https://youtu.be/tTt3F9PwTJ8
https://youtu.be/PlnoGtcmYXE