mirror of
https://github.com/dholerobin/Lecture_Notes.git
synced 2025-07-01 04:56:29 +00:00
Update TimSort.md
This commit is contained in:
Aakash Panchal
GitHub
parent
8d80812df1
commit
3da1198afb
@ -21,7 +21,7 @@ As we know Quick sort, Merge sort and Heap sort has the time complexity of $\mat
|
||||
|
||||
Tim sort is an **adaptive algorithm**, which means that it changes its behavior based on the patterns observed in the input data. That is why it is an **intelligent algorithm** and more optimal than other sorting algorithms.
|
||||
|
||||
Tim sort is a **stable algorithm** and we take too much care about the stability.
|
||||
Tim sort is a **stable algorithm** and we take too much care to maintain stability.
|
||||
|
||||
## Brief explanation of tim sort algorithm
|
||||
|
||||
@ -33,32 +33,37 @@ Tim sort is a **stable algorithm** and we take too much care about the stability
|
||||
|
||||
Now, let's discuss the entire algorithm.
|
||||
|
||||
- **Run**: Run is an ordered(sorted) sub-array. It can be non-decreasing or decreasing. The input array is to be splitted into Runs.
|
||||
```cpp
|
||||
## Run
|
||||
|
||||
_Run_ is an ordered(sorted) sub-array. It can be non-decreasing or decreasing. The input array is to be splitted into _Runs_.
|
||||
|
||||
Below is a structure for _Run_.
|
||||
```cpp
|
||||
struct run {
|
||||
// Starting address of a run
|
||||
int base_address;
|
||||
// length of run
|
||||
int len;
|
||||
run(){
|
||||
run() {
|
||||
base_address = 0;
|
||||
len = 0;
|
||||
}
|
||||
run(int a, int b)
|
||||
{
|
||||
run(int a, int b) {
|
||||
base_address = a;
|
||||
len = b;
|
||||
}
|
||||
};
|
||||
```
|
||||
- **Minimum Run length:** Minimum run length is the minimum length of such run. Its value is decided based on the number of elements in the list. Let's see the algorithm to find it out.
|
||||
```
|
||||
|
||||
## Minimum Run length
|
||||
Minimum run length is the minimum length of such run. Its value is decided based on the number of elements($n$) in the list. Let's see the algorithm to find it out.
|
||||
|
||||
Roughly the computation is:
|
||||
Roughly the computation is:
|
||||
- If $n < 64$, return $n$
|
||||
- Else if n is an exact power of $2$, return $32$.
|
||||
- Else return an integer $k$, $32 <= k <= 64$, such that $n/k$ is close to, but strictly less than, an exact power of $2$.
|
||||
|
||||
- If n < 64, return n
|
||||
- Else if n is an exact power of 2, return 32.
|
||||
- Else return an integer k, 32 <= k <= 64, such that n/k is close to, but strictly less than, an exact power of 2.
|
||||
```cpp
|
||||
```cpp
|
||||
int compute_minrun(int n)
|
||||
{
|
||||
int r = 0;
|
||||
@ -69,15 +74,16 @@ Now, let's discuss the entire algorithm.
|
||||
}
|
||||
return n + r;
|
||||
}
|
||||
```
|
||||
Here, 64 is a standard value decided by the inventor such that the **min-run length**, for a list of size greater than 63, will turned out to be in the range 32 to 64 inclusive.
|
||||
```
|
||||
|
||||
The main reason for this is that we are going to use modified insertion sort to sort this small chunk of data and insertion sort performs better on an array of small size.
|
||||
Here, 64 is a standard value decided by the inventor such that the **min-run length**, for a list of size greater than 63, will turned out to be in the range 32 to 64 inclusive.
|
||||
|
||||
**Why do we find runs?**
|
||||
The main reason for this is that we are going to use modified insertion sort to sort this small chunk of data and insertion sort performs better on an array of small size.
|
||||
|
||||
## Why do we find runs?
|
||||
There is no need to sort already sorted data, therefore inorder to take advantage of already sorted data present in the list. We find runs.
|
||||
|
||||
### How to find a "Run"?
|
||||
## How to find a "Run"?
|
||||
|
||||
A **Run** can be increasing or decreasing, so minimum length of a **Run** is $2$, because any sequence of length 2 is either increasing or decreasing.
|
||||
|
||||
@ -123,7 +129,7 @@ If the length of the run is less than the constant **min-run length**, then we u
|
||||
|
||||
As we know the main idea of Insertion sort is to take an element and insert it at the correct position. Now, we are going to use binary search to find correct position, rather than a simple loop.
|
||||
|
||||
After finding the correct index, we shift the data from that index by 1 towards right and insert the element at the correct index.
|
||||
After finding the correct index, we right-shift the data from that index by 1 and insert the element at the correct index.
|
||||
|
||||
```cpp
|
||||
// @param start: Position of next element to be inserted
|
||||
@ -163,7 +169,7 @@ void binaryInsertionsort(vector<int>& data, int start, int low, int high)
|
||||
}
|
||||
```
|
||||
|
||||
Now, we have understood how to find runs. Next we are going to how to merge them?
|
||||
Now, we have understood how to find runs. Next we are going to see how to merge them?
|
||||
|
||||
## Merging
|
||||
|
||||
@ -173,17 +179,19 @@ For example, [2 3 4], [1 2 5], [2 4 5] are three consecutive runs, so if we merg
|
||||
|
||||
Now, to maintain information about runs, we are going to use an array, which is used as a stack, so whenever a new run comes, we insert it at the top. Now, let's discuss some criteria about merging this runs.
|
||||
|
||||
It is a proven that merging lists of similar sizes tends to perform better than the other case. It is called **"balanced merge"**. Therefore, we use some criteria to merge runs such that we end up having merges of similar sizes, later on.
|
||||
Merging lists of similar sizes tends to perform better than the other case, due to relatively simple logic. It is called **"balanced merge"**.
|
||||
|
||||
IMG
|
||||
Now, let's discuss some criteria we use to merge runs efficiently.
|
||||
|
||||
Criterion 1: If the stack-size is greater than equal to 3 and $|Z| <= |Y| + |X|$ is true, then if $|Z|<|X|$, then merge $|Z|$ and $|Y|$ otherwise merge $|X|$ and $|Y|$.
|
||||
|
||||
|
||||
Criterion 1: If the stack-size is greater than or equal to 3 and $|Z| <= |Y| + |X|$ is true, then if $|Z|<|X|$, then merge $|Z|$ and $|Y|$ otherwise merge $|X|$ and $|Y|$.
|
||||
|
||||
Criterion 2: $|Y|<=|X|$ then merge them.
|
||||
|
||||
Whenever we push a new run into the stack, we check for these criteria and we merge runs accordingly until none of these criteria satisfy. And then we wait for a next run.
|
||||
Whenever we push a new run into the stack, we check for these criteria and we merge runs accordingly until none of these criteria satisfy.
|
||||
|
||||
**Note:** <code>mergeAt()</code> function in the below code will be discussed.
|
||||
**Note:** <code>mergeAt()</code> function is used to merge two runs, will be discussed.
|
||||
```cpp
|
||||
// This method is called each time a new run is pushed onto the stack
|
||||
void mergecollapse(vector<int>& data) {
|
||||
@ -203,10 +211,11 @@ void mergecollapse(vector<int>& data) {
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
There are two things to take a note:
|
||||
|
||||
1. In order to have balanced merge, we are delaying the merge, by waiting for a next run.
|
||||
2. If we want to have advantage of cache memory, i.e. fresh runs are already in the cache, so merging them have less memory overhead.
|
||||
1. In order to have balanced merge, we are delaying the merge by waiting for a next run.
|
||||
2. If we want to take advantage of cache memory, that is fresh runs are already in the cache therefore merging them have less memory overhead, then we should merge the fresh runs as soon as possible.
|
||||
|
||||
So, by taking care of both the things, criteria are decided.
|
||||
|
||||
@ -227,59 +236,80 @@ void mergeForceCollapse(vector<int>& data)
|
||||
}
|
||||
```
|
||||
|
||||
Now, let's discuss a new concept Galloping.
|
||||
Now, let's discuss a new concept **Galloping**.
|
||||
|
||||
## Galloping
|
||||
|
||||
Standard merging procedure for merging two sorted arrays [10] and [1,2,3,4,6,8,9,14] goes as below:
|
||||
Standard merging procedure for merging two sorted arrays array_1: [10] and array_2: [1,2,3,4,6,9,14] goes as below:
|
||||
|
||||
IMGs
|
||||
CREATE GIF
|
||||
|
||||
|
||||
As you can see we are consistently taking element from array-2 until we reach $14$. But can we do better? Yes, use galloping.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As you can see we are consistently taking element from array_2 until we reach $14$. But can we do better? Yes, use galloping.
|
||||
|
||||
The idea of galloping is to perform exponential search to find a correct position, rather than comparing elements one by one.
|
||||
|
||||
**For example**, if we find correct position of 10 in array-2 using exponential search, then it will be a huge win in terms of **taken time**.
|
||||
**For example**, if we find correct position of $10$ in array_2 using exponential search, then it will be a huge win in terms of **taken time**.
|
||||
|
||||
**Does Galloping work in every situation?**
|
||||
|
||||
Certainly no, when we are not achieving a sufficient index jump by performing exponential search, then it might be worse then classical merge procedure. But what is sufficient jump?
|
||||
Certainly no, when we are not achieving a sufficient index jump by performing exponential search, then we might end up having more comparisons than classical merge procedure. But what is a sufficient jump?
|
||||
|
||||
It is decided to be a constant 7, it is called **minimum gallop**(MIN_GALLOP).
|
||||
It is decided to be a constant $7$, it is called **minimum gallop**(MIN_GALLOP). Other than that we have a new variable called _current_min_gallop_, which is assigned to constant MIN_GALLOP(7) at the start of the algorithm.
|
||||
|
||||
We use galloping mode during merging two sorted arrays.
|
||||
We use galloping mode during merge. To avoid the drawbacks of galloping mode, we perform actions as below:
|
||||
|
||||
We have a new variable called _min_gallop_, which is assigned to MIN_GALLOP(7) at the start of the algorithm.
|
||||
1. If we find that we are taking elements from one array more than or equal to _current_min_gallop_ times consistently, then we enter into galloping mode.
|
||||
2. After entering into galloping mode, we continue to remain into galloping mode if and only if we find that we have a jump of more than or equal to MIN_GALLOP(7). Otherwise we exit gallop mode. If galloping mode is a success, then we decrease _current_min_gallop_ by $1$ per one success. It promotes galloping mode.
|
||||
**Note:** In galloping mode we are using MIN_GALLOP constant to check for the success of galloping mode and to enter into galloping mode we are using _current_min_gallop_ variable.
|
||||
3. After we exit galloping mode, we increase _current_min_gallop_ by one, to discourage a return to galloping mode again. This is to make sure that next we recover the lost of more comparisons.
|
||||
|
||||
To avoid the drawbacks of galloping mode, we perform actions as below:
|
||||
So, **we are trying to balance the whole situation by taking the advantage of galloping.** Sometimes, the value of _current_min_gallop_ becomes so large that we never enter into galloping mode.
|
||||
|
||||
1. If we find that we are taking elements from one array more than or equal to _min_gallop_ times consistently, then we enter into galloping mode.
|
||||
2. After entering into galloping mode, We continue to remain into galloping mode if and only if we find that we have a jump of more than or equal to MIN_GALLOP(7). Otherwise we exit gallop mode. If galloping mode is a success, then we decrease _min_gallop_ by 1 per one success.
|
||||
**Note:** If we are in galloping mode, then we are using MIN_GALLOP constant to check for the success of galloping mode and to enter into galloping mode we are using _min_gallop_ variable.
|
||||
3. After we exit galloping mode, we increase _min_gallop_ by one, to discourage a return to galloping mode again.
|
||||
|
||||
So, we are trying to balance the whole situation by taking the advantage of galloping.
|
||||
|
||||
Sometimes, the value of _min_gallop_ becomes so large that we never enter into galloping mode.
|
||||
The whole procedure discussed above will be helpful in merge procedure. Now, we come back to pure galloping.
|
||||
|
||||
Note that we can do galloping(exponential search) from any side of the array, either left or right, because we are just intended to find a position and that can be approached by doing exponential search from any side.
|
||||
|
||||
The starting position for the search is called a **hint**. Sometimes it is better to search from left and sometimes from right.
|
||||
The starting position for the search is called a "$hint$". Sometimes it is better to search from left and sometimes from right. For the given array below,
|
||||
|
||||
IMG
|
||||
|
||||
|
||||
Procedure for _galloping_:
|
||||
If we want to find position for $3$, then starting from index 0($hint = 0$) is efficient, but if we are looking for position of $13$ then starting from index $len-1$($hint = len-1$) is more efficient.
|
||||
|
||||
1. If $key < data[base+hint]$, then it indicates that we should do galloping towards left side, because element at the starting position is greater than key.
|
||||
**Procedure for _galloping_:**
|
||||
|
||||
1. If $key < data[base+hint]$, then it indicates that we should do galloping towards left side, because element at the starting position is greater than the key.
|
||||
2. Otherwise, we do galloping towards right side.
|
||||
3. To do galloping. we first find range for the key using the procedure we use in exponential search.
|
||||
4. At last, we do binary search over the range to find the correct position.
|
||||
|
||||
We have two types of galloping function _gallopRight_ and _gallopLeft_, the main difference between them is, _gallopRight_ and _gallopLeft_ returns rightmost index and leftmost index respectively in case if there are equal elements.
|
||||
We have two types of galloping function `gallopRight` and `gallopLeft`, the main difference between them is, `gallopRight` and `gallopLeft` returns rightmost index and leftmost index respectively in case if there are equal elements.
|
||||
|
||||
IMG
|
||||
For example, for the array given below,
|
||||
|
||||
Now, to maintain stability while merging in galloping mode, If we are doing galloping for run2's element, then we are going to use _gallopRight_, otherwise we will use _gallopLeft_. In simple mode it is trivial to maintain stability.
|
||||
|
||||
|
||||
If you find a position of 13 by using galloping, then `gallopLeft` will return 6, but `gallopRight` will return 9.
|
||||
|
||||
In simple mode it is trivial to maintain stability, but to maintain stability while merging in galloping mode, we sometimes use `gallopRight` and sometimes `gallopLeft`. Just to get the basic idea, see the below example, it will be more clear when we will the merge procedure.
|
||||
|
||||
|
||||
|
||||
Now, if we are finding a position of run2[0] in run1, then we will use `gallopRight`, but if we are finding a position for run1[3] in run2, then we will use `gallopLeft`.
|
||||
|
||||
We have a slightly modified binary search methods for both of them.
|
||||
|
||||
@ -363,7 +393,7 @@ int gallopLeft(vector<int>& data, int key, int base, int len, int hint)
|
||||
int maxofs = len - hint;
|
||||
|
||||
// Gallop towards right side
|
||||
while (ofs < maxofs && key >= data[base + hint + ofs]) {
|
||||
while (ofs < maxofs && key > data[base + hint + ofs]) {
|
||||
lastofs = ofs;
|
||||
ofs = (ofs << 1) + 1;
|
||||
}
|
||||
@ -387,30 +417,50 @@ int gallopLeft(vector<int>& data, int key, int base, int len, int hint)
|
||||
|
||||
```
|
||||
|
||||
Now let's discuss _mergeAt_ procedure, which is used to merge two runs-at the top of the stack.
|
||||
Now let's discuss `mergeAt` procedure, which is used to merge two runs.
|
||||
|
||||
Let $base_i$ and $len_i$ are base address and length of $run_i$, respectively.
|
||||
|
||||
We perform two operation before merging two runs:
|
||||
|
||||
|
||||
- Find index of the first element of $run_2$ into run1. If the index turns out to be the last, then no merging is required. Otherwise just increment the base address for run_1, because the elements before this index are already in place.
|
||||
- Similarly, find index of the last element of run1 in run2. If the index turns out to be the first, then no merging is required. Otherwise set len2 to this index, because the elements after this index are already in place.
|
||||
We perform two operations before merging two runs:
|
||||
|
||||
This steps may lead to a very efficient merging.
|
||||
1. Find index of the first element of $run_2$ into $run_1$. If the index turns out to be the last, then no merging is required.
|
||||
|
||||
|
||||
|
||||
Otherwise just increment the base address for $run_1$, because the elements before this index are already in place.
|
||||
|
||||
|
||||
|
||||
2. Similarly, find index of the last element of $run_1$ in $run_2$. If the index turns out to be the first, then no merging is required.
|
||||
|
||||
|
||||
|
||||
Otherwise set $len_2$ to this index, because the elements after this index are already in place.
|
||||
|
||||
|
||||
|
||||
After performing this operation you notice that all elements of $run_2$ are less than last element of $run_1$ and first element fo $run_1$ is greater than first element of $run_2$, i.e. $run_1[base_1] > run_2[base_2]$. These implies two things:
|
||||
|
||||
Conclusion 1. The last element of $run_1$ is the largest element.
|
||||
Conclusion 2. The first element of $run_2$ is the smallest element.
|
||||
|
||||
After performing this operation you notice that all elements of run2 are less than last element fo run1 and first element fo run1 is greater than first element of run2, i.e. $run1[base1] > run2[base2]$.
|
||||
We will see how useful these conclusions are! Just keep it in mind.
|
||||
|
||||
Let say we are merging two sorted arrays of size _len1_ and _len2_. In traditional merge procedure, we create a new array of size len1+len2. But in Tim sort's merge procedure, we just create a new temporary array of size $min(len1,len2)$ and we copy the smaller array into this temporary array.
|
||||
|
||||
|
||||
Now, Let say we are merging two sorted arrays of size _$len_1$_ and _$len_2$_. In traditional merge procedure, we create a new array of size $len_1$+$len_2$. But in Tim sort's merge procedure, we just create a new temporary array of size $min(len1,len2)$ and we copy the smaller array into this temporary array.
|
||||
|
||||
The main intention behind it is to decrease **merge space overhead**, because it reduces the number of required element movements.
|
||||
|
||||
IMG
|
||||
|
||||
|
||||
Notice that we can do merging in both directions: **left-to-right**, as in the traditional mergesort, or **right-to-left**.
|
||||
|
||||
Now, suppose the len1 is less than len2, then we will create a temporary copy of run1. To merge them, we are not going to allocate any more memory, but we will merge them directly into the main array, in **left-to-right** direction. In the other case(len2 < len1), we will merge them in **right-to-left** direction.
|
||||
Now, suppose the $len_1$ is less than $len_2$, then we will create a temporary copy of $run_1$. To merge them, we are not going to allocate any more memory, but we will merge them directly into the main array, in **left-to-right** direction. In the other case($len_2$ < $len_1$), we will merge them in **right-to-left** direction.
|
||||
|
||||
**The reason** for different directions is that, by doing this we are able to do merging in the main array itself.
|
||||
**The reason for different directions is that, by doing this we are able to do merging in the main list itself.** You will be able to see this when we will see `merge_LtoR` and `merge_RtoL`.
|
||||
|
||||
```cpp
|
||||
|
||||
@ -454,32 +504,29 @@ void mergeAt(vector<int>& data, int i)
|
||||
}
|
||||
```
|
||||
|
||||
Now, let's discuss _merge_LtoR_ and _merge_RtoL_.
|
||||
Now, let's discuss `merge_LtoR` and `merge_RtoL`.
|
||||
|
||||
Note that the first element of run1 is greater than first element of run2 and last element of run1 is greater than all elements of run2, implies two things:
|
||||
|
||||
Conclusion 1. The last element of run1 is the largest element.
|
||||
Conclusion 2. The first element of run2 is the smallest element.
|
||||
|
||||
The generic procedure for both of this functions is as below:
|
||||
The generic procedure for both of this functions is as below, most of the things we have already discussed in galloping part.
|
||||
|
||||
1. First copy the smaller run into temporary array.
|
||||
2. According to above two conclusions start by
|
||||
3. Start by merging them in a classical way, until one run seems to be contributing elements consistently. Let's call it a win, If one run wins for more than minGallop times consistently, then we enter into galloping mode.
|
||||
4. We stay into galloping mode as far as it is performing good.
|
||||
5. If we exit galloping mode, then we panalize by incrementing minGallop. Then start from step-2 again.
|
||||
2. Start by merging them in a classical way, until one run seems to be contributing elements consistently, let's call it a win. If one run wins for more than _cur_minGallop_ times consistently, then we enter into galloping mode.
|
||||
3. We stay into galloping mode as far as it is performing good.
|
||||
4. If we exit galloping mode, then we penalize by incrementing _cur_minGallop_. Then start from step-2 again.
|
||||
|
||||
There are two degenerate cases we check again and again in the run of the function, which shows that merge procedure is at its end. They are based on the two conclusions discussed above.
|
||||
**There are two degenerate cases, we check again and again while executing the function, which shows that merge procedure is at its end.** They are based on the two conclusions we have alredy discussed:
|
||||
|
||||
Conclusion 1. The last element of $run_1$ is the largest element.
|
||||
Conclusion 2. The first element of $run_2$ is the smallest element.
|
||||
|
||||
For _merge_LtoR_,
|
||||
1. len2 == 0 $\implies$ Only run1 elements are left.
|
||||
2. len1 == 1 $\implies$ According to conc. 1, all remaining elements of run2 are smaller than the remaining element in run1.
|
||||
1. $len_2$ == 0 $\implies$ Only $run_1$ elements are left.
|
||||
2. $len_1$ == 1 $\implies$ According to conclusion 1, all remaining elements of $run_2$ are smaller than the remaining element in $run_1$.
|
||||
|
||||
For _merge_RtoL_,
|
||||
1. len1 == 0 $\implies$ Only run2 elements are left.
|
||||
2. len2 == 1 $\implies$ According to conc. 2, all remaining elements of run1 are larger than the remaining element in run2.
|
||||
1. $len_1$ == 0 $\implies$ Only $run_2$ elements are left.
|
||||
2. $len_2$ == 1 $\implies$ According to conclusion 2, all remaining elements of $run_1$ are larger than the remaining element in $run_2$.
|
||||
|
||||
Finally, implementation:
|
||||
Finally, implementation.
|
||||
```cpp
|
||||
// If len1 <= len2 the mergeLo is called
|
||||
// First element of run1 must be greater than first element of run2
|
||||
@ -509,12 +556,11 @@ void merge_LtoR(vector<int>& data, int base1, int len1, int base2, int len2)
|
||||
return;
|
||||
}
|
||||
|
||||
// Used to end merge in degenerate case
|
||||
bool done = false;
|
||||
|
||||
// cur_minGallop is a global variable which is
|
||||
// used to keep track of minimum Gallop required
|
||||
// to enter into galloping mode for this merge call
|
||||
// It will updated after this merge procedure
|
||||
// cur_minGallop is a global variable
|
||||
// Copy it to a local variable for performance
|
||||
int minGallop = cur_minGallop;
|
||||
while (true) {
|
||||
int count1 = 0; // Number of times in a row that first run won
|
||||
@ -643,7 +689,11 @@ void merge_RtoL(vector<int>& data, int base1, int len1, int base2, int len2)
|
||||
return;
|
||||
}
|
||||
|
||||
// Used to end merge in degenerate case
|
||||
bool done = false;
|
||||
|
||||
// cur_minGallop is a global variable
|
||||
// Copy it to a local variable for performance
|
||||
int minGallop = cur_minGallop;
|
||||
while (true) {
|
||||
int count1 = 0; // Number of times in a row that first run won
|
||||
@ -747,7 +797,7 @@ void merge_RtoL(vector<int>& data, int base1, int len1, int base2, int len2)
|
||||
}
|
||||
```
|
||||
|
||||
Tim sort function is as below.
|
||||
Tim sort function is as below. It is trivial to understand, if you have understood the whole thing above.
|
||||
|
||||
```cpp
|
||||
void Timsort(vector<int>& data)
|
||||
@ -839,5 +889,3 @@ In the **worst case**, Timsort takes $\mathcal{O}(NlogN)$.
|
||||
Average complexity is $\mathcal{O}(NlogN)$.
|
||||
|
||||
Space complexity is $\mathcal{O}(N)$.
|
||||
|
||||
**Note:** Comparison is an expensive operation. Here, expensive is in terms of the computer resources used by the operation.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user