mirror of
https://github.com/dholerobin/Lecture_Notes.git
synced 2025-03-15 21:59:56 +00:00
Update KMP.md
This commit is contained in:
Aakash Panchal
GitHub
parent
3092769295
commit
da4058ef5b
@ -1,25 +1,30 @@
|
||||
Prefix function and KMP
|
||||
Prefix function and KMP-algorithm
|
||||
|
||||
KMP-algorithm is a widely used **string-matching algorithm**, which is used to find a place where a string is found within a larger string. It uses the value of **prefix function** for a given string.
|
||||
|
||||
Let's first see what is a **prefix function**.
|
||||
|
||||
# Prefix function
|
||||
|
||||
Prefix function for a given string($S$) of length $n$ returns an array $\Pi$ of length $n$, such that $i^{th}$ element of an array contains length of longest proper prefix of $S[0...i]$ which is also suffix of $S[0...i]$.
|
||||
Prefix function for a given string($s$) of length $n$ returns an array $\Pi$ of length $n$, such that $i^{th}$ element of an array contains length of longest proper prefix of $S[0,i]$ which is also suffix of $S[0,i]$.
|
||||
|
||||
Where, $S[l...r]$ represents substring of $S$ starting at index $l$ and ending at index $r$.
|
||||
Where, $S[l,r]$ represents substring of $S$ starting at index $l$ and ending at index $r$.
|
||||
|
||||
**Note:** A proper prefix of a string is a prefix that is not equal to the string itself.
|
||||
|
||||
**For example**, $\text{prefix function("aaab")}$ returns $\Pi = [0,1,2,0]$. Can you figure out why it has value $2$ at index $2$(0-based)?
|
||||
|
||||
Because $S[0...1]$ is equal to $S[1...2]$.
|
||||
Because $S[0,1]$ is equal to $S[1,2]$.
|
||||
|
||||
Note that **$\Pi[0]$ is always zero.**
|
||||
|
||||
Mathematically, it can be written as,
|
||||
|
||||
$\Pi[i] = \max_ {k = 0 \dots i} \{k : s[0 \dots k-1] = s[i-(k-1) \dots i] \}$
|
||||
$\Pi[i] = \max_ {k = 0 \dots i} \{k : s[0, k-1] = s[i-(k-1), i] \}$
|
||||
|
||||
## Trivial Algorithm
|
||||
|
||||
Basic algorithm to find such a value say at index $i$ is to compare all prefixes and suffixes of substring $S[0...i]$ one by one, of course, of same lengths.
|
||||
Basic algorithm to find such a value say at index $i$ is to compare all prefixes and suffixes of substring $S[0, i]$ one by one, of course, of the same lengths.
|
||||
|
||||
```
|
||||
for(len = 1; len <= i; len++)
|
||||
@ -51,13 +56,13 @@ Can we do better?
|
||||
|
||||
## Efficient Algorithm
|
||||
|
||||
We will take advantage of previously computed values as much as possible. We are going to reach at the most efficient version($O(n)$) step by step.
|
||||
We will take advantage of previously computed values as much as possible. We are going to reach the most efficient version($O(n)$) step by step.
|
||||
|
||||
### Step 1
|
||||
|
||||
The value of prefix function at any index $i$ can either increase by at most 1 or decrease by certain amount. That means, $\Pi[i] <= \Pi[i-1] + 1$. Why?
|
||||
|
||||
Suppose, for a given string $"s_0s_1s_2s_3s_4s_5s_6s_7s_8"$, we are told that $\Pi[7]=3$, means $"s_0s_1s_2"$ and $"s_5s_6s_7"$ are equal substrings.
|
||||
Suppose, for a given string "$s_0s_1s_2s_3s_4s_5s_6s_7s_8$", we are told that $\Pi[7]=3$, means "$s_0s_1s_2$" and "$s_5s_6s_7$" are equal substrings.
|
||||
|
||||
|
||||
|
||||
@ -69,20 +74,22 @@ $\Pi[8]$ will be 4. We can see that $\Pi[8] = \Pi[7]+1$. What if $s_3$ and $s_8$
|
||||
|
||||
We can not say anything about value of $\Pi[8]$, because depending on the value of $s_8$, there are many possibilities like,
|
||||
|
||||
1. $s_0s_1s_2 = s_6s_7s_8$ OR
|
||||
2. $s_0s_1 = s_7s_8$ OR
|
||||
3. $s_0 = s_8$ OR
|
||||
1. "$s_0s_1s_2$" = "$s_6s_7s_8$" OR
|
||||
2. "$s_0s_1$" = "$s_7s_8$" OR
|
||||
3. "$s_0$" = "$s_8$" OR
|
||||
4. $\Pi[8]=0$
|
||||
|
||||
But, now it will certainly be less than or equal to $\Pi[7]$.
|
||||
**But, now it will certainly be less than or equal to $\Pi[7]$.**
|
||||
|
||||
Therefore, $\Pi[i] <= \Pi[i-1] + 1$ is always true.
|
||||
|
||||
A formal proof for at most one increase can be given as below,
|
||||
Formal proof for at most one increase can be given as below,
|
||||
|
||||
If $\Pi[i+1]>\Pi[i]+1$, then we can take this suffix ending in position $i+1$ with the length $\Pi[i+1]$ and remove the last character from it. We end up with a suffix ending in position $i$ with the length $\Pi[i+1]−1$, which is better than $\Pi[i]$, that means we get a contradiction.
|
||||
1. If $\Pi[i+1]>\Pi[i]+1$, then we can take the suffix ending in position $i$ with the length $\Pi[i]$ and remove the last character from it.
|
||||
2. We will end up with a suffix ending in position $i-1$ with the length $\Pi[i]−1$, which is better than $\Pi[i]$, that means we get a contradiction.
|
||||
3. Because $\Pi[i]$ represents the length of **the longest** proper prefix which is also a suffix.
|
||||
|
||||
By finding the value of prefix function using this truth, we end up having only $O(N)$ string comparisons because the value of prefix function(in total) increases by at most $N$ steps and also decreases by at most $N$ steps(see in reverse mode). Therefore, total complexity now is $O(N*N)$-as string comparison takes $O(N)$.
|
||||
By finding the value of prefix function using the truth discussed above, we will end up having only $O(N)$ string comparisons because of the value of prefix function(in total) increases by at most $N$ steps and also decreases by at most $N$ steps(see in reverse mode). Therefore, total complexity now is $O(N*N)$-as one string comparison takes $O(N)$.
|
||||
|
||||
```cpp
|
||||
vector<int> prefix_function(string& s) {
|
||||
@ -104,7 +111,7 @@ vector<int> prefix_function(string& s) {
|
||||
|
||||
In step-1, we have seen that the value of prefix function overall increases by at most n steps, but due to string comparisons we ended up having $O(N^2)$ complexity. Here we will eliminate string comparisons completely.
|
||||
|
||||
Now, suppose that we are finding value of $\Pi[i]$ for string $s$, then from the value of $\Pi[i-1]$ we know that prefix of $s[0....i-1]$ of length $\Pi[i-1]$ is equal to the suffix of substring $s[0....i-1]$ of length $\Pi[i-1]$ i.e. $s[0...(\Pi[i-1]-1)] = s[(i-\Pi[i-1])...i-1]$
|
||||
Now, suppose that we are finding value of $\Pi[i]$ for string $s$, then from the value of $\Pi[i-1]$ we know that prefix of $s[0,i-1]$ of length $\Pi[i-1]$ is equal to the suffix of substring $s[0,i-1]$ of length $\Pi[i-1]$ i.e. $s[0,(\Pi[i-1]-1)] = s[(i-\Pi[i-1]),i-1]$
|
||||
|
||||
|
||||
|
||||
@ -112,13 +119,13 @@ As discussed in step-1, if $s[\Pi[i-1]]$ is equal to $s[i]$, then $\Pi[i] = \Pi[
|
||||
|
||||
|
||||
|
||||
If $s[\Pi[i-1]] != s[i]$, then we are in search for a next **longest prefix**(say has lenght $k$) which supports $s[0...(k-1)]=s[(i-k)...(i-1)]$, such that $s[k]=s[i]$ may turn out to be true.
|
||||
If $s[\Pi[i-1]] != s[i]$, then we are in search for a next **longest prefix**(say has lenght $k$) which supports $s[0,k-1]=s[i-k,i-1]$, such that $s[k]=s[i]$ may turn out to be true.
|
||||
|
||||
|
||||
|
||||
Note that now anyway, value of $\Pi[i]$ will be less than or equal to $\Pi[i-1]$, as discussed in step-1.
|
||||
Note that now anyway, the value of $\Pi[i]$ will be less than or equal to $\Pi[i-1]$, as discussed in step-1.
|
||||
|
||||
If $s[k] = s[i]$, then we can say that $\Pi[i]=k+1$ because $s[0...k]$ is the **longest prefix** which matches with a suffix and that is the definition of $\Pi[i]$.
|
||||
If $s[k] = s[i]$, then we can say that $\Pi[i]=k+1$ because $s[0,k]$ is the **longest prefix** which matches with a suffix and that is the definition of $\Pi[i]$.
|
||||
|
||||
|
||||
|
||||
@ -126,13 +133,13 @@ The value-$k$ we are searching is exactly what $\Pi[\Pi[i-1]-1]$ represents. Why
|
||||
|
||||
Points to note:
|
||||
|
||||
1. We know that $s[0...(\Pi[i-1]-1)] = s[(i-\Pi[i-1])...i-1]$.
|
||||
1. We know that $s[0,(\Pi[i-1]-1)] = s[(i-\Pi[i-1]),i-1]$.
|
||||
2. If $s[\Pi[i-1]] != s[i]$, then value of $\Pi[i] <= \Pi[i-1]$ - from step-1.
|
||||
3. We are searching for $k$, whose value must be less than $\Pi[i-1]$ because in case $s[k] = s[i]$(i.e. $\Pi[i]=k+1$), then
|
||||
|
||||
$\Pi[i] <= \Pi[i-1] \implies k+1$ $<=$ $\Pi[i-1] \implies k < \Pi[i-1]$
|
||||
|
||||
Therefore, basically we are searching for the **longest proper prefix** of length less than $\Pi[i-1]$ such that it matches a suffix of $s[0...(\Pi[i-1]-1)]$(which equals to $s[(i-\Pi[i-1])...i-1]$ **from point-1 above**), which is the definion of the prefix function at index $\Pi[i-1] -1$ i.e. $\Pi[\Pi[i-1]-1]$.
|
||||
Therefore, basically we are searching for the **longest proper prefix** of length less than $\Pi[i-1]$ such that it matches a suffix of $s[0,(\Pi[i-1]-1)]$(which equals to $s[(i-\Pi[i-1]),i-1]$ **from point-1 above**), which is the definion of the prefix function at index $\Pi[i-1] -1$ i.e. $\Pi[\Pi[i-1]-1]$.
|
||||
|
||||
|
||||
|
||||
@ -156,9 +163,9 @@ vector<int> prefix_function(string& s) {
|
||||
}
|
||||
```
|
||||
|
||||
**Time complexity:** $O(N)$, where $N$ is the length of the string $s$. Because, now we are not doing any string comparisons.
|
||||
**Time complexity:** $O(N)$, where $N$ is the length of the string $s$. Because now we are not doing any string comparisons.
|
||||
|
||||
## Knuth-Morris-Pratt Algorithm(KMP-algo)
|
||||
## Knuth-Morris-Pratt Algorithm(KMP-algorithm)
|
||||
|
||||
KMP algorithm is used to search all occurrences of pattern-string $p$ in a string $s$ in $O(N)$.
|
||||
|
||||
@ -168,13 +175,13 @@ For example, `p = "ab"` and `s = "abbbabab"`, then KMP will find us `[0,4,6]` be
|
||||
|
||||
Basic idea here is to create a new string having $p$ as a prefix and $s$ as a suffix i.e. `new_str = p + '#' + s`.
|
||||
|
||||
**To make sure that the value of prefix function does not exceed length of $p$, we will add an additional character which is never going to appear in string $s$.**
|
||||
**To make sure that the value of prefix-function does not exceed the length of $p$, we add a character that is never going to appear in string $s$ like `'#'`**.
|
||||
|
||||
Now, we will find prefix function of `new_str`.
|
||||
|
||||
Let say $m$ is the length of $p$.
|
||||
|
||||
$\Pi[i] = m$, means that `new_str[0..m-1]` is equal to `new-str[i-m...i]`, which is bacially means $p$(=`new_str[0...m-1]`) is equal to `new_str[i-m...i]`.
|
||||
$\Pi[i] = m$, means that `new_str[0,m-1]` is equal to `new-str[i-m,i]`, which is bacially means $p$(=`new_str[0,m-1]`) is equal to `new_str[i-m,i]`.
|
||||
|
||||
And therefore **all indices-$i$ where the values of prefix function $\Pi[i]$ equals to the length of $p$ means it is an occurrence of $p$ in $s$.**
|
||||
|
||||
@ -209,13 +216,13 @@ int main()
|
||||
|
||||
## Find a period of a string
|
||||
|
||||
Period of string is the shortest length such that a larger string $s$ can be represented as a concatenation of one or more copies of a substring($t$).
|
||||
Period of a string is the shortest length such that a larger string $s$ can be represented as a concatenation of one or more copies of a substring($t$).
|
||||
|
||||
For example, `s = "ababab"` has a period of $2$, where `t = "ab"`.
|
||||
|
||||
Let's see how to find period of $s$ using value of prefix function of $s$.
|
||||
Let's see how to find a period of $s$ using the value of prefix function of $s$.
|
||||
|
||||
**First of all note that length of string $s$($n$) is divisible by period of string.**
|
||||
**First of all note that length of string $s$($n$) is divisible by a period of string.**
|
||||
|
||||
As we know for a string of length $n$, $\Pi[n-1]$ represents the length of longest proper prefix which is also suffix. Now, let $k = n - \Pi[n-1]$.
|
||||
|
||||
@ -225,13 +232,13 @@ $k=n-\Pi[n-1] \implies \Pi[n-1] = n-k$
|
||||
|
||||
That means, prefix of length $n-k$ and suffix of length $n-k$ are equal, as per the definition of $\Pi[n-1]$.
|
||||
|
||||
If you compare all blocks from the start and the end, then it turns out that all blocks of k size are equal. Which means that $k$ is the period of $s$, as same blocks of size $k$ repeats in $s$.
|
||||
If you compare all blocks from the start and the end, then it turns out that all blocks of k size are equal. This means that $k$ is the period of $s$, as the same blocks of size $k$ repeats in $s$.
|
||||
|
||||
Now, why $k$ is the smallest such period?
|
||||
|
||||
Because otherwise the value of $\Pi[i-1]$ will be greater than $n-k$.
|
||||
Because otherwise, the value of $\Pi[i-1]$ will be greater than $n-k$.
|
||||
|
||||
If $k$ does not divide $n$, then string is not periodic as we cannot divide string into equivalent blocks.
|
||||
If $k$ does not divide $n$, then the string is not periodic as we cannot divide the string into equivalent blocks.
|
||||
|
||||
```cpp
|
||||
int main()
|
||||
@ -261,7 +268,7 @@ int main()
|
||||
|
||||
Now, we know how to find a period of a string and therefore we can compress string as only one block of size $k$ which repeats all over again and again in $s$.
|
||||
|
||||
To retrive the string back from compressed version, we can attatch its real length i.e. length of $s$.
|
||||
To retrieve the string back from the compressed version, we can attach its real length i.e. length of $s$.
|
||||
|
||||
```cpp
|
||||
int main()
|
||||
@ -290,19 +297,19 @@ int main()
|
||||
|
||||
## Number of unique substrings in a string
|
||||
|
||||
**Problem statement:** Find number of unique substrings in a given string $s$.
|
||||
**Problem statement:** Find the number of unique substrings in a given string $s$.
|
||||
|
||||
**Brief idea:** Basic idea here is to take an empty string $t$ and add characters one by one from string $s$ and along with that check how many new substrings are created, due to addition of a character in $t$, using prefix-function.
|
||||
**Brief idea:** Basic idea here is to take an empty string $t$ and add characters one by one from string $s$ and along with that check how many new substrings are created, due to the addition of a character in $t$, using prefix-function.
|
||||
|
||||
Let say we have already added some characters to $t$ from $s$ and $k$ is the number of distinct substrings currently. Now, we are a adding character $c$ to $t$, $t = t+c$.
|
||||
|
||||
Note that total number of new substrings created by appending a character to any string($t$) is equal to the length of new string($t=t+c$) created. **For example, Appending `'d'` in `"abc"` creates 4 new substrings: `"d"`, `"cd"`, `"bcd"`, `"abcd"`.**
|
||||
|
||||
But how to find number of new unique substrings created by addition of $c$ **using prefix function**?
|
||||
But how to find the number of new unique substrings created by the addition of $c$ **using prefix function**?
|
||||
|
||||
**Hint:** Reverse $t$.
|
||||
|
||||
By reversing $t$, our task burn down into computing how many prefixes there are that don't appear anywhere else in $t$, which can be done by finding prefix function of $t$.
|
||||
By reversing $t$, our task burns down into computing how many prefixes there are that don't appear anywhere else in $t$, which can be done by finding the prefix function of $t$.
|
||||
|
||||
After finding value of prefix function, we will find maximum value $\Pi_{max}$($\Pi_{max} = max\{\Pi[i]\}, \forall i$) in the prefix function of reversed $t$, which shows the length of longest prefix which is already in $t$ as a substring and it also implies that all smaller prefixes are already present as substrings in $t$.
|
||||
|
||||
@ -310,7 +317,7 @@ Therefore, we will deduct this number of already present substrings i.e. $\Pi_{m
|
||||
|
||||
Where $|t|$ is the length of $t$.
|
||||
|
||||
Finally, number of new unique substrings created by addition of a character turns out to be $|t|-\Pi_{max}$.
|
||||
Finally, the number of new unique substrings created by the addition of a character turns out to be $|t|-\Pi_{max}$.
|
||||
|
||||
**Note that $|t|$ is the length of $t$ after adding a character.**
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user