You are on page 1of 40

# String Algorithms

Jaehyun Park
CS 97SI
Stanford University

Outline

## String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

within T
Notations:

## n and m: lengths of P and T

: set of alphabets (of constant size)
Pi : ith letter of P (1-indexed)
a, b, c: single letters in
x, y, z: strings

## String Matching Problem

Example

T = AGCATGCTGCAGTCATGCTTAGGCTA

P = GCT

## A naive method takes O(mn) time

Initiate string comparison at every starting point
Each comparison takes O(m) time

Outline

## String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Hash Table

Hash Function

## A function that takes a string and outputs a number

A good hash function has few collisions
i.e., If x 6= y, H(x) 6= H(y) with high probability

## An easy and powerful hash function is a polynomial mod some

prime p
Consider each letter as a number (ASCII value is fine)
H(x1 . . . xk ) = x1 ak1 + x2 ak2 + + xk1 a + xk (mod p)
How do we find H(x2 . . . xk+1 ) from H(x1 . . . xk )?

Hash Table

Hash Table

## Main idea: preprocess T to speedup queries

Hash every substring of length k
k is a small constant

## For each query P , hash the first k letters of P to retrieve all

the occurrences of it within T

## Dont forget to check collisions!

Hash Table

Hash Table

Pros:
Easy to implement
Significant speedup in practice

Cons:
Doesnt help the asymptotic efficiency

difficult

Hash Table

Outline

## String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

## A linear time (!) algorithm that solves the string matching

problem by preprocessing P in (m) time
Main idea is to skip some comparisons by using the previous
comparison result

## Uses an auxiliary array that is defined as the following:

[i] is the largest integer smaller than i such that P1 . . . P[i] is
a suffix of P1 . . . Pi

10

## [i] is the largest integer smaller than i such that P1 . . . P[i]

is a suffix of P1 . . . Pi
e.g.,  = 4 since abab is a suffix of ababab
e.g.,  = 0 since no prefix of length 8 ends with c

11

## T = ABC ABCDAB ABCDABCDABDE

P = ABCDABD

= (0, 0, 0, 0, 1, 2, 0)

12

## We matched k = 3 letters so far, and [k] = 0

Thus, there is no point in starting the comparison at T2 , T3
(crucial observation)

## Shift P by k [k] = 3 letters

Mismatch at T4 again!

13

## Shift P by k [k] = 1 letter (we define  = 1)

Mismatch at T11 !

14

15

16

17

Mismatch at T18

18

## Currently all 7 letters are matched

After recording this match (at T16 . . . T22 , we shift P again in
order to find other matches

19

Computing

## Observation 1: if P1 . . . P[i] is a suffix of P1 . . . Pi , then

P1 . . . P[i]1 is a suffix of P1 . . . Pi1

## Observation 2: all the prefixes of P that are a suffix of

P1 . . . Pi can be obtained by recursively applying to i

Well, obviously...

## e.g., P1 . . . P[i] , P1 . . . , P[[i]] , P1 . . . , P[[[i]]] are all

suffixes of P1 . . . Pi

## Knuth-Morris-Pratt (KMP) Algorithm

20

Computing

A non-obvious conclusion:
First, lets write (k) [i] as [] applied k times to i
e.g., (2) [i] = [[i]]
[i] is equal to (k) [i 1] + 1, where k is the smallest integer
that satisfies P(k) [i1]+1 = Pi

## Intuition: we look at all the prefixes of P that are suffixes of

P1 . . . Pi1 , and find the longest one whose next letter
matches Pi

## Knuth-Morris-Pratt (KMP) Algorithm

21

Implementation

pi = -1;
int k = -1;
for(int i = 1; i <= m; i++) {
while(k >= 0 && P[k+1] != P[i])
k = pi[k];
pi[i] = ++k;
}

22

## Pattern Matching Implementation

int k = 0;
for(int i = 1; i <= n; i++) {
while(k >= 0 && P[k+1] != T[i])
k = pi[k];
k++;
if(k == m) {
// P matches T[i-m+1..i]
k = pi[k];
}
}

23

Outline

## String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Suffix Trie

24

Suffix Trie

## Suffix trie of a string T is a rooted tree that stores all the

suffixes (thus all the substrings)

to x

Suffix Trie

25

Example

## (Figure modified from Ukkonens original paper)

Suffix Trie

26

Incremental Construction

## Given the suffix tree for T1 . . . Tn

Then we append Tn+1 = a to T , creating necessary nodes

## Start at node u corresponding to T1 . . . Tn

Create an a-transition to a new node v

## Take the suffix link at u to go to u , corresponding to

T2 . . . Tn
Create an a-transition to a new node v
Create a suffix link from v to v

Suffix Trie

27

Incremental Construction

## Repeat the previous process:

Take the suffix link at the current node
Make a new a-transition there
Create the suffix link from the previous node

## Stop if the node already has an a-transition

Because from this point, all nodes that are reachable via suffix

Suffix Trie

28

Construction Example

## Given the suffix trie for aba

We want to add a new letter c

Suffix Trie

29

Construction Example

Suffix Trie

30

Construction Example

Suffix Trie

31

Construction Example

Suffix Trie

32

Construction Example

Suffix Trie

33

Construction Example

Suffix Trie

34

Construction Example

## Construction time is linear in the tree size

But the tree size can be quadratic in n
e.g., T = aa . . . abb . . . b

Suffix Trie

35

Construction Example

## To find P , start at the root and keep following edges labeled

with P1 , P2 , etc.

Suffix Trie

36

Outline

## String Matching Problem

Hash Table
Knuth-Morris-Pratt (KMP) Algorithm
Suffix Trie
Suffix Array

Suffix Array

37

Suffix Array

Suffix Array

38

Suffix Array

## Has the same computational power as suffix trie

Can be constructed in O(n) time (!)

## But its hard to implement

If you want to see how it works, read the paper on the course
website
http://cs97si.stanford.edu/suffix-array.pdf

Suffix Array

39

## If a problem involves rotations of some string, consider

concatenating it with itself and see if it helps

## Stanford team notebook has implementations of suffix arrays

and the KMP matcher

Suffix Array

40
{"ft_lang_label":"__label__en","ft_lang_prob":0.7863256,"math_prob":0.9668856,"size":1219,"snap":"2019-51-2020-05","text_gpt3_token_len":437,"char_repetition_ratio":0.07901235,"word_repetition_ratio":0.11818182,"special_character_ratio":0.3847416,"punctuation_ratio":0.15498155,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.98324096,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2019-12-11T22:36:07Z\",\"WARC-Record-ID\":\"<urn:uuid:9bfb0bf4-28f2-4772-8817-61be9f1f1627>\",\"Content-Length\":\"262650\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:5edbf080-93dc-4822-9c19-f0cc068aab04>\",\"WARC-Concurrent-To\":\"<urn:uuid:d4532e83-9a2b-4694-b84e-b0ed4ac7f60f>\",\"WARC-IP-Address\":\"151.101.250.152\",\"WARC-Target-URI\":\"https://www.scribd.com/document/302599387/StringAlgorithms-pdf\",\"WARC-Payload-Digest\":\"sha1:F45ACLDLUI36YFLKTOXBRXPYC5CQXLDQ\",\"WARC-Block-Digest\":\"sha1:G24I6TST4U4MT4KCT25HDSGZIFGIPOCJ\",\"WARC-Identified-Payload-Type\":\"application/xhtml+xml\",\"warc_filename\":\"/cc_download/warc_2019/CC-MAIN-2019-51/CC-MAIN-2019-51_segments_1575540533401.22_warc_CC-MAIN-20191211212657-20191212000657-00239.warc.gz\"}"}