Knuth-Morris-Pratt(KMP)

String Matching Problem:

             

            Suppose a Text is an array T[1..n] of length n and pattern is also an array P[1..m] (m <= n). We say pattern P occurs with shift s in Text T if 0 <= s <= n-m and T[s+1..s+m] = P[1..m] i.e. T[s + j] = P[j] for all 1 <= j <=m.

             

                 If P occurs with shift s in T the s is a valid shift otherwise it is an invalid shift.The string matching problem is the problem of finding all valid shifts with which a given Pattern P occurs in a given Text T.

 
                                   

Notation and Terminology:

Prefix : W is a prefix of string x if x = wy for some string y.

Suffix : W is suffix of string x if x = yw for some string y.

xy – string concatenation   | x | - length of string x

Knuth-Morris-Pratt Algorithm:

Knuth, Morris and Pratt proposed a linear time algorithm for the string matching problem.

A matching time of O(n) is achieved by avoiding comparisons with elements of T that have previously been involved in comparison with some elements of the pattern P to be matched. i.e., backtracking on the string T never occurs.

Components of KMP Algorithm:

Prefix Function Π: Prefix function Π for a pattern encapsulate knowledge about how the pattern matches against shifts of itself.

         

Knowing these q text characters allows us to determine immediately that certain shifts are invalid since the first character a would be aligned with a text character that is known to match with second pattern character b. So next shift should be s = s + 2.

           

 Given P[1..q] matches T[s+1..s+q] what is the least shift s1 > s such that

 P[1..k] = T[s1+1 . . s1+k] where s1 + k = s + q

Prefix function Π [q] = max ( k : k < q and P_k is suffix of P_q  ) where P_k  is P[1..k] i.e.  " Π[q] is the length of longest prefix of P that is a proper suffix P_q ."

My implementation of Prefix function is as follows:

int* Prefix(char *pattern)

{

    int len = strlen(pattern);

    int *pre = new int[len];

    pre[0] = 0;

    int k = 0;

    for(int i=1; i<len; ++i)

    {

        while(k>0 && pattern[k] != pattern[i])

            k = pre[k];

        if(pattern[k] == pattern[i])

            ++k;

        pre[i] = k;

    }

    return pre;

}

KMP Matcher:  With text T, pattern P and prefix function Π as inputs, it finds all the occurrence of P  in T and returns the number of shifts of  P after which occurrence is found.

Implementation of KMP Match function is as follows:

 bool Match(char *text, char *pattern, vector<int>& shifts)

{

    int textLength = strlen(text);

    int patrnLength = strlen(pattern);

    int* prefixArray = Prefix(pattern); // Π function

    bool found = false;

    int q = 0;

    for(int i=0; i<textLength; ++i)

    {

        while( q>0 && pattern[q] != text[i])

            q = prefixArray[q-1]; //Next character does not macth

        if(pattern[q] == text[i])

            ++q;  //Next character Matched

        if(q == patrnLength) //All matched

        {

            shifts.push_back(i-patrnLength);

            found = true;

            q = prefixArray[q-1];  //Look for next match

        }

       

    }

    return found;

}

You can see both prefix( ) function and Match function takes linear time to execute.

Count Sort

This sorting algorithm takes advantage of the range of the numbers in the array to be sorted.
Say the range is 0..k then it can sort the array in O(k). If k is less than or equal to n, we are in great advantage because we can sort this array in linear time i.e. O(n).

The algorithm first determine for each element x the number of elements less than x. One of the advantage of this algorithm is its stability.

Following is my implementation:

void countSort(int *a, int size)
{
    int max = Max(a);
    int *c = new int[max+1];
    int *sorted = new int[size];
    for( int i=0; i<size; ++c[a[i++]] );   // Step 1
    for( i=1; i<max; c[i++] += c[i-1] );     // Step 2:Get the number of elements less than each number
    for(i=size-1; i >= 0; --i)    // Step 3
    {
        sorted[c[a[i]]-1] = a[i];
        --c[a[i]];
    }
    for(i=0; i<size; ++i)
        a[i] = sorted[i];
}

Example --

Input Array   

2

5

3

0

2

3

0

3

Step 1: Array C

2

0

2

3

0

1

Step 2: Array C

2

2

4

7

7

8

Step 3:

 a[7] = 3  c[3] = 7 sorted [6] = 3 => c[3] = 6

3

a[6] = 0  c[0] = 2  sorted[1] = 0  => c[0] = 1

0

3

a[5] = 3  c[3] = 6  sorted[5] = 3  => c[3] = 5

0

3

3

.

.

.

.

In the End Sorted array

0

0

2

2

3

3

3

5

Pancake sorting

Given an an unsorted array, sort the given array. You are allowed to do only following operation on array:
rev(arr, i): Reverse arr from 0 to i

Implementation:

void rev(int *arr, int i)
{
     int start = 0;
     while(start < i)
     {
                 swap(arr[start], arr[i]);
                 --i;
                 ++start;
     }
}

int findMaxIndex(int *arr, int n)
{
    int max = 0;
    for(int i = 0; i < n; ++i)
    {
            if(arr[i] > arr[max])
                      max = i;
    }
    return max;
}

void panCakeSort(int *arr, int n)
{
     for(int currSize = n; currSize > 1; --currSize)
     {
             int mi = findMaxIndex(arr, currSize);
             if(mi != currSize - 1)
             {
                   rev(arr, mi);
                   rev(arr, currSize - 1);
             }
     }

}

Microsoft Question: Nuts and Bolts problem

Question: Given an array of nuts of different sizes and array bolts of different sizes. There is a one-one mapping between nuts and bolts. Match nuts and bolts efficiently with the given constraint that comparison of a nut to another nut or a bolt to another bolt is not allowed. It means nut can only be compared with bolt and bolt can only be compared with nut to see which one is bigger/smaller.

Solution: Tweak quick sort to achieve it.

Implementation:

int split(int *arr, int low, int high, int pivot)

{

        int j = low;

        for(int i = low; i < high; ++i)

        {

                if(arr[i] < pivot)

                {

                        std::swap(arr[i], arr[j]);

                        ++j;

                }

                else if(arr[i] == pivot)

                {

                        std::swap(arr[i], arr[high]);

                        --i;

                }

        }

        std::swap(arr[j], arr[high]);

        return j;

}

void matchNutsBolts(int *nuts, int *bolts, int low, int high)

{

        if(!nuts || !bolts)

                return;

        if(low < high)

        {

                int pivot = split(nuts, low, high, bolts[high]);

                split(bolts, low, high, nuts[pivot]);

                matchNutsBolts(nuts, bolts, low, pivot - 1);

                matchNutsBolts(nuts, bolts, pivot + 1, high);

        }

}

Complexity: O(nlogn)

Wave sort an array.

Question: Given an unsorted array, sort it in wave form. Wave sorted array looks like as follows:
arr[0] >= arr[1] <= arr[2] >= arr[3] <= arr[4]...

Solution: Traverse all elements at even positions in the array and compare nth element with it's previous element(n-1th), if it is greater than previous, swap previous and current.
Now, again compare the nth element with its next element(n + 1th), if it is greater than next, swap next and current.

Implementation:

void sortWave(int* arr, int len)

{

        if(arr == NULL || len == 0 || len == 1)

                return;

        for(int i = 0; i < len; i+=2)

        {

                if(i>0 && arr[i-1] > arr[i])

                        std::swap(arr[i], arr[i-1]);

                if(i<len-1 && arr[i] < arr[i+1])

                        std::swap(arr[i], arr[i+1]);

        }

}

Complexity: O(n)

Alternative approach: Sort the array and swap adjacent elements. But the complexity of this approach will be O(nlogn)

Binary Search Tree

The property that makes a Binary Tree into Binary Search Tree is that for every node, X, in the tree, the values of all the keys in the left subtree are smaller than the key value in X and the values of all the keys in the right subtree are larger than or equal to key value in X.

Example of a Binary Search Tree is:

The class to implement Binary Search Tree :

// Node class

template < class Etype >
class TreeNode
{
public:
        Etype element;
        TreeNode* Left;
        TreeNode* Right;
        TreeNode( Etype e = 0, TreeNode* l = NULL, TreeNode* r = NULL) : 
        element ( e), left ( l ), right ( r ) { }
        friend int height ( TreeNode *T);
        friend void inOrderTraversal ( TreeNode *T);
        friend void preOrderTraversal ( TreeNode *T);
        friend void postOrderTraversal ( TreeNode *T);
        friend class BinarySearchTree < Etype >;
};

// Binary Search Tree Class

template<class Etype>
class BinarySearchTree
{
public: 
         BinarySearchTree ( ) : Root ( NULL ) { }
         void insert ( const Etype & x ) {  insert ( x, root ); }
         void remove ( const Etype& x ) { Remove (  x, root ); }
private:
         void insert ( const Etype& x, TreeNode<Etype>* & T );
         void remove ( const Etype& x, TreeNode<Etype>* & T);
         TreeNode *root;
};

Height:

int height ( TreeNode *T)
{
        return ( T ? 1 + Max ( height ( T-> left ), height ( T-> right ) ) : 0 ) ; 
}

Inorder Traversal:

void inOrderTraversal ( TreeNode *T)
{
       if ( T )
       {
                inOrderTraversal ( T-> left );
                cout<< T-> element;
                inOrderTraversal ( T-> right);
        }
}

Preorder Traversal:


void preOrderTraversal ( TreeNode *T)
{
       if ( T )
       {
                cout<< T-> element;
                preOrderTraversal ( T-> left );
                preOrderTraversal ( T-> right);

        }

}

Postorder Traversal:


void postOrderTraversal ( TreeNode *T)
{
       if ( T )
       {
                postOrderTraversal ( T-> left );
                postOrderTraversal ( T-> right);
                cout<< T-> element;
        }

}




Insert:

Following image shows how to insert an element in Binary Search Tree:

Following is the implementation of insert( ) function:

template < class Etype >
void BinarSearchTree< Etype > :: insert ( const Etype & x, TreeNode<Etype> * & T)
{
         if ( T = = NULL )
                 T = new TreeNode<Etype> ( x );
        else
        {
                 if ( x < T -> element )
                          insert ( x, T->left );
                 else if ( x > T-> element)
                          insert ( T-> right )
         }
}


Remove:

Following image shows how to remove an element in Binary Search Tree:

Following is the implementation of remove( ) function:

template < class Etype >
void BinarySearchTree< Etype > :: remove ( const Etype & x, TreeNode<Etype> * & T)
{
        TreeNode<Etype> *tmpCell;
   
        if ( T = = NULL )
                 cout << " Element Not Found ";
        else if ( x < T-> element )
                 remove ( x, T-> left );
        else if ( x > T-> element )
                 remove ( x, T-> right );
        else       // element found
        {
                  if ( T-> left != NULL && T -> right != NULL )
                  {
                            tmpCell = findMin ( T-> right );   // Minimum element
                            T-> element = tmpCell -> element;
                            remove ( T->element, T->right );
                  }
                 else
                 {
                          tmpCell = T;
                          if ( T-> left)
                                     T = T-> left;
                         else if ( T-> right )
                                     T = T-> right;
                         delete tmpCell;
                  }
        }
}

Linked List

Well the first Data Structure that we studied is Linked List and it also provides a way to implement various Advance Data Structures. It can be defined in the following way --

                    "A Linked List is a Data Structure that consists of a sequence of Data Records such that in each record there is a field that contains a reference or link to the next record in the sequence."

Above is the Singly Linked List in which each node has data and a link to the next node in the List. Head node is the start of the List. Next of the last node in the List is NULL.

To implement Singly Linked List, I used following way (C++) --

template <class Etype>

class List

{

     struct Node

     {

            Etype element;

            Node *next;

            Node(Etype E = 0, Node *N = NULL) : element ( E ), next ( N ) { }

      };

    

     Node *head; // start position of head 

     Node *curr;  // To optimize and support many operation, think what we can do with this

   // then various operations, constructors, destructors etc.

};

* In My implementation, head is always in the list even in the case of empty list and head will remain same for lifetime of a list object.


Insert ::
//Insert after curr

template < class Etype  >
void List < Etype > :: insert ( const Etype & data )
{
     curr->next = new Node (data, curr->next );
}

See the following figure for more clarification, it shows insert( 30 ) --

Remove ::
template < class Etype >
int List < Etype > :: remove ( const Etype & data )
{
      if ( Find_Prev( data ) )   // set curr to the node whose next node's element is data
      {
              Node *toDelete = curr->next;
              curr->next = toDelete->next;
              delete toDelete;
               return 1;
      }
      return 0;
}

Following figure shows remove( 8 )