Count ways to choose K equal strings - algorithm

Question

Given a string S consisting of N lowercase English alphabets. Suppose we have a list L consisting of all non empty substrings of the string S.
Now we need to answer Q queries. For ith query, I need to count the number of ways to choose exactly K equal strings from the list L.
NOTE: For each K we will have different value of K.
To avoid overflow I need to take it modulo 10^9+7.
Example : Let S=ababa and we have 2 Queries. Value of K for each query is :
2
3
Then answer for first query is 7 and for second query its 1.
As List L = {"a", "b", "a", "b", "a", "ab", "ba", "ab", "ba", "aba", "bab", "aba", "abab", "baba", "ababa"}
For Query 1 : There are seven ways to choose two equal strings ("a", "a"), ("a", "a"), ("a", "a"), ("b", "b"), ("ab", "ab"), ("ba", "ba"), ("aba", "aba").
For Query 2 : There is one way to choose three equal strings - ("a", "a", "a").
Now the problem is that N<=5000 and Queries can be 100000. So brute solution won't work. What can be better way to do it.

Answer 1

One way to optimize search would be to reduce the set of possible candidates for pairs.
Basic idea would be: For each substring of length n > 2, that matches the constraints (the substring appears atleast K times in the input-string), 2 substrings of length n - 1 that match requirements must exist. Example: if ("abab" , "abab") is a pair of substrings of the input-string, ("aba" , "aba") and ("bab" , "bab") must aswell be pairs of substrings of the input-string.
This can be used to eliminate candidates for pairs in the following way:
Start with an initial set of substrings of length = 1, such that the set only contains substrings for which atleast K - 1 equal substrings can be found. Extend each of these substrings by adding the next character. Now we can eliminate substrings, for which not enough matches can be found. Repeat this until all substrings are eliminated.
Now from theory to praxis:
This basic datastructure simply represents a substring by it's starting and end-point (inclusive) in the input string.
define substr:
int start , end
A helper method for getting the string represented by a substr:
define getstr:
input: string s , substr sub
return string(s , sub.start , sub.end)
First generate a lookup-table for all characters and their position in the string. The table will be needed later on.
define posMap:
input: string in
output: multimap
multimap pos
for int i in [0 , length(in)]
put(pos , in[i] , i)//store the position of character in[i] in the map
return pos
Another helper-method generates a set of all indices of characters that only appear once in the input-string
define listSingle:
input: multimap pos
output: set
set single
for char c in keys(pos)
if length(get(pos , c)) == 1
add(single , get(get(pos , c) , 0)
return single
A method that creates the initial set of matching pairs. These pairs consist of substrings of length 1. The pairs themself aren't specifed; the algorithm only maps the text of a substring to all occurences. (NOTE: I'm using pair here, though the correct term would be set of length K)
define listSinglePairs:
input: multimap pos
output: multimap
multimap result
for char key in keys(pos)
list ind = get(pos , key)
if length(ind) < 2
continue
string k = toString(key)
for int i in ind
put(result , k , substr(i , i))
return result
Furthermore a method is required to list all substrings that contain the same string as a given string:
define matches:
input: string in , substr sub , multimap charmap
output: list
list result
string txt = getstr(in , sub)
list candidates = get(charmap , txt[0])
for int i in [1 , length(txt)[
//increment all elements in candidates
for int c in [0 , size(candidates)[
replace(candidates , c , get(candidates , c) + 1)
list next = get(charmap , txt[i])
//since the indices of all candidates were incremented (index of the previous character in
//in) they now are equal to the indices of the next character in the substring, if it matches
candidates = intersection(candidates , next)
if isEmpty(candidates)
return EMPTY
//candidates now holds the indices of the end of all substrings that
//match the given substring -> convert to list of substr
for int i in candidates
add(result , substr(i - length(txt) , i))
return result
This is the main-routine that does the work:
define listMatches:
input: string in , int K
output: multimap
multimap chars = posMap(in)
set single = listSingle(chars)
multimap clvl = listSinglePairs(chars , K)
multimap result
while NOT isEmpty(clvl)
multimap nextlvl
for string sub in clvl
list pairs = get(clvl , sub)
list tmp
//extend all substrings by one character
//substrings that end in a character that only appears once in the
//input string can be ignored
for substr s in pairs
if s.end + 1 > length(in) OR contains(single , s.end + 1)
continue
add(tmp , substr(s.start , s.end + 1)
//map all substrs to their respective string
while NOT isEmpty(tmp)
substr s = get(tmp , 0)
string txt = getstr(s , in)
list match = matches(in , s , chars)
//this substring doesn't have enough pairs
if size(match) < K
continue
//save all matches as solution and candidates for the next round
for substr m in match
put(result , txt , m)
put(nextlvl , txt , m)
//overwrite candidates for the next round with the given candidates
clvl = nextlvl
return result
NOTE: this algorithm generates a map of all substrings, for which pairs exist to the position of the substrings.
I hope this is comprehensible (i'm horrible in explaining things).