Efficient program to keep track of movies and search frequency? - algorithm

Question

In studying for my exam I came across this question.
A website streams movies to customers’ TVs or other devices. Movies are in one of several genres such as action, drama, mystery, etc. Every movie is in exactly one genre (so that if a movie is an action movie as well as a comedy, it is in a genre called “action-Comedy”). The site has around 10 million customers, and around 25,000 movies, but both are growing rapidly. The site wants to keep track of the most popular movies streamed. You have been hired as the lead engineer to develop a tracking program.
i) Every time a movie is streamed to a customer, its name (e.g. “Harold and Kumar: Escape from Guantanamo Bay”) and genre (“Comedy”) is sent to your program so it can update the data structures it maintains.
(Assume your program can get the current year with a call to an appropriate Java class, in O(1) time.)
ii) Also, every once in a while, customers want to know what were the top k most streamed movies in genre g in year y. (If y is the current year, then accounting is done up to the current date.) For example, what were the top 10 most streamed comedy movies in 2010? Here k = 10, g=”comeday” and y = 2010. This query is sent to your program which should output the top k movie names.
Describe the data structures and algorithms used to implement both requirements. For (i), analyze the big O running time to update the data structures, and for (ii) the big O running time to output the top k streamed movies.
My thought process was to create a hash table, with every new movie added to its respective genre in the hash table in a linked list. As for the second part, my only idea is to keep the linked list sorted but that seems way too expensive. What is a better alternative?

Answer 1

I use a heap to keep track of the top k objects of a class (k fixed). You can find the details of this data structure in any CS text, but basically it's a binary tree in which every node is smaller than either of its children. The main operation, which we will call reheap(node) assumes that both the children of node are heaps, compares node with the smaller of its two children, does the swap if necessary, and recursively calls reheap for the modified child. The class needs to have an overloaded operator< or the equivalent defined to do this.
At any point in time, the heap holds the top k objects with the smallest of these at the top of the heap. When a new object arrives which is bigger than the top of the heap, it replaces that object on the heap, and then
reheap is called. This can also happen at a node other than the top node if an object already on the heap becomes bigger than its smaller child. Another type of update occurs if an object already on the heap becomes smaller than its parent (this probably won't happen in the case you describe). Here it gets swapped with its parent and we then compare recursively against the grandparent, etc.
All of these updates have complexity O(log(k)). If you need to output the heap sorted from the top down, the same structure works well in time
O(k log(k)). (This process is known as heapsort).
Since swapping objects can be expensive, I usually keep the objects in a fixed array somewhere, and implement the heap as an array, A, of pointers, where the children of A[i] are A[2i+1] and A[2i+2].

Answer 2

You could do this in O(1) using one hash table "HT1" to map from (genre, year, movie_title) to an iterator into a linked list of (num_times_streamed, hash table of movie titles). You use the iterator to see if the next element in the list is for one greater streaming count and if so insert your movie title there and remove it from the other table (which if empty can be removed from the list), otherwise if the existing hash table has no other titles then increment the num_times_streamed, otherwise insert a new hash table in the list and add your title. Update the record of the iterator in HT1 as necessary.
Note that as described above the operations in the list use the end-points or an existing iterator to step through by no more than one position as the num_times_streamed value is incremented, so O(1).
To get the top k titles you'll need a hash table HT2 from { genre, year } to each of the linked lists: simply iterate from the end of the list and you'll encounter a hash table with a movie or movies with the highest streaming count, then the next highest and so on. If the year's just changed, you may not find k entries, handle that however you like. If when looking up a movie title it's found not to exist in HT1, you'd add a new list for that genre and the current year to HT2.
More visually, using { } around hash tables (whether mappings or sets), [ ] around linked lists, and ( ) around grouped struct/tuple data:
HT2 = { "comedy 2015": [ (1, { "title1", "title2" }),
(2, { "title3" }), <--------\
(4, { "title4" }) ], |
"drama 2012": [ (1, { "title5" }), |
(3, { "title6" }) ], |
... | .
}; | .
| .
HT1 = { "title3", -----------------------------------/ |
"title2", ---------------------------------------/
...
};