Build tree structure from the records - algorithm

Question

I query database for records in structure as follows
ID | Term | ParentID
In C# code I have following class
public class Tree
{
public string Id { get; set; }
public string Term { get; set; }
public string ParentId { get; set; }
public int Level { get; set; }
public IList<Tree> ChildItems { get; set; }
}
Query returns 5 000 000 records.
I need to build tree of Tree items and populate it.
First at all, I select all items where ParentID is null, and then for every element search parent (if parent doesn't exist I build parent of the parent and so on) and build tree using recursion.
I'm not happy with my algorithm because It takes more than 5 minutes.
Please, let me some advice how to do that, what to use and so on.
This is how the code is now implemented:
private string Handle2(List<Tree> originalTree)
{
IList<Tree> newTree = new List<TreeTest.Models.Tree>();
IList<Tree> treeWithoutParents = originalTree.Where(x => String.IsNullOrEmpty(x.ParentID)).OrderBy(x => x.Term).ToList();
foreach(Tree item in treeWithoutParents)
{
Tree newItem = new Tree { Id = item.ID, Term = item.Term, ParentId = item.ParentID, Level = 0 };
newTree.Add(newItem);
InsertChilds(newItem, originalTree, 0);
}
return "output";
}
private void InsertChilds(Tree item, List<Tree> origTree, int level)
{
++level;
IList<Tree> childItems = origTree.Where(x => x.ParentID == item.Id).ToList();
origTree.RemoveAll(x => x.ParentID == item.Id);
foreach (Tree i in childItems)
{
origTree.Remove(i);
}
foreach (Tree tItem in childItems)
{
if (item.ChildTree == null)
{
item.ChildTree = new List<TreeTest.Models.Tree>();
}
Tree itemToAdd = new Tree { Id = tItem.ID, Term = tItem.Term, ParentId = tItem.ParentID, Level = level };
this.InsertChilds(itemToAdd, origTree, level);
item.ChildTree.Add(itemToAdd);
}
}

Answer 1

Try using a map (C# Dictionary) of ID (string, although I'm curious why that isn't int) to node (Tree object) to store your tree nodes.
This would allow you to get the node corresponding to an ID with expected O(1) complexity, rather than your current O(n) complexity.
Beyond that, I suggest you rethink your approach a bit - try to write code which involves you only going through the input data once, just use a single Dictionary - if the parent doesn't exist yet, you could just create a filler-item for the parent, which has its members populated only when you get to that item.

Answer 2

I would use a dictionary (hash table) to make this faster. Here is my algorithm in pseudocode:
- create a dictionary mapping ID to IList<Tree> // mapping a node to its children
- create Queue<string,string> of IDs //item (3,5) in the queue corresponds to a node with ID=3 that has a parent with ID=5
- initialize the queue with all the codes with no parent
- List<Tree> withoutParent = dictionary[null]
- for each item in withoutParent:
- add (item.Id, null) to the queue
- while queue is not empty:
- (id,pid) = delete an item from the queue
- make a new node t
- t.Id = id
- t.parentId = pid
- t.ChildItems = dictionary[id]
- for each child in t.ChildItems:
- add (child.Id, id) to the queue

Answer 3

is the column ID a unique identifier. If it is then you can try the following. Instead of using a List, use a Set or a hashmap. This is because if a parent has too many child, lookup in a list can slow down your operations. If you use a Set, you can do a quick lookup and you can also do a quick addition of your elements.
Also, can you check how much time an order by clause will take . This might really help you speed up your process. If ID is a clustered index, you will get a fast sort by(as the data is already sorted) , else your query will still use the same index
When a parent does not exist , you are creating a parent of a parent. I would try to avoid that. What you can do is in case a child's parent does not exist in the tree, add it to a separate list. After you have gone through the original list , make a second pass to find orphaned elements. The advantage is that you do not need to resize your tree every time you create a parent of parent and then find out that the parent was just at the end of the list