AngularJS browser memory keeps climbing when adding a removing controller - memory-management

Question

I've created a webapp using angular js and the routeProvider. I don't load partials inside ng-view but I highjack the routParam changes that it causes to create my own object indicating what part of the app is active see below:
lrApp.config(function ($routeProvider) {
$routeProvider
.when('/',{
redirectTo:'/share'
})
.when('/:action',{
controller: "viewController",
template: ' '
})
.when('/:action/:section',{
controller: "viewController",
template: ' '
})
.when('/:action/:section/:subsection',{
controller: "viewController",
template: ' '
})
.otherwise({
redirectTo:'/'
});
});
The empty spaces for template and including an empty in your body trigger the routeParams to change I then call viewController which is included on the body and includes a globally accessible object i.e. ng-show="url.section == 'contacts'"
So I use this global object to ng-show or ng-switch or ui-if certain parts of the app into place when applicable. This also allows me to create 3 levels of deep linking.
I'm creating a factory for each major section of the app that contains an empty object I then store the growing data for each controller inside of a factory so that when the controllers are reloaded via ng-switch the data is still there and I don't have to get it again...
MY ISSUE / QUESTIONS
Is this the appropriate way to be storing my data? or should I be coupling this with cacheFactory and storing that object in the cacheFactory and then returning the cacheFactory in the controller?
What's the difference between storing the data in cacheFactory and just storing the data in the factory and having it outside of the controller ... I'm confused?
When I do this and I switch to a different section using ng-switch the info is taken out of the DOM and when I switch back the data is put back in because it still stored in the factory BUT the memory it shows for my tab in chrome just keeps climbing higher and higher every time I switch back even though I'm not loading any new data... what am I doing wrong?
This is a rough sample of how I build my app... test app I built

Answer 1

I didn't understand enough about angularjs at the time but my first issue was that I was de-rendering and rendering large chunks of data out of the DOM and then back in and what I thought was a memory leak was really just the DOM memory climbing because the angular template engine was having to re-render templates.