I am working on using a real time application in java, I have a data structure that looks like this.
HashMap<Integer, Object> myMap;
now this works really well for storing the data that I need but it kills me on getting data out. The underlying problems that I run into is that if i call
Collection<Object> myObjects = myMap.values();
Iterator<object> it = myObjects.iterator();
while(it.hasNext(){ object o = it.next(); }
I declare the iterator and collection as variable in my class, and assign them each iteration, but iterating over the collection is very slow. This is a real time application so need to iterate at least 25x per second.
Looking at the profiler I see that there is a new instance of the iterator being created every update.
I was thinking of two ways of possibly changing the hashmap to possibly fix my problems.
1. cache the iterator somehow although i'm not sure if that's possible.
2. possibly changing the return type of hashmap.values() to return a list instead of a collection
3. use a different data structure but I don't know what I could use.
If this is still open use Google Guava collections. They have things like multiMap for the structures you are defining. Ok, these might not be an exact replacement, but close:
From the website here: https://code.google.com/p/guava-libraries/wiki/NewCollectionTypesExplained
Every experienced Java programmer has, at one point or another, implemented a Map> or Map>, and dealt with the awkwardness of that structure. For example, Map> is a typical way to represent an unlabeled directed graph. Guava's Multimap framework makes it easy to handle a mapping from keys to multiple values. A Multimap is a general way to associate keys with arbitrarily many values.
Related
(By heavy I meant objects with several lines)
Hi, I got this doubt, I got a class EntityVO, and is getting quite big lately, 250+ lines, however are mostly private variables with getters and setters.
My doubt is, iterating through them, would consume more time because of the size of the class rather than if I iterate through a lightweight class?
Thanks.
No. The number of fields in a class does not affect performance of the class in any way (other than when you create the class).
When you iterate through an array of classes, you are really just throwing around a reference to the actual instance, not all the data stored in it.
In C# on Framework 4, I have a List and List. They can be joined on the JoinId property. ParentObj will have 2 ChildObj matches, sometimes 10.
I would like to take each Parent and all Children and serialize to a single XML entity. I am having a hard time figuring out where to start, because I also need to serialize the objects in a custom way. Can I use Linq-to-XML in this case to get each object written correctly? XmlSerializer? Not sure.
Thanks.
Can I use Linq-to-XML in this case to get each object written correctly?
Yes. This is exactly what you need. A basic example.
XmlSerializer?
This will work too, but this approach is older and I think it is less appropriate and more complicated in this case.
I'm caching a whole bunch of static metadata in my app at startup. It's from a db and there are tons of Foreign Key relationships. I'm looking into the best way of modelling them.
I've just started off with LINQ.
It's easy for me to declare a class
public class AllData {
public static IQueryable<libm_ColPurpose> IQ_libm_ColPurpose = libm_ColPurpose.All();
public static IQueryable<libm_ColType> IQ_libm_ColType = libm_ColType.All();
...
(I'm using SubSonic 3 to generate my classes, but that's beside the point).
Then I can use the IQueryable<T> members to get access to anything I want, for example:
libm_ColType ct = AllData.IQ_libm_ColType.SingleOrDefault(x => x.ColTypeStr == this.DefaultJetColTypeStr);
Prior to using IQueryable I was using Dictionaries to store FK relationships, so to mimic the above code I'd code the following from a preexisting List<libm_ColType> list
Dictionary<string, libm_ColType> colTypeByColTypeStr = new Dictionary<string, libm_ColType>();
foreach (libm_ColType x in list) { rtn.Add(x.ColTypeStr, x); }
and then I could use
libm_ColType ct = colTypeByColTypeStr[this.DefaultJetColTypeStr];
OK, so finally we get to the question !
The Dictionary lookup by ID is extremely efficient, however the IQueryable solution is far more flexible and elegant.
I'm wondering how much of a performance hit I'm going to get using IQueryable. I suspect I am doing a linear scan of the list each time I call it, and that's really gonna add up over repeat calls if there are a lot of records involved.
It woul be great if I could identify unique-valued columns and have a hashtable generated and cached after the first lookup, but I suspect this is not gonna be part of the offering.
This is a bit of a dealbreaker for me regarding using LINQ.
Note (I'll repeat it again) that I'm NOT pulling data from a database, it's already in memory and I'm querying it there, so I'm only interesting in looking up the in-memory IQueryable<T>.
IQueryable represents a collection in a data-store, so you probably don't have the collections in memory. If you explicitly want in-memory collections, then I would go back to your dictionaries. Remember, this doesn't prevent you from using LINQ queries over the data.
I'm trying to put a HashMap<Object, List<Object>> into my dataModel, but when i call the template.process() method, I get the following exception:
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at freemarker.template.SimpleCollection$SimpleTemplateModelIterator.next(SimpleCollection.java:142)
at freemarker.core.IteratorBlock$Context.runLoop(IteratorBlock.java:157)
at freemarker.core.Environment.visit(Environment.java:351)
at freemarker.core.IteratorBlock.accept(IteratorBlock.java:95)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.IteratorBlock$Context.runLoop(IteratorBlock.java:172)
at freemarker.core.Environment.visit(Environment.java:351)
at freemarker.core.IteratorBlock.accept(IteratorBlock.java:95)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.Environment.process(Environment.java:176)
at freemarker.template.Template.process(Template.java:232)
After looking over some articles and older questions, I've tried to use a ConcurrentHashMap instead, to the same result. I've also tried making a copy using new HashMap<Object, List<Object>>(oldHashMap). Are there any other common fixes to this problem I could try?
EDIT: I know the general cause of ConcurrentModificationExceptions. Please only reply if you can help me understand why the framework Freemarker is throwing these exceptions, mkay? =)
Thanks!
The ConcurrentModificationException is caused by using an invalid iterator after the underlying collection has been changed. The only way to fix this is not changing the collection you are iterating over. In most cases this is not caused by multi-threading.
Simple Example:
//throws an exception in the second iteration
for(String s: list){
list.remove(s);//changes the collection
}
fix 1, not supported by all iterators:
Iterator<String> iter = list.iterator();
while(iter.hasNext()){
iter.next();
iter.remove();//iterator still valid
}
fix 2:
List<String> toRemove = ...;
for(String s: list){
toRemove.add(s);
}
list.removeAll(toRemove);
The exception means that, while you're iterating over the map, something has changed the map's contents.
Your best course of action is figure out what that "something" is. For example, it could be another thread, or it could be that you have a foreach loop and modify the map from within the loop.
It is very hard to give advice on how to best fix the problem until we understand what exactly is causing it and what the desired behaviour is.
You'll get this kind of problem on List and Map when doing something like this:
List<A> list = ...; //a list with few elements
for(A anObject : list){
list.add(anotherObject); //modify list inside the loop
}
The same goes with maps. The solution is to look for possible places where you might be modifying the map inside the loop over that map. Or if you are using a multi-threaded application, then it's possible that another thread is looping over the map while you are modifying it (or visa-versa). In such case you'll need to synchronize access to the map in both places: looping code and map modifying code.
There some info on it in the Java API for TreeMap here.
The iterators returned by the iterator
method of the collections returned by
all of this class's "collection view
methods" are fail-fast: if the map is
structurally modified at any time
after the iterator is created, in any
way except through the iterator's own
remove method, the iterator will throw
a ConcurrentModificationException.
Thus, in the face of concurrent
modification, the iterator fails
quickly and cleanly, rather than
risking arbitrary, non-deterministic
behavior at an undetermined time in
the future.
Note that the fail-fast behavior of an
iterator cannot be guaranteed as it
is, generally speaking, impossible to
make any hard guarantees in the
presence of unsynchronized concurrent
modification. Fail-fast iterators
throw ConcurrentModificationException
on a best-effort basis. Therefore, it
would be wrong to write a program that
depended on this exception for its
correctness: the fail-fast behavior of
iterators should be used only to
detect bugs.
Synchronise access to the hashmap so that only one thread can be accessing the hashmap at once.
I need to store a lookup table as an instance member in one of my classes. The table will be initialized when the object is constructed. Each "row" will have 3 "columns":
StringKey (e.g., "car")
EnumKey (e.g., LookupKeys.Car)
Value (e.g, "Ths is a car.")
I want to pick the data structure that will yield the best performance for doing lookups either by the StringKey or the EnumKey.
It's kind of awkward having 2 keys for the same dictionary value. I've never encountered this before, so I'm wondering what the norm is for this type of thing.
I could make a Key/Value/Value structure instead of Key/Key/Value, but I'm wondering what type of performance impact that would have.
Am I thinking about this all wrong?
Well ... "Wrong" is a harsh way of putting it. I think that because the most common dictionary is "single key to value", and a lot of effort goes into providing efficient data structures for that (maps), it's often best to just use two of those, sharing the memory for the values if at all possible.
You have two hashmaps.
One from StringKey to value.
One from EnumKey to value.
You do not have to duplicate all the Value instances, those objects can be shared between the two hashmaps.
If it's a LOT of items, you might want to use two treemaps instead of two hashmaps. But the essential principle ("Share the Values") applies to both structures. One set of Values with two maps.
Is it really necessary to key into the same structure with both types of key? You probably don't need to rebuild a complex data structure yourself. You could do some sort of encapsulation for the lookup table so that you really have two lookup tables if memory is not an issue. You could use this encapsulating structure to simulate being able to pull out the value from the "same" structure with either type of key.
OR
If there is some way to map between the enum value and the string key you could go that route with only having one type of lookup table.
LINQ's ILookup(TKey, TElement) interface may help. Assuming your Dictionary is something like:
Dictionary<carKey, carValue> cars;
You could use:
ILookUp<carValue, carKey> lookup = cars.ToLookup(x => x.Value, x => x.Key);
(...actually I think I might have slightly misread the question - but an ILookUp might still fit the bill, but the key/value set might need to be the key and the enum.)
If every value is guaranteed to be accessible by both types of keys, another idea would be to convert one type of key to another. For example:
public Value getValue(String key)
{
dictionary.get(key); // normal way
}
public Value getValue(Enum enumKey)
{
String realKey = toKey(enumKey);
getValue(realKey); // use String key
}
You could have your Enum implement a toKey() method that returns their String key, or maybe have another dictionary that maps Enum values to the String counterparts.