I've setup a simple cache, using an Integer for the key and a Double for the value. After populating the cache, the ratio cache.calculateInMemorySize() / cache.getMemoryStoreSize() is constant at 344 bytes per element. I expect overhead, but my payload is (32 + 64) 96 bits, or 12 bytes, so the overhead is a whopping 332 bytes - or am I completely misunderstanding how this work? If not, what, if anything, can I do to bring down the overhead?
The cache is meant to be a memory-only store. We want to fit everything in there, so overflow and expiry is not needed, and as we can populate fairly quickly from the external data source (just not fast enough to use it as the primary data source), persistence is not needed either.
Using version 2.4.0.
I'm assuming the payload is the things actually being cached. Are you including the size of the keys as well? I believe the calculateInMemorySize also includes the keys.
Based on your requirements:
The cache is meant to be a memory-only store. We want to fit
everything in there, so overflow and expiry is not needed, and as we
can populate fairly quickly from the external data source (just not
fast enough to use it as the primary data source), persistence is not
needed either.
I conclude that you don't need to use any 3rd party caching framework at all. Instead you can get away with simple HashMap...since your cached elements never expire and always fit in memory. Also, you will not to include EhCache jars in your classpath and load its classes!
Here is sample code:
public class MyCustomCache
{
private Map<Integer, Double> myMap = new HashMap<Integer, Double>();
public Double getCachedValueByKey(Integer key)
{
return myMap.get(key);
}
public void putValue2Cache(Integer key, Double value)
{
myMap.put(key, value);
}
public Double removeValueFromCache(Integer key)
{
return myMap.remove(key);
}
}
Related
I am implementing a cache in Golang. Let's say the cache could be implemented as sync.Map with integer key and value as a struct:
type value struct {
fileName string
functionName string
}
Huge number of records have the same fileName and functionName. To save memory I want to use string pool. Go has immutable strings and my idea looks like:
var (
cache sync.Map
stringPool sync.Map
)
type value struct {
fileName string
functionName string
}
func addRecord(key int64, val value) {
fileName, _ := stringPool.LoadOrStore(val.fileName, val.fileName)
val.fileName = fileName.(string)
functionName, _ := stringPool.LoadOrStore(val.functionName, val.functionName)
val.functionName = functionName.(string)
cache.Store(key, val)
}
My idea is to keep every unique string (fileName and functionName) in memory once. Will it work?
Cache implementation must be concurrent safe. The number of records in the cache is about 10^8. The number of records in the string pool is about 10^6.
I have some logic that removes records from the cache. There is no problem with main cache size.
Could you please suggest how to manage string pool size?
I am thinking about storing reference count for every record in the string pool. It will require additional synchronizations or probably global locks to maintain it. I would like to implementation as simple as possible. You can see in my code snippet I don't use additional mutexes.
Or may be I need to follow completely different approach to minimize memory usage for my cache?
What you are trying to do with stringPool is commonly known as string interning. There are libraries like github.com/josharian/intern that provide "good enough" solutions to that kind of problem, and that do not require you to manually maintain the stringPool map. Note that no solution (including yours, assuming you eventually remove some elements from stringPool) can reliably deduplicate 100% of strings without incurring impractical levels of CPU overhead.
As a side note, it's worth pointing out that sync.Map is not really designed for update-heavy workloads. Depending on the keys used, you may actually experience significant contention when calling cache.Store. Furthermore, since sync.Map relies on interface{} for both keys and values, it normally incurs much more allocations that a plain map. Make sure to benchmark with realistic workloads to ensure that you picked the right approach.
The documentation of OSData says that "...You can add bytes to them and overwrite portions of the byte array.". I can see a method to append bytes, but I don't understand how I am able to overwrite a portion of the buffer.
Another option would be to use IONewZero to allocate a number of elements of the type I need. I my case I just need this for ints.
Example:
T* dataBuffer = IONewZero(T, SIZE);
And then deallocate with:
IOSafeDeleteNULL(dataBuffer_, T, SIZE);
What are the advantages of using an OSData object compared to the solution with IONewZero / IOSafeDeleteNULL?
I think the documentation might just be copy-pasted from the kernel variant of OSData. I've seen that in a bunch of places, especially USBDriverKit.
OSData is mostly useful for dealing with plist-like data structures (i.e. setting and getting properties on service objects) in conjunction with the other OSTypes: OSArray, OSDictionary, OSNumber, etc. It's also used for in-band (<= 4096 byte) "struct" arguments of user client external methods.
The only use I can see outside of those scenarios is when you absolutely have to reference-count a blob of data. But it's certainly not a particularly convenient or efficient container for data-in-progress. If you subsequently need to send the data to a device or map it to user space, IOBufferMemoryDescriptor is probably a better choice (and also reference counted) though it's even more heavyweight.
We are using gsoap for C client and server webservices implemented for blackfin running Linux.
We don't use any malloc in the application. But we see memory usage climbs over time. We are using soap_end to do a cleanup at the end of the call. But when the calls are invoked repeatedly memory usage slowly increasing, may be because of memory fragmentation. This is also impacting performance of the system
What's the preferred usage of gsoap where soap_malloc is not used much. For eg: If we use static arrays etc will it help?
Thanks,
nkr
I would not recommend using static data, there is no need for that.
To debug memory use, compile all your sources files with -DDEBUG. When you run your application you will see three files:
SENT.log the messages sent
RECV.log the messages received
TEST.log the debug log
The TEST.log is useful to check on messaging issues.
The other valuable information produced at runtime are error messages related to memory leaks or heap memory that is damaged (e.g. overruns) in your code. It is unlikely these will happen in the gSOAP engine, but better check.
To ensure proper allocation and deallocation of managed data:
soap_destroy(soap);
soap_end(soap);
I am using the auto-generated functions to allocate managed data:
SomeClass *obj = soap_new_SomeClass(soap);
and sporadically use soap_malloc for raw managed allocation, or to allocate an array of pointers, or a C string:
const char *s = soap_malloc(soap, 100);
but better is to allocate strings with:
std::string *s = soap_new_std__string(soap);
and arrays can be allocated with the second parameter, e.g. an array of 10 strings:
std::string *s = soap_new_std__string(soap, 10);
All managed allocations are deleted with soap_destroy() followed by soap_end(). After that, you can start allocating again and delete again, etc.
If you want to preserve data that otherwise gets deleted with these calls, use:
soap_unlink(soap, obj);
Now obj can be removed later with delete obj. But be aware that all pointer members in obj that point to managed data have become invalid after soap_destroy() and soap_end(). So you may have to invoke soap_unlink() on these members or risk dangling pointers.
A new cool feature of gSOAP is to generate deep copy and delete function for any data structures automatically, which saves a HUGE amount of coding time:
SomeClass *otherobj = soap_dup_SomeClass(NULL, obj);
This duplicates obj to unmanaged heap space. This is a deep copy that checks for cycles in the object graph and removes such cycles to avoid deletion issues. You can also duplicate the whole (cyclic) managed object to another context by using soap instead of NULL for the first argument of soap_dup_SomeClass.
To deep delete:
soap_del_SomeClass(obj);
This deletes obj but also the data pointed to by its members, and so on.
To use the soap_dup_X and soap_del_X functions use soapcpp2 with options -Ec and -Ed, respectively.
In principle, static and stack-allocated data can be serialized just as well. But consider using the managed heap instead.
Hope this helps.
I have a 1.5 GB file that contains a serialized HashMap inside it.
I have a setup() method in the Mapper class where I am reading this into a HashMap variable.
It looks like it is able to go to the read method, but immediately throws a java heap space error for the tasks.
I read over many discussions that we may need to set the mapred.child.opts parameter and I am doing that inside the main program code.
I am using:
conf.set("mapred.child.java.opts.", "-Xmx1024M");
I even tried to increase the number. Why does it still keep throwing the same error at the time it is trying to read the serialized file into a HashMap variable?
Here is the code in my setup() method:
try {
test="hello";
Path pt=new Path("hdfs://localhost:9000/user/watsonuser/topic_dump.tsv");
FileSystem fs = FileSystem.get(new Configuration());
}catch(Exception e) {System.out.println("Exception while reading the nameMap
file."); e.printStackTrace();}
InputStream is = fs.open(pt);
ObjectInputStream s = new ObjectInputStream(is);
nameMap = (HashMap<String, String>) s.readObject();
s.close();
}catch(Exception e) {
System.out.println("Exception while reading the nameMap file.");
e.printStackTrace();
}
As you're using the serialized version of the hash map, and the final output size of the file is 1.5GB, i'm guessing that the amount of memory your JVM is going to need is at least 1.5GB.
You should be able to test this with a small program to load in your file (as you already have), but keep increasing the -Xmx value until you no longer see the memory error - this will be your baseline (you'll probably still need to add some more when running within a hadoop mapper as it has buffer size requirements for spills sorting etc.
Do you also know how many bins and items are represented in this hash map? The implementation of HashMap is just an array of bins with linked entry items that hash to that bin number. The number of bins also has to be a power of two, so as you put more and more items in your map, the memory requirements for the actual backing array double when the map reaches its threshold value / load factor (0.75). With this in mind, i imagine the problems your seeing is that such a large hash map (1.5GB serialized) will require an as large, if not larger memory footprint when deserialized into memory
The bottleneck of my current project is heap allocation... profiling stated about 50% of the time one critical thread spends with/in the new operator.
The application cannot use stack memory here and needs to allocate a lot of one central job structure—a custom job/buffer implementation: small and short-lived but variable in size. The object are itself heap memory std::shared_ptr/std::weak_ptr objects and carry a classic C-Array (char*) payload.
Depending on the runtime configuration and workload in different parts 300k-500k object might get created and are in use at the same time (but this should usually not happen). Since its a x64 application memory fragmentation isn't that big a deal (but it might get when also targeted at x86).
To increase speed and packet throughput and as well be save to memory fragmentation in the future I was thinking about using some memory management pool which lead me to boost::pool.
Almost all examples use fixed size object... but I'm unsure how to deal with a variable lengthed payload? A simplified object like this could be created using a boost::pool but I'm unsure what to do with the payload? Is it usable with a boost:pool at all?
class job {
public:
static std::shared_ptr<job> newObj();
private:
delegate_t call;
args_t * args;
unsigned char * payload;
size_t payload_size;
}
Usually the objects are destroyed when all references to the shared_ptr run out of scope and I wouldn't want to change the shared-ptr back to a c-ptr. A deferred destruction of the objects should also work to increase performance and from what I read should work better with a boost:pool. I haven't found if the pool supports an interaction with the smart_ptr? The alternative but quirky way would be to save a reference to the shared_ptr on creation together with the pool and release them in blocks.
Does anyone have experiences with the two? boost:pool usage with variable sized objects and smart pointer interaction?
Thank you!