Im trying to figure out of how to correctly work with the Extra Bytes that you can let windows allocate for your windows and window classes.
If i'm reading the docs correctly, you can tell windows to allocate a specified amount of memory for your window or window class.
But there is only two methods i could find to access and modify said data, SetWindowLongPtr and GetWindowLongPtr.
Problem is, with those methods you can only set a LongPtr full of data, so 64 / 32 bits depending on your system.
Can somebody explain this to me, is there a method i am missing or is this as it should be?
(Get|Set)WindowLong() accesses a value at the specified nIndex as a whole LONG.
(Get|Set)WindowLongPtr() accesses a value at the specified nIndex as a whole LONG_PTR.
So yes, this does mean that (Get|Set)WindowLongPtr() will access a different number of bytes depending on whether you are compiling your project for 32bit or 64bit. As such, if you want to read/write a smaller number of bytes, you will have to read/write a whole LONG/_PTR and do some bit shifting as needed.
Even though you can specify an arbitrary byte count for the WNDCLASS/EX::cbWndExtra field, in reality it needs to be large enough to hold at least sizeof(LONG/_PTR) number of bytes at the last byte offset you intend to specify in the nIndex parameter.
This is stated in the GetWindowLong()/SetWindowLong() and GetWindowLongPtr()/SetWindowLongPtr documentations:
nIndex
Type: int
The zero-based offset to the value to be retrieved. Valid values are in the range zero through the number of bytes of extra window memory, minus four; for example, if you specified 12 or more bytes of extra memory, a value of 8 would be an index to the third 32-bit integer.
nIndex
Type: int
The zero-based offset to the value to be retrieved. Valid values are in the range zero through the number of bytes of extra window memory, minus the size of a LONG_PTR.
From the documentation for both Extra Class Memory and Extra Window Memory:
Because extra memory is allocated from the system's local heap, an application should use extra [class or window] memory sparingly. The RegisterClassEx function fails if the amount of extra [class or window] memory requested is greater than 40 bytes. If an application requires more than 40 bytes, it should allocate its own memory and store a pointer to the memory in the extra [class or window] memory.
Related
I'm want to understand what happens when created an empty slice with make([]int, 0). I do this code for test:
emptySlice := make([]int, 0)
fmt.Println(len(emptySlice))
fmt.Println(cap(emptySlice))
fmt.Println(unsafe.Sizeof(emptySlice))
The size and capacity return is obvious, both are 0, but the size of slice is 24 bytes, why?
24 bytes should be 3 int64 right? One internal array for a slice with 24 bytes should be something like: [3]int{}, then why one empty slice have 24 bytes?
If you read the documentation for unsafe.Sizeof, it explains what's going on here:
The size does not include any memory possibly referenced by x. For instance, if x is a slice, Sizeof returns the size of the slice descriptor, not the size of the memory referenced by the slice.
All data types in Go are statically sized. Even though slices have a dynamic number of elements, that cannot be reflected in the size of the data type, because then it would not be static.
The "slice descriptor", as implied by the name, is all the data that describes a slice. That is what is actually stored in a slice variable.
Slices in Go have 3 attributes: The underlying array (memory address), the length of the slice (memory offset), and the capacity of the slice (memory offset). In a 64-bit application, memory addresses and offsets tend to be stored in 64-bit (8-byte) values. That is why you see a size of 24 (= 3 * 8 ) bytes.
unsafe.Sizeof is the size of the object in memory, exactly the same as sizeof in C and C++. See How to get memory size of variable?
A slice has size, but also has the ability to resize, so the maximum resizing ability must also be stored somewhere. But being resizable also means that it can't be a static array but needs to store a pointer to some other (possibly dynamically allocated) array
The whole thing means it needs to store its { begin, end, last valid index } or { begin, size, capacity }. That's a tuple of 3 values which means its in-memory representation is at least 3×8 bytes on 64-bit platforms, unless you want to limit the maximum size and capacity to much smaller than 264 bytes
It's exactly the same situation in many C++ types with the same dynamic sizing capability like std::string or std::vector is also a 24-byte type although on some implementations 8 bytes of padding is added for alignment reasons, resulting in a 32-byte string type. See
C++ sizeof Vector is 24?
Why is sizeof array(type string) 24 bytes with a single space element?
Why is sizeof(string) == 32?
Why is sizeof array(type string) 24 bytes with a single space element?
In fact golang's strings.Builder which is the closest to C++'s std::string has size of 32 bytes. See demo
In my application I have a memory pool. I allocate all memory at startup in the form of a uint64 array (64 bit machine). Then construct objects in this array using placement new. So object 1 starts at position pool[0] object 2 start at position pool[1] so on so forth. Since each object will span at least 64bits or multiples of sizeof(uint64) (if it needs more uint64 slots to allocate).
Am I correct to assume that all memory returned from the pool will be aligned correctly? Since each uint64 in the array will be properly aligned by the compiler. If so, does using uint32 on a 32bit machine in the same manner will work?
You are correct to assume that there will not be any padding. Compilers mostly pack in 2 bytes or 4 bytes boundaries (this can be controlled).
You should verify this on your specific target using __alignof__.
The keyword __alignof__ allows you to inquire about how an object is aligned, or the minimum alignment usually required by a type. Its syntax is just like sizeof.
However, an allocation of 8 bytes, might not be aligned to 64bit, if the array was allocated starting with a 32bit offset address.
You could use aligned_alloc(8, size) to allocate the memory, then cast it to array of uint64.
I have trouble understanding how in say a 32-bit computer byte addressing is achieved:
Is the ram itself byte addressable meaning the first byte has address 0 and the second 1 etc? In this case, wouldn't is take 4 read cycles to read a 32-bit word and waste the width of the data bus?
Or does the ram consist of 32-bit words meaning address 0 points to the first 4 bytes and address 2 points to bytes 5 to 8? In this case I would expect the ram interface to make byte addressing possible (from the cpu's point of view)
Think of RAM as 8 bit wide structure with N entries. N is often the size quoted when referring to memory (256 MB - 256M entries, 2GB - 2G entries etc, B is for bytes). When you access this memory, the smallest unit you can address is one of these entries which is 8 bits (1 byte). Since you can only access it at byte level, we call it byte addressable memory.
Now coming to your question about accessing this memory, we do not just access a byte. Most of the time, memory accesses are sent through caches which are there to reduce memory access latency. Caches store data at a higher granularity than a byte or word, normally it is multiple of words. In doing so, caches explore a property called "locality". Locality means, there is a high chance that we either access this data item or a near by data item very soon. So fetching not just the byte, but all the adjacent bytes is not a waste. Think of it as an investment for future, saves you multiple data fetches that you would have done otherwise.
Memory addresses in RAM start with 0th address and they are accessed using the registers with capacity of 8 bit register or 32 bit registers. Based on these registers the value from specific address is accessed by the CPU. If you really need to understand how it works, you will need to run couple of programs using Assembly language to navigate in the physical memory by reading the values directly using registers and register move commands.
In a byte addressed space with 32bit addressing, it takes up 32bits of memory to reference 8 bits?
So the addressing is the major portion of Memory?
Am I conceptualizing this correctly?
No. It takes 32 bits to reference a contiguous region of any size. If you have a 1 megabyte buffer, you're not going to store a pointer to every byte inside it, you'll just store a pointer to the beginning of it.
I've been doing some work on high memory issues, and I've been doing a lot of heap analysis in windbg, and I was curious what the different columns really mean in "!heap -flt -s xxxx" command.
I read What do the 'size' numbers mean in the windbg !heap output?, and I looked in my "Windows Internals" book, but I still had a bunch of questions. So the columns and my questions are below.
**HEAP_ENTRY** - What does this pointer really point to? How is it different than UserPtr?
**Size** - What does this size mean? How is it different than UserSize?
**Prev** - This just appears to be the negative offset to get to the previous heap entry. Still not sure exactly how it's used.
**Flags** - Is there any documentation on these flags?
**UserPtr** - What is the user pointer? In all cases I've seen it's always 8 bytes higher than the HEAP_ENTRY, but I don't really know what it points to.
**UserSize** - This appears to be the size of the actual allocation.
**state** - This just tells you what state of this heap entry is (free, busy, etc....)
Example:
HEAP_ENTRY Size Prev Flags UserPtr UserSize - state
0015eeb0 0044 0000 [07] 0015eeb8 00204 - (busy)
HEAP_ENTRY
Heaps store allocated blocks in contiguous Segments of memory, each allocated block starts with a 8-bytes header followed by the actual allocated data. The HEAP_ENTRY column is the address of the beginning of the header of the allocated block.
Size
The heap manager handles blocks in multiple of 8 bytes. The column is the number of 8 bytes chunk allocated. In your sample, 0044 means that the block takes 0x220 bytes (0x44*8).
Prev
Multiply per 8 to have the negative offset in bytes to the previous heap block.
Flags
This is a bitmask that encodes the following information
0x01 - HEAP_ENTRY_BUSY
0x02 - HEAP_ENTRY_EXTRA_PRESENT
0x04 - HEAP_ENTRY_FILL_PATTERN
0x08 - HEAP_ENTRY_VIRTUAL_ALLOC
0x10 - HEAP_ENTRY_LAST_ENTRY
UserPtr
This is the pointer returned to the application by the HeapAlloc (callbed by malloc/new) function. Since the header is always 8 bytes long, it is always HEAP_ENTRY +8.
UserSize
This is the size passed the HeapAlloc function.
state
This is a decoding of the Flags column, telling if the entry is busy, freed, last of its segment, …
Be aware that in Windows 7/2008 R2, heaps are by default using a front-end named LFH (Low fragmented heap) that uses the default heap manager to allocate chunks in which it dispatched user allocated data. For these heaps, UserPtr and UserSize will not point to real user data.
The output of !heap -s displays which heaps are LFH enabled.
From looking at the !heap documentation in the Debugging Tools for Windows help file and the heap docs on MSDN and a great excerpt from Advanced Windows Debugging, here's what I've been able to put together:
HEAP_ENTRY: pointer to entry within the heap. As you found, there is an 8 byte header which contains the data for the HEAP_ENTRY structure. The size of the HEAP_ENTRY structure is 8 bytes which defines the "heap granularity" size. This is used for determining the...
SIZE: size of the entry in terms of the granularity (i.e. the allocation size / 8)
FLAGS: these are defined in winbase.h with explanations found the in MSDN link.
USERPTR: the actual pointer to the allocated (or freed) object
Well, the main difference between HEAP_ENTRY and UserPtr is the due to the fact that heaps have to be indexed, allocated, filled with metadata (like the allocated length made available to user)... otherwise, how could you free(p) something without providing how many bytes were allocated? Same thing with the two size fields: one thing is how big the structure indexing the heap is, one thing is how big is the memory region made available to the user.
The FLAGS, in turn, basically specify which properties of the allocated memory block, if it is committed or just reserved, and, I guess, used by the kernel to rearrange or share memory regions if needed (but as nithins specifies they are documented in MSDN).
The PREV ptr is used to keep track of all the allocated regions and the first pointer is stored in the PEB structure so both user-space and kernel-space code is aware of the allocated heap pools.