gcc 4.3.4 bug with structure size? - gcc

What's wrong with this code when I compile it with -DPORTABLE?
#include <stdio.h>
#include <stdlib.h>
typedef struct {
unsigned char data[11];
#ifdef PORTABLE
unsigned long intv;
#else
unsigned char intv[4];
#endif
} struct1;
int main() {
struct1 s;
fprintf(stderr,"sizeof(s.data) = %d\n",sizeof(s.data));
fprintf(stderr,"sizeof(s.intv) = %d\n",sizeof(s.intv));
fprintf(stderr,"sizeof(s) = %d\n",sizeof(s));
return 0;
}
The output I get on 32 bit GCC:
$ gcc -o struct struct.c -DPORTABLE
$ ./struct
sizeof(s.data) = 11
sizeof(s.intv) = 4
sizeof(s) = 16
$ gcc -o struct struct.c
$ ./struct
sizeof(s.data) = 11
sizeof(s.intv) = 4
sizeof(s) = 15
Where did the extra byte came from?
I always thought 11+4 = 15 not 16.

Nothing's wrong with the code; those sizes are correct. The compiler may add padding to structs at its discretion. The size of a struct is only guaranteed to be large enough to hold its elements, so adding the sizes of its elements is not a reliable way to get the size of the struct.
Such padding can be helpful in keeping elements and the structs themselves aligned to specific boundaries, both to avoid alignment errors (perhaps why it's enabled with -DPORTABLE) and as a speed optimization, as Als points out.

This is due to structure padding.
Compilers are free to add extra padding bytes to structures to optimize the access time.
This is the reason You should always use sizeof operator and never manually calculate size of structures.

It's called alignment. That's especially added padding at end of structures to decrease cache misses. If you want to disable it, you can use something like that:
#pragma pack(push) /* push current alignment to stack */
#pragma pack(1) /* set alignment to 1 byte boundary */
typedef struct {
unsigned char data[11];
#ifdef PORTABLE
unsigned long intv;
#else
unsigned char intv[4];
#endif
} struct1;
#pragma pack(pop) /* restore original alignment from stack */

Related

Use field of union in intialiser of another variable

I am building records for unit testing a software module. Record data is serialised before sending it to the UUT.
The records contain bitfields, so I would like to build serialised records using these same bitfields at compile-time (to prevent having to account for little- and big-endian issues and where the bits in bitfield go) and use a union to access the (serialised) data. I have to calculate a checksum over the record, so I need the bitfields as bytes to do so.
My attempt so far is:
/* defines for 64 bit valid record */
#define REC3_ID EEID_ARRAY_FIRST
#define REC3_SIZE 1
#define REC3_INDEX 248
#define REC3_SI0 MAKE_SIZE_INDEX0(REC3_SIZE,REC3_INDEX)
#define REC3_SI1 MAKE_SIZE_INDEX1(REC3_SIZE,REC3_INDEX)
#define REC3_VALUE0 0xf2
#define REC3_VALUE1 0x4f
#define REC3_VALUE2 0xb8
#define REC3_VALUE3 0xa0
#define REC3_DATA \
MAKE_CHKSUM7(REC3_ID,REC3_SI0,REC3_SI1,REC3_VALUE0,REC3_VALUE1,REC3_VALUE2,REC3_VALUE3),\
REC3_ID,REC3_SI0,REC3_SI1,REC3_VALUE0,REC3_VALUE1,REC3_VALUE2,REC3_VALUE3
#define CHKSUM_SEED (0x2a)
#define MAKE_CHKSUM7(v0,v1,v2,v3,v4,v5,v6) (0x100-(((v0)+(v1)+(v2)+(v3)+(v4)+(v5)+(v6)+CHKSUM_SEED)%0x100))
typedef union
{
uint8_t si[2];
struct
{
uint16_t s: 6;
uint16_t i: 10;
} b;
} si_t;
MAKE_SIZE_INDEX0(size,index) ((si_t){.b.s=size,.b.i=index}).si[0]
MAKE_SIZE_INDEX1(size,index) ((si_t){.b.s=size,.b.i=index}).si[1]
static uint8_t rec3[] = {REC3_DATA};
The problem lies with the macros MAKE_SIZE_INDEX0 and MAKE_SIZE_INDEX1. I can't get them to compile (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.11))
The problem can be simplified to:
uint8_t rec[] = {0x12, (si_t){.b.s=4,.b.i=8}.si[0], (si_t){.b.s=4,.b.i=8}.si[1], 0x34};
But that results in error:
error: initializer element is not constant
I know I can create my records at run-time, but I wondered whether it is possible to let the preprocessor handle it.
My alternative is something like:
#if defined (TGT_ARCHITECTURE_x86_64)
#define MAKE_SIZE_INDEX0(size,index) (((size)&0x3f)+(((index)<<6)&0xc0))
#define MAKE_SIZE_INDEX1(size,index) ((index>>2)&0xff)
#endif
But this depends on whether the target is little-endian or big-endian and how it stores bitfields.
static uint8_t rec3[] = { (si_t){.b.s=4,.b.i=8}.si[0] };
Variables with static storage duration must be initialized only using a static initializer - it has to be a constant expression. There is a list of what is allowed in a constant expression - using array subscript operator on a array embedded in a compound literal is not allowed in a constant expression. It's basically the same as you can't do static int a[] = {1, 2}; static int b = a[1];
On a side note, the standard says that implementations are allowed to accept custom forms of constant expression. So the code may happen to work with a different compiler and even a different gcc version (as with newest gcc versions you may initialize variable with static storage duration with const qualived variable, which is an extension).
The compiler errors with "initializer element is not constant", as the element used to initialize variable with static storage duration is not a constant expression.
Using bit-fields to extract a bit-mask of a variable is compiler dependent, compiler options dependent (gcc storage layout) and shouldn't be used in portable code. Compiler is free to reorder the bitfields in your struct and is free to add padding between bit fields members. As advertised on stackoverflow many, many times, use bitmasks - they work every time.

What is the modern C++ way of having a constexpr const pointer to a volatile memory location for embedded applications?

In building libraries for controlling hardware on embedded microprocessors, a common task is manipulating bits at specific memory locations for controlling hardware features.
In AVR processors, Atmel (now Microchip) provides macros that expand to something like this:
#define PORTA (*(volatile uint8_t *)(0x25))
Which enables things like:
PORTA |= 1;
Now in C++11 (and newer), it is desirable to replace almost any usage of #define with constexpr.
In older versions of the GCC C++ compiler (4.9.2), the following compiled:
#include <avr/io.h>
constexpr volatile uint8_t *const PortA = &PORTA;
In version 8.2.0, the above does not compile and gives errors:
error: 'reinterpret_cast<volatile uint8_t* {aka volatile unsigned char*}>(37)' is not a constant expression
I'm not looking for explanations of why you cannot use reinterpret_cast inside a constexpr context or why integer to pointer conversion is illegal.
What is the correct way to have a constexpr pointer to volatile memory in modern C++?
I've seen suggestions of storing the memory address of PORTA in a constexpr uintptr_t and then reinterprect_casting that to volatile uint8_t * const at runtime for bit manipulation.
For instance, this works and even compiles to a single sbi instruction in avr-gcc as expected.
#include <stdint.h>
constexpr uintptr_t PortA = 0x25;
void set() { *((volatile uint8_t *)(PortA)) |= 1; }
However it takes a decent amount of ugly boilerplate to use PortA as the pointer it is intended to be.
This also has the problem that it seem to be impossible to use the PORTA macro directly. We're instead forced to hard-code the memory address 0x25 which breaks certain desirable portability features.
It feels like I'm missing something obvious buy my searches have not yielded anything fruitful.
For instance, this feels like an "address constant expression", but that seems to relate to referring to statically allocated const values like which is not quite what I want.
const char str[] = "FooBar";
constexpr const char * x = str + 2;
You can't create a constexpr pointer initialized by a non-constant expression. However, you can create a static const pointer:
static uint8_t volatile* const PortA = &PORTA;
Or, even better, a static reference:
static uint8_t volatile& PortA = PORTA;

alignment in user defined sections in gcc

It seems that top level objects in gcc targeting x86 that are >= 32 bytes automatically get 32 byte alignment. This may be nice for performance, but I'm collecting an array of thingies from all my object files in a user-defined section, and the extra alignment gaps play havoc with this array. Is there any way to prevent this object alignment?
To clarify, I have a low-aligned struct, and different object files define
data in the form of an array of that struct in a user defined section, with
the purpose to make one application wide array.
As soon as one of those arrays is >= 32, the object alignment and with that the section alignment is pushed to 32 and when the linker concatenates the separate sections from the object files into the executable, it creates alignment fillers at the module boundaries in that section.
The following program illustrates a possible solution, assuming GCC
extensions are acceptable to you:
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#define ALIGNMENT // __attribute__ ((aligned (8)))
struct A {
char arr[40];
} ;
struct A a __attribute__ ((section ("my_data"))) ALIGNMENT = {{'a'}};
struct A b __attribute__ ((section ("my_data"))) ALIGNMENT = {{'b'}};
struct A c __attribute__ ((section ("my_data"))) ALIGNMENT = {{'c'}};
int main(int argc, char **argv)
{
assert(sizeof(struct A) == 40);
printf("%c\n",a.arr[0]);
printf("%c\n",b.arr[0]);
printf("%c\n",c.arr[0]);
printf("%lu\n",(unsigned long)(&a));
printf("%lu\n",(unsigned long)(&b));
printf("%lu\n",(unsigned long)(&c));
return 0;
}
My output is:
a
b
c
6295616
6295680
6295744
Note that in my (64-bit) executable each of the three 40-byte structures
is 64-byte aligned.
Now uncomment // __attribute__ ((aligned (8))), rebuild and rerun. My
output then is:
a
b
c
6295616
6295656
6295696
Now the structures are 8-byte aligned, without gaps.

MinGW and packed struct alignment using C++11

For the below structure, the actual (with no padding) size of the structure is 54. On a 64-bit (Windows 7) machine with MinGW (GCC) 4.8.1 x86_64, I get sizeof(BMPHeader) as 56, which is understandable. As per the requirement of the BMP file format, the structure should've no padding. I've three options (priority ordered):
C++11's alignas(1)
struct __attribute__ ((packed)) BMPHeader
#pragma pack(1)
However the last option (with least priority) alone seems to work giving me 54. Is this a bug in the compiler or I've completely mistook something here? The SSCCE
#include <iostream>
struct alignas(1) BMPHeader
{
// BMP header
uint16_t magic;
uint32_t fileSize;
uint32_t reserved;
uint32_t dataOffset;
// DIB header
uint32_t dibHeaderLength;
uint32_t width;
uint32_t height;
uint16_t numColourPlanes;
uint16_t bitsPerPixels;
uint32_t biBitFields;
uint32_t dataSize;
uint32_t physicalWidth;
uint32_t physicalHeight;
uint32_t numPaletteColours;
uint32_t numImportantColours;
};
int main()
{
std::cout << sizeof(BMPHeader) << std::endl;
}
alignas cannot be used in this situation as Martinho notes, since we're asking for an alignment less stricter than the natural alignment of the struct. This is specified in the standard under dcl.align (emphasised the relevant part):
When multiple alignment-specifiers are specified for an entity, the alignment requirement shall be set to the strictest specified alignment.
The combined effect of all alignment-specifiers in a declaration shall not specify an alignment that is less strict than the alignment that would be required for the entity being declared if all alignment-specifiers were omitted (including those in other declarations).
The alignment of BMPHeader as returned by alignof(BMPHeader) is 4 and thus any alignment less stricter (less wider) than that wouldn't be honoured.
__attribute__ ((packed)) is certainly the right way when using GCC as specified in its manual to make a struct tightly packed. However, this doesn't work due to a bug in MinGW and works fine when using GCC.
So currently the only way in MinGW is to make-do with #pragma pack(1). See #pragma pack effect for more details on this method.
See Also
Cross-platform ALIGN(x) macro?

KbdLayerDescriptor pVkToWcharTable returns NULL on Win64

I am running out of ideas here. I have a piece of code adapted from http://thetechnofreak.com/technofreak/keylogger-visual-c/ to convert keycodes to unicode chars. It works fine in all situations except when you try to run the 32-bit version from 64-bit Windows. For some reason pKbd->pVkToWcharTable keeps returning NULL. I have tried __ptr64 as well as explicitly specifying SysWOW64 and System32 for the kbd dll path. I have found several items across the internet referring to this exact or very similar problem but I cannot seem to get any of the solutions to work (See: KbdLayerDescriptor returns NULL at 64bit architecture) The following is my test code that was compiled with mingw-32 on Windows XP (gcc -std=c99 Wow64Test.c) and then executed on Windows 7 64-bit. On Windows XP I am getting a valid pointer, however on Windows 7 I am getting NULL.
***Update: So it looks like the problems I am having are due to mingw not implementing __ptr64 correctly as the sizeof operation gives 4 bytes instead of the 8 bytes given by visual studio. So the real solution would be figuring out a way to make the size of KBD_LONG_POINTER dynamic or at least 64-bits but I am not sure if thats possible. Any ideas?
#include <windows.h>
#include <stdio.h>
#define KBD_LONG_POINTER __ptr64
//#define KBD_LONG_POINTER
typedef struct {
BYTE Vk;
BYTE ModBits;
} VK_TO_BIT, *KBD_LONG_POINTER PVK_TO_BIT;
typedef struct {
PVK_TO_BIT pVkToBit;
WORD wMaxModBits;
BYTE ModNumber[];
} MODIFIERS, *KBD_LONG_POINTER PMODIFIERS;
typedef struct _VK_TO_WCHARS1 {
BYTE VirtualKey;
BYTE Attributes;
WCHAR wch[1];
} VK_TO_WCHARS1, *KBD_LONG_POINTER PVK_TO_WCHARS1;
typedef struct _VK_TO_WCHAR_TABLE {
PVK_TO_WCHARS1 pVkToWchars;
BYTE nModifications;
BYTE cbSize;
} VK_TO_WCHAR_TABLE, *KBD_LONG_POINTER PVK_TO_WCHAR_TABLE;
typedef struct {
DWORD dwBoth;
WCHAR wchComposed;
USHORT uFlags;
} DEADKEY, *KBD_LONG_POINTER PDEADKEY;
typedef struct {
BYTE vsc;
WCHAR *KBD_LONG_POINTER pwsz;
} VSC_LPWSTR, *KBD_LONG_POINTER PVSC_LPWSTR;
typedef struct _VSC_VK {
BYTE Vsc;
USHORT Vk;
} VSC_VK, *KBD_LONG_POINTER PVSC_VK;
typedef struct _LIGATURE1 {
BYTE VirtualKey;
WORD ModificationNumber;
WCHAR wch[1];
} LIGATURE1, *KBD_LONG_POINTER PLIGATURE1;
typedef struct tagKbdLayer {
PMODIFIERS pCharModifiers;
PVK_TO_WCHAR_TABLE pVkToWcharTable;
PDEADKEY pDeadKey;
PVSC_LPWSTR pKeyNames;
PVSC_LPWSTR pKeyNamesExt;
WCHAR *KBD_LONG_POINTER *KBD_LONG_POINTER pKeyNamesDead;
USHORT *KBD_LONG_POINTER pusVSCtoVK;
BYTE bMaxVSCtoVK;
PVSC_VK pVSCtoVK_E0;
PVSC_VK pVSCtoVK_E1;
DWORD fLocaleFlags;
BYTE nLgMax;
BYTE cbLgEntry;
PLIGATURE1 pLigature;
DWORD dwType;
DWORD dwSubType;
} KBDTABLES, *KBD_LONG_POINTER PKBDTABLES;
typedef PKBDTABLES(CALLBACK *KbdLayerDescriptor) (VOID);
int main() {
PKBDTABLES pKbd;
HINSTANCE kbdLibrary = NULL;
//"C:\\WINDOWS\\SysWOW64\\KBDUS.DLL"
//"C:\\WINDOWS\\System32\\KBDUS.DLL"
kbdLibrary = LoadLibrary("C:\\WINDOWS\\SysWOW64\\KBDUS.DLL");
KbdLayerDescriptor pKbdLayerDescriptor = (KbdLayerDescriptor) GetProcAddress(kbdLibrary, "KbdLayerDescriptor");
if(pKbdLayerDescriptor != NULL) {
pKbd = pKbdLayerDescriptor();
printf("Is Null? %d 0x%X\n", sizeof(pKbd->pVkToWcharTable), pKbd->pVkToWcharTable);
}
FreeLibrary(kbdLibrary);
kbdLibrary = NULL;
}
It might be late for you, but here is a solution for anyone having the same problem. This demo and incomplete explanation helps, but only works in Visual Studio:
http://www.codeproject.com/Articles/439275/Loading-keyboard-layout-KbdLayerDescriptor-in-32-6
The pointers in the structures in kbd.h all have the KBD_LONG_POINTER macro, which is defined as *__ptr64* on 64 bit operating systems. In Visual Studio, this makes the pointers take up 8 bytes instead of the usual 4 of 32 bit programs. Unfortunately in MinGW, *__ptr64* is defined to not do anything.
As written in the linked explanation, the KbdLayerDescriptor function returns pointers differently on 32 bit and 64 bit Windows. The size of pointers seem to depend on the operating system and not on the running program. Actually, the pointers are still 4 bytes on a 64 bit operating system for a 32 bit program, but in VS, the __ptr64 keyword lies that they are not.
For example some structures look like this in kbd.h:
typedef struct {
BYTE Vk;
BYTE ModBits;
} VK_TO_BIT, *KBD_LONG_POINTER PVK_TO_BIT;
typedef struct {
PVK_TO_BIT pVkToBit;
WORD wMaxModBits;
BYTE ModNumber[];
} MODIFIERS, *KBD_LONG_POINTER PMODIFIERS;
This can't work neither in MinGW nor in VS for 32 bit programs on 64 bit Windows. Because the pVkToBit member in MODIFIERS is only 4 bytes without __ptr64. The solution is to forget about KBD_LONG_POINTER (you could even remove them all) and define structures similar to the above. i.e. :
struct VK_TO_BIT64
{
BYTE Vk;
BYTE ModBits;
};
struct MODIFIERS64
{
VK_TO_BIT64 *pVkToBit;
int _align1;
WORD wMaxModBits;
BYTE ModNumber[];
};
(You could use VK_TO_BIT and not define your own VK_TO_BIT64, as they are the same, but having separate definitions help understanding what's going on.)
The member pVkToBit still takes up 4 bytes, but KbdLayerDescriptor pads pointers to 8 bytes on a 64 bit OS, so we have to insert some padding (int _align1).
You have to do the same thing for the other structures in kbd.h. For example this will replace KBDTABLES:
struct WCHARARRAY64
{
WCHAR *str;
int _align1;
};
struct KBDTABLES64
{
MODIFIERS64 *pCharModifiers;
int _align1;
VK_TO_WCHAR_TABLE64 *pVkToWcharTable;
int _align2;
DEADKEY64 *pDeadKey;
int _align3;
VSC_LPWSTR64 *pKeyNames;
int _align4;
VSC_LPWSTR64 *pKeyNamesExt;
int _align5;
WCHARARRAY64 *pKeyNamesDead;
int _align6;
USHORT *pusVSCtoVK;
int _align7;
BYTE bMaxVSCtoVK;
int _align8;
VSC_VK64 *pVSCtoVK_E0;
int _align9;
VSC_VK64 *pVSCtoVK_E1;
int _align10;
DWORD fLocaleFlags;
byte nLgMax;
byte cbLgEntry;
LIGATURE64_1 *pLigature;
int _align11;
DWORD dwType;
DWORD dwSubType;
};
(Notice that the _align8 member does not come after a pointer.)
To use this all, you have to check whether you are running on 64 bit windows with this: http://msdn.microsoft.com/en-us/library/ms684139%28v=vs.85%29.aspx
If not, use the original structures from kbd.h, because the pointers behave correctly. They take up 4 bytes. In case the program is running on a 64 bit OS, use the structures you created. You can achieve it with this:
typedef __int64 (CALLBACK *LayerDescriptor64)(); // Result should be cast to KBDTABLES64.
typedef PKBDTABLES (CALLBACK *LayerDescriptor)(); // This is used on 32 bit OS.
static PKBDTABLES kbdtables = NULL;
static KBDTABLES64 *kbdtables64 = NULL;
And in some initialization function:
if (WindowsIs64Bit()) // Your function that checks the OS version.
{
LayerDescriptor64 KbdLayerDescriptor = (LayerDescriptor64)GetProcAddress(kbdLibrary, "KbdLayerDescriptor");
if (KbdLayerDescriptor != NULL)
kbdtables64 = (KBDTABLES64*)KbdLayerDescriptor();
else
kbdtables64 = NULL;
}
else
{
LayerDescriptor KbdLayerDescriptor = (LayerDescriptor)GetProcAddress(kbdLibrary, "KbdLayerDescriptor");
if (KbdLayerDescriptor != NULL)
kbdtables = KbdLayerDescriptor();
else
kbdtables = NULL;
}
This solution does not use __ptr64 at all, and works both in VS and MinGW. The things you have to watch out for are:
The structures should be aligned on 8 byte boundaries. (This is the default in current VS or MinGW, at least for C++.)
Don't define KBD_LONG_POINTER to __ptr64, or remove it from everywhere. Although you are better off not changing kbd.h.
Understand how alignment for structure members work. (I have compiled this as C++ and not C. I'm not sure whether alignment rules would be any different for C.)
Use the correct variable (either kbdtables or kbdtables64) depending on the OS.
This solution is obviously not needed when compiling a 64 bit program.

Resources