alignment in user defined sections in gcc - gcc

It seems that top level objects in gcc targeting x86 that are >= 32 bytes automatically get 32 byte alignment. This may be nice for performance, but I'm collecting an array of thingies from all my object files in a user-defined section, and the extra alignment gaps play havoc with this array. Is there any way to prevent this object alignment?
To clarify, I have a low-aligned struct, and different object files define
data in the form of an array of that struct in a user defined section, with
the purpose to make one application wide array.
As soon as one of those arrays is >= 32, the object alignment and with that the section alignment is pushed to 32 and when the linker concatenates the separate sections from the object files into the executable, it creates alignment fillers at the module boundaries in that section.

The following program illustrates a possible solution, assuming GCC
extensions are acceptable to you:
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#define ALIGNMENT // __attribute__ ((aligned (8)))
struct A {
char arr[40];
} ;
struct A a __attribute__ ((section ("my_data"))) ALIGNMENT = {{'a'}};
struct A b __attribute__ ((section ("my_data"))) ALIGNMENT = {{'b'}};
struct A c __attribute__ ((section ("my_data"))) ALIGNMENT = {{'c'}};
int main(int argc, char **argv)
{
assert(sizeof(struct A) == 40);
printf("%c\n",a.arr[0]);
printf("%c\n",b.arr[0]);
printf("%c\n",c.arr[0]);
printf("%lu\n",(unsigned long)(&a));
printf("%lu\n",(unsigned long)(&b));
printf("%lu\n",(unsigned long)(&c));
return 0;
}
My output is:
a
b
c
6295616
6295680
6295744
Note that in my (64-bit) executable each of the three 40-byte structures
is 64-byte aligned.
Now uncomment // __attribute__ ((aligned (8))), rebuild and rerun. My
output then is:
a
b
c
6295616
6295656
6295696
Now the structures are 8-byte aligned, without gaps.

Related

Unexpected value appears on stack when attempting buffer overflow

I am trying to learn more about cyber security, in this case about buffer overflows. I have a simple code that I want to change flow of:
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
void win()
{
printf("code flow successfully changed\n");
}
int main(int argc, char **argv)
{
volatile int (*fp)();
char buffer[64];
fp = 0;
gets(buffer);
if(fp) {
printf("calling function pointer, jumping to 0x%08x\n", fp);
fp();
}
}
By using some tools I have determined that function pointer (fp) gets it value updated after 72 characters have entered the buffer. The function win() is located at value 0xe5894855 so after 72 characters I need to provide that value to buffer for it to jump to the desired function.
However I am facing this issue:
By putting Python3's print("A"*18*4 + "UH" + "\x89" + "\xe5") into input of given C code, I should be getting desired value 0xe5894855 in section marked with red. But instead, I am getting highlighted malformed hex from somewhere. (89 is getting extra C2 and incorrect e5 value is overflowing to next part of stack) (value in those parts of stack are zero initially, but changed into that once overflow is attempted).
Why is this happening? Am I putting hex values into C program incorrectly?
Edit: Still have not figured out why passing hex through python did not work, but I found a different method, by using Perl: perl -e 'print "A"x4x18 . "\x55\x48\x89\xe5"', which did work, and address I needed to jump to was also incorrect (which I also fixed)

Use field of union in intialiser of another variable

I am building records for unit testing a software module. Record data is serialised before sending it to the UUT.
The records contain bitfields, so I would like to build serialised records using these same bitfields at compile-time (to prevent having to account for little- and big-endian issues and where the bits in bitfield go) and use a union to access the (serialised) data. I have to calculate a checksum over the record, so I need the bitfields as bytes to do so.
My attempt so far is:
/* defines for 64 bit valid record */
#define REC3_ID EEID_ARRAY_FIRST
#define REC3_SIZE 1
#define REC3_INDEX 248
#define REC3_SI0 MAKE_SIZE_INDEX0(REC3_SIZE,REC3_INDEX)
#define REC3_SI1 MAKE_SIZE_INDEX1(REC3_SIZE,REC3_INDEX)
#define REC3_VALUE0 0xf2
#define REC3_VALUE1 0x4f
#define REC3_VALUE2 0xb8
#define REC3_VALUE3 0xa0
#define REC3_DATA \
MAKE_CHKSUM7(REC3_ID,REC3_SI0,REC3_SI1,REC3_VALUE0,REC3_VALUE1,REC3_VALUE2,REC3_VALUE3),\
REC3_ID,REC3_SI0,REC3_SI1,REC3_VALUE0,REC3_VALUE1,REC3_VALUE2,REC3_VALUE3
#define CHKSUM_SEED (0x2a)
#define MAKE_CHKSUM7(v0,v1,v2,v3,v4,v5,v6) (0x100-(((v0)+(v1)+(v2)+(v3)+(v4)+(v5)+(v6)+CHKSUM_SEED)%0x100))
typedef union
{
uint8_t si[2];
struct
{
uint16_t s: 6;
uint16_t i: 10;
} b;
} si_t;
MAKE_SIZE_INDEX0(size,index) ((si_t){.b.s=size,.b.i=index}).si[0]
MAKE_SIZE_INDEX1(size,index) ((si_t){.b.s=size,.b.i=index}).si[1]
static uint8_t rec3[] = {REC3_DATA};
The problem lies with the macros MAKE_SIZE_INDEX0 and MAKE_SIZE_INDEX1. I can't get them to compile (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.11))
The problem can be simplified to:
uint8_t rec[] = {0x12, (si_t){.b.s=4,.b.i=8}.si[0], (si_t){.b.s=4,.b.i=8}.si[1], 0x34};
But that results in error:
error: initializer element is not constant
I know I can create my records at run-time, but I wondered whether it is possible to let the preprocessor handle it.
My alternative is something like:
#if defined (TGT_ARCHITECTURE_x86_64)
#define MAKE_SIZE_INDEX0(size,index) (((size)&0x3f)+(((index)<<6)&0xc0))
#define MAKE_SIZE_INDEX1(size,index) ((index>>2)&0xff)
#endif
But this depends on whether the target is little-endian or big-endian and how it stores bitfields.
static uint8_t rec3[] = { (si_t){.b.s=4,.b.i=8}.si[0] };
Variables with static storage duration must be initialized only using a static initializer - it has to be a constant expression. There is a list of what is allowed in a constant expression - using array subscript operator on a array embedded in a compound literal is not allowed in a constant expression. It's basically the same as you can't do static int a[] = {1, 2}; static int b = a[1];
On a side note, the standard says that implementations are allowed to accept custom forms of constant expression. So the code may happen to work with a different compiler and even a different gcc version (as with newest gcc versions you may initialize variable with static storage duration with const qualived variable, which is an extension).
The compiler errors with "initializer element is not constant", as the element used to initialize variable with static storage duration is not a constant expression.
Using bit-fields to extract a bit-mask of a variable is compiler dependent, compiler options dependent (gcc storage layout) and shouldn't be used in portable code. Compiler is free to reorder the bitfields in your struct and is free to add padding between bit fields members. As advertised on stackoverflow many, many times, use bitmasks - they work every time.

MinGW and packed struct alignment using C++11

For the below structure, the actual (with no padding) size of the structure is 54. On a 64-bit (Windows 7) machine with MinGW (GCC) 4.8.1 x86_64, I get sizeof(BMPHeader) as 56, which is understandable. As per the requirement of the BMP file format, the structure should've no padding. I've three options (priority ordered):
C++11's alignas(1)
struct __attribute__ ((packed)) BMPHeader
#pragma pack(1)
However the last option (with least priority) alone seems to work giving me 54. Is this a bug in the compiler or I've completely mistook something here? The SSCCE
#include <iostream>
struct alignas(1) BMPHeader
{
// BMP header
uint16_t magic;
uint32_t fileSize;
uint32_t reserved;
uint32_t dataOffset;
// DIB header
uint32_t dibHeaderLength;
uint32_t width;
uint32_t height;
uint16_t numColourPlanes;
uint16_t bitsPerPixels;
uint32_t biBitFields;
uint32_t dataSize;
uint32_t physicalWidth;
uint32_t physicalHeight;
uint32_t numPaletteColours;
uint32_t numImportantColours;
};
int main()
{
std::cout << sizeof(BMPHeader) << std::endl;
}
alignas cannot be used in this situation as Martinho notes, since we're asking for an alignment less stricter than the natural alignment of the struct. This is specified in the standard under dcl.align (emphasised the relevant part):
When multiple alignment-specifiers are specified for an entity, the alignment requirement shall be set to the strictest specified alignment.
The combined effect of all alignment-specifiers in a declaration shall not specify an alignment that is less strict than the alignment that would be required for the entity being declared if all alignment-specifiers were omitted (including those in other declarations).
The alignment of BMPHeader as returned by alignof(BMPHeader) is 4 and thus any alignment less stricter (less wider) than that wouldn't be honoured.
__attribute__ ((packed)) is certainly the right way when using GCC as specified in its manual to make a struct tightly packed. However, this doesn't work due to a bug in MinGW and works fine when using GCC.
So currently the only way in MinGW is to make-do with #pragma pack(1). See #pragma pack effect for more details on this method.
See Also
Cross-platform ALIGN(x) macro?

gcc 4.3.4 bug with structure size?

What's wrong with this code when I compile it with -DPORTABLE?
#include <stdio.h>
#include <stdlib.h>
typedef struct {
unsigned char data[11];
#ifdef PORTABLE
unsigned long intv;
#else
unsigned char intv[4];
#endif
} struct1;
int main() {
struct1 s;
fprintf(stderr,"sizeof(s.data) = %d\n",sizeof(s.data));
fprintf(stderr,"sizeof(s.intv) = %d\n",sizeof(s.intv));
fprintf(stderr,"sizeof(s) = %d\n",sizeof(s));
return 0;
}
The output I get on 32 bit GCC:
$ gcc -o struct struct.c -DPORTABLE
$ ./struct
sizeof(s.data) = 11
sizeof(s.intv) = 4
sizeof(s) = 16
$ gcc -o struct struct.c
$ ./struct
sizeof(s.data) = 11
sizeof(s.intv) = 4
sizeof(s) = 15
Where did the extra byte came from?
I always thought 11+4 = 15 not 16.
Nothing's wrong with the code; those sizes are correct. The compiler may add padding to structs at its discretion. The size of a struct is only guaranteed to be large enough to hold its elements, so adding the sizes of its elements is not a reliable way to get the size of the struct.
Such padding can be helpful in keeping elements and the structs themselves aligned to specific boundaries, both to avoid alignment errors (perhaps why it's enabled with -DPORTABLE) and as a speed optimization, as Als points out.
This is due to structure padding.
Compilers are free to add extra padding bytes to structures to optimize the access time.
This is the reason You should always use sizeof operator and never manually calculate size of structures.
It's called alignment. That's especially added padding at end of structures to decrease cache misses. If you want to disable it, you can use something like that:
#pragma pack(push) /* push current alignment to stack */
#pragma pack(1) /* set alignment to 1 byte boundary */
typedef struct {
unsigned char data[11];
#ifdef PORTABLE
unsigned long intv;
#else
unsigned char intv[4];
#endif
} struct1;
#pragma pack(pop) /* restore original alignment from stack */

Preventing recursive C #include

I roughly understand the rules with what #include does with the C preprocessor, but I don't understand it completely. Right now, I have two header files, Move.h and Board.h that both typedef their respective type (Move and Board). In both header files, I need to reference the type defined in the other header file.
Right now I have #include "Move.h" in Board.h and #include "Board.h" in Move.h. When I compile though, gcc flips out and gives me a long (what looks like infinite recursive) error message flipping between Move.h and Board.h.
How do I include these files so that I'm not recursively including indefinitely?
You need to look into forward declarations, you have created an infinite loops of includes, forward declarations are the proper solution.
Here's an example:
Move.h
#ifndef MOVE_H_
#define MOVE_H_
struct board; /* forward declaration */
struct move {
struct board *m_board; /* note it's a pointer so the compiler doesn't
* need the full definition of struct board yet...
* make sure you set it to something!*/
};
#endif
Board.h
#ifndef BOARD_H_
#define BOARD_H_
#include "Move.h"
struct board {
struct move m_move; /* one of the two can be a full definition */
};
#endif
main.c
#include "Board.h"
int main() { ... }
Note: whenever you create a "Board", you will need to do something like this (there are a few ways, here's an example):
struct board *b = malloc(sizeof(struct board));
b->m_move.m_board = b; /* make the move's board point
* to the board it's associated with */
Include guards would be part of the solution to this issue.
Example from wikipedia:
#ifndef GRANDFATHER_H
#define GRANDFATHER_H
struct foo {
int member;
};
#endif
http://en.wikipedia.org/wiki/Include_guard
The other part as noted by several others is forward referencing. (http://en.wikipedia.org/wiki/Forward_Reference)
You can partially declare one of the structures above the other one like so:
#ifndef GRANDFATHER_H
#define GRANDFATHER_H
struct bar;
struct foo {
int member;
};
#endif
Like so:
//Board.h
#ifndef BOARD_H
#define BOARD_H
strunct move_t; //forward declaration
typedef struct move_t Move;
//...
#endif //BOARD_H
//Move.h
#ifndef MOVE_H
#define MOVE_H
#include "Move.h"
typedef struct board_t Board;
//...
#endif //MOVE_H
This way Board.h can be compiled without dependency on move.h and you can include board.h from move.h to make its content available there.
First, you seem to lack include guards in your .h files, so you're including them recursively. That is bad.
Second, you can do a forward declaration. In Move.h:
/* Include guard to make sure your header files are idempotent */
#ifndef H_MOVE_
#define H_MOVE_
#include "Board.h"
/* Now you can use struct Board */
struct Move { struct Board *board; };
#endif
In Board.h:
#ifndef H_BOARD_
#define H_BOARD_
struct Move; /* Forward declaration. YOu can use a pointer to
struct Move from now on, but the type itself is incomplete,
so you can't declare an object of the type itself. */
struct Board { struct Move *move; }; /* OK: since move is a pointer */
#endif
Note that if you need to declare struct Move and struct Board objects (rather than pointer to one of them) in both the files, this method won't work. This is because one of the types is an incomplete type at the time of parsing of one of the files (struct Move in the above example).
So, if you need to use the types in both the files, you will have to separate out the type definitions: have header files that define struct Move and struct Board, and nothing else (something like my example above), and then use another header file that references both struct Move and struct Board.
Of course, you can't have struct Move contain a struct Board and struct Board contain a struct Move at the same time—that will be infinite recursion, and the struct sizes will be infinite as well!
You need to have one of them first. Make a forward decl in one of them and have that one for for example
#ifndef move
struct move;
#endif
could be part of the board.h file.
and
#ifndef board
struct board;
#endif
could be part of the move.h file
then you could add them in either order.
edit
As was pointed out in the comments... I was assuming the use of the typedef construct as follows for the board struct
typedef struct {…} board;
since I've never seen anyone using structs in C without a typedef I made this assumption... maybe things have changed since the last time I coded in C (yikies.... it was like 15 years ago)
Circular dependencies are a pain in the ass and should be eliminated wherever feasible. In addition to the forward declaration suggestions given so far (Alok's is the best example), I'd like to throw another suggestion into the works: break the mutual dependency between Board and Move by introducing a third type (call it BoardMoveAssoc for illustration; I'm sure you can come up with a less sucky name):
#ifndef H_BOARD_MOVE_ASSOC
#define H_BOARD_MOVE_ASSOC
#include "Move.h"
#include "Board.h"
struct BoardMoveAssoc {
Move m;
Board b;
};
...
#endif
Under this scheme, Board and Move don't have to know anything about each other; any associations between the two are managed by the BoardMoveAssoc type. The exact structure will depend on how Move and Board are supposed to be related; e.g., if multiple moves are mapped to a single board, the structure may look more like
struct BoardMoveAssoc {
Move m[NUM_MOVES] // or Move *m;
Board b;
};
This way, you don't have to worry about forward declarations or incomplete types. You are introducing a third type into the mix, but I believe this will be easier to understand and maintain.
From K&R The C Programming Language (p 91 "Conditional Inclusion" in my copy), with some tweaks for you:
#if !defined (BOARD_H)
#define BOARD_H
/* contents of board.h go here */
#endif
and the same for Move.h
In this way, once a header has been included once, it will not be included again, as the 'BOARD_H' name has already been defined for the preprocessor.

Resources