How to avoid C++ code bloat issued by template instantiation and symbol table? - c++11

I'd started a bare-metal (Cortex-M) project some years ago. At project setup we decided to use gcc toolchain with C++11 / C++14 etc. enabled and even for using C++ exceptions and rtti.
We are currently using gcc 4.9 from launchpad.net/gcc-arm-embedded (having some issue which prevent us currently to update to a more recent gcc version).
For example, I'd wrote a base class and a derived class like this (see also running example here):
class OutStream {
public:
explicit OutStream() {}
virtual ~OutStream() {}
OutStream& operator << (const char* s) {
write(s, strlen(s));
return *this;
}
virtual void write(const void* buffer, size_t size) = 0;
};
class FixedMemoryStream: public OutStream {
public:
explicit FixedMemoryStream(void* memBuffer, size_t memBufferSize): memBuffer(memBuffer), memBufferSize(memBufferSize) {}
virtual ~FixedMemoryStream() {}
const void* getBuffer() const { return memBuffer; }
size_t getBufferSize() const { return memBufferSize; }
const char* getText() const { return reinterpret_cast<const char*>(memBuffer); } ///< returns content as zero terminated C-string
size_t getSize() const { return index; } ///< number of bytes really written to the buffer (max = buffersize-1)
bool isOverflow() const { return overflow; }
virtual void write(const void* buffer, size_t size) override { /* ... */ }
private:
void* memBuffer = nullptr; ///< buffer
size_t memBufferSize = 0; ///< buffer size
size_t index = 0; ///< current write index
bool overflow = false; ///< flag if we are overflown
};
So that the customers of my class are now able to use e.g.:
char buffer[10];
FixedMemoryStream ms1(buffer, sizeof(buffer));
ms1 << "Hello World";
Now I'd want to make the usage of the class a bit more comfortable and introduced the following template:
template<size_t bufferSize> class FixedMemoryStreamWithBuffer: public FixedMemoryStream {
public:
explicit FixedMemoryStreamWithBuffer(): FixedMemoryStream(buffer, bufferSize) {}
private:
uint8_t buffer[bufferSize];
};
And from now, my customers can write:
FixedMemoryStreamWithBuffer<10> ms2;
ms2 << "Hello World";
But from now, I'd observed increasing size of my executable binary. It seems that gcc added symbol information for each different template instantiation of FixedMemoryStreamWithBuffer (because we are using rtti for some reason).
Might there be a way to get rid of symbol information only for some specific classes / templates / template instantiations?
It's ok to get a non portable gcc only solution for this.
For some reason we decided to prefer templates instead of preprocessor macros, I want to avoid a preprocessor solution.

First of all, keep in mind that compiler also generates separate v-table (as well as RTTI information) for every FixedMemoryStreamWithBuffer<> type instance, as well as every class in the inheritance chain.
In order to resolve the problem I'd recommend using containment instead of inheritance with some conversion function and/or operator inside:
template<size_t bufferSize>
class FixedMemoryStreamWithBuffer
{
uint8_t buffer[bufferSize];
FixedMemoryStream m_stream;
public:
explicit FixedMemoryStreamWithBuffer() : m_stream(m_buffer, bufferSize) {}
operator FixedMemoryStream&() { return m_stream; }
FixedMemoryStream& toStream() { return m_stream; }
};

Yes, there's a way to bring the necessary symbols almost down to 0: using the standard library. Your OutStream class is a simplified version of std::basic_ostream. Your OutStream::write is really just std::basic_ostream::write and so on. Take a look at it here. Overflow is handled really closely, though, for completeness' sake, it also deals with underflow i.e. the need for data retrieval; you may leave it as undefined (it's virtual too).
Similarly, your FixedMemoryStream is std::basic_streambuf<T> with a fixed-size (a std::array<T>) get/put area.
So, just make your classes inherit from the standard ones and you'll cut off on binary size since you're reusing already declared symbols.
Now, regarding template<size_t bufferSize> class FixedMemoryStreamWithBuffer. This class is very similar to std::array<std::uint8_t, bufferSize> as for the way memory is specified and acquired. You can't optimize much about that: each instantiation is a different type with all what that implies. The compiler cannot "merge" or do anything magic about them: each instantiation must have its own type.
So either fall back on std::vector or have some fixed-size specialized chunks, like 32, 128 etc. and for any values in between would choose the right one; this can be achieved entirely at compile-time, so no runtime cost.

Related

how to fix wrong GCC ARM startup code pointer to initialized and zero variables?

[skip to UPDATE2 and save some time :-)]
I use ARM Cortex-M4, with CMSIS 5-5.7.0 and FreeRTOS, compiling using GCC for ARM (10_2021.10)
My variables are not initialized as they should.
My startup code is pretty simple, the entry point is the reset handler (CMSIS declared startup_ARMCM4.s as deprecated and recommend using the C code startup code so this is what I do).
Here is my code:
__attribute__((__noreturn__)) void Reset_Handler(void)
{
DataInit();
SystemInit(); /* CMSIS System Initialization */
main();
}
static void DataInit(void)
{
typedef struct {
uint32_t const* src;
uint32_t* dest;
uint32_t wlen;
} __copy_table_t;
typedef struct {
uint32_t* dest;
uint32_t wlen;
} __zero_table_t;
extern const __copy_table_t __copy_table_start__;
extern const __copy_table_t __copy_table_end__;
extern const __zero_table_t __zero_table_start__;
extern const __zero_table_t __zero_table_end__;
for (__copy_table_t const* pTable = &__copy_table_start__; pTable < &__copy_table_end__; ++pTable) {
for(uint32_t i=0u; i<pTable->wlen; ++i) {
pTable->dest[i] = pTable->src[i];
}
}
for (__zero_table_t const* pTable = &__zero_table_start__; pTable < &__zero_table_end__; ++pTable) {
for(uint32_t i=0u; i<pTable->wlen; ++i) {
pTable->dest[i] = 0u;
}
}
}
__copy_table_start__, __copy_table_end__ etc. have the wrong values an so no data is copied to the appropriate place in RAM.
I tried adding __libc_init_array() before DataInit(), as suggested in this answer, and remove the nostartfiles flag from the linker, but at some point __libc_init_array() jumps to an illegal address and I get a HardFault interrupt.
Is there a different method to fix it? maybe one where I can use the nostartfiles flag?
UPDATE:
Looking at the memory, where __copy_table_start__ is located, I see the data there is valid (even without the use of __libc_init_array()). It seems that pTable doesn't get the correct value.
I tried using __data_start__, __data_end__, __bss_start__, __bss_end__ and __etext instead of the above variables, in the linker file it is said they can be used in code without definition, but they cannot (maybe that's a clue?). In any case they didn't work either.
UPDATE2:
found the actual problem
all struct members get the same value (modifying one changes all others), it happens with every struct. I have no idea how this is possible. In other words the value of __copy_table_start__.src is, for example, 0x14651234, __copy_table_start__.dest is 0x00100000, and __copy_table_start__.wlen is 0x0365. When looking at pTable all members are 0x14651234.

C++ Check if generic object has member function matching signature

first post, so hopefully not violating any etiquette. Feel free to give suggestions for making the question better.
I've seen a few posts similar to this one: Check if a class has a member function of a given signature, but none do quite what I want. Sure it "works with polymorphism" in the sense that it can properly check subclass types for the function that comes from a superclass, but what I'd like to do is check the object itself and not the class. Using some (slightly tweaked) code from that post:
// Somewhere in back-end
#include <type_traits>
template<typename, typename T>
struct HasFunction {
static_assert(integral_constant<T, false>::value,
"Second template parameter needs to be of function type."
);
};
template<typename C, typename Ret, typename... Args>
class HasFunction<C, Ret(Args...)> {
template<typename T>
static constexpr auto check(T*) -> typename is_same<
decltype(declval<T>().myfunc(declval<Args>()...)), Ret>::type;
template<typename>
static constexpr false_type check(...);
typedef decltype(check<C>(0)) type;
public:
static constexpr bool value = type::value;
};
struct W {};
struct X : W { int myfunc(double) { return 42; } };
struct Y : X {};
I'd like to have something like the following:
// somewhere else in back-end. Called by client code and doesn't know
// what it's been passed!
template <class T>
void DoSomething(T& obj) {
if (HasFunction<T, int(double)>::value)
cout << "Found it!" << endl;
// Do something with obj.myfunc
else cout << "Nothin to see here" << endl;
}
int main()
{
Y y;
W* w = &y; // same object
DoSomething(y); // Found it!
DoSomething(*w); // Nothin to see here?
}
The problem is that the same object being viewed polymorphically causes different results (because the deduced type is what is being checked and not the object). So for example, if I was iterating over a collection of W*'s and calling DoSomething I would want it to no-op on W's but it should do something for X's and Y's. Is this achievable? I'm still digging into templates so I'm still not quite sure what's possible but it seems like it isn't. Is there a different way of doing it altogether?
Also, slightly less related to that specific problem: Is there a way to make HasFunction more like an interface so I could arbitrarily check for different functions? i.e. not have ".myfunc" concrete within it? (seems like it's only possible with macros?) e.g.
template<typename T>
struct HasFoo<T> : HasFunction<T, int foo(void)> {};
int main() {
Bar b;
if(HasFoo<b>::value) b.foo();
}
Obviously that's invalid syntax but hopefully it gets the point across.
It's just not possible to perform deep inspection on a base class pointer in order to check for possible member functions on the pointed-to type (for derived types that are not known ahead of time). Even if we get reflection.
The C++ standard provides us no way to perform this kind of inspection, because the kind of run time type information that is guaranteed to be available is very limited, basically relegated to the type_info structure.
Your compiler/platform may provide additional run-time type information that you can hook into, although the exact types and machinery used to provide RTTI are generally undocumented and difficult to examine (This article by Quarkslab attempts to inspect MSVC's RTTI hierarchy)

When initializing an atomic class member it requires a 'deleted' function, but adding it would make it no longer trivially copyable

When initializing an atomic class member it requires a 'deleted' function, but adding it would make it no longer trivially copyable which is a requirement for an object/struct to be atomic. Am I just not understanding how to do this correctly, or is this a problem in the c++ standard?
Take the example below:
#include <atomic>
#include <cstdint>
template<typename T>
struct A
{
T * data;
std::atomic<uintptr_t> next;
};
template<typename T>
class B
{
std::atomic<A<T>> myA;
public:
B ( A<T> & a ) noexcept
{
myA.store(a, std::memory_order_relaxed );
}
};
int main ()
{
A<int> a;
B<int> b(a);
return 0;
}
Trying to compile this with g++ gives error: use of deleted function 'A<int>::A(const A<int>&)' myA.store(a, std::memory_order_relaxed);. My understanding of this error is that the atomic::store method is looking for that constructor in my struct A but not finding it.
Now here is what happens when I add that constructor:
#include <atomic>
#include <cstdint>
template<typename T>
struct A
{
T * data;
std::atomic<uintptr_t> next;
A(const A<T>& obj) { }
A( ) { }
};
template<typename T>
class B
{
std::atomic<A<T>> myA;
public:
B ( A<T> & a ) noexcept
{
myA.store(a, std::memory_order_relaxed );
}
};
int main ()
{
A<int> a;
B<int> b(a);
return 0;
}
I no longer receive the above compiler error but a new one coming from the requirements of the atomic class required from 'class B<int>' .... error: static assertion failed: std::atomic requires a trivially copyable type ... In other words by adding the used-defined constructors I have made my struct A a non-trivially copyable object which cannot be initialized in class B. However, without the user-defined constructors I cannot use the store method in myA.store(a, std::memory_order_relaxed).
This seems like a flaw in the design of the std::atomic class. Now maybe I am just doing something wrong because I don't have a lot of experience using C++11 and up (I'm old school). Since 11 there have been a lot of changes and the requirements seem to be a lot stricter. I'm hoping someone can tell me how to achieve what I want to achieve.
Also I cannot change std::atomic<A<T>> myA; to std::atomic<A<T>> * myA; (changed to pointer) or std::atomic<A<T>*> myA;. I realize this will compile but it will destroy the fundamental design of a class I am trying to build.
The problem here resides in the fact that std::atomic requires a trivially copiable type. This because trivially copyable types are the only sure types in C++ which can be directly copied by copying their memory contents directly (eg. through std::memcpy). Also non-formerly trivially copyable types could be safe to raw copy but no assumption can be made on this.
This is indeed important for std::atomic since copy on temporary values is made through std::memcpy, see some implementation details for Clang for example.
Now at the same time std::atomic is not copy constructible, and this is for reasonable reasons, check this answer for example, so it's implicitly not trivially copyable (nor any type which contains them).
If, absurdly, you would allow a std::atomic to contain another std::atomic, and the implementation of std::atomic contains a lock, how would you manage copying it atomically? How should it work?

Another void* topic; I just have to ask because I am confused

Ok, muddling though Stack on the particulars about void*, books like The C Programming Language (K&R) and The C++ Programming Language (Stroustrup). What have I learned? That void* is a generic pointer with no type inferred. It requires a cast to any defined type and printing void* just yields the address.
What else do I know? void* can't be dereferenced and thus far remains the one item in C/C++ from which I have discovered much written about but little understanding imparted.
I understand that it must be cast such as *(char*)void* but what makes no sense to me for a generic pointer is that I must somehow already know what type I need in order to grab a value. I'm a Java programmer; I understand generic types but this is something I struggle with.
So I wrote some code
typedef struct node
{
void* data;
node* link;
}Node;
typedef struct list
{
Node* head;
}List;
Node* add_new(void* data, Node* link);
void show(Node* head);
Node* add_new(void* data, Node* link)
{
Node* newNode = new Node();
newNode->data = data;
newNode->link = link;
return newNode;
}
void show(Node* head)
{
while (head != nullptr)
{
std::cout << head->data;
head = head->link;
}
}
int main()
{
List list;
list.head = nullptr;
list.head = add_new("My Name", list.head);
list.head = add_new("Your Name", list.head);
list.head = add_new("Our Name", list.head);
show(list.head);
fgetc(stdin);
return 0;
}
I'll handle the memory deallocation later. Assuming I have no understanding of the type stored in void*, how do I get the value out? This implies I already need to know the type, and this reveals nothing about the generic nature of void* while I follow what is here although still no understanding.
Why am I expecting void* to cooperate and the compiler to automatically cast out the type that is hidden internally in some register on the heap or stack?
I'll handle the memory deallocation later. Assuming I have no understanding of the type stored in void*, how do I get the value out?
You can't. You must know the valid types that the pointer can be cast to before you can dereference it.
Here are couple of options for using a generic type:
If you are able to use a C++17 compiler, you may use std::any.
If you are able to use the boost libraries, you may use boost::any.
Unlike Java, you are working with memory pointers in C/C++. There is no encapsulation whatsoever. The void * type means the variable is an address in memory. Anything can be stored there. With a type like int * you tell the compiler what you are referring to. Besides the compiler knows the size of the type (say 4 bytes for int) and the address will be a multiple of 4 in that case (granularity/memory alignment). On top, if you give the compiler the type it will perform consistency checks at compilation time. Not after. This is not happening with void *.
In a nutshell, you are working bare metal. The types are compiler directives and do not hold runtime information. Nor does it track the objects you are dynamically creating. It is merely a segment in memory that is allocated where you can eventually store anything.
The main reason to use void* is that different things may be pointed at. Thus, I may pass in an int* or Node* or anything else. But unless you know either the type or the length, you can't do anything with it.
But if you know the length, you can handle the memory pointed at without knowing the type. Casting it as a char* is used because it is a single byte, so if I have a void* and a number of bytes, I can copy the memory somewhere else, or zero it out.
Additionally, if it is a pointer to a class, but you don't know if it is a parent or inherited class, you may be able to assume one and find out a flag inside the data which tells you which one. But no matter what, when you want to do much beyond passing it to another function, you need to cast it as something. char* is just the easiest single byte value to use.
Your confusion derived from habit to deal with Java programs. Java code is set of instruction for a virtual machine, where function of RAM is given to a sort of database, which stores name, type, size and data of each object. Programming language you're learning now is meant to be compiled into instruction for CPU, with same organization of memory as underlying OS have. Existing model used by C and C++ languages is some abstract built on top of most of popular OSes in way that code would work effectively after being compiled for that platform and OS. Naturally that organization doesn't involve string data about type, except for famous RTTI in C++.
For your case RTTI cannot be used directly, unless you would create a wrapper around your naked pointer, which would store the data.
In fact C++ library contains a vast collection of container class templates that are useable and portable, if they are defined by ISO standard. 3/4 of standard is just description of library often referred as STL. Use of them is preferable over working with naked pointers, unless you mean to create own container for some reason. For particular task only C++17 standard offered std::any class, previously present in boost library. Naturally, it is possible to reimplement it, or, in some cases, to replace by std::variant.
Assuming I have no understanding of the type stored in void*, how do I get the value out
You don't.
What you can do is record the type stored in the void*.
In c, void* is used to pass around a binary chunk of data that points at something through one layer of abstraction, and recieve it at the other end, casting it back to the type that the code knows it will be passed.
void do_callback( void(*pfun)(void*), void* pdata ) {
pfun(pdata);
}
void print_int( void* pint ) {
printf( "%d", *(int*)pint );
}
int main() {
int x = 7;
do_callback( print_int, &x );
}
here, we forget thet ype of &x, pass it through do_callback.
It is later passed to code inside do_callback or elsewhere that knows that the void* is actually an int*. So it casts it back and uses it as an int.
The void* and the consumer void(*)(void*) are coupled. The above code is "provably correct", but the proof does not lie in the type system; instead, it depends on the fact we only use that void* in a context that knows it is an int*.
In C++ you can use void* similarly. But you can also get fancy.
Suppose you want a pointer to anything printable. Something is printable if it can be << to a std::ostream.
struct printable {
void const* ptr = 0;
void(*print_f)(std::ostream&, void const*) = 0;
printable() {}
printable(printable&&)=default;
printable(printable const&)=default;
printable& operator=(printable&&)=default;
printable& operator=(printable const&)=default;
template<class T,std::size_t N>
printable( T(&t)[N] ):
ptr( t ),
print_f( []( std::ostream& os, void const* pt) {
T* ptr = (T*)pt;
for (std::size_t i = 0; i < N; ++i)
os << ptr[i];
})
{}
template<std::size_t N>
printable( char(&t)[N] ):
ptr( t ),
print_f( []( std::ostream& os, void const* pt) {
os << (char const*)pt;
})
{}
template<class T,
std::enable_if_t<!std::is_same<std::decay_t<T>, printable>{}, int> =0
>
printable( T&& t ):
ptr( std::addressof(t) ),
print_f( []( std::ostream& os, void const* pt) {
os << *(std::remove_reference_t<T>*)pt;
})
{}
friend
std::ostream& operator<<( std::ostream& os, printable self ) {
self.print_f( os, self.ptr );
return os;
}
explicit operator bool()const{ return print_f; }
};
what I just did is a technique called "type erasure" in C++ (vaguely similar to Java type erasure).
void send_to_log( printable p ) {
std::cerr << p;
}
Live example.
Here we created an ad-hoc "virtual" interface to the concept of printing on a type.
The type need not support any actual interface (no binary layout requirements), it just has to support a certain syntax.
We create our own virtual dispatch table system for an arbitrary type.
This is used in the C++ standard library. In c++11 there is std::function<Signature>, and in c++17 there is std::any.
std::any is void* that knows how to destroy and copy its contents, and if you know the type you can cast it back to the original type. You can also query it and ask it if it a specific type.
Mixing std::any with the above type-erasure techinque lets you create regular types (that behave like values, not references) with arbitrary duck-typed interfaces.

Overloaded "operator new" wants to see the type it's allocating

This seems like it ought to be obvious, but I'm blanking on it. I have
class SimpleMemoryPool {
char buffer[10000];
size_t idx;
void *Alloc(size_t nbytes) { idx += nbytes; return &buffer[idx - nbytes]; }
};
inline void* operator new (size_t size, SimpleMemoryPool& pool)
{
return pool.Alloc(size);
}
inline void* operator new[] (size_t size, SimpleMemoryPool& pool)
{
return pool.Alloc(size);
}
The idea is that I can allocate new objects out of my SimpleMemoryPool and then they'll all be "released" when the SimpleMemoryPool is destroyed:
void foo()
{
SimpleMemoryPool pool;
int *arr = new (pool) int[10];
double *arr2 = new (pool) double(3.14);
...do things with arr and arr2...
return; // and arr, arr2 are "released" at this point
}
One nitpick I've simplified away: The above code is sketchy because the double won't be 8-byte-aligned. Don't worry about that; my real SimpleMemoryPool code returns maxaligned chunks.
Here's the next thing you're probably thinking at this point: "Who calls the destructors?!" I.e., if I accidentally write
std::string *arr3 = new (pool) std::string;
then I'm in a world of hurt, because the compiler will generate a call to std::string::string() for me, but nobody will ever call std::string::~string(). Memory leaks, bad stuff follows.
This is the problem I want to solve. What I want to do is basically
class SimpleMemoryPool {
...
// (std::enable_if omitted for brevity)
template<typename T, typename... Args>
T *New(Args... args) {
static_assert(std::is_trivially_destructible<T>::value, "T must be trivially destructible!");
void *ptr = this->Alloc(nelem * sizeof (T));
return new (ptr) T(std::forward<Args>(args)...);
}
template<typename ArrayT>
auto NewArray(size_t nelem) -> std::remove_extent<ArrayT>::type {
typedef typename std::remove_extent<ArrayT>::type T;
static_assert(std::is_trivially_destructible<T>::value, "T must be trivially destructible!");
void *ptr = this->Alloc(nelem * sizeof (T));
return new (ptr) T[ nelem ];
}
};
...
int *arr = pool.NewArray<int>(10);
double *arr2 = pool.New<double>(3.14);
std::string *arr3 = pool.New<string>(); // fails the static_assert, hooray!
The problem with this approach is that it's ugly. It looks bad, and it invites later maintainers to come along and "fix" the code by adding a "proper" operator new, at which point we lose the safety of the static_assert.
Is there any way to get the best of both worlds — type-safety via the static_assert, and also a nice syntax?
You may assume C++11. I also welcome C++14 answers, even though they won't be immediately useful to me.
Adding a member operator new to all my classes (in this example int and double) is not acceptable. Whatever I do has to work out-of-the-box without changing a million lines of code.
This is probably a duplicate of Get type of object being allocated in operator new but I'd still like answers tailored to this particular use-case. There might be some nice idiom of which I'm not aware.

Resources