From /snap/dlang/43/usr/include/dlang/dmd/std/process.d:
string[string] toAA() #trusted
{
import std.conv : to;
string[string] aa;
version (Posix)
{
auto environ = getEnvironPtr;
for (int i=0; environ[i] != null; ++i)
{
import std.string : indexOf;
immutable varDef = to!string(environ[i]);
immutable eq = indexOf(varDef, '=');
assert(eq >= 0);
immutable name = varDef[0 .. eq];
immutable value = varDef[eq+1 .. $];
// In POSIX, environment variables may be defined more
// than once. This is a security issue, which we avoid
// by checking whether the key already exists in the array.
// For more info:
// http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/environment-variables.html
if (name !in aa) aa[name] = value;
}
}
// ...
}
But getEnvironPtr() is defined as follows:
extern(C) extern __gshared const char** environ;
const(char**) getEnvironPtr() #trusted
{
return environ;
}
The above code seems for me beign not thread safe because of use __gshared and the fact that environ extern variable is modifyable. Is this a bug in D? Or what may I misunderstand?
This is the only way to do it in Posix. The variable is marked const, so you can't modify it, but you can declare another extern(C) variable that can modify the same memory. So don't do that.
D does a lot to make things safe, but the only way to make this truly safe would be to eliminate extern(C). Or to get rid of Posix and rebuild an OS from scratch in D. Those are both drastic measures disproportionate to the size of the problem.
Related
I would like to know the best/correct way to get back to the initial values of an object without playing with delete and new (everything must stay on the stack)
With this 2 classes:
static const int defaultValue{15};
class Foo
{
private:
int val1{defaultValue};
short val2{4};
}
class LongStandingObject
{
public:
void resetFoo(int index);
private:
Foo foos[100];
}
If I need to reset some foos to their default values, what is the best way?
Create reset method in Foo
void Foo::reset()
{
val1 = defaultValue;
val2 = 4;
}
I don't really like the idea to have the values coming from 2 differents places and I do like to have the defaults values in the header next to the variable declaration.
Replace by a locally created Foo
void LongStandingObject::resetFoo(int index)
{
foos[index] = Foo();
}
Am I heading to trouble when the local variable is destroyed?
Use memcpy
void LongStandingObject::resetFoo(int index)
{
Foo foo;
memcpy(foos[index], &foo, sizeof(Foo));
}
Maybe less readable...
Any other method?
Your #2 is just fine, and probably the most legible.
void LongStandingObject::resetFoo(int index)
{
foos[index] = Foo();
}
There are no object lifetime issues here: the assignment operator is called on foos[index] to change its values to match the temporary object materialized from Foo(). That is, the code is exactly equivalent to
{
Foo tmp;
foos[index].val1 = tmp.val1;
foos[index].val2 = tmp.val2;
}
And if optimizations are enabled, almost any compiler will be able to just modify foos[index] directly without actually creating the temporary Foo.
If you do want a Foo::reset() function as well, you can use the same idea for it:
void Foo::reset()
{
*this = Foo();
}
I would avoid using memcpy, since the program would become incorrect if you ever make changes to Foo that make it no longer trivially copyable.
What you can do is to use std::pair on each variable. Initialized with with variable.first = variable.second = value. After, every time you want to update the variable you set: variable.second = new_value. When you want to restore to the original, you set: variable.second = variable.first. You can improve it by writing a macro RestoreDefault(var) to make the code more readable.
For example:
#define RestoreDefault(var) ((var).second = (var).first)
// int val1{180}; // Instead of this line
std::pair<int,int> val1{ 180,180}; // <- this line
val1.second = 456;
RestoreDefault(val1);
If you want to hardcoded block any possibility to re-set later the default value, write:
std::pair<const int,int> val1{ 180,180}; // <- this line
-
Addition: Same principle for array:
class Foo
{
public:
int x = 100;
int y = 200;
};
#define RestoreArrDefault(var) ((var).second.fill((var).first))
// Use:
std::pair<Foo, std::array<Foo, 100>> FooAr, FooAr2;
// You can define different arrays with different defaults:
FooAr.first = { 180,360 }; // <- Customize FooAr defaults
// In that point FooAr default is 180,360 and FooAr2 default is 100,200
FooAr.second[3] = { 2,10 }; // <- Edit FooAr index-3 item
RestoreArrDefault(FooAr); // <- Restore Default
first of all I admit I'm a newbie in C++ addons for node.js.
I'm writing my first addon and I reached a good result: the addon does what I want. I copied from various examples I found in internet to exchange complex data between the two languages, but I understood almost nothing of what I wrote.
The first thing scaring me is that I wrote nothing that seems to free some memory; another thing which is seriously worrying me is that I don't know if what I wrote may helps or creating confusion for the V8 garbage collector; by the way I don't know if there are better ways to do what I did (iterating over js Object keys in C++, creating js Objects in C++, creating Strings in C++ to be used as properties of js Objects and what else wrong you can find in my code).
So, before going on with my job writing the real math of my addon, I would like to share with the community the nan and V8 part of it to ask if you see something wrong or that can be done in a better way.
Thank you everybody for your help,
iCC
#include <map>
#include <nan.h>
using v8::Array;
using v8::Function;
using v8::FunctionTemplate;
using v8::Local;
using v8::Number;
using v8::Object;
using v8::Value;
using v8::String;
using Nan::AsyncQueueWorker;
using Nan::AsyncWorker;
using Nan::Callback;
using Nan::GetFunction;
using Nan::HandleScope;
using Nan::New;
using Nan::Null;
using Nan::Set;
using Nan::To;
using namespace std;
class Data {
public:
int dt1;
int dt2;
int dt3;
int dt4;
};
class Result {
public:
int x1;
int x2;
};
class Stats {
public:
int stat1;
int stat2;
};
typedef map<int, Data> DataSet;
typedef map<int, DataSet> DataMap;
typedef map<float, Result> ResultSet;
typedef map<int, ResultSet> ResultMap;
class MyAddOn: public AsyncWorker {
private:
DataMap *datas;
ResultMap results;
Stats stats;
public:
MyAddOn(Callback *callback, DataMap *set): AsyncWorker(callback), datas(set) {}
~MyAddOn() { delete datas; }
void Execute () {
for(DataMap::iterator i = datas->begin(); i != datas->end(); ++i) {
int res = i->first;
DataSet *datas = &i->second;
for(DataSet::iterator l = datas->begin(); l != datas->end(); ++l) {
int dt4 = l->first;
Data *data = &l->second;
// TODO: real population of stats and result
}
// test result population
results[res][res].x1 = res;
results[res][res].x2 = res;
}
// test stats population
stats.stat1 = 23;
stats.stat2 = 42;
}
void HandleOKCallback () {
Local<Object> obj;
Local<Object> res = New<Object>();
Local<Array> rslt = New<Array>();
Local<Object> sts = New<Object>();
Local<String> x1K = New<String>("x1").ToLocalChecked();
Local<String> x2K = New<String>("x2").ToLocalChecked();
uint32_t idx = 0;
for(ResultMap::iterator i = results.begin(); i != results.end(); ++i) {
ResultSet *set = &i->second;
for(ResultSet::iterator l = set->begin(); l != set->end(); ++l) {
Result *result = &l->second;
// is it ok to declare obj just once outside the cycles?
obj = New<Object>();
// is it ok to use same x1K and x2K instances for all objects?
Set(obj, x1K, New<Number>(result->x1));
Set(obj, x2K, New<Number>(result->x2));
Set(rslt, idx++, obj);
}
}
Set(sts, New<String>("stat1").ToLocalChecked(), New<Number>(stats.stat1));
Set(sts, New<String>("stat2").ToLocalChecked(), New<Number>(stats.stat2));
Set(res, New<String>("result").ToLocalChecked(), rslt);
Set(res, New<String>("stats" ).ToLocalChecked(), sts);
Local<Value> argv[] = { Null(), res };
callback->Call(2, argv);
}
};
NAN_METHOD(AddOn) {
Local<Object> datas = info[0].As<Object>();
Callback *callback = new Callback(info[1].As<Function>());
Local<Array> props = datas->GetOwnPropertyNames();
Local<String> dt1K = Nan::New("dt1").ToLocalChecked();
Local<String> dt2K = Nan::New("dt2").ToLocalChecked();
Local<String> dt3K = Nan::New("dt3").ToLocalChecked();
Local<Array> props2;
Local<Value> key;
Local<Object> value;
Local<Object> data;
DataMap *set = new DataMap();
int res;
int dt4;
DataSet *dts;
Data *dt;
for(uint32_t i = 0; i < props->Length(); i++) {
// is it ok to declare key, value, props2 and res just once outside the cycle?
key = props->Get(i);
value = datas->Get(key)->ToObject();
props2 = value->GetOwnPropertyNames();
res = To<int>(key).FromJust();
dts = &((*set)[res]);
for(uint32_t l = 0; l < props2->Length(); l++) {
// is it ok to declare key, data and dt4 just once outside the cycles?
key = props2->Get(l);
data = value->Get(key)->ToObject();
dt4 = To<int>(key).FromJust();
dt = &((*dts)[dt4]);
int dt1 = To<int>(data->Get(dt1K)).FromJust();
int dt2 = To<int>(data->Get(dt2K)).FromJust();
int dt3 = To<int>(data->Get(dt3K)).FromJust();
dt->dt1 = dt1;
dt->dt2 = dt2;
dt->dt3 = dt3;
dt->dt4 = dt4;
}
}
AsyncQueueWorker(new MyAddOn(callback, set));
}
NAN_MODULE_INIT(Init) {
Set(target, New<String>("myaddon").ToLocalChecked(), GetFunction(New<FunctionTemplate>(AddOn)).ToLocalChecked());
}
NODE_MODULE(myaddon, Init)
One year and half later...
If somebody is interested, my server is up and running since my question and the amount of memory it requires is stable.
I can't say if the code I wrote really does not has some memory leak or if lost memory is freed at each thread execution end, but if you are afraid as I was, I can say that using same structure and calls does not cause any real problem.
You do actually free up some of the memory you use, with the line of code:
~MyAddOn() { delete datas; }
In essence, C++ memory management boils down to always calling delete for every object created by new. There are also many additional architecture-specific and legacy 'C' memory management functions, but it is not strictly necessary to use these when you do not require the performance benefits.
As an example of what could potentially be a memory leak: You're passing the object held in the *callback pointer to the function AsyncQueueWorker. Yet nowhere in your code is this pointer freed, so unless the Queue worker frees it for you, there is a memory leak here.
You can use a memory tool such as valgrind to test your program further. It will spot most memory problems for you and comes highly recommended.
One thing I've observed is that you often ask (paraphrased):
Is it okay to declare X outside my loop?
To which the answer actually is that declaring variables inside of your loops is better, whenever you can do it. Declare variables as deep inside as you can, unless you have to re-use them. Variables are restricted in scope to the outermost set of {} brackets. You can read more about this in this question.
is it ok to use same x1K and x2K instances for all objects?
In essence, when you do this, if one of these objects modifies its 'x1K' string, then it will change for all of them. The advantage is that you free up memory. If the string is the same for all these objects anyway, instead of having to store say 1,000,000 copies of it, your computer will only keep a single one in memory and have 1,000,000 pointers to it instead. If the string is 9 ASCII characters long or longer under amd64, then that amounts to significant memory savings.
By the way, if you don't intend to modify a variable after its declaration, you can declare it as const, a keyword short for constant which forces the compiler to check that your variable is not modified after declaration. You may have to deal with quite a few compiler errors about functions accepting only non-const versions of things they don't modify, some of which may not be your own code, in which case you're out of luck. Being as conservative as possible with non-const variables can help spot problems.
I know that using atomics is dangerous (I watched Herb Sutter's 3hr lecture a few days ago), but the following use case seems reasonable to me, in terms of being simple and well contained.
Questions: (a) Is there something wrong with this? (Surely, there must be.) (b) Is there a name for this kind of hybrid atomic/mutex based approach? (c) Is there a simpler way of achieving the same thing?
The goal is to have a thread-safe counter class which we can call attempt_invalidation() on, knowing that it will only set its invalid flag to true if the count is at zero. There will be no other public methods on the class, but we will have a friend class specially designed to do RAII incrementing/decrementing of the counter.
class hybrid_counter{
friend class hybrid_counter_user;
bool invalidated = false;
int counter_a = 0;
std::atomic_int counter_b;
std::mutex mu;
bool increment_safely(){
std::lock_guard<std::mutex> gaurd(mu);
if ( !invalidated )
counter_a++;
return invalidated;
};
void increment_dangerously(){
counter_b++;
};
void decrement(){
counter_b--;
};
public:
bool attempt_invalidation(){
if(counter_a + counter_b == 0){
std::lock_guard<std::mutex> gaurd(mu);
if(counter_a + counter_b == 0)
invalidated = true;
}
return invalidated;
};
};
This is the friend class that knows how to use the counter correctly:
class hybrid_counter_user{
public:
hybrid_counter_user(hybrid_counter& hc){
if(hc.increment_safely() == false) // is not yet invalidated
c = &hc;
else
c = nullptr;
};
~hybrid_counter_user(){
if(c)
c->decrement();
};
hybrid_counter_user(hybrid_counter_user&& old){
c = old.c;
old.c = nullptr;
}
hybrid_counter_user(hybrid_counter_user& other){
c = other.c;
if(c)
c->increment_dangerously();
}
private:
hybrid_counter* c;
};
Note that the copy constructor uses the fact that hybrid_counter remains valid while other is in scope and other's destructor cannot be reordered with increment_dangerously because both involve the same atomic var.
The move constructor is simply transferring responsibility for decrementing.
I am trying to make the producer-consumer method using c++11 concurrency. The wait method for the condition_variable class has a predicate as second argument, so I thought of using a lambda function:
struct LimitedBuffer {
int* buffer, size, front, back, count;
std::mutex lock;
std::condition_variable not_full;
std::condition_variable not_empty;
LimitedBuffer(int size) : size(size), front(0), back(0), count(0) {
buffer = new int[size];
}
~LimitedBuffer() {
delete[] buffer;
}
void add(int data) {
std::unique_lock<std::mutex> l(lock);
not_full.wait(l, [&count, &size]() {
return count != size;
});
buffer[back] = data;
back = (back+1)%size;
++count;
not_empty.notify_one();
}
int extract() {
std::unique_lock<std::mutex> l(lock);
not_empty.wait(l, [&count]() {
return count != 0;
});
int result = buffer[front];
front = (front+1)%size;
--count;
not_full.notify_one();
return result;
}
};
But I am getting this error:
[Error] capture of non-variable 'LimitedBuffer::count'
I don't really know much about c++11 and lambda functions so I found out that class members can't be captured by value. By value though, I am capturing them by reference, but it seems like it's the same thing.
In a display of brilliance I stored the struct members values in local variables and used them in the lambda function, and it worked! ... or not:
int ct = count, sz = size;
not_full.wait(l, [&ct, &sz]() {
return ct != sz;
});
Obviously I was destroying the whole point of the wait function by using local variables since the value is assigned once and the fun part is checking the member variables which may, should and will change. Silly me.
So, what are my choices? Is there any way I can make the wait method do what it has to do, using the member variables? Or I am forced to not use lambda functions so I'd have to declare auxiliary functions to do the work?
I don't really get why I can't use members variables in lambda functions, but since the masters of the universe dessigned lamba functions for c++11 this way, there must be some good reason.
count is a member variable. Member variables can not be captured directly. Instead, you can capture this to achieve the same effect:
not_full.wait(l, [this] { return count != size; });
Okay, I believe in defensive programming. I assume that if I get a pointer it might be null (especially when using GSOAP). Therefore before I try to use the value of the pointer, I always check to make sure the pointer is not null.
In my current code, this is leading to a lot of nearly identical statements.
if (res->A) {
item.out_trace->a = *res->A;
}
if (res->B) {
item.out_trace->b = *res->B;
}
if (res->C) {
item.out_trace->b = *res->C;
}
I realize that I could always go and define a macro for this, but I am wondering if there is a neat C++11 trick to do that. I would love something like the C# ??
// Set y to the value of x if x is NOT null; otherwise,
// if x = null, set y to -1.
int y = x ?? -1;
Thanks.
Perhaps a template like this would meet your need:
template<typename T>
T safe_get( T const *ptr, T defval = T{} ) {
return ptr ? *ptr : std::move(defval);
}
It could be used like this:
item.out_trace->a = safe_get( rez->A );
Ideally it would be inlined and effectively zero-overhead (other than the inherent overhead of doing the safety check and having a branch, of course).