question updated !
I got a std::string object, and std::thread supposed to copy it into it's internal storage, why the code below still doesn't work?(FIXED)
If I detach t(f, 5, std::string(buf)), there are no output at all ! (FIXED)
#include <thread>
#include <iostream>
#include <unistd.h>
using namespace std;
void f(int i, const std::string &str) {
printf("i = %d, str = %s\n", i, str.c_str());
}
void oops(int ival) {
char *buf = new char[32]{0};
snprintf(buf, 32, "%i", ival);
// maybe danling pointer, if it's a danling pointer, the content of buf is undefined
// in this program it happens to be 1999.
std::thread t(f, 3, buf);
t.detach();
delete [] buf;
}
void not_oops(int ival) {
char *buf = new char[32]{0};
snprintf(buf, 32, "%i", ival);
std::thread t(f, 5, std::string(buf));
t.detach();
delete [] buf;
}
int main() {
oops(1992); // This is buggy
not_oops(1999);
}
expected output:
i = 3, str = 1992
i = 5, str = 1999
actual output:
i = 3, str = 1999
i = 5, str = 1999
Your program finishes before it is complete, your function exits scope before thread is complete, killing it.
void oops(int ival) {
char *buf = new char[32]{0};
snprintf(buf, 32, "%i", ival);
std::thread t(f, 3, buf); // maybe danling pointer
t.detach(); // no more waiting for it
delete [] buf; // thread is not started yet (probably)
} // thread is destroyed , and std::string dies.
Only way to make it work without vast changes to code, is to allow threads to be managed outside. Also you have to ensure that a new std::string is created, or thread appears to use same storage for both.
void f(int i, const std::string &str) {
printf("i = %d, str = %s\n", i, str.c_str());
}
std::thread *t;
void oops(int ival) {
char *buf = new char[32]{0};
snprintf(buf, 32, "%i", ival);
t = new std::thread(f, 3, std::string(buf)); // maybe danling pointer
//t.detach();
delete [] buf;
}
void not_oops(int ival) {
char *buf = new char[32]{0};
snprintf(buf, 32, "%i", ival);
std::thread t(f, 5, std::string(buf)); // FIXME: why it won't work ?
t.join();
delete [] buf;
}
int main() {
oops(1992); // This is buggy
not_oops(1999);
t->join();
delete t;
}
Related
Consider the functions
#include <iostream>
#include <boost/bind.hpp>
#include <boost/asio.hpp>
void foo(const uint64_t begin, uint64_t *result)
{
uint64_t prev[] = {begin, 0};
for (uint64_t i = 0; i < 1000000000; ++i)
{
const auto tmp = (prev[0] + prev[1]) % 1000;
prev[1] = prev[0];
prev[0] = tmp;
}
*result = prev[0];
}
void batch(boost::asio::thread_pool &pool, const uint64_t a[])
{
uint64_t r[] = {0, 0};
boost::asio::post(pool, boost::bind(foo, a[0], &r[0]));
boost::asio::post(pool, boost::bind(foo, a[1], &r[1]));
pool.join();
std::cerr << "foo(" << a[0] << "): " << r[0] << " foo(" << a[1] << "): " << r[1] << std::endl;
}
where foo is a simple "pure" function that performs a calculation on begin and writes the result to the pointer *result.
This function gets called with different inputs from batch. Here dispatching each call to another CPU core might be beneficial.
Now assume the batch function gets called several 10 000 times. Therefore a thread pool would be nice which is shared between all the sequential batch calls.
Trying this with (for the sake of simplicity only 3 calls)
int main(int argn, char **)
{
boost::asio::thread_pool pool(2);
const uint64_t a[] = {2, 4};
batch(pool, a);
const uint64_t b[] = {3, 5};
batch(pool, b);
const uint64_t c[] = {7, 9};
batch(pool, c);
}
leads to the result
foo(2): 2 foo(4): 4
foo(3): 0 foo(5): 0
foo(7): 0 foo(9): 0
Where all three lines appear at the same time, while the computation of foo takes ~3s.
I assume that only the first join really waits for the pool to complete all jobs.
The others have invalid results. (The not initialized values)
What is the best practice here to reuse the thread pool?
The best practice is not to reuse the pool (what would be the use of pooling, if you keep creating new pools?).
If you want to be sure you "time" the batches together, I'd suggest using when_all on futures:
Live On Coliru
#define BOOST_THREAD_PROVIDES_FUTURE_WHEN_ALL_WHEN_ANY
#include <iostream>
#include <boost/bind.hpp>
#include <boost/asio.hpp>
#include <boost/thread.hpp>
uint64_t foo(uint64_t begin) {
uint64_t prev[] = {begin, 0};
for (uint64_t i = 0; i < 1000000000; ++i) {
const auto tmp = (prev[0] + prev[1]) % 1000;
prev[1] = prev[0];
prev[0] = tmp;
}
return prev[0];
}
void batch(boost::asio::thread_pool &pool, const uint64_t a[2])
{
using T = boost::packaged_task<uint64_t>;
T tasks[] {
T(boost::bind(foo, a[0])),
T(boost::bind(foo, a[1])),
};
auto all = boost::when_all(
tasks[0].get_future(),
tasks[1].get_future());
for (auto& t : tasks)
post(pool, std::move(t));
auto [r0, r1] = all.get();
std::cerr << "foo(" << a[0] << "): " << r0.get() << " foo(" << a[1] << "): " << r1.get() << std::endl;
}
int main() {
boost::asio::thread_pool pool(2);
const uint64_t a[] = {2, 4};
batch(pool, a);
const uint64_t b[] = {3, 5};
batch(pool, b);
const uint64_t c[] = {7, 9};
batch(pool, c);
}
Prints
foo(2): 2 foo(4): 4
foo(3): 503 foo(5): 505
foo(7): 507 foo(9): 509
I would consider
generalizing
message queuing
Generalized
Make it somewhat more flexible by not hardcoding batch sizes. After all, the pool size is already fixed, we don't need to "make sure batches fit" or something:
Live On Coliru
#define BOOST_THREAD_PROVIDES_FUTURE_WHEN_ALL_WHEN_ANY
#include <iostream>
#include <boost/bind.hpp>
#include <boost/asio.hpp>
#include <boost/thread.hpp>
#include <boost/thread/future.hpp>
struct Result { uint64_t begin, result; };
Result foo(uint64_t begin) {
uint64_t prev[] = {begin, 0};
for (uint64_t i = 0; i < 1000000000; ++i) {
const auto tmp = (prev[0] + prev[1]) % 1000;
prev[1] = prev[0];
prev[0] = tmp;
}
return { begin, prev[0] };
}
void batch(boost::asio::thread_pool &pool, std::vector<uint64_t> const a)
{
using T = boost::packaged_task<Result>;
std::vector<T> tasks;
tasks.reserve(a.size());
for(auto begin : a)
tasks.emplace_back(boost::bind(foo, begin));
std::vector<boost::unique_future<T::result_type> > futures;
for (auto& t : tasks) {
futures.push_back(t.get_future());
post(pool, std::move(t));
}
for (auto& fut : boost::when_all(futures.begin(), futures.end()).get()) {
auto r = fut.get();
std::cerr << "foo(" << r.begin << "): " << r.result << " ";
}
std::cout << std::endl;
}
int main() {
boost::asio::thread_pool pool(2);
batch(pool, {2});
batch(pool, {4, 3, 5});
batch(pool, {7, 9});
}
Prints
foo(2): 2
foo(4): 4 foo(3): 503 foo(5): 505
foo(7): 507 foo(9): 509
Generalized2: Variadics Simplify
Contrary to popular believe (and honestly, what usually happens) this time we can leverage variadics to get rid of all the intermediate vectors (every single one of them):
Live On Coliru
void batch(boost::asio::thread_pool &pool, T... a)
{
auto launch = [&pool](uint64_t begin) {
boost::packaged_task<Result> pt(boost::bind(foo, begin));
auto fut = pt.get_future();
post(pool, std::move(pt));
return fut;
};
for (auto& r : {launch(a).get()...}) {
std::cerr << "foo(" << r.begin << "): " << r.result << " ";
}
std::cout << std::endl;
}
If you insist on outputting the results in time, you can still add when_all into the mix (requiring a bit more heroics to unpack the tuple):
Live On Coliru
template <typename...T>
void batch(boost::asio::thread_pool &pool, T... a)
{
auto launch = [&pool](uint64_t begin) {
boost::packaged_task<Result> pt(boost::bind(foo, begin));
auto fut = pt.get_future();
post(pool, std::move(pt));
return fut;
};
std::apply([](auto&&... rfut) {
Result results[] {rfut.get()...};
for (auto& r : results) {
std::cerr << "foo(" << r.begin << "): " << r.result << " ";
}
}, boost::when_all(launch(a)...).get());
std::cout << std::endl;
}
Both still print the same result
Message Queuing
This is very natural to boost, and sort of skips most complexity. If you also want to report per batched group, you'd have to coordinate:
Live On Coliru
#include <iostream>
#include <boost/asio.hpp>
#include <memory>
struct Result { uint64_t begin, result; };
Result foo(uint64_t begin) {
uint64_t prev[] = {begin, 0};
for (uint64_t i = 0; i < 1000000000; ++i) {
const auto tmp = (prev[0] + prev[1]) % 1000;
prev[1] = prev[0];
prev[0] = tmp;
}
return { begin, prev[0] };
}
using Group = std::shared_ptr<size_t>;
void batch(boost::asio::thread_pool &pool, std::vector<uint64_t> begins) {
auto group = std::make_shared<std::vector<Result> >(begins.size());
for (size_t i=0; i < begins.size(); ++i) {
post(pool, [i,begin=begins.at(i),group] {
(*group)[i] = foo(begin);
if (group.unique()) {
for (auto& r : *group) {
std::cout << "foo(" << r.begin << "): " << r.result << " ";
std::cout << std::endl;
}
}
});
}
}
int main() {
boost::asio::thread_pool pool(2);
batch(pool, {2});
batch(pool, {4, 3, 5});
batch(pool, {7, 9});
pool.join();
}
Note this is having concurrent access to group, which is safe due to the limitations on element accesses.
Prints:
foo(2): 2
foo(4): 4 foo(3): 503 foo(5): 505
foo(7): 507 foo(9): 509
I just ran into this advanced executor example which is hidden from the documentation:
I realized just now that Asio comes with a fork_executor example which does exactly this: you can "group" tasks and join the executor (which represents that group) instead of the pool. I've missed this for the longest time since none of the executor examples are listed in the HTML documentation – sehe 21 mins ago
So without further ado, here's that sample applied to your question:
Live On Coliru
#define BOOST_BIND_NO_PLACEHOLDERS
#include <boost/asio/thread_pool.hpp>
#include <boost/asio/ts/executor.hpp>
#include <condition_variable>
#include <memory>
#include <mutex>
#include <queue>
#include <thread>
// A fixed-size thread pool used to implement fork/join semantics. Functions
// are scheduled using a simple FIFO queue. Implementing work stealing, or
// using a queue based on atomic operations, are left as tasks for the reader.
class fork_join_pool : public boost::asio::execution_context {
public:
// The constructor starts a thread pool with the specified number of
// threads. Note that the thread_count is not a fixed limit on the pool's
// concurrency. Additional threads may temporarily be added to the pool if
// they join a fork_executor.
explicit fork_join_pool(std::size_t thread_count = std::thread::hardware_concurrency()*2)
: use_count_(1), threads_(thread_count)
{
try {
// Ask each thread in the pool to dequeue and execute functions
// until it is time to shut down, i.e. the use count is zero.
for (thread_count_ = 0; thread_count_ < thread_count; ++thread_count_) {
boost::asio::dispatch(threads_, [&] {
std::unique_lock<std::mutex> lock(mutex_);
while (use_count_ > 0)
if (!execute_next(lock))
condition_.wait(lock);
});
}
} catch (...) {
stop_threads();
threads_.join();
throw;
}
}
// The destructor waits for the pool to finish executing functions.
~fork_join_pool() {
stop_threads();
threads_.join();
}
private:
friend class fork_executor;
// The base for all functions that are queued in the pool.
struct function_base {
std::shared_ptr<std::size_t> work_count_;
void (*execute_)(std::shared_ptr<function_base>& p);
};
// Execute the next function from the queue, if any. Returns true if a
// function was executed, and false if the queue was empty.
bool execute_next(std::unique_lock<std::mutex>& lock) {
if (queue_.empty())
return false;
auto p(queue_.front());
queue_.pop();
lock.unlock();
execute(lock, p);
return true;
}
// Execute a function and decrement the outstanding work.
void execute(std::unique_lock<std::mutex>& lock,
std::shared_ptr<function_base>& p) {
std::shared_ptr<std::size_t> work_count(std::move(p->work_count_));
try {
p->execute_(p);
lock.lock();
do_work_finished(work_count);
} catch (...) {
lock.lock();
do_work_finished(work_count);
throw;
}
}
// Increment outstanding work.
void
do_work_started(const std::shared_ptr<std::size_t>& work_count) noexcept {
if (++(*work_count) == 1)
++use_count_;
}
// Decrement outstanding work. Notify waiting threads if we run out.
void
do_work_finished(const std::shared_ptr<std::size_t>& work_count) noexcept {
if (--(*work_count) == 0) {
--use_count_;
condition_.notify_all();
}
}
// Dispatch a function, executing it immediately if the queue is already
// loaded. Otherwise adds the function to the queue and wakes a thread.
void do_dispatch(std::shared_ptr<function_base> p,
const std::shared_ptr<std::size_t>& work_count) {
std::unique_lock<std::mutex> lock(mutex_);
if (queue_.size() > thread_count_ * 16) {
do_work_started(work_count);
lock.unlock();
execute(lock, p);
} else {
queue_.push(p);
do_work_started(work_count);
condition_.notify_one();
}
}
// Add a function to the queue and wake a thread.
void do_post(std::shared_ptr<function_base> p,
const std::shared_ptr<std::size_t>& work_count) {
std::lock_guard<std::mutex> lock(mutex_);
queue_.push(p);
do_work_started(work_count);
condition_.notify_one();
}
// Ask all threads to shut down.
void stop_threads() {
std::lock_guard<std::mutex> lock(mutex_);
--use_count_;
condition_.notify_all();
}
std::mutex mutex_;
std::condition_variable condition_;
std::queue<std::shared_ptr<function_base>> queue_;
std::size_t use_count_;
std::size_t thread_count_;
boost::asio::thread_pool threads_;
};
// A class that satisfies the Executor requirements. Every function or piece of
// work associated with a fork_executor is part of a single, joinable group.
class fork_executor {
public:
fork_executor(fork_join_pool& ctx)
: context_(ctx), work_count_(std::make_shared<std::size_t>(0)) {}
fork_join_pool& context() const noexcept { return context_; }
void on_work_started() const noexcept {
std::lock_guard<std::mutex> lock(context_.mutex_);
context_.do_work_started(work_count_);
}
void on_work_finished() const noexcept {
std::lock_guard<std::mutex> lock(context_.mutex_);
context_.do_work_finished(work_count_);
}
template <class Func, class Alloc>
void dispatch(Func&& f, const Alloc& a) const {
auto p(std::allocate_shared<exFun<Func>>(
typename std::allocator_traits<Alloc>::template rebind_alloc<char>(a),
std::move(f), work_count_));
context_.do_dispatch(p, work_count_);
}
template <class Func, class Alloc> void post(Func f, const Alloc& a) const {
auto p(std::allocate_shared<exFun<Func>>(
typename std::allocator_traits<Alloc>::template rebind_alloc<char>(a),
std::move(f), work_count_));
context_.do_post(p, work_count_);
}
template <class Func, class Alloc>
void defer(Func&& f, const Alloc& a) const {
post(std::forward<Func>(f), a);
}
friend bool operator==(const fork_executor& a, const fork_executor& b) noexcept {
return a.work_count_ == b.work_count_;
}
friend bool operator!=(const fork_executor& a, const fork_executor& b) noexcept {
return a.work_count_ != b.work_count_;
}
// Block until all work associated with the executor is complete. While it
// is waiting, the thread may be borrowed to execute functions from the
// queue.
void join() const {
std::unique_lock<std::mutex> lock(context_.mutex_);
while (*work_count_ > 0)
if (!context_.execute_next(lock))
context_.condition_.wait(lock);
}
private:
template <class Func> struct exFun : fork_join_pool::function_base {
explicit exFun(Func f, const std::shared_ptr<std::size_t>& w)
: function_(std::move(f)) {
work_count_ = w;
execute_ = [](std::shared_ptr<fork_join_pool::function_base>& p) {
Func tmp(std::move(static_cast<exFun*>(p.get())->function_));
p.reset();
tmp();
};
}
Func function_;
};
fork_join_pool& context_;
std::shared_ptr<std::size_t> work_count_;
};
// Helper class to automatically join a fork_executor when exiting a scope.
class join_guard {
public:
explicit join_guard(const fork_executor& ex) : ex_(ex) {}
join_guard(const join_guard&) = delete;
join_guard(join_guard&&) = delete;
~join_guard() { ex_.join(); }
private:
fork_executor ex_;
};
//------------------------------------------------------------------------------
#include <algorithm>
#include <iostream>
#include <random>
#include <vector>
#include <boost/bind.hpp>
static void foo(const uint64_t begin, uint64_t *result)
{
uint64_t prev[] = {begin, 0};
for (uint64_t i = 0; i < 1000000000; ++i) {
const auto tmp = (prev[0] + prev[1]) % 1000;
prev[1] = prev[0];
prev[0] = tmp;
}
*result = prev[0];
}
void batch(fork_join_pool &pool, const uint64_t (&a)[2])
{
uint64_t r[] = {0, 0};
{
fork_executor fork(pool);
join_guard join(fork);
boost::asio::post(fork, boost::bind(foo, a[0], &r[0]));
boost::asio::post(fork, boost::bind(foo, a[1], &r[1]));
// fork.join(); // or let join_guard destructor run
}
std::cerr << "foo(" << a[0] << "): " << r[0] << " foo(" << a[1] << "): " << r[1] << std::endl;
}
int main() {
fork_join_pool pool;
batch(pool, {2, 4});
batch(pool, {3, 5});
batch(pool, {7, 9});
}
Prints:
foo(2): 2 foo(4): 4
foo(3): 503 foo(5): 505
foo(7): 507 foo(9): 509
Things to note:
executors can overlap/nest: you can use several joinable fork_executors on a single fork_join_pool and they will join the distinct groups of tasks for each executor
You can get that sense easily when looking at the library example (which does a recursive divide-and-conquer merge sort).
I had a similar problem and ended up using latches. In this case the code would would be (I also switched from bind to lambdas):
void batch(boost::asio::thread_pool &pool, const uint64_t a[])
{
uint64_t r[] = {0, 0};
boost::latch latch(2);
boost::asio::post(pool, [&](){ foo(a[0], &r[0]); latch.count_down();});
boost::asio::post(pool, [&](){ foo(a[1], &r[1]); latch.count_down();});
latch.wait();
std::cerr << "foo(" << a[0] << "): " << r[0] << " foo(" << a[1] << "): " << r[1] << std::endl;
}
https://godbolt.org/z/oceP6jjs7
I am writing a sample kernel module which reads data sent through ioctl call from application and prints them.
I am passing structure "ioctl_struct" through ioctl from the application and in the kernel module, I will be printing its member variables.
this works absolutely fine in a few machines. In a few machines
"BUG: unable to handle kernel paging request at"
the error is thrown while accessing "name and testStruct's id1 and id2".
I don't think this module is hardware/kernel dependent.
I am not sure where it's going wrong. any help would be appreciated.
thanks.
Driver.c kernel module
static const char DEVICE_NAME[]="testipc";
static struct proc_dir_entry * proc_ipc = NULL;
struct test
{
int id1;
int id2;
};
struct ioctl_struct
{
__user struct test *testStruct;
__user int * id;
__user char * name;
int cmd;
};
static int __init etx_driver_init(void);
static void __exit etx_driver_exit(void);
static long etx_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
static struct file_operations fops =
{
.owner = THIS_MODULE,
.read = etx_read,
.write = etx_write,
.open = etx_open,
.unlocked_ioctl = etx_ioctl,
.release = etx_release,
.unlocked_ioctl = etx_ioctl,
};
static long etx_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
printk("reached ioctl....\n");
struct ioctl_struct buf;
if (copy_from_user(&buf, (void *)arg, sizeof(buf)))
return -EFAULT;
printk("succes..2\n");
printk("id %d\n",buf.id);
printk("cmd %d\n",buf.cmd);
printk("filename %s\n",buf.name);
printk("token %d\n",buf.testStruct->id1);
printk("token %d\n",buf.testStruct->id2);
return 0;
}
static int __init etx_driver_init(void)
{
printk("new test driver loaded..");
proc_ipc = proc_create(DEVICE_NAME, 0, NULL, &fops);
if (!proc_ipc)
{
printk(KERN_ALERT "Unable to create /proc/%s\n", DEVICE_NAME);
return 1;
}
return 0;
}
void __exit etx_driver_exit(void)
{
if (proc_ipc)
proc_remove(proc_ipc);
proc_ipc = NULL;
}
module_init(etx_driver_init);
module_exit(etx_driver_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("lin");
MODULE_DESCRIPTION("A simple driver");
MODULE_VERSION("1.0");
and following is my application file
#include <stdio.h>
#include<sys/ioctl.h>
# define __user
static int fd=NULL;
#define TEST_IOCTL _IOWR('z', 80, struct ioctl_struct)
struct test
{
int id1;
int id2;
};
struct ioctl_struct
{
__user struct test *testStruct;
__user int * id;
__user char * name;
int cmd;
};
void init()
{
printf("\nOpening Driver\n");
fd = open("/proc/testipc", O_RDWR);
if(fd < 0) {
printf("Cannot open device file...\n");
return 0;
}
}
void send()
{
int id=5;
int *pid=id;
char name[10]={'H','e','l','l','o'};
struct test testStruct;
testStruct.id1=44;
testStruct.id2=33;
struct ioctl_struct request;
request.name = name ;
request.id = pid;
request.cmd = 33;
request.testStruct = &testStruct;
ioctl(fd, TEST_IOCTL, &request);
}
void finish()
{
printf("Closing Driver\n");
close(fd);
}
int main()
{
init();
send();
finish();
return 0;
}
In dmesg,
id 5,
cmd 33,
Hello,
44,
33,
should be printed
I am trying to achieve the same as RapidCheck: call any Callable no matter its arguments.
Here is an example from RapidCheck:
#include <rapidcheck.h>
int main() {
rc::check("Addition is commutative.",
[](int a, int b) {
RC_ASSERT(a + b == b + a);
});
return 0;
}
Unlike RapidCheck, which uses random data to call the Callable, I
would like to use a source of data. In particular, I am casting a uint8_t array to whatever type I need. (See the example below.)
I am okay with using C++17 but would prefer C++11. I currently only have a C++17 example.
My example below works for an arbitrary number of arguments to a Callable but not for arbitrary types of argument. And certainly not for mixed types of arguments.
I am doing this, so that I can use the awesome API of RapidCheck with libFuzzer from LLVM. Similar to my previous approach here (Example).
What I have so far with some comments (online):
// Compiles with Clang(trunk)/GCC(7.2) using -std=c++17!
#include <cassert>
#include <functional>
#include <iostream>
#include <mutex>
#include <string>
#include <type_traits>
/** Provides access to a uint8_t array as specific types.
*
* Fulfills thus the LLVMFuzzerTestOneInput-interface, which uses
* (uint8_t *Data, size_t Size) as input.
*/
class RawQueue {
public:
RawQueue(uint8_t *Data, size_t Size)
: data_(Data), size_(Size / sizeof(uint8_t)), index_(0){};
/** Takes one element of type T from queue.
*
* Throws if empty.
*
* NOTE: Little endianess means that uint8_t {1, 0, 0, 0} == int {1}.
*/
template <typename T> T pop() {
assert(data_);
std::scoped_lock<std::mutex> lock(data_mutex_);
const size_t new_index = index_ + sizeof(T) / sizeof(uint8_t);
if (new_index > size_) {
std::runtime_error(
"Queue depleted!"); // TODO: Thou shall not use plain runtime_error!
}
const T val = *reinterpret_cast<const T *>(&(data_[index_]));
index_ = new_index;
return val;
}
private:
const uint8_t *data_; ///< Warning: Ownership resides outside of RawQueue.
std::mutex data_mutex_;
const size_t size_;
size_t index_;
};
template <> std::string RawQueue::pop<std::string>() {
return std::string("Left-out for brevity.");
};
template <typename T, typename F, typename... Args>
decltype(auto) call(RawQueue *Data, F &&f, Args &&... args) {
if constexpr (std::is_invocable<F, Args...>::value) {
return std::invoke(f, args...);
} else {
assert(Data);
// Is there a way to deduce T automatically and for each argument
// independently?
auto val = Data->pop<T>();
return call<T>(Data, f, val, args...);
}
}
int adder(int a, int b, int c) { return a + b + c; }
std::string greeter(const std::string &name) { return "Hello, " + name + "!"; }
int mixed_arguments(int i, float f, const std::string &s) { return 42; }
int main() {
constexpr size_t Size = 16;
uint8_t Data[Size] = {1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0};
RawQueue data(Data, Size);
auto res_int = call<int>(&data, adder);
std::cout << "Integer result: " << res_int << std::endl;
auto res = call<std::string>(&data, greeter);
std::cout << "String result: " << res << std::endl;
// Impossible with current approach:
// std::cout << "Mixed-types: " << call(&data, mixed_arguments) << std::endl;
return 0;
}
You could use variadic templates.
If you want to call, at runtime, an arbitrary function of arbitrary signature with arbitrary arguments, you should consider using libffi (a foreign function interface library which knows your ABI and calling conventions).
I found a solution using callable.hpp. Answers which do not rely on an external library are still welcome!
The relevant new addition is this:
constexpr size_t pos = sizeof...(args);
typedef typename callable_traits<F>::template argument_type<pos> T;
auto val = Data->pop<T>();
Complete Example:
#include <cassert>
#include <functional>
#include <iostream>
#include <mutex>
#include <string>
#include <type_traits>
#include "external/callable/callable.hpp"
class RawQueue {
public:
RawQueue(uint8_t *Data, size_t Size)
: data_(Data), size_(Size / sizeof(uint8_t)), index_(0){};
/** Takes one element of type T from queue.
*
* Throws if empty.
*
* NOTE: Little endianess means that uint8_t {1, 0, 0, 0} == int {1}.
*/
template <typename T> T pop() {
assert(data_);
std::scoped_lock<std::mutex> lock(data_mutex_);
const size_t new_index = index_ + sizeof(T) / sizeof(uint8_t);
if (new_index > size_) {
// TODO: Thou shall not use plain runtime_error!
std::runtime_error("Queue depleted!");
}
const T val = *reinterpret_cast<const T *>(&(data_[index_]));
index_ = new_index;
return val;
}
private:
const uint8_t *data_; ///< Warning: Ownership resides outside of RawQueue.
std::mutex data_mutex_;
const size_t size_;
size_t index_;
};
template <> std::string RawQueue::pop<std::string>() {
std::scoped_lock<std::mutex> lock(data_mutex_);
assert(data_);
assert(index_ < size_);
size_t string_length = data_[index_]; // Up-to 255 ought to be enough.
const size_t new_index =
index_ + string_length + 1; // +1 b/c first value is length of string.
if (new_index > size_) {
// TODO: Thou shall not use plain runtime_error!
std::runtime_error("Queue depleted!");
}
const std::string val(reinterpret_cast<const char *>(&(data_[index_ + 1])),
string_length);
index_ = new_index;
return val;
};
template <typename F, typename... Args>
decltype(auto) call(RawQueue *Data, F &&f, Args &&... args) {
if constexpr (std::is_invocable<F, Args...>::value) {
return std::invoke(f, args...);
} else {
assert(Data);
constexpr size_t n_already = sizeof...(args);
constexpr size_t n_needed = callable_traits<F>::argc;
static_assert(n_needed >= n_already, "Too many arguments!");
constexpr size_t pos = n_already;
typedef typename callable_traits<F>::template argument_type<pos> T;
auto val = Data->pop<T>();
return call(Data, f, args..., val);
}
}
int adder(int a, int b, int c) { return a + b + c; }
std::string greeter(std::string a) { return "hello " + a; };
void mixed(int i, float f, std::string s) {
std::cout << "Mixed: " << i << ", " << f << ", " << s << std::endl;
}
int main() {
constexpr size_t Size = 28;
// clang-format off
uint8_t Data[Size] = {
3, 'A', 'd', 'a',
1, 0, 0, 0,
2, 0, 0, 0,
4, 0, 0, 0,
42, 0, 0, 0,
0xDA, 0x0F, 0x49, 0x40, // 3.141...
3, 'P', 'i', '!'};
// clang-format on
RawQueue data(Data, Size);
std::cout << "String: " << call(&data, greeter) << std::endl;
std::cout << "Integers: " << call(&data, adder) << std::endl;
call(&data, mixed);
call(&data, []() { std::cout << "Nothing to do!" << std::endl; });
return 0;
}
Prints:
String: hello Ada
Integers: 7
Mixed: 42, 3.14159, Pi!
Nothing to do!
I am following this guide.
I used the following code to display LDAP servers in the domain:
#include <dns_sd.h>
#include <stdio.h>
#include <pthread.h>
void ResolveCallBack(DNSServiceRef sdRef,
DNSServiceFlags flags,
uint32_t interfaceIndex,
DNSServiceErrorType errorCode,
const char *fullname,
const char *hosttarget,
uint16_t port, /* In network byte order */
uint16_t txtLen,
const unsigned char *txtRecord,
void *context) {
}
void BrowserCallBack(DNSServiceRef inServiceRef,
DNSServiceFlags inFlags,
uint32_t inIFI,
DNSServiceErrorType inError,
const char* inName,
const char* inType,
const char* inDomain,
void* inContext) {
DNSServiceErrorType err = DNSServiceResolve(&inServiceRef,
0, // Indicate it's a shared connection.
inIFI,
inName,
inType,
inDomain,
ResolveCallBack,
NULL);
printf("DNSServiceResolve err = %x, name = %s, type=%s, domain=%s\n",
err, inName, inType, inDomain);
}
int main() {
DNSServiceRef ServiceRef;
DNSServiceErrorType err = DNSServiceBrowse(&ServiceRef, // Receives reference to Bonjour browser object.
kDNSServiceFlagsDefault, // Indicate it's a shared connection.
kDNSServiceInterfaceIndexAny, // Browse on all network interfaces.
"_ldap._tcp", // Browse for service types.
NULL, // Browse on the default domain (e.g. local.).
BrowserCallBack, // Callback function when Bonjour events occur.
NULL); // Callback context.
printf("err = 0x%x\n", err);
int sockfd = DNSServiceRefSockFD(ServiceRef);
printf("sockfd = %d\n", sockfd);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, 0);
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, 0);
struct timeval timeout;
timeout.tv_sec = 1;
timeout.tv_usec = 0;
fd_set descriptors;
FD_ZERO(&descriptors);
FD_SET(sockfd, &descriptors);
int r = select(sockfd + 1, &descriptors, NULL, NULL, &timeout);
printf("r = %d\n", r);
fflush(stdout);
if (r > 0) {
if (FD_ISSET(sockfd, &descriptors)) {
// This function will call the appropiate callback to process the
// event, in this case the BrowseReply static method.
err = DNSServiceProcessResult(ServiceRef);
if (err != kDNSServiceErr_NoError) {
printf("Error on process an event in event loop, e = 0x%x\n", err);
}
}
} else if (r == -1) {
printf("The select() call failed");
}
return 0;
}
However, this didn't give me any LDAP server.
Any Help on this?
Thanks in advance
N.B:
This command returns results:
$nslookup -type=any _ldap._tcp
So there is LDAP servers in the domain.
When I tried "_http._tcp" as the registration type this returns
results.
Operating system is Mac OS X 10.9.
I expect to get numbers from 0 to 4 in random order, but instead, I have some unsynchronized mess
What i do wrong?
#include <iostream>
#include <windows.h>
#include <process.h>
using namespace std;
void addQuery(void *v );
HANDLE ghMutex;
int main()
{
HANDLE hs[5];
ghMutex = CreateMutex( NULL, FALSE, NULL);
for(int i=0; i<5; ++i)
{
hs[i] = (HANDLE)_beginthread(addQuery, 0, (void *)&i);
if (hs[i] == NULL)
{
printf("error\n"); return -1;
}
}
printf("WaitForMultipleObjects return: %d error: %d\n",
(DWORD)WaitForMultipleObjects(5, hs, TRUE, INFINITE), GetLastError());
return 0;
}
void addQuery(void *v )
{
int t = *((int*)v);
WaitForSingleObject(ghMutex, INFINITE);
cout << t << endl;
ReleaseMutex(ghMutex);
_endthread();
}
You have to read and write the shared variable inside the lock. You are reading it outside of the lock and thus rendering the lock irrelevant.
But even that's not enough since your shared variable is a loop variable that you are writing to without protection of the lock. A much better example would run like this:
#include <iostream>
#include <windows.h>
#include <process.h>
using namespace std;
void addQuery(void *v );
HANDLE ghMutex;
int counter = 0;
int main()
{
HANDLE hs[5];
ghMutex = CreateMutex( NULL, FALSE, NULL);
for(int i=0; i<5; ++i)
{
hs[i] = (HANDLE)_beginthread(addQuery, 0, NULL);
if (hs[i] == NULL)
{
printf("error\n"); return -1;
}
}
printf("WaitForMultipleObjects return: %d error: %d\n",
(DWORD)WaitForMultipleObjects(5, hs, TRUE, INFINITE), GetLastError());
return 0;
}
void addQuery(void *v)
{
WaitForSingleObject(ghMutex, INFINITE);
cout << counter << endl;
counter++;
ReleaseMutex(ghMutex);
_endthread();
}
If you can, use a critical section rather than a mutex because they are simpler to use and more efficient. But they have the same semantics in that they only protect code inside the locking block.
Note: Jerry has pointer out some other problems, but I've concentrated on the high level trheading and serialization concerns.
Your synchronization has some issues as you want to get numbers from 0 to 4 in random order.
The problem is that the variable i is write outside the lock and every time the addQuery method get called by the execution of a thread, it get the modified version of variable i. That why you may see 5 as the value at the output for all.
So, here is my fix for this scenario. Instead of pass the address of variable i in parameters of the function addQuery, you should pass it's value. Hope it helps:
#include <iostream>
#include <windows.h>
#include <process.h>
using namespace std;
void addQuery(void *v);
HANDLE ghMutex;
int main()
{
HANDLE hs[5];
ghMutex = CreateMutex(NULL, FALSE, NULL);
for (int i = 0; i<5; ++i)
{
hs[i] = (HANDLE)_beginthread(addQuery, 0, (void *)i);
if (hs[i] == NULL)
{
printf("error\n"); return -1;
}
}
printf("WaitForMultipleObjects return: %d error: %d\n",
(DWORD)WaitForMultipleObjects(5, hs, TRUE, INFINITE), GetLastError());
return 0;
}
void addQuery(void *v)
{
int t = (int)v;
WaitForSingleObject(ghMutex, INFINITE);
cout << t << endl;
ReleaseMutex(ghMutex);
_endthread();
}