I want to write a thread which runs tasks from an unlimited-size container of tasks.
While the task-list is empty the thread trying to get a task should be blocked.
Coming from Linux I wanted to use condition variable which will be signaled on task adding and will be waited while the list is empty.
I found that CONDITION_VARIABLE is available only from windows Vista, so this is out of question.
Semaphores are problematic too due to the unlimited-size restriction.
Is there any apropriate subtitution?
Thanks
Why do you say that semaphores are problematic? Linux/Windows both have semaphores with a maximum count that can be realistically be described as 'Unlimited'.
Use James' suggestion on Windows - it will work fine. Init. your semaphore with zero count. Add a task to your big (thread-safe), container, then signal the semaphore. In the thread, wait on the semaphore, then get a task from your container and process it. You can pass the semaphore instance to multiple threads if you wish - that will work OK as well.
Rgds,
Martin
Sounds like you want a Win32 kernel event. See CreateEvent.
WaitForSingleObject and CreateSemaphore?
Thanks all,
thats my conclusion:
void ThreadPool::ThreadStartPoint(ThreadPool* tp)
{
while (1)
{
WaitForSingleObject(tp->m_taskCountSemaphore,INFINITE); // while (num of tasks==0) block; decreament num of tasks
BaseTask* current_task = 0;
// get top priority task
EnterCriticalSection (&tp->m_mutex);
{
current_task = tp->m_tasksQue.top();
tp->m_tasksQue.pop();
}
LeaveCriticalSection (&tp->m_mutex);
current_task->operator()(); // this is not critical section
current_task->PostExec();
}
}
void ThreadPool::AddTask(BaseTask& _task)
{
EnterCriticalSection (&m_mutex);
{
m_tasksQue.push(&_task);
_task.PrepareTask(m_mutex);
}
LeaveCriticalSection (&m_mutex);
if (!ReleaseSemaphore(m_taskCountSemaphore,
1, // increament num of tasks by 1
NULL // don't store previuos num of tasks value
))
{//if failed
throw ("semaphore release failed");
}
}
Related
Background
It is possible to perform a software-controlled disconnection of the power adapter of a Mac laptop by creating an DisableInflow power management assertion.
Code from this answer to an SO question can be used to create said assertion. The following is a working example that creates this assertion until the process is killed:
#include <IOKit/pwr_mgt/IOPMLib.h>
#include <unistd.h>
int main()
{
IOPMAssertionID neverSleep = 0;
IOPMAssertionCreateWithName(kIOPMAssertionTypeDisableInflow,
kIOPMAssertionLevelOn,
CFSTR("disable inflow"),
&neverSleep);
while (1)
{
sleep(1);
}
}
This runs successfully and the power adapter is disconnected by software while the process is running.
What's interesting, though, is that I was able to run this code as a regular user, without root privileges, which wasn't supposed to happen. For instance, note the comment in this file from Apple's open source repositories:
// Disables AC Power Inflow (requires root to initiate)
#define kIOPMAssertionTypeDisableInflow CFSTR("DisableInflow")
#define kIOPMInflowDisableAssertion kIOPMAssertionTypeDisableInflow
I found some code which apparently performs the actual communication with the charger; it can be found here. The following functions, from this file, appears to be of particular interest:
IOReturn
AppleSmartBatteryManagerUserClient::externalMethod(
uint32_t selector,
IOExternalMethodArguments * arguments,
IOExternalMethodDispatch * dispatch __unused,
OSObject * target __unused,
void * reference __unused )
{
if (selector >= kNumBattMethods) {
// Invalid selector
return kIOReturnBadArgument;
}
switch (selector)
{
case kSBInflowDisable:
// 1 scalar in, 1 scalar out
return this->secureInflowDisable((int)arguments->scalarInput[0],
(int *)&arguments->scalarOutput[0]);
break;
// ...
}
// ...
}
IOReturn AppleSmartBatteryManagerUserClient::secureInflowDisable(
int level,
int *return_code)
{
int admin_priv = 0;
IOReturn ret = kIOReturnNotPrivileged;
if( !(level == 0 || level == 1))
{
*return_code = kIOReturnBadArgument;
return kIOReturnSuccess;
}
ret = clientHasPrivilege(fOwningTask, kIOClientPrivilegeAdministrator);
admin_priv = (kIOReturnSuccess == ret);
if(admin_priv && fOwner) {
*return_code = fOwner->disableInflow( level );
return kIOReturnSuccess;
} else {
*return_code = kIOReturnNotPrivileged;
return kIOReturnSuccess;
}
}
Note how, in secureInflowDisable(), root privileges are checked for prior to running the code. Note also this initialization code in the same file, again requiring root privileges, as explicitly pointed out in the comments:
bool AppleSmartBatteryManagerUserClient::initWithTask(task_t owningTask,
void *security_id, UInt32 type, OSDictionary * properties)
{
uint32_t _pid;
/* 1. Only root processes may open a SmartBatteryManagerUserClient.
* 2. Attempts to create exclusive UserClients will fail if an
* exclusive user client is attached.
* 3. Non-exclusive clients will not be able to perform transactions
* while an exclusive client is attached.
* 3a. Only battery firmware updaters should bother being exclusive.
*/
if ( kIOReturnSuccess !=
clientHasPrivilege(owningTask, kIOClientPrivilegeAdministrator))
{
return false;
}
// ...
}
Starting from the code from the same SO question above (the question itself, not the answer), for the sendSmartBatteryCommand() function, I wrote some code that calls the function passing kSBInflowDisable as the selector (the variable which in the code).
Unlike the code using assertions, this one only works as root. If running as a regular user, IOServiceOpen() returns, weirdly enough, kIOReturnBadArgument (not kIOReturnNotPrivileged, as I would have expected). Perhaps this has to do with the initWithTask() method above.
The question
I need to perform a call with a different selector to this same Smart Battery Manager kext. Even so, I can't even get to the IOConnectCallMethod() since IOServiceOpen() fails, presumably because the initWithTask() method prevents any non-root users from opening the service.
The question, therefore, is this: how is IOPMAssertionCreateWithName() capable of creating a DisableInflow assertion without root privileges?
The only possibility I can think of is if there's a root-owned process to which requests are forwarded, and which performs the actual work of calling IOServiceOpen() and later IOConnectCallMethod() as root.
However, I'm hoping there's a different way of calling the Smart Battery Manager kext which doesn't require root (one that doesn't involve the IOServiceOpen() call.) Using IOPMAssertionCreateWithName() itself is not possible in my application, since I need to call a different selector within that kext, not the one that disables inflow.
It's also possible this is in fact a security vulnerability, which Apple will now fix in a future release as soon as it is alerted to this question. That would be too bad, but understandable.
Although running as root is a possibility in macOS, it's obviously desirable to avoid privilege elevation unless absolutely necessary. Also, in the future I'd like to run the same code under iOS, where it's impossible to run anything as root, in my understanding (note this is an app I'm developing for my own personal use; I understand linking to IOKit wipes out any chance of getting the app published in the App Store).
I have created a kernel thread using kthread_run in a kernel module.
The thread is very simple, just like bellow.
static int my_thread_func(void * data)
{
int a;
DBG_PRINT("policy:%lu; prio:%d", current->policy, current->prio);
while (!kthread_should_stop())
{
a++;
}
}
However, after I loaded the module, the system did not response any more.
So I wonder what's the schedule policy and priority of this kernel thread.
Then I try to print out the schedule policy and priority of this kernel thread,
and got bellow output.
policy:0; prio:120
policy:0 means SCHED_NORMAL;
prio:120 this is also not high.
While the thread does not have a SCHED_FIFO or SCHED_RR schedule policy, Why can it hog up the cpu?
And I also found that if I insert some sleep code in the loop body of the thread, the system could remain responsive.
And I also found when I run a userspace program implemented as bellow, the system remained responsive, too.
int main(int argc, char *argv[])
{
int a;
while (1) a++;
return 0;
}
So who can tell me, why the kernel thread could hog up the cpu.
When you say the priority is 120, are you observing the priority of kthreadd, or the actual kernel thread that was created for you?
Please see http://lxr.free-electrons.com/source/kernel/kthread.c#L310 for the function that is used to create new kernel threads.
Excerpt:
struct task_struct *kthread_create_on_node(int (*threadfn)(void *data),
void *data, int node,
const char namefmt[],
...)
{
...
if (!IS_ERR(task)) {
static const struct sched_param param = { .sched_priority = 0 };
...
/*
* root may have changed our (kthreadd's) priority or CPU mask.
* The kernel thread should not inherit these properties.
*/
sched_setscheduler_nocheck(task, SCHED_NORMAL, ¶m);
set_cpus_allowed_ptr(task, cpu_all_mask);
}
...
}
It appears that the sched_priority is set to 0.
Now, please look at http://lxr.free-electrons.com/source/include/linux/sched/prio.h#L9
Excerpt:
/*
* Priority of a process goes from 0..MAX_PRIO-1, valid RT
* priority is 0..MAX_RT_PRIO-1, and SCHED_NORMAL/SCHED_BATCH
* tasks are in the range MAX_RT_PRIO..MAX_PRIO-1. Priority
* values are inverted: lower p->prio value means higher priority.
*
* The MAX_USER_RT_PRIO value allows the actual maximum
* RT priority to be separate from the value exported to
* user-space. This allows kernel threads to set their
* priority to a value higher than any user task. Note:
* MAX_RT_PRIO must not be smaller than MAX_USER_RT_PRIO.
*/
Please note the second paragraph: This allows kernel threads to set their priority to a value higher than any user task.
Summary:
It appears that newly created kernel threads are set to sched_priority of 0 even with SCHED_NORMAL. Priority of 0 is not a normal priority for SCHED_NORMAL, and thus the kernel thread will take priority over any other thread (that isn't using an RT policy---and I don't think any processes in the kernel use RT by default).
Addendum:
Note: I am not 100% sure if this is the reason. But, if you look at the comments in the kernel for kernel threads, they all seem to imply that a kernel thread keeps running UNLESS:
The kernel thread itself yields or calls do_exit.
Someone else calls kthread_should_stop().
Which to me, sounds like the kernel thread runs as long as it wants, until it decides to stop, or someone else explicitly tells it to stop.
I created a method to prevent the system from sleeping as follows:
public static void KeepSystemAwake(bool bEnable)
{
if (bEnable)
{
EXECUTION_STATE state = SetThreadExecutionState(EXECUTION_STATE.ES_DISPLAY_REQUIRED | EXECUTION_STATE.ES_CONTINUOUS);
}
else
{
EXECUTION_STATE state = SetThreadExecutionState(EXECUTION_STATE.ES_CONTINUOUS);
}
}
The method prevents the system from sleep but when I call the ES_CONTINUOUS part of the method,the system does not sleep at all when I want it behave normally. What am I missing? I'm running this code in a different thread (Timer)
I'm running this code in a different thread (Timer)
If you're using something like a System.Threading.Timer callback, it will be called on different (read: arbitrary) threads.
From MSDN:
The callback method executed by the timer should be reentrant, because it is called on ThreadPool threads
Make sure you're calling SetThreadExecutionState for the same thread. Ideally, you'll serialise calls onto one thread (like the main thread).
The run method of my worker role is:
public override void Run()
{
Message msg=null;
while (true)
{
msg = queue.GetMessage();
if(msg!=null && msg.DequeueCount==1){
//delete message
...
//execute operations
...
}
else if(msg!=null && msg.DequeueCount>1){
//delete message
...
}
else{
int randomTime = ...
Thread.Sleep(randomTime);
}
}
}
For performance tests I would that a message could be analysed only by a worker (I don't consider failure problems on workers).
But seems by my tests, that two workers can pick up the same message and read DequeueCount equals to 1 (both workers). Is it possible?
Does exist a way that allow just a worker to read a message in a "mutex" way?
How is your "getAMessage(queue)" method defined? If you do PeekMessage(), a message will be visible by all workers. If you do GetMessage(), the message will be got only by the worker which firsts get it. But for the invisibility timeout either specified or the default (30 sec.). You have to delete the message before the invisibility timeout comes.
Check out the Queue Service API for more information. I am sure that there is something wrong in your code. I use queues and they behave as by documentation in dev storage and in production storage. You may want to explicitly put higher value of the Visibility Timeout when you do GetMessage. And make sure you do not sleep longer than the visibility timeout.
In C++ Windows app, I launch several long running child processes (currently I use CreateProcess(...) to do this.
I want the child processes to be automatically closed if my main processes crashes or is closed.
Because of the requirement that this needs to work for a crash of the "parent", I believe this would need to be done using some API/feature of the operating system. So that all the "child" processes are cleaned up.
How do I do this?
The Windows API supports objects called "Job Objects". The following code will create a "job" that is configured to shut down all processes when the main application ends (when its handles are cleaned up). This code should only be run once.:
HANDLE ghJob = CreateJobObject( NULL, NULL); // GLOBAL
if( ghJob == NULL)
{
::MessageBox( 0, "Could not create job object", "TEST", MB_OK);
}
else
{
JOBOBJECT_EXTENDED_LIMIT_INFORMATION jeli = { 0 };
// Configure all child processes associated with the job to terminate when the
jeli.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
if( 0 == SetInformationJobObject( ghJob, JobObjectExtendedLimitInformation, &jeli, sizeof(jeli)))
{
::MessageBox( 0, "Could not SetInformationJobObject", "TEST", MB_OK);
}
}
Then when each child process is created, execute the following code to launch each child each process and add it to the job object:
STARTUPINFO info={sizeof(info)};
PROCESS_INFORMATION processInfo;
// Launch child process - example is notepad.exe
if (::CreateProcess( NULL, "notepad.exe", NULL, NULL, TRUE, 0, NULL, NULL, &info, &processInfo))
{
::MessageBox( 0, "CreateProcess succeeded.", "TEST", MB_OK);
if(ghJob)
{
if(0 == AssignProcessToJobObject( ghJob, processInfo.hProcess))
{
::MessageBox( 0, "Could not AssignProcessToObject", "TEST", MB_OK);
}
}
// Can we free handles now? Not sure about this.
//CloseHandle(processInfo.hProcess);
CloseHandle(processInfo.hThread);
}
VISTA NOTE: See AssignProcessToJobObject always return "access denied" on Vista if you encounter access-denied issues with AssignProcessToObject() on vista.
One somewhat hackish solution would be for the parent process to attach to each child as a debugger (use DebugActiveProcess). When a debugger terminates all its debuggee processes are terminated as well.
A better solution (assuming you wrote the child processes as well) would be to have the child processes monitor the parent and exit if it goes away.
Windows Job Objects sounds like a good place to start. The name of the Job Object would have to be well-known, or passed to the children (or inherit the handle). The children would need to be notice when the parent dies, either through a failed IPC "heartbeat" or just WFMO/WFSO on the parent's process handle. At that point any child process could TermianteJobObject to bring down the whole group.
You can keep a separate watchdog process running. Its only task is watching the current process space to spot situations like you describe. It could even re-launch the original application after a crash or provide different options to the user, collect debug information, etc. Just try to keep it simple enough so that you don't need a second watchdog to watch the first one.
You can assign a job to the parent process before creating processes:
static HANDLE hjob_kill_on_job_close=INVALID_HANDLE_VALUE;
void init(){
hjob_kill_on_job_close = CreateJobObject(NULL, NULL);
if (hjob_kill_on_job_close){
JOBOBJECT_EXTENDED_LIMIT_INFORMATION jobli = { 0 };
jobli.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
SetInformationJobObject(hjob_kill_on_job_close,
JobObjectExtendedLimitInformation,
&jobli, sizeof(jobli));
AssignProcessToJobObject(hjob_kill_on_job_close, GetCurrentProcess());
}
}
void deinit(){
if (hjob_kill_on_job_close) {
CloseHandle(hjob_kill_on_job_close);
}
}
JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE causes all processes associated with the job to terminate when the last handle to the job is closed. By default, all child processes will be assigned to the job automatically, unless you passed CREATE_BREAKAWAY_FROM_JOB when calling CreateProcess. See https://learn.microsoft.com/en-us/windows/win32/procthread/process-creation-flags for more information about CREATE_BREAKAWAY_FROM_JOB.
You can use process explorer from Sysinternals to make sure all processes are assigned to the job. Just like this:
You'd probably have to keep a list of the processes you start, and kill them off one by one when you exit your program. I'm not sure of the specifics of doing this in C++ but it shouldn't be hard. The difficult part would probably be ensuring that child processes are shutdown in the case of an application crash. .Net has the ability to add a function that get's called when an unhandled exception occurs. I'm not sure if C++ offers the same capabilities.
You could encapsulate each process in a C++ object and keep a list of them in global scope. The destructors can shut down each process. That will work fine if the program exits normally but it it crashes, all bets are off.
Here is a rough example:
class myprocess
{
public:
myprocess(HANDLE hProcess)
: _hProcess(hProcess)
{ }
~myprocess()
{
TerminateProcess(_hProcess, 0);
}
private:
HANDLE _hProcess;
};
std::list<myprocess> allprocesses;
Then whenever you launch one, call allprocessess.push_back(hProcess);