I have a process, set a timer to run some dynamic scripts to scrape data, each of those scripts run as a sub-process of my process. And I set cgroup limit to my process to limit cpu/mem, intend to limit some bad instruct in these scripts.
But some script includes some script like raising up a down process, these processes will be my process's child, but they should not be limited by my cgroup conf.
How to set to avoid these processes to be limited by my process's cgroup?
Related
I want to limit the memory usage of a child process using rlimit. Currently our code is as follows:
old_rlimit := get_rlimit()
set_rlimit(child_process_rlimit)
cmd.Start()
set_rlimit(old_rlimit)
cmd.Wait()
However, sometimes Golang runtime will report out of memory error at cmd.Start(). It seems that in cmd.Start() current process will allocate some memory, and if current memory usage is higher than child_process_rlimit, an error will be raised.
I want to know is there any way to limit the memory usage of child process without affecting current one?
You need to apply rlimit to the child process only rather than relying on rlimit inheritance. Your main blocker here is clearly spelled out in the setrlimit man page:
an unprivileged process may set only its soft limit to a value in the
range from 0 up to the hard limit, and (irreversibly) lower its hard
limit
The standard way to do this is through fork/exec, something along the lines of:
child_pid := fork()
if pid != 0 {
// in child process
setrlimit(...)
execve(...)
}
If you don't want to do that, you still have a few options:
run as privileged user
cmd.Run a small wrapper (eg: bash ulimit -m ...) that calls the child process. note: -m is not honored by many systems.
call prlimit (linux-specific and no syscall wrapper. see this thread for details)
If we are using the taskset for setting the particular process to run on a particular core by using the process pid. I want to know that all the child processes which will generated by the pid will also run on the particular core or uses any core.
taskset calls sched_setaffinity(2) (here) to specify the set of CPUs a thread is eligible to run on. As sched_setaffinity(2)'s manual states:
A child created via fork(2) inherits its parent's CPU affinity mask.
The affinity mask is preserved across an execve(2).
the child processes are allowed to run on the same set of CPUs as their parent.
Suppose in a two process environment, one process is scheduled for execution by the kernel, and it demanded for some data which is not available in the RAM. So the cpu will indicate the kernel that something is not available and the process will be suspended. Then after kernel loads the second process for execution through the CPU and start investigating about the data in secondary memory location (say virtual memory) and gets it, puts it back to main memory by a swap to the memory data which is currently inactive, and puts the process back in the ready queue for execution.
We know that everything in computer system is get manipulated by CPU only and if CPU is busy executing continuously the process code then who is executing the kernel code to perform the tasks done by kernel?
Please let me know if i am able to explain the scenario.
At any point in time, CPU (/s) will be
Running a process in User Mode.
Running on behalf of a process in Kernel Mode to execute previleged instruction or access hardware (for example when system call read / write is issued).
Running in repsonse to a hardware interrupt. i.e. running in interrupt context. (Not associated with any process in particular) and yes in kernel mode.
Running some kernel threads to serve deferred work like soft irq. (Tasklet / Softirq)
Running CPU idle thread if nothing is there to execute.
If you are in particular asking about scheduling, then
Suppose a process is running and now it has issued a read call to retrieve data from hard disk, say, then process is removed from cpu and kernel invokes schedule() functions. So here, first process issues read system call, which results in switching from user mode to kernel mode. The kernel which is running on behalf of the process prepares for the hard disk read operation and then calls schedule() function
Suppose a hardware interrupt has come, then currently running process is removed, and interrupt service handler for that interrupt begins to execute in kernel mode (obviously).
Basically, kernel runs in between user processes !!
Clear now ?
Shash
The kernel runs either as a result of a hardware interrupt, or as a result of being invoked by a process to do something. In both cases the code which was executing at that moment stops running until the kernel finishes its job.
It is similar to a function call: when function A calls function B, function A has to wait until function B is done doing what it does, and returns control to function A. You do not need multiple CPUs, or any kind of magic to accomplish this.
The CPU is not continuously executing process code. The CPU is interrupted to perform various operations. Interrupts can occur for various reasons: a resource becomes available, a previous action completes, or simply a timer goes off.
I recommend this series of videos for more in-depth information: http://academicearth.org/courses/operating-systems-and-system-programming
When you code a data supplier app in C for Unix that uses shared memory when do you detach the shared memory only when the server exits or when you are finished updating the shared memory ?
AFAIK, keeping it attached will not bother.
However since the attachment tracks the number of processes attached, if that count is >0, then you won't be allowed to destroy your shm until that count get back to 0 (in other words when all process are detached).
If you have a main process attached, I'm not sure you will be able to destroy it from an external "administrative" process for you shm.
In my personnal experience, I don't detach the SHM after write operations, only at process exit.
I want to write a program, which will launch a child process. The child process may be windows mode or console mode program.
I want to monitor the child process status and resource usage. e.g. I want to know the child process is still running or terminated. If it terminated, I want to know the reason (is terminated normally or because of crash?).
And during the child process running and/or it terminated, I want to know its resource usage, especially CPU time (user time, system) and memory usage (virtual size and/or rss). It is OK if the numbers are not very accurate.
In Unix terminology, I want to fork, exec, waitpid and getrusage . And fork+setrusage+exec can limit child's resource usage. But I don't know how to do these on the Windows platform.
Please point me the Windows API name. I could study the rest myself.
Prefer not using library other than the Windows API. Prefer it is not parent working as debugger and attaching to child process. Just not prefer, but still acceptable.
When you call CreateProcess, it returns a handle to the process.
WaitForSingleObject on a process handle will block until the process has exited or time-out has expired. A timeout of zero will return immediately and indicate if the process is still running.
BOOL IsProcessRunning(HANDLE process)
{
return WaitForSingleObject(process, 0) != WAIT_OBJECT_0;
}
void WaitForProcessToExit(HANDLE process)
{
WaitForSingleObject(process, INFINITE);
}
To get the exit code of a running process, you can use GetExitCodeProcess. You'll need to interpret what the error code means, however. 0xC0000005 is typical for an access violation, but not all crashes result in this error code.
For resource usage, you can call GetProcessTimes to get total CPU time, GetGuiResources to get GDI handle info, GetProcessMemoryInfo to get memory stats, and GetProcessIoCounters to get IO info.