So as we know when page fault occurs:
(1)the hardware traps to the kernel
(2) saves program counter on the stack
(3)assembly-code routine saves all registers on corresponding process table entry
(4)page fault handler discovers which virtual page is needed
(5)if everything is valid and no page frames are free, the page replacement algorithm is run to select a victim
(6) If the page frame selected is dirty, the page is scheduled for transfer to the disk, and a context switch takes place, suspending the faulting process and letting another one run until the disk transfer has completed
(7)and after step 6 is another 4 steps to bring in the corresponding page from backing store in RAM and so on ......
what i dont understand is that when at step 6 context switch occurs this means that page fault handler must suspend until disk transfer has completed , but how is it then posible to resume page fault handler at step 7>? as i know all handlers(Interrupt , trap , fault) are running in kernel and they are not processes
Related
Windows Context Switching
The scheduler maintains a queue of executable threads for each
priority level. These are known as ready threads. When a processor
becomes available, the system performs a context switch. The steps in
a context switch are:
Save the context of the thread that just finished executing.
Place the thread that just finished executing at the end of the queue for its priority.
Find the highest priority queue that contains ready threads.
Remove the thread at the head of the queue, load its context, and execute it.
I don't know much about the topic yet, so I don't know how to elaborate on my question. Where is a thread's context saved, and can it be accessed (edit: read) programmatically (without modifying the kernel)?
If you have a handle to a thread with the required access rights you can suspend the thread and then call GetThreadContext. When a thread is running the values are in the real CPU registers, when it is not running the context is stored in memory not accessible from usermode.
The context stores the values of various CPU registers, it is only useful to debuggers and advanced features like code injection and error logging.
I read this question
I/O Completion Ports *LAST* called callback, or: where it's safe to cleanup things
And i can not get my issue solved. The answer does not fully cover this method.
I have also searched a lot here and in google but can not find a solution so i am opening a question here hope that is not duplicated.
In a multi-threaded IO Completion ports design when to increase the RefCount of the Per Socket structure? ie the CompletionKey. Currently i do increase it before calling WSARecv and if the return value of the call is not 0 or ERROR_IO_PENDING by last error, i decrease it and call a cleanup function, this function will check if the RefCount is 0, if it is then it will free the Per Socket structure. Else it will just free the Per IO structure (the one of OVERLAPPED), i also increase it before issuing any WSASend using the same way mentioned above. This RefCount is atomic using CRITCAL_SECTION. Upon returning from GetQueuedCompletionStatus i also decrease the RefCount.
However i have some questions about this method
I have a function that sends files from the main thread, the function is reading the file and issuing a PostQueuedCompletionStatus to do a send using WSASend through the IO Worker threads, the function sends file in chunks and when each chunk completes the IO Worker threads will inform the main thread with PostMessage to issue another send of the next chunk.
Now where i am supposed to increase this RefCount? in the main thread just before issuing a call to PostQueuedCompletionStatus? but what if a return from GetQueuedCompletionStatus returned and freed the Per Socket structure and the main thread still using it? (Example the main thread is executing the send function but not yet increased the RefCount) i tried to increase RefCount in the WSASend function in the IO Worker threads but it is the same issue
For instance: what if a thread woke up from GetQueuedCompletionStatus as a socket closure(caused by the outstanding WSARecv) and decrement the RefCount and it became 0 so it will free the Per Socket structure while a WSASend is executing in another IO Worker thread but not yet increased the RefCount? then obviously the thread that is about to issue a WSASend call will crash with access violation whenever it tries to enter the critical section.
Any idea how to synchronize access to this structure between IO Worker threads and the main thread?
I have a working GUI and now need to add some code that will need to run continuously and update the GUI with data. Where should this code go? I know that it should not go into the message loop because it might block incoming messages to the window, but I'm confused on where in my window process this code could run.
You have a choice: you can use a thread and post messages back to the main thread to update the GUI (or update the GUI directly, but don't try this if you used MFC), or you can use a timer that will post you messages periodically, you then simply implement a handler for the timer and do whatever you need to there.
The thread is best for a complicated, slow process that might block. If the process of getting data is quick (and/or can be set to timeout on error) then a timer is simpler.
Have you looked into threading at all?
Typically, you would create one thread that performs the background task (in this case, reading the voltage data) and storing it into a shared buffer. The GUI thread simply reads that buffer every so often (on redraw, every 30 seconds, when the user clicks refresh, etc) and displays the data.
Your background thread runs on its own schedule, getting CPU time from the OS, and is not bound to the UI or message pump. It can use some type of timer to monitor the data source and read things in as necessary.
Now, since the threads run separately and may run at the same time, you need to make them aware of one another. This can be done with locks (look into mutexes). For example:
The monitor reads the current voltage and stores it in the buffer.
The background/monitor thread locks the buffer holding the latest sample.
The monitor copies the internal buffer to the shared one.
The monitor unlocks the buffer.
Simultaneously, but separately, the UI thread:
Gets a redraw call.
Waits for the buffer to be unlocked, then reads the value.
Draws the UI with the buffer value.
Setting up a new thread and using it, in most Windows GUI-producing languages, is pretty simple. C/++ and C# both have very simple APIs for creating a new thread and having it work on some task, you usually just need to provide a function for the thread to process. See the MSDN docs on CreateThread for a C example.
The concept of threading and locking is for the most part language-agnostic, and similar in most C-inspired languages. You'll need to have your main (in this case, probably UI) thread control the lifetime of the worker: start the worker after the UI is created, and kill it before the UI is shut down.
This approach has a little bit of overhead up front, especially if your data fetch is very simple. If your data source changes (a network request, some blocking data source, reading over actual wires from a physical sensor, etc) then you only need to change the monitor thread and the UI doesn't need to know.
In my kernel configuration CONFIG_PREEMPT is not set. Since schedule() is not allowed in interrupt handler how does round robin type of scheduling is implemented in linux kernel. i.e. Who calls the scheduler so often. In entry_32.S it calls preempt_schedule_irq only if CONFIG_PREEMPT is set.
What happens is the timer on the CPU is set to interrupt the kernel every so often. But we can't just call schedule from interrupt context right? So what the kernel does is a neat trick. It changes the currently executing task while executing the handler and then returns. What this effectively does is switch out the context from underneath the handler so the handler completes but at the same time the next context to run is now the next task that will execute. Read up on do_context_switch (IIRC I think that's what it's called) and you will see that it switches it's stack and context from underneath the current execution and resumes the same function in another context.
And CONFIG_PREEMPT only applies to kernel code preemption in kernel context. Userspace tasks will always preempt. All this means is that any kernel code that starts to execute runs to completion (unless you call schedule() yourself or block waiting for I/O, etc....). Normally the kernel can preempt as long as it does not hold any locks except in certain cases where acquiring a lock can put the thread to sleep.
I have an application with the following pattern:
2 long running processes that go into hibernate after some idle time
and their memory consumption goes down as expected
N (0 < N < 100) worker processes that do some work and hibernate when idle more than
10 seconds or terminate if idle more than two hours
during the night,
when there is no activity the process memory goes back to almost the
same value that was at the application start, which is expected as
all the workers have died.
The issue is that "system" section keeps growing (around 1GB/week).
My question is how can I debug what is stored there or who's allocating memory in that area and is not freeing it.
I've already tested lists:keysearch/3 and it doesn't seem to leak memory, as that is the only native thing I'm using (no ports, no drivers, no NIFs, no BIFs, nothing). Erlang version is R15B03.
Here is the current erlang:memory() output (slight traffic, app started on Feb 03):
[{total,378865650},
{processes,100727351},
{processes_used,100489511},
{system,278138299},
{atom,1123505},
{atom_used,1106100},
{binary,4493504},
{code,7960564},
{ets,489944},
{maximum,402598426}]
This is a 64-bit system. As you can see, "system" section has ~270MB and "processes" is at around 100MB (that drops down to ~16MB during the night).
It seems that I've found the issue.
I have a "process_killer" gen_server where processes can subscribe for periodic GC or kill. Its subscribe functions are called on each message received by some processes to postpone the GC/kill (something like re-arm).
This process performs an erlang:monitor if not already monitored to catch a dead process and remove it from watch list. If I comment our the re-subscription line on each handled message, "system" area seems to behave normally. That means it is a bug in my process_killer that does leak monitor refs (remember you can call erlang:monitor multiple times and each call creates a reference).
I was lead to this idea because I've tested a simple module which was calling erlang:monitor in a loop and I have seen ~13 bytes "system" area grow on each call.
The workers themselves were OK because they would die anyway taking their monitors along with them. There is one long running (starts with the app, stops with the app) process that dispatches all the messages to the workers that was calling GC re-arm on each received message, so we're talking about tens of thousands of monitors spawned per hour and never released.
I'm writing this answer here for future reference.
TL;DR; make sure you are not leaking monitor refs on a long running process.