I read about this in one of my textbooks, where it is mentioned quite briefly, but didn't quite understand what it meant and how it is defined. What is process control block file format? Is it XML? And is it similar to manifest.xml files in ros packages?
The "Process Control Block" is an invention of some terrible books on operating systems to describe multiple data structures in a kernel. The initialism PCB in operating systems normally means Process Context Block, which is a data structure defined by a processor in which registers are stored in a context switch. XML does not appear in kernel data structures in any rationally designed operating system.
The process control block is never stored as a file, and therefore doesn't have a file format.
Instead, it'd be a structure (e.g. struct process_control_block { ... in C) with various fields (process name, number of threads, amount of CPU time consumed, ...) that are set when the process is created, and then used and modified while the process is running, and then discarded/de-allocated when the process is terminated.
Note that Linux doesn't call it "PCB", it's called "Task Control Block" (because it's used for both processes and threads). You can find it relatively easily: https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h#L723
Also; XML is so inefficient that anyone who uses it should probably be banned from touching a computer ever again. For a trivial example, a simple integer like 12345, which could be stored in 2 bytes with no need to waste CPU time parsing it; will become something like <integer>1234</integer> that takes 10 times as much space and 1000 times as many CPU cycles to parse. Advocates like to say it's "more human readable", but it's not (e.g. no way to tell if it's supposed to be kilowatts or micro-farads, or which range of numbers is acceptable, and it's unreadable for people who can't read English) - it's just an excuse not to have a decent tool (with internationalization, help, etc).
Related
We have a Windows application that interfaces with a sensor array. It reads out the array of 37 elements 10 times per second and appends the set of 35 32-bit integers and 2 16-bit integers to a csv file in the Documents folder on the C: drive
We have neither the application source nor access to the developer, who left the company a couple of years ago, nor specifications for the array to system protocols. We now want to perform real time analysis of the data, but all of the communications between the code and the array are effectively a black box.
I'm not a Windows system programmer, but a million years (in IT time) ago I was a designer for IBM OS/360 so I have a basic understanding of file system structures and it seems to me that it should be possible to somehow intercept file "open" and "write" calls to the OS and perform "near" real time analysis. Any good ideas how to do it? Preferably explained in terms that an 80 year old whio only dabbles in Python and C/C++ would comprehend? I've thought of a disassembler. or executing in a debugging environment that might be able to trap the I/O calls and pass control to an analysis routine, but I have no idea of what tools might be available these days in the Windows environment.
By the way, one other thing occurred to me - the app also outputs a plot of the data from each sensor - not sure if that's something we could get at .
There's no standard/supported way to hook into file I/O operations.
For this specific problem, the ideal solution will likely to be to use ReadDirectoryChangesW to watch the file for changes and read them out between updates; 100 ms should be more than enough time to pull out the data, unless it's a network drive or similar. This obviously won't work if the application is preventing you from reading the file between writes, though.
In an absolute worst case scenario, you can hook into the application's writes by injecting the process with a DLL that overwrites the first instructions of WriteFile (or whatever it's using to write) in kernel32.dll with a hook on load. You can read more about this process here.
xmalloc can be used in the process environment only when I write a AIX kernel extension.
what's the memory allocation functions can be called from the interrupt environment in AIX?
thanks.
The network memory allocation routines. Look in /usr/include/net/net_malloc.h. The lowest level is net_malloc and net_free.
I don't see much documentation in IBM's pubs nor the internet. There are a few examples in various header files.
There is public no prototype that I can find for these.
If you look in net_malloc.h, you will see MALLOC and NET_MALLOC macros defined that call it. Then if you grep in all the files under /usr/include, you will see uses of these macros. From these uses, you can deduce the arguments to the macros and thus deduce the arguments to net_malloc itself. I would make one routine that is a pass through to net_malloc that you controlled the interface to.
On your target system, do "netstat -m". The last bucket size you see will be the largest size you can call net_malloc with the M_NOWAIT flag. M_WAIT can be used only at process time and waits for netm to allocate more memory if necessary. M_NOWAIT returns with a 0 if there is not enough memory pinned. At interrupt time, you must use M_NOWAIT.
There is no real checking for the "type" but it is good to pick an appropriate type for debugging purposes later on. The netm output from kdb shows the type.
In a similar fashion, you can figure out how to call net_free.
Its sad IBM has chosen not to document this. An alternative to get this information officially is to pay for an "ISV" question. If you are doing serious AIX development, you want to become an ISV / Partner. It will save you lots of heart break. I don't know the cost but it is within reach of small companies and even individuals.
This book is nice to have too.
As the title says, I want to associate a random bit of data (ULONG) with a running process on the local machine. I want that data persisted with the process it's associated with, not the process thats reading & writing the data. Is this possible in Win32?
Yes but it can be tricky. You can't access an arbitrary memory address of another process and you can't count on shared memory because you want to do it with an arbitrary process.
The tricky way
What you can do is to create a window (with a special and known name) inside the process you want to decorate. See the end of the post for an alternative solution without windows.
First of all you have to get a handle to the process with OpenProcess.
Allocate memory with VirtualAllocEx in the other process to hold a short method that will create a (hidden) window with a special known name.
Copy that function from your own code with WriteProcessMemory.
Execute it with CreateRemoteThread.
Now you need a way to identify and read back this memory from another process other than the one that created that. For this you simply can find the window with that known name and you have your holder for a small chunk of data.
Please note that this technique may be used to inject code in another process so some Antivirus may warn about it.
Final notes
If Address Space Randomization is disabled you may not need to inject code in the process memory, you can call CreateRemoteThread with the address of a Windows kernel function with the same parameters (for example LoadLibrary). You can't do this with native applications (not linked to kernel32.dll).
You can't inject into system processes unless you have debug privileges for your process (with AdjustTokenPrivileges).
As alternative to the fake window you may create a suspended thread with a local variable, a TLS or stack entry used as data chunk. To find this thread you have to give it a name using, for example, this (but it's seldom applicable).
The naive way
A poor man solution (but probably much more easy to implement and somehow even more robust) can be to use ADS to hide a small data file for each process you want to monitor (of course an ADS associated with its image then it's not applicable for services and rundll'ed processes unless you make it much more complicated).
Iterate all processes and for each one create an ADS with a known name (and the process ID).
Inside it you have to store the system startup time and all the data you need.
To read back that informations:
Iterate all processes and check for that ADS, read it and compare the system startup time (if they mismatch then it means you found a widow ADS and it should be deleted.
Of course you have to take care of these widows so periodically you may need to check for them. Of course you can avoid this storing ALL these small chunk of data into a well-known location, your "reader" may check them all each time, deleting files no longer associated to a running process.
If one process does a write() of size (and alignment) S (e.g. 8KB), then is it possible for another process to do a read (also of size and alignment S and the same file) that sees a mix of old and new data?
The writing process adds a checksum to each data block, and I'd like to know whether I can use a reading process to verify the checksums in the background. If the reader can see a partial write, then it will falsely indicate corruption.
What standards or documents apply here? Is there a portable way to avoid problems here, preferably without introducing lots of locking?
When a function is guaranteed to complete without there being any chance of any other process/thread/anything seeing things in a half finished state, it's said to be atomic. It either has or hasn't happened, there is no part way. While I can't speak to Windows, there are very few file operations in POSIX (which is what Linux/BSD/etc attempt to stick to) that are guaranteed to be atomic. Reading and writing are not guaranteed to be atomic.
While it would be pretty unlikely for you to write 2 bytes to a file and another process only see one of those bytes written, if by dumb luck your write straddled two different pages in memory and the VM system had to do something to prepare the second page, it's possible you'd see one byte without the other in a second process. Usually if things are page aligned in your file, they will be in memory, but again you can't rely on that.
Here's a list someone made of what is atomic in POSIX, which is pretty short, and I can't vouch for it's authenticity. (I can't think of why unlink isn't listed, for example).
I'd also caution you against testing what appears to work and running with it, the moment you start accessing files over a network file system (NFS on Unix, or SMB mounts in Windows) a lot of things that seemed to be atomic before no longer are.
If you want to have a second process calculating checksums while a first process is writing the file, you may want to open a pipe between the two and have the first process write a copy of everything down the pipe to the checksumming process. That may be faster than dealing with locking.
I write a pdf viewer that uses various libraries written in C. This C code is potentially easy to exploit. And there are just too many lines to check. I will have to assume that this code may contain exploitable bugs.
The thing is that the C code is quite straightforward. A stream of bytes go in at one end, and a bitmap (also a stream of bytes) comes out at the other.
Inspired by google chrome, I am thinking to create a separate process that does the decoding and page rendering. Ideally this should be executed in a process that has absolutely no rights to do anything except reading the one input stream it has, and outputting to a stream of bytes (some uncompresed bitmap) at the other end.
What I think the process should not be able to do is:
any disk access
open sockets
limited amount of memory use
access shared memory with other processes
load other dll's
... anything else?
Is that possible? Is this described somewhere?
If you have the source code - you may check it doesn't do the described things.
Well, limiting available memory is a bit more difficult. You may however use SetProcessWorkingSetSize.
Also after you've built the executable you may check its DLL import table (by dependencies walker) to ensure it doesn't access any file/socket function.
This isn't really possible. Ultimately any potential exploit code will be running with whatever privileges this process runs with. If you run it as a standard user then you will limit the damage that could be done, but your best bet is to just fix the code as much as possible.