Is there any way in the kernel to get a list of blocks that have been physically written to and the order in which they were written?
Looks like blktrace can produce this information.
Related
New to this forum - looks great!
I have some Processing code that periodically reads data wirelessly from remote devices and writes that data as bytes to a file, e.g. data.dat. I want to write an Objective C program on my Mac Mini using Xcode to read this file, parse the data, and act on the data if data values indicate a problem. My question is: can my two different programs access the same file asynchronously without a problem? If this is a problem can you suggest a technique that will allow these operations?
Thanks,
Kevin H.
Multiple processes can read from the same file at a time without any problem. A process can also read from a file while another writes without problem, although you'll have to take care to ensure that you read in any new data that was written. Multiple processes should not write to the same file at at the same time, though. The OS will let you do it, but the ordering of data will be undefined, and you'll like overwrite data—in general, you're gonna have a bad time if you do that. So you should take care to ensure that only one process writes to a file at a time.
The simplest way to protect a file so that only one process can write to it at a time is with the C function flock(), although that function is admittedly a bit rudimentary and may or may not suit your use case.
Say I have app TestApp.exe
While TestApp.exe is running I want a separate program to be able to read the executable code that is resident in memory. I'd like to ignore stack and heap and anything else that is tangential.
Put another way, I guess I'm asking how to determine where the memory-side equivalent of the .exe binary data on disk resides. I realize it's not a 1:1 stuffing into memory.
Edit: I think what I'm asking for is shown as Image in the following screenshot of vmmap.exe
Edit: I am able to get from memory all memory that is tagged with any protect flag of Execute* (PAGE_EXECUTE, etc) using VirtualQueryEx and ReadProcessMemory. There are a couple issues with that. First, I'm grabbing about 2 megabytes of data for notepad.exe which is a 189 kilobyte file on disk. Everything I'm grabbing has a protect flag of PAGE_EXECUTE. Second, If I run it on a different Win7 64bit machine I get the same data, only split in half and in a different order. I could use some expert guidance. :)
Edit: Also, not sure why I'm at -1 for this question. If I need to clear anything up please let me know.
Inject a DLL to the target process and call GetModuleHandle with the name of the executable. That will point to its PE header that has been loaded in the memory. Once you have this information, you can parse the PE header manually and find where .text section is located relative to the base address of the image in the memory.
no need to inject a dll
use native api hooking apis
I learned a ton doing this project. I ended up parsing the PE header and using that information to route me all over. In the end I accomplished what I set out to and I am more knowledgeable as a result.
I have an application that traces program execution through memory. I tried to use readelf --debug-dump=decodedline to get memory address / line # information, but the memory addresses I see don't match up often with the ones given by that dump. I wrote something to match up each address with the "most recent" one appearing in the DWARF data -- this seemed to clean some things up but I'm not sure if that's the "official" way to interpret this data.
Can someone explain the exact process to map a program address to line number using DWARF?
Have a look at the program addr2line. It can probably give you some guidance on how to do this, if not solving your problem entirely (e.g. by shelling out to it, or linking its functionality in).
Indeed, as mentioned by Phil Miller's answer, addr2line is your friend. I have a gist where I show how I get the line number in the (C++) application source code from an address obtained from a backtrace.
Following this process will not show you the process you mention, but can give you an idea of how the code gets mapped into the object code (in an executable or a library/archive). Hope it helps.
I'm using the GoAsm assembler on a Windows 7 - 64 bit OS and I'll be asking you a few (not so dumb) questions.
First question :
How can I find the physical address of a file ?
Let's suppose file "Text.txt" is at the root of my C:\ partition.
Is there a way to get the exact memory address where this file is ?
Second question :
Is it possible to call a routine which will just do like if I invoked a C function ?
(i.e. : Consider a C function "WriteToScreen", is it possible to have the same function, but in assembler format, that means without having the need to use high-level invokes to do that work ?
Third question :
Are there somewhere on the net some include files for GoAsm containing useful routines like (move, copy, edit, erase) commands ? I've first thought of ms-dos interrupts but I can't manage to get them to work without crashing the program. I guess it just not compatible with Windows OS even though the command prompt acts like ms-dos... ?
Fourth question :
I've heard from different sources and myself that NASM works pretty bad on Win7 x64, is it just true, or am I doing it the wrong way ?
1
An hard drive, from a logical point of view, can be seen as a sequence of "blocks" (the more common name is sectors). How these blocks are organized physically on the disks can be disregarded, but the driver must know someway how to get data of course, though you send to modern hd driver "high level" commands that, as far as you know, are not strongly related to where data physically are (you can say "read the block 123", but there's no extern evidence of where that block lives).
However this way you can "name" a block with a number, and say e.g. that block 0 is the MBR. Each block contains several bytes (512, 1024...). Not all used blocks contain actual data of a file, in fact there are metainformations of any sort, depending on the filesystem but even related to the "structure" of the hd (I mean, partitions).
A file located on an hd is not automatically loaded into memory, so it has no memory address. Once you read it, piece of it if not all are of course copied into the memory you give, which is not an intrinsic property of the file. (Filesystems retrieve the blocks belonging to the file and "show" them as we are used to see them, as a single "unit", the file)
Summarizing: files have no memory address. The physical address could be the set of blocks holding data (and metadata, like inodes ) of the file, or just the first block (but if a block of data is N, N+1 could not belong to the same file - the blocks need no to be one next to the other). To know them, you have to analyse the structure of the filesystem you use. I don't know if there's an API to retrieve them easily, but in the worst case you can analyse the source code of the filesystem... good luck!
2
C functions are translated into assembly. If you respect the C calling convention, you can write a "C function" directly in assembly. Try reading this and this for x86.
3
You can call windows API from asm. Forget MS-DOS, MS-DOS is dead, MS-DOS is not Windows, the cmd is a sort of "emulation"... indeed no, not an emulation but just a command line interface that resemble the one MS-DOS users was used to. But it is not exaclty the same, i.e. there are no MS-DOS system interrupt you can use. Iczelion's assembly tutorials, though old, could be an interesting resource. (If links expire, try with the wayback machine)
4
I do not own Win7 and never installed nasm on windows, so I can't say anything about.
For the first question just drag the file into the address bar in the browser
I'm trying gather measurements of cycle counts for a particular sys call (sys_clone) in the linux kernel. That said, my process won't be the only one calling it and I can't know my pid ahead of time; so I'll have to record every invocation of it for every pid.
The problem that I've got is that the only ways I can figure out how to output this data (debugfs, sysfs, procfs) involve statically sized buffers, which will be quickly overwritten with irrelevant data from other processes calling sys_clone.
So, does anyone know how to append an arbitrary number of lines to a user space accessible file in linux?
You can take the printk()/klogd approach, and use a circular buffer that is exported via /proc. A user-space process blocks on reading your /proc file, and once it reads something that is removed from the buffer. In fact, you could take a look whether klogd/syslogd can be modified to also read your /proc file, thus you wouldn't need to implement the userspace part.
If you are good with something simpler, just printk() your information in a normalized form with some prefix, and then just filter it out from your syslog using this prefix.
There are a few more possibilities (e.g. using netlink to send messages to userspace), but writing to a file from the kernel is not something I'd recommend.
You could stash the counts in the right task_struct, and make it visible through a per-process file in /proc/<pid>/.