Is there any way to create a block device via user space in OSX (10.8+), without 3rd party libraries (FUSE, etc)?
I am trying to create a userspace tool. The idea is the user supplies a file to the tool and the tool creates a virtual interface. Whenever the interface is written to, the tool applies an operation to the data and then writes to the original file. Whenever the interface is read from, the tool reads from the original file and applies the inverse operation.
I don't necessarily need a block device. The tool needs to create some kind of virtual interface that can be treated as a file, i.e. it can be opened and saved to by another application.
I looked at the userspace routines of the I/O Kit, but nothing seemed aplicable, as creating a virtual USB/FireWire/MMC/SCSI device seems excessive.
The closest thing I can think of without going into kernel space is a fifo or named pipe - this is essentially a pipe with a file name. Unfortunately, being a pipe, you can't seek, etc. - you can just open the fifo for reading in one process and for writing in another. You create these either with the mkfifo command line utility or the mkfifo() C function.
There are also UNIX domain sockets, which are similar to IP sockets but again are identified by a file name, not a networking construct. These can be read and written from both ends, but again no seeking.
As for actually implementing the ops functions for a vnode (the in-kernel representation of a file in OSX) I believe you do have to drop to the kernel. Note that the I/O kit isn't strictly necessary for creating a block device - if the BSD device nodes are sufficient, and you don't need support for hardware or ejecting volumes etc. you can simply create a node with bdevsw_add(), supplying the ops vector as a parameter. (for an I/O Kit based storage device, the IOMediaBSDClient does this automatically, along with creating the character device with cdevsw_add_with_bdev()) Another, rather more elaborate option is to implement your own file system.
Related
I currently have a resource light "unmanaged" Windows C++ application using:
CreateMutex to ensure there is at most a single instance of the application.
COPYDATASTRUCT to pass command line parameters over to that single instance, should another instance attempt to start (after sending data, it will then quit).
The use case is a simple one: launch a new application instance to open a file specified on the command line, or otherwise open the file in a previously-launched application instance.
How could I achieve the same behaviour in a Haskell program?
It looks like CreateMutex isn't offered in the Win32 package that comes with GHC. You'll need to either bind to it yourself using the FFI or emulate it in a cross-platform way. One trick commonly used on Linux is to choose a particular filename to act as the "mutex"; if it doesn't exist, you create it and write down how to contact you then delete it when you exit, while if it does, you read the contact information out of it to find out how to connect to the already running process. This approach does have downsides: if either your app or the computer as a whole goes down, you'll have stale contact information.
For IPC, I think there's a number of cross-platform interfaces. Wikipedia has an excellent list; especially suitable for doing this task in Haskell are sockets, (named) pipes, and dbus. For the latter two, you may need Cygwin, since they ultimately depend on getting the unix package installed.
For a variety of reasons, revolving around cost of copy and the travails of the Windows filesystem, I need to mount a stream encoder as a drive, so that incoming data can simply be blindly directed at this "drive", aggregated, and encoded, without the source program being any the wiser. This would be basically trivial on Linux, but seems to be an uphill struggle on windows.
Specifically, I'd like to be able to "mount" a tar builder, which I know sounds odd, but there's a compelling reason for doing this. Is there a utility or library that deals with this? Perhaps an obscure part of the Windows API?
This looks promising... but it seems intended to mount folders or similar, rather than "devices." I do have control of where the data is written, so I can specify an arbitrary path.
Having experience with virtual drives (see our Virtual Storage product line) I can say that your task needs some redefining. As said in comments, drives (or, better say, filesystems) in Windows are expected to be namely filesystems (unlike Unix world), and as such they must support certain reading and enumeration operations, which is not something you'd expect.
Probably the closest you can do is a virtual drive in memory whose contents are then passed to your application in some way. The user will drag the data to your drive and on unmounting (or on other command) drive contents are passed to other program.
Several of our products can be used for your task (see CallbackDisk, Callback File System and SolFS OS Edition on Virtual Storage page), yet they are all commercial products. If you have a one-time or short-term task, you can build something for your use with a trial key.
There exist free approaches to your task, namely Pismo File Mount and Dokan, but I don't know how well they fit.
I am writing a kernel extention for Mac os, and i need to delete a tmp file created by
vnode_open(),but i can not find out how to delete a file and list all files in a dir.
http://developer.apple.com/library/mac/#documentation/Darwin/Reference/KernelIOKitFramework/vnode_h/index.html
Thanks!
It is better to create user-space daemon and to communicate with it using mach RPC.
The method of deletion of file in kext (that i'm aware of) depends on the version of kernel because it requires lookup of the offset of v_op (vnode operations vector. It is not visible from kernel extensions) within vnode structure. The vnode structure may change from kernel to kernel. And even if one finds vnop_remove_desc and vnop_rmdir_desc i think using it may be dangerous due to possible changes of the logic of using these callbacks.
UNIX file-locking is dead-easy: The operating system assumes that you know what you are doing and lets you do what you want:
For example, if you try to delete a file which another process has opened the operating system will usually let you do it. The original process still keeps it's file-handles until it terminates - at which point the the file-system will quietly re-cycle the disk-resources. No fuss, that's the way I like it.
How different things are on Windows: If I try to delete a file which another process is using I get an Operating-System error. The file is untouchable until the original process releases it's lock on the file. That was great back in the single-user days of MS-DOS when any locking process was likely to be on the same computer that contained the files, however on a network it's a nightmare:
Consider what happens when a process hangs while writing to a shared file on a Windows file-server. Before the file can be deleted we have to locate the computer and ID the process on that computer which originally opened the file. Only then can we kill the process and delete our unwanted file.
What a nuisance!
Is there a way to make this better? What I want is for file-locking on Windows to behave a like file-locking in UNIX. I want the operating system to just let me do what I want because I'm in charge and I know what I'm doing...
...so can it be done?
No. Windows is designed for the "average user", that is people who don't understand anything about a computer. Therefore, the OS tries to be smart to avoid PEBKACs. To quote Bill Gates: "There are no issues with Windows that any number of people want to be fixed." Of course, he knows that 99.9999% of all Windows users can't tell whether the program just did something odd because of them or the guy who wrote it.
Unix was designed when the world was more simple and anyone close enough to a computer to touch it, probably knew how to assemble it from dirty sand. Therefore, the OS usually lets you do what you want because it assumes that you know better (and if you didn't, you will next time).
Technical answer: Unix allocates an "i-nodes" if you create a file. I-nodes can be shared between processes. If two processes create the same file (that is, two processes call create() with the same path), then you end up with two i-nodes. This is by design. It allows for a fancy security feature: You can create files which no one can open but yourself:
Open a file
Delete it (but keep the file handle)
Use the file any way you like
Close the file
After step #2, the only process in the universe who can access the file is the one who created it (unless you want to read the hard disk block by block). The OS will keep the data alive until you either close the file or your process dies (at which time Unix will clean up after you).
This design is the foundation of all Unix filesystems. The Windows file system NTFS works much the same way but the high level API is different. Many applications open files in exclusive mode (which prevents anyone, even backup programs) to read the file. This is even true for applications which just display information like PDF viewers.
That means you'll have to fix all the Windows applications to achieve the desired effect. If you have access to the source, you can create a file in a shared mode. That would allow other processes to access it at the same time but then, you will have to check before every read/write if the file still exists, whether someone has made changes, etc.
According to MSDN you can specify to CreateFile() 3rd parameter (dwSharedMode) shared mode flag FILE_SHARE_DELETE which:
Enables subsequent open operations on a file or device to request delete access.
Otherwise, other processes cannot open the file or device if they request delete access.
If this flag is not specified, but the file or device has been opened for delete access, the function fails.
Note Delete access allows both delete and rename operations.
http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx
So if you're can control your applications you can use this flag.
Note that Process Explorer allow for force closing of file handles (for processes local to the box on which you are running it) via Handle -> Close Handle.
Unlocker purports to do a lot more, and provides a helpful list of other tools.
Also deleting on reboot is an option (though this sounds like not what you want)
That doesn't really help if the hung process still has the handle open. It won't release the resources until that hung process releases the handle. But anyway, in Windows it is possible to force close a file out from under a process that's using it. Process Explorer from sysinternals.com will let you look at and close handles that a process has open.
I need to modify the MBR of Windows, and I would really like to do this from Windows.
Here are my questions. I know that I can get a handle on a physical device with a call to CreateFile. Will the MBR always be on \\.\PHYSICALDRIVE0? Also, I'm still learning the Windows API to read directly from the disk. Is readabsolutesectors and writeabsolutesectdors the two functions I'm going to need to use to read/write to the disk sectors which contain the MBR?
Edit from from what I've learned on my own.
The MBR will not always be on \\.\PHYSICALDRIVE0. Also, you can write to the bootsector (at least as Administrator on XP) by call CreateFile with the device name of the drive that contains the MBR. Also, you can write to this drive by simply calling WriteFile and passing the handle of the device created by calling CreateFile.
Edit to address Joel Coehoorn.
I need to edit the MBR because I'm working on a project that needs to modify hardware registers after POST in BIOS, but before Windows will be allowed to boot. Our plan is to make these changes by modifying the bootloader to execute our code before Windows boots up.
Edit for Cd-MaN.
Thanks for the info. There isn't anything in your answer, though, that I didn't know and your answer doesn't address my question. The registry in particular absolutely will not do what we need for multiple reasons. The big reason being that Windows is the highest layer among multiple software layers that will be running with our product. These changes need to occur even before the lower levels run, and so the registry won't work.
P.S. for Cd-MaN.
As I understand it, the information you give isn't quite correct. For Vista, I think you can write to a volume if the sectors being written to are boot sectors. See http://support.microsoft.com/kb/942448
Once the OS is started the MBR is typically protected for virus reasons - this is one of the oldest virus tricks in the books - goes back to passing viruses from floppy to floppy.
Even if it wasn't restricted, you have to write low level code - it isn't part of the file system, but exists on a specific location on the hard drive.
Due to that, you pretty much are restricted to writing low level (most programs implement this in assembly) or C code targeting 16 bit DOS.
Most of these programs use the BIOS interface (13h, I believe) to access the sectors of the disk directly. You can access these in C using some inline assembly, or compiler provided interfaces. You will generally not get access to BIOS without the cooperation of the OS, though, so your program, again, will be restricted to DOS. If you can access these you're almost home free - the nice thing about BIOS is you don't have to worry about what type of HD is in the system - even RAID cards often insert themselves into the BIOS routines so they can be accessed without knowing where in memory the ATA or SATA controller is, and executing commands on that low level.
If you absolutely must access it within an OS, though, you pretty much have to write a device driver to access the BIOS or the memory space where the HD controllers exist. I wouldn't recommend it, though, as this is very tricky to deal with - modern computers put the HD controllers in different spots in memory, with different IRQs, and each chipset has become a little more esoteric because they can provide a minimum interface to bios for bootup, and then a specific driver for Windows. They skip all the other interface niceties that would be considered compatible with other controllers because it's more expensive to be compatible.
You may find that at the driver level inside windows you'll have methods for accessing the drive sectors directly (or pseudo directly), but again, they are likely very well protected due to the aforementioned virus issues.
Good luck!
Modifying the bootloader is bad, bad idea. Here are just a few of the possible gotcha's:
it will potentially kill full disk encryption products (Truecrypt, PGP, Vista's BitLocker, etc)
it will potentially trip up AV products (scaring users)
it will potentially kill complicated booting scenarios (chained boot loaders, etc)
it will kill off the chain of trust when using the TPM module (because it checks the MBR for change before executing it)
direct disk access is not allowed starting from Vista (only using drivers)
Alternatives (like modifying the hardware register during the Windows bootup via a driver which is set to load at boot time or after Windows has booted) should really be considered. If the modification is as simple as writing to a port, ie:
OUT AX, BL
then drivers exists for all versions of Window which can do this (reading/writing a value from/to a certain port) which can be called from user mode.
Maybe a PXE boot scenario could help you? Simply boot on your crafted PXE image which modify the hardware registers you need to modify, and then return the control to the Master Boot Record or to the active partition's boot record.
This way you don't have to modify the boot records.