We have this application that does not write to the Windows registry or store its configuration files (such as an INI file) in the user's profile; instead, it stores its configuration files in the program's directory. Wikipedia has this statement
A portable application (portable app) is a computer software program designed to run independently from an operating system. This type of application is stored on a removable storage device such as a CD, USB flash drive, flash card - storing its program files, configuration information and data on the storage medium alone.
so our question is, does this make our application a true portable application (portable app)?
I should point out that if the application is on a write protected medium we use the function below, so it doesn't try to write to that medium.
function GetTempFile(): string;
var
Buffer: array[0..MAX_PATH] of Char;
begin
Windows.ZeroMemory(#Buffer, System.SizeOf(Buffer));
SysUtils.StrPCopy(Buffer, SysUtils.ExcludeTrailingBackslash(SysUtils.ExtractFilePath(System.ParamStr(0))));
Windows.GetTempFileName(Buffer, '~', 0, Buffer);
Result := string(Buffer);
end;
function IsMediumWriteProtected(): Boolean;
var
ErrorMode: Word;
hHandle: THandle;
begin
ErrorMode := Windows.SetErrorMode(SEM_FAILCRITICALERRORS);
try
hHandle := Windows.CreateFile(PChar(GetTempFile()), GENERIC_WRITE, 0, nil,
CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY or FILE_FLAG_DELETE_ON_CLOSE, 0);
try
Result := (hHandle = INVALID_HANDLE_VALUE);
finally
Windows.CloseHandle(hHandle);
end;
finally
Windows.SetErrorMode(ErrorMode);
end;
end;
A truly portable application, if that is truly your question, should:
Not require any installation whatsoever.
Work anywhere you copy it to.
My own gestalt definition of a portable app does not always require that the app work when it is located on a read-only place, and other than this wikipedia reference, I am not sure that it's standard in the wild. It's great that you're trying to follow the letter of the law as spelled out in Wikipedia articles. It irks me that the quote you have given from the Wikipedia article contains a bit of an internal contradiction; if the definition both expects that a CDROM must be supported (or your app is not portable) and also specifies that the data should be stored on that media alone. You have run into that contradiction, and are now violating one part of the letter of the law, to avoid violating the first part of it. So the definition either prevents you from saving anything at all in that case, or that definition isn't in fact, a law unto itself, and you're free to do as you like, provided the two points I mention above are followed.
A minimalist definition of portable app is the one I gave above, and what you do beyond those two points, should be whatever is right for your users.
I really don't see how anyone on SO can give you a green light on portability. You can only truly verify portability by testing, and ideally testing by a group of people, such as beta test or fieldtest users. I also don't see StackOverflow as a good place for requests for help testing binaries. This is a site for asking programming questions, not for forming beta tests, field-tests etc, or for promoting your app. In fact I don't see how we could safely download a binary from you and not be risking our systems running binary code we downloaded from links here. I recommend you remove the link to the binaries.
Tests I would do:
Put it on a usb key, and run it on several different PCs on several different clean windows systems, including XP, Vista, Win 7, Windows Server 2008 at several service pack levels (no SP, Sp1, Sp2, Sp3 on XP etc), and on several different dirty systems (systems with visual studio installed, delphi installed, office installed, and various anti-virus systems).
Be sure you require no external runtime BPLs/DLLs or other runtime stuff, except those located in the same folder as your executable (side by side configuration). That way the user's path does not have to be modified for your app to work, and you don't have to worry about DLL hell confusion.
Check that no registry writes or filesystem writes outside the app folder occur in your app. SysInternals Process Monitor is great for this.
Do not use Windows.GetCurrentDirectory() to retrieve the executable file's directory. It returns the process's current working directory, which can change dynamically during the process's lifetime, so you are not guaranteed to get the correct directory every time. To get the app's directory reliably, use Sysutils.ExtractFilePath(Sysutils.ParamStr(0)) instead. Internally, this uses Windows.GetModuleFileName(nil) to get the full path of your app and then truncates off the executable name, leaving the desired directory path.
Related
I know that ntdll is always present in the running process but is there a way (not necessarily supported/stable/guaranteed to work) to create a file/key without ever invoking ntdll functions?
NTDLL is at the bottom of the user-mode hierarchy, some of its functions switch to kernel mode to perform their tasks. If you want to duplicate its code then I suppose there is nothing stopping you from decompiling NtCreateFile to figure out how it works. Keep in mind that on 32-bit Windows there are 3 different instructions used to enter kernel mode (depending on the CPU type), the exact way and where the transition code lives changes between versions and the system call ids change between versions (and even service packs). You can find a list of system call ids here.
I assume you are doing this to avoid people hooking your calls? Detecting your calls? Either way, I can't recommend that you try to do this. Having to test on a huge set of different Windows versions is unmanageable and your software might break on a simple Windows update at any point.
You could create a custom kernel driver that does the work for you but then you are on the hook for getting all the security correct. At least you would have documented functions to call in the kernel.
Technically, registry is stored in %WINDIR%\System32\config / %WINDIR%\SysWOW64\config, excepted your own user's registry which is stored in your own profile, in %USERPROFILE%\NTUSER.DAT.
And now, the problems...
You don't normally have even a read access to this folder, and this is true even from an elevated process. You'll need to change (and mess up a lot...) the permissions to simply read it.
Even for your own registry, you can't open the binary file - "Sharing violation"... So, for system/local machine registries... You can't in fact open ANY registry file for the current machine/session. You would need to shut down your Windows and mount its system drive in another machine/OS to be able to open - and maybe edit - registry files.
Real registry isn't a simple file like the .reg files. It's a database (you can look here for some elements on its structure). Even when having a full access to the binary files, it won't be fun to add something inside "from scratch", without any sotware support.
So, it's technically possible - after all, Windows does it, right? But I doubt that it can be done in a reasonable amount of time, and I simply can't see any benefit from doing that since, as you said, ntdll is ALWAYS present, loaded and available to be used.
If the purpose is to hack the current machine and/or bypass some lack of privileges, it's a hopeless approach, since you'll need even more privileges to do it - like being able to open your case and extract the system drive or being able to boot on another operating system on the same machine... If it's possible, then there is already tools to access the offline Windows, found on a well-known "Boot CD", so still no need to write in registry without any Windows support.
We have a 3rd party native application (written in C I believe) which we want to run multiple instances of on a machine.
however the application reads and writes from one particular registry key in order to find the location of a config file. It reads this location continuously during its running. The registry key is in HKLM. this means that if we try and run 2 different instances of the app with 2 different locations for the config file the processes tread on each others toes.
Is it possible to 'virtualise' the registry (or run each process in a sandbox) that the processes are using so that they can both think they are writing to a single location, but actually they are writing and reading from somewhere different and they won't step on each others toes?
There are several options to virtualize a program:
https://en.wikipedia.org/wiki/Portable_application_creators
Creating your own virtualization software is much more complicated and would require an entire coarse on programming and hooking library calls using the windows SDK.
However an easier option that doesn't require setting up and running additional software for each copy of the program I suggest creating multiple copies of the program and hex editing each executable.
Make as many copies of the application as you need to run, then open the application file in a hex editor and search for the name of the registry key, ie:
HKLM\System\CurrentControlSet\Control\Session Manager
Then change the last byte to a digit for each different version (1 byte, 0-9) ie:
HKLM\System\CurrentControlSet\Control\Session Manage1
HKLM\System\CurrentControlSet\Control\Session Manage2
HKLM\System\CurrentControlSet\Control\Session Manage3
For more than 10 differences (2 bytes, 00-99) use the last two bytes:
HKLM\System\CurrentControlSet\Control\Session Manag01
HKLM\System\CurrentControlSet\Control\Session Manag02
HKLM\System\CurrentControlSet\Control\Session Manag03
While the solution from Joshua will work for this particular application, it might not work for others (f.e. where the registry path is constructed in code or when the application is signed).
Therefore, I would suggest using DLL injection and intercept calls to RegOpenKey(Ex), RegCreateKey(Ex), etc. That way, you can fiddle with the registry path before passing the call down to the real Windows Advapi32.dll.
Some great articles about API hooking:
API Hooking and DLL Injection on Windows
API Hooking with MS Detours
Yes, Sandboxie can run multiple instances of an app, each in it's own "Sandbox" which it believes to be the entire universe. But you can also access the data directly through the normal ways if you need to.
So in other words, Sandboxie lets you see all the registry changes that were made in the app's operations, and you can roll them back if you like.
Yes, you can virtualize the application, this technology is called Application Virtualization.
Try http://www.cameyo.com/. Cameyo is a software used to build virtual application.
A virtual application is a single EXE file that holds an entire application including files, DLLs and registry. Virtual apps are isolated from your system and can be copied & moved from one computer to another without installation.
I was asked such a question in an interview:
In windows, suppose there is an exe which depends on some dlls, when you start
the exe, and then the dependent dlls will be loaded, are these dlls
loaded in kernel mode or user mode?
I am not quite sure about the question, not the mention the answer - could you help to explain?
Thanks.
I'm not an expert about how Windows internally works, but for what i know the correct answer is user mode, simply because only the processes related to your Operative System are admitted in the kernel space http://en.wikibooks.org/wiki/Windows_Programming/User_Mode_vs_Kernel_Mode
Basically if it's not an OS process, it's going to be allocated in the user space.
The question is very imprecise/ambiguous. "In Windows" suggests something but isn't clear what. Likely the interviewer was referring to the Win32 subsystem - i.e. the part of Windows that you usually get to see as an end-user. The last part of the question is even more ambiguous.
Now while process and section objects (in MSDN referred to as MMF, loaded PE images such as .exe and .dll and .sys) are indeed kernel objects and require some assistance from the underlying executive (and memory manager etc) the respective code in the DLL (including that in DllMain) will behave exactly the same as for any other user mode process, when called from a user mode process. That is, each thread that is running code from the DLL will transition to kernel mode to make use of OS services eventually (opening files, loading PE files, creating events etc) or do some stuff in user mode whenever that is sufficient.
Perhaps the interviewer was even interested in the memory ranges that are sometimes referred to as "kernel space" and "user space", traditionally at the 2 GB boundary for 32bit. And yes, DLLs usually end up below the 2 GB boundary, i.e. in "user space", while other shared memory (memory mapped files, MMF) usually end up above that boundary.
It is even possible that the interviewer fell victim to a common misunderstanding about DLLs. The DLL itself is merely a dormant piece of memory, it isn't running anything on its own ever (and yes, this is also true for DllMain). Sure, the loader will take care of all kinds of things such as relocations, but in the end nothing will run without being called explicitly or implicitly (in the context of some thread of the process loading the DLL). So for all practical purposes the question would require you to ask back.
Define "in Windows".
Also "dlls loaded in kernel mode or user mode", does this refer to the code doing the loading or to the end result (i.e. where the code runs or in what memory range it gets loaded)? Parts of that code run in user mode, others in kernel mode.
I wonder whether the interviewer has a clear idea of the concepts s/he is asking about.
Let me add some more information. It seems from the comments on the other answer that people have the same misconception that exists about DLLs also about drivers. Drivers are much closer to the idea of DLLs than to that of EXEs (or ultimately "processes"). The thing is that a driver doesn't do anything on its own most of the time (though it can create system threads to change that). Drivers are not processes and they do not create processes.
The answer is quite obviously User mode for anybody who does any kind of significant application development for windows. Let me explain two things.
DLL
A dynamic link library is closely similar to a regular old link library or .lib. When your application uses a .lib it pastes in function definitions just after compile time. You typically use a .lib to store API's and to modify the functions with out having to rebuild the whole project, just paste new .lib with same name over the old and as long as the interface(function name and parameters) hasn't changed it still works. Great modularity.
A .dll does exactly the same thing however it doesn't require re-linking or any compilation. You can think of a .dll as essentially a .lib which gets compiled to an .exe just the same as applications which use it. Simply put the new .dll which shares the name and function signatures and it all just works. You can update your application simply by replacing .dlls. This is why most windows software consists of .dlls and a few exe's.
The usage of a .dll is done in two ways
Implicit linking
To link this way if you had a .dll userapplication.dll you would have an userapplication.lib which defines all the entry points in the dll. You simply link to the static link library and then include the .dll in the working directory.
Explicit linking
Alernatively you can programmatically load the .dll by first calling LoadLibrary(userapplication.dll) which returns a handle to your .dll. Then GetProcAddress(handle, "FunctionInUserApplicationDll") which returns a function pointer you can use. This way your application can check stuff before attempting to use it. c# is a little different but easier.
USER/KERNEL MODES
Windows has two major modes of execution. User mode and Kernel modes (kernel further divided into system and sessions). For user mode the physical memory address is opaque. User mode makes use of virtual memory which is mapped to real memory spaces. User mode driver's are coincidentally also .dll's. A user mode application typically gets around 4Gb of virtual addressing space to work with. Two different applications can not meaningfully use those address because they are with in context of that application or process. There is no way for a user mode application to know it's physical memory address with out falling back to kernel mode driver. Basically everything your used to programming (unless you develop drivers).
Kernel mode is protected from user mode applications. Most hardware drivers work in the context of kernel mode and typically all windows api's are broken into two categories user and kernel. Kernel mode drivers use kernel mode api's and do not use user mode api's and hence don't user .dll's(You can't even print to a console cause that is a user mode api set). Instead they use .sys files which are drivers and essentially work exactly the same way in user mode. A .sys is an pe format so basically an .exe just like a .dll is like an .exe with out a main() entry point.
So from the askers perspective you have two groups
[kernel/.sys] and [user/.dll or .exe]
There really isn't .exe's in kernel because the operating system does everything not users. When system or another kernel component starts something they do it by calling DriverEntry() method so I guess that is like main().
So this question in this sense is quite simple.
For testing and development purposes, it would be nice to somehow simulate (spurious) file access errors to local files. For example, even if an application has correctly opened a file with the appropriate restrictive sharing flags, it still can happen that an attempt to access the file (through any of the Win32 API functions or your favourite framework, which internally will just call any of the Win32 API functions) can fail.
The only example I ever was able to track down was the virus scanner on a machine, but I guess there could be other reasons. (In this question's comment, Luke mentions something about "File system filter drivers".)
FWIW, I know of a few possibilities to "simulate" file problems, that I do not consider good solutions, either because they require to much manual work or because they don't fit for every app/file:
Place a file on a network drive or removable storage device - that way you can just mess up the device (unplug, disk-full, ...).
Open the application process in Process Explorer and close the handle of the file you want to test.
So the question really is if there are any ((semi)automated) tools that can mess up file access (on an NTFS drive) even though an application has already opened a file with appropriate (for the app) sharing flags.
Holodeck purports to allow Win32 API hooking, which would enable you to manipulate return codes as needed for Fault Injection.
If your API set of interest is well-defined, you could probably do this yourself using the Import Address Table approach described here.
Linux has a feature called namespaces, which let you give a different "view" of the filesystem to different processes. In Windows terms, this would be useful for example if you had a legacy program "floyd" that always loaded its configuration from C:\floyd\floyd.ini. If Windows had namespaces, you could write a wrapper script which would create a namespace in which to run floyd, making it so when Alice ran the script, floyd would start up in an environment where C:\floyd existed but actually pointed to C:\Users\Alice\Floyd.
Now you may be thinking, "OK, just use soft or hard links and make C:\floyd an alias for C:\Users\Alice." But with namespaces, Bob can also run the startup script, but his instance of floyd (on the same computer, running at the same time) will see C:\floyd with the contents of, say, C:\Users\Bob\Program Settings\Floyd Config (or any other path we like).
You can do this on Linux with namespaces. Is there something similar or analogous on Windows? It's fine if it requires writing a C program, and it's OK if it only works on recent versions of Windows.
NTFS hard links are really a simple case of reparse points. Reparse points are typed, and can include more advanced behavior. For instance, they're also used for "offline storage" (transparent migration of files to and from secondary storage). You can therefore also use reparse points to implement per-user symbolic links, by creating a new reparse type.
The reparse point type even has an explicit "Name surrogate" bit, which (if set) indicates that reparse points of those types are some kind of symbolic link.
You can even have multiple reparse points in a path. Therefore files inside your symbolic namepace can still be migrated to secondary storage - you'd just have two reparse points in the path.
I think Virtual Store does this automatically for legacy programs that try to write to nonstandard directories. So the legacy program writes to a user- and program-specific directory instead to C:\floyd.
This sounds like Windows Vista's file system virtualization. For example, it can silently redirect c:\Program Files\Floyd to c:\Users\<username>\AppData\Local\VirtualStore\Program Files\Floyd. However, file system virtualization isn't nearly as configurable as Linux namespaces. From what I can tell from reading, file system virtualization should apply any time a 32-bit interactive process opens for writing a file, folder, or registry key that's only writable by administrators. (So you typically end up with some read-only files under c:\Program Files and some per-user writable files under c:\Users\<username>\AppData\Local\VirtalStore.)
An application virtualization product can probably also do this, although those are often more complicated and more expensive.
You can use hardlinks for that, but only with NTFS. http://en.wikipedia.org/wiki/Hard_link
I think windows doesn't have virtual FS view per process.
The most relevant thing is probably special environment folders, such as %temp%, %appdata%, %localappdata%. Not that they're equivalent, but they fulfil the same purpose.
You can define your own environment variables then use '%myspecialplace%\myfile.txt' to access them.
As a horrid kludge (and here I book my passage to programming hell) could you use a NamedPipe for C:\Floyd that mapped the IO operations onto a file specific to the current process user?
I know it's not pretty and I don't know enough about NamedPipes (FIFOs in other dialects) on Windows to know how feasible this is.
Dan
There are several things that come to my mind.
First of all you can create a file system filter driver (or use a ready driver, such as our CallbackFilter product) that would redirect all file system calls, coming from the application, to other location. This is close to virtualization that you mentioned but this won't change the list of drive letters though. Such approach is both powerful and non-trivial, so see the other option first of all.
And the other option is:
there exist several products (Thinstall, Molebox if memory serves) that "wrap" the application redirecting it's file I/O to some other location. There was also some SDK to do the same, but I don't remember it's name at all.
e.g. http://www.msigeek.com/4819/file-re-direction-using-correctfilepaths-shim-to-fix-broken-applications
But I think it's not configurable per user, although the target can vary per-user based on environment variable substitution.
Most programs, though, store configuration in the registry, in which case RegOverridePredefKey would do the trick.
There's a shortage of good solutions for this. For simplicity, I can't improve on using NTFS soft links (junctions) for this - as you correctly point out, this creates issues if you want per-user configuration. As MSalters correctly says, all NTFS soft and hard links are just special cases of reparse points, so you could do something more general by impplementing a new reparse type, if you don't mind some work digging into NTFS..
(Junction is a pretty useful tool when experimenting with NTFS soft links: http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx )
You could just take a direct approach - give each user (or your program initialization if you only care about one piece of software) a logon script that sets up the appropriate junction into their user directory (and make sure you clean it up afterwards). But it's clumsy.
In general the right Windows approach is to put things into the folders pointed at by the %localappdata% (from Vista on) and, more generally, %userprofile% system variables.
Win file system virtualization is intended to ensure this in the cases where it applies.