Recently I get a test task from one company. And one question is:
Suppose you are given a task to write a simple debugger (for a
proprietary operating system) that is capable of setting a break-point
in an application and running it. What would be key design decisions
you make in such a task?
I think I don't know something but I have absolutely no idea about answer. I understand how debuggers work (INT 3 - should have access to virtual space of debuggie) but I suppose the answer about "proprietary operating system".
This question, as usual at the interview should make you asking more questions about the system and requirements.
Does operating system already provide some kind of primitive tracing tool you can use?
What language are tested applications written in?
Some inspection tools, for example valgrind, run inspected programs in their own environment, which seems a good way to go in your case. The other approach is to instrument the binary with tracing instructions, communicating with your debugger - this is probably more suitable and easier to do when your applications run under a VM.
Related
I am running Windows 7 on which I want to do kernel debugging and I do not want to mess with boot loader. So I've downloaded LiveKd as suggested here and make it run and seems it is working. If I understand correct it is some kind of read only debugging. Here is mentioned that it is very limited and even breakpoint cannot be used. I would like to ask if is possible in this mode to periodically dump all the instructions that are being executed or basically all events which are happening on current OS? I would like to have some system wide strace (Linux users know) and to do some statistical analysis on this. I suppose it depends on more factors like installed debug symbols to begin able resolve addresses etc.
I'm not sure if debugger is the best tool you can use for tracing live system calls. As you've mentioned LiveKd session is quite limited and you are not allowed to place breakpoints in it (otherwise you would hang your own system). However, you still can create memory dumps using the .dump command (check windbg help: .hh .dump). Keep in mind though that getting a full dump (/f) of a running system might take a lot of time.
Moving back to the subject of your question, by using the "dump approach" you will miss many system calls as you will have only snapshots of a system at given points in time. So if you are looking for something similar to Linux strace I would recommend checking those tools:
Process Monitor (procmon) - it's a tool which will show you all I/O requests in the system, as well as operations performed on the registry or process activity events
Windows Performance Toolkit - it contains tools for collecting (WPR) and analysing (WPA) system and application tracing events. It might be a lot of events and it's really important to filter them accordingly to your needs. ETW (Event Tracing for Windows) is a huge subject and you probably will need to read some tutorials or books before you will be able to use it effectively (but it's really worth it!).
API Monitor - it's one of many (I consider it as one of the best) tracing applications - this tool will allow you to trace method calls in any of the running processes. It has a nice interface and even allows you to place breakpoints on methods you'd like to intercept.
There are many other tools which might be used for tracing on Windows, but I would start with the ones I listed above. You may also check a great book on this subject: Inside Windows Debugging. Good luck! :)
I have a question about a family of softwares, of which one example is INtime, which lets you run a real-time operating system in parallel with Windows.
I have a reasonable grasp on how Windows works, including kernel/driver/application security rings etc. Similarly, I know how a RTOS runs on a dedicated system.
The Simple Question:
How do these go about existing together without fighting over hardware or other similar problems? How is the allocation or resources made, and how is this integrated with Windows?
Slightly more complicated:
What are the steps I would have to take if I wanted to develop something similar myself? Are there some open-source embodiment's of this paradigm I can inspect to glean some more understanding?
I'd like to ask for your advice about improving security of executing a compiled program on a client's computer. The idea is that we send a compiled program to a client but the program has been written and compiled by a third-party. How to make sure that the program won't make any harm to a client's operating system while running? What would be the best to achieve that goal and not decrease dramatically performance of executing a program?
UPDATE:
I assume that third-party don't want to harm client's OS but it can happen that they make some mistake or their program is infected by someone else.
The program could be compiled to either bytecode or native, it depends on third-party.
There are two main options, depending on whether or not you trust the third party.
If you trust the 3rd party, then you just care that it actually came from them, and that it hasn't changed in transit. Code signing is a good solution here. If the third party signs the code, and you check the signature, then you can check nothing has changed in the middle, and prove it was them who wrote it.
If you don't trust the third party, then it is a difficult problem. The usual solution is to run code in a "sandbox", where it is allowed to perform a limited set of operations. This concept has been implemented for a number of languages - google "sandbox" and you'll find a lot about it. For Perl, see SafePerl, for Java see "Java Permissions". Variations exist for other languages too.
Depending on the language involved and what kind of permissions are required, you may be able to use the language's built in sandboxing capabilities. For example, earlier versions of .NET have a "Trust Level" that can be set to control how much access a program has when it's run (newer versions have a similar feature called Code Access Security (CAS)). Java has policy files that control the same thing.
Another method that may be helpful is to run the program using (Microsoft) Sysinternals process monitor, while scanning all operations that the program is doing.
If it's developed by a third party, then it's very difficult to know exactly what it's going to do without reviewing the code. This may be more of a contractual solution - adding penalties into the contract with the third-party and agreeing on their liability for any damages.
sign it. Google for 'digital signature' or 'code signing'
If you have the resources, use a virtual machine. That is -- usually -- a pretty good sandbox for untrusted applications.
If this happens to be a Unix system, check out what you can do with chroot.
The other thing is that don't underestimate the value of thorough testing. you can run the app (in a non production environment) and verify the following (escalating levels of paranoia!)
CPU/Disk usage is acceptable
doesn't talk to any networked hosts it shouldn't do - i.e no 'phone home capability'
Scan with your AV program of choice
you could even hook up pSpy or something to find out more about what it's doing.
additionally, if possible run the application with a low privileged user. this will offer some degree of 'sandboxing', i.e the app won't be able to interfere with other processes
..also don't overlook the value of the legal contracts with the vendor that may often give you some kind of recompense if there is a problem. of course, choosing a reputable vendor in the first place offers a level of assurance as well.
-ace
I couldn't find a suitable title for this. I'm going to express my query with examples.
Consider following softwares:
Process explorer from sysinternals (an advanced task manager)
Resource Manager : resmon.exe (lists each and every fine detail about resource usage about each process).
For me these softwares seems like miracles. I wonder how these are even made. C'mon how a user process can know such fine details about other processes? Who tells this software, what processes are running and what all resources are utilized? Which dlls are used? etc..
Does windows operating system give these software that information? I mean though (obviously the most lower level api) WIN32API. Are there some functions,which on calling return these values
abstractly say:
GetAllRunningProcesses()
GetMemoryUsedByProcess(Process* proc)
etc..
Other similar applications are
network Packet Capture software. How does it get information about all those packets? It clearly sits just infront of the NIC card. How is it possible?
Anti-virus: It scans memory for viruses. Intercepts other processes. Acts like a sandbox for the user application space. How? How??
If its WIN32API. I swear, I'm going to master it.
I don't want to create a multi-threaded application. I want to get information about other multithreaded applications.
I don't want to create a program which communicates using sockets. I want to learn how to learn how to capture all communication packets.
I actually want to work at the lower level. But I don't know, what should I learn. Please guide me in proper direction.
This is really a pretty open-ended question. For things like a list of running processes, look up "PSAPI" or "Toolhelp32". For memory information about a particular process, you can use VirtualQuery.
Capturing network packets is normally done by installing a device driver. If you look, you should be able to find a fair amount about how to write device drivers, though don't expect to create wonders overnight, and do expect to crash your machine a few times in the process (device drivers run in kernel mode, so it's easy for a mistake to crash the machine hard).
I can't say as much with any certainty about anti-virus, because I've never tried to write one. My immediate guess would be that their primary technique is API hooking. There's probably more to it than that, but offhand I've never spent enough time looking at them to know what.
Mark Russinovich's classic, Windows Internals, is the go-to book if you want to get deep in this kind of stuff. I notice that the just-released 5th edition includes Vista. Here's a sample chapter to peek at.
If you like Process Explorer, this is the guy who wrote that, and there are lots of examples using it in the book.
Plus, at 1232 hardcover pages, you can use it to press your clothes.
I'm looking for the concept to spawn a process such that:
it has only access to certain libraries/APIs
it cannot acess the file system or only specific parts
it can do least harm should malicious code run in it
This concept is known as sandbox or jail.
It is required to do this for each major Operating system (Windows, MacOSX and Linux) and the question is conceptual (as in what to do, which APIs to use and and what to observe) rather then language specific.
answer requirements
I really want to accept an answer and give you 20 points for that. I cannot accept my own answer, and I don't have it yet anyway. So if you really want your answer to be accepted, please observe:
The answer has to be specific and complete
With specific I mean that it is more then a pointer to some resource on the internet. It has to summarize what the resource says about the topic at least.
It may or may not contain example code, but if it does please write it in C
I cannot accept an answer that is 2/3 complete even if the 2/3 that are there are perfect.
this question FAQ
Is this homework? No.
Why do you ask this like a homework question? If you ask a specific question and you want to get a specific answer, and you know how that answer should look like, even though you don't know the answer, that's the style of question you get.
If you know how it should look like, why do you ask? 1) because I don't know all the answer 2) because on the internet there's no single place that contains all the details to this question in one place. Please also read the stackoverflow FAQ
Why is the main part of your question how to answer this question? Because nobody reads the FAQ.
Mac OS X has a sandbox facility code-named Seatbelt. The public API for it is documented in the sandbox(7), sandbox_init(3), and related manual pages. The public API is somewhat limited, but the facility itself is very powerful. While the public API only lets you choose from some pre-defined sandboxes (e.g. “All sockets-based networking is prohibited”), you can also use the more powerful underlying implementation which allows you to specify exactly what operating system resources are available via a Scheme-like language. For example, here is an excerpt of the sandbox used for portmap:
(allow process-exec (regex #"^/usr/sbin/portmap$"))
(allow file-read-data file-read-metadata (regex
#"^/etc"
#"^/usr/lib/.*\.dylib$"
#"^/var"
#"^/private/var/db/dyld/"
#"^/dev/urandom$"))
(allow file-write-data (regex
#"^/dev/dtracehelper$"))
You can see many sandboxes used by the system in /usr/share/sandbox. It is easy to experiment with sandboxes by using the sandbox-exec(1) command.
For Windows, you may want to have a look at David LeBlanc’s “Practical Sandboxing” talk given at Black Hat USA 2007. Windows has no built-in sandboxing technology per se, so the techniques described leverage an incomplete mechanism introduced with Windows 2000 called SAFER. By using restricted tokens, one can create a process that has limited access to operating system resources.
For Linux, you might investigate the complicated SELinux mechanism:
SELinux home,
a HOWTO. It is used by Red Hat, for example, to harden some system services in some of their products.
For Windows there is a sandbox in Google Chrome. You may want to investigate it. It uses liberal BSD-like license.
For Linux there would be good old chroot or more sophisticated http://plash.beasts.org/wiki/.
OS X since Leopard has some SELinux-like protection available.
The site codepad.prg has a good "About" page on how they safely allow the execution of any code snippets..
Code execution is handled by a supervisor based on geordi. The strategy is to run everything under ptrace, with many system calls disallowed or ignored. Compilers and final executables are both executed in a chroot jail, with strict resource limits. The supervisor is written in Haskell.
When your app is remote code execution, you have to expect security problems. Rather than rely on just the chroot and ptrace supervisor, I've taken some additional precautions:
The supervisor processes run on virtual machines, which are firewalled such that they are incapable of making outgoing connections.
The machines that run the virtual machines are also heavily firewalled, and restored from their source images periodically.
FreeBSD has specific concepts of jails, and Solaris has containers. Depending on what you're looking for, these may help.
chroot jails can help to limit what an application can do (though any app with root privileges can escape a jail), and they're available on most UNIXen, including OS X.
As for Windows, I'm not sure. If there was an easy way to sandbox a Windows app, most of them would be a lot more secure by now, I'm sure.
On windows (2000 and later) you can use Job objects to restrict processes.
If you really want a technique that will work with all these platforms, as opposed to a separate solution for each platform, then I think your only answer is to set up a virtual machine for each testing environment. You can restore back to a snapshot at any time.
Another big advantage of using virtualization is that you can have all of the testing environments with their guest operating systems all on the same box.
For Linux, there is AppArmor. Unfortunately, the project is somewhat on hiatus.
Another sandboxing-alternative is VServer, which uses virtualization.
Generally any virtual private server will do:
Linux VServer
http://linux-vserver.org/Welcome_to_Linux-VServer.org
Parallels Virtuozzo Containers
http://www.parallels.com/products/pvc/
and as was mentioned FreeBSD and Solaris has own implementations.
Oh. actually I've noticed you're asking it to work on ANY OS. Well, that might be complicated a bit as the I think less effort is just to reuse some VM that can support some level of sandboxing like:
Java
.NET
I'm not an expert on the topic, but i think the standard answer for linux is to define a SeLinux policy with the right capabilities for the process.