I am doing a research on how someone can detect whether another machine is running Windows or Linux, if it runs inside a virutal machine, if it's behind NAT\proxy\VPN and what not.
I saw some interesting tools like p0f, which does passive detection.
Basically, I know there are implementation differences in TCP and other underlying protocols, not to mention application level stuff that exist on Windows and not Linux, but I can't find any, and worse still, I really do not know what terms to look for in Google.
Do you have links or tutorials explaining what to detect to differentiate Linux and Windows?
This is not a subject about which I have a great depth of knowledge but try looking for "TCP stack fingerprinting" and also have a look at nmap's source code. I believe nmap is capable of using stack fingerprinting which involves testing for differences in TCP implementation in order to attempt to determine the environment on a host.
I've not used nmap to any great extent so I can't vouch for the quality or accuracy of it's implementation, but source code is available.
For the web, the differences between web browsers are fairly extensively documented. One example of a difference between browsers is the way in which an XmlHttpRequest object is created. By trying to instantiate an XmLHttpObject in different ways and testing which ones succeed, you may be able to determine the browser or JavaScript engine running the code.
here - http://nmap.org/book/osdetect.html
It's all been done before. How about starting with nmap?
Nmap -O is great for detecting systems operating systems by probing.
p0f is another tool that can be set in listener mode, and detects the operating system on systems that make connections to you.
Related
I want to set system wide proxy settings on my windows machine. I know about the settings from Internet Explorer but dont want to do it that way. Is there a way to set up a proxy which will be used by all the applications on my machine(especially firefox, I dont want to have to set Use System proxy Settings in the Firefox options menu)?
In windows, that is the preferred way to set up the proxy settings.
But you can have a look at this for command line options
https://superuser.com/questions/419696/in-windows-7-how-to-change-proxy-settings-from-command-line
How can this be achieved theoretically
I am going to provide a somewhat unusual answer, because I've noticed that this particular 'way' of solving this problem has (for some reason) not crossed people's minds so far.
If you want to really make all apps without exception send internet traffic through your proxy, you are going to have to use a special technology known as TUN/TAP devices.
In short, these are special drivers, which when installed appear to a system as a network adapter (just like your local Ethernet or Wireless card), but they are in fact built in such a way so as to be easy to control from a software level.
Basically, when you install such a driver on the system, the system now regards that device as a fully functional Network Adapter. Therefore, if you now set this network adapter as the default gateway, all apps (without knowing it or being able to prevent it) will automatically pass through it, the same way as all apps pass through a generic Wireless Adapter / Ethernet.
Practical ways of achieving this / How can I use this with proxies?
Now that you have a basic idea of what redirecting system traffic through a TAP/TUN device means, there are a couple of ways of doing this.
Before I start, I really recommend that even if you stray from the suggested resources here, you stick to using OpenVPN's open source TAP device, since it has been extensively tested and confirmed to work on many systems, and is very widely used now (Some basics are available at https://openvpn.net/tuntap, and I trust you should find it embedded in any latest version of OpenVPN, the only files you need are the compiled drivers (.inf), you don't need to have the entirety of OpenVPN installed to use them).
The project that instantly comes to mind when thinking of using SOCKS proxies as the endpoint of a TAP device is badvpn/tun2socks. The project basically does exactly what is outlined here, so I definitely recommend you read the source code, or use it as a standalone utility (If you need some help with usage, I suggest you check out this wiki page.
What if any are the drawbacks of using this approach?
First of all, speaking of compatibility, performance and bugs, there are no drawbacks of using this approach at all, it is if anything more reliable and easier to use then even the ways of doing this provided by the system.
The only two drawbacks I can see at this point would be:
You have to be careful to make sure whatever proxy/intermediate host you are using, it is capable of handling at least the majority of system traffic, because if an app sends incompatible internet traffic, it will still be redirected through the TAP device (that is it's purpose).
The code base may be larger than in other cases
An alternative, 'unclean' way of doing this for Firefox in particular
If you are interested in only setting this proxy for Firefox, there are a couple of unclean ways of doing this: For instance, via the command line. It is, however (in my opinion), a very cheap and dirty way of achieving this, as this does not provide any compatibility whatsoever (basically a hack).
Conclusion
While implementing this may take a while, and the code base may be large:
It is not really possible, through any other means to achieve the same effect as VPNs achieve when they tunnel the entirety of your machine's traffic through the OpenVPN server.
If you want to achieve this kind of behavior, it is recommended that you use the approach outlined above, as it is a lot cleaner then 'alternative' methods of doing so (e.g. Socksifying traffic by intercepting it at a software level)
I need to write an SNMP agent on Windows for our company product.
SnmpApi.lib - It's my understanding that SnmpApi.lib allows you to create a full agent from scratch and probably requires more work. This, however, takes over the SNMP port and doesn't allow other agents to run and will take longer to code.
MgmtAPI.lib - This lib will allow one to create an extension to the already running Windows SNMP service and seems a more elegant approach to creating an Windows agent. However, I'm findining very little official MS documentation/examples on how to code such an agent. I've also read on another forum that this might be deprecated.
I've seen an Oreilly book out in the wild called "Windows NT SNMP" but I heard it uses deprecated libs or techniques. I don't know if this is true or not.
I've also seen libraries like SNMP++ that use the SnmpAPI.lib to create an agent, but again, this isn't as elegant as letting multiple sub-agents on the system via MgmtAPI.lib, it seems.
A few questions:
I heard that MgmtAPI.lib is legacy/deprecated. Is this true?
What's the best practice for creating a Windows SNMP agent?
What library should i be using?
Any other suggestions?
Thank you!
The default windows SNMP stack and framework is very limited in its ability. Unfortunately, your best solutions are probably found elsewhere (of which there are a number of external libraries and stacks). If you want one that is modular and allows sub-agents, then look into a stack that supports the AgentX protocol. That's a standardized SNMP subagent protocol and will let multiple sub-agents attach to a master agent.
But whatever you do, please pick a stack that supports SNMPv3 and use the security in it.
If you want a random stack example that is known to work well on windows, try Net-SNMP which is fairly popular among the "replace the SNMP stack on windows" crowd. But I'm also biased and associated with the project, so I'll end with saying: "it doesn't matter too much; pick any stack that supports both AgentX and SNMPv3".
I'd like to write a packet sniffer and editor for Windows. I want to able to see the contents of all packets entering and leaving my system and possibly modify them. Any language is fine but I'd like it to run fast enough that it won't burden the system.
I've read a little about WinPcap but the documentation claims that you can't use WinPcap to create a firewall because it can't drop packets. What tools will help me write this software?
Been there, done that :-) Back in 2000 my first Windows program ever was a filter hook driver.
What I did was implementing the filter hook driver and writing a userspace application that prepared a filter table on what to allow and what to disallow. When you get around your initial set of blue screens (see below for my debug tip in kernel mode) the filter mode driver is quite easy to use ... it gives each packet to a function you wrote and depending on the return code drops it or lets it pass.
Unfortunatley packets at that level are QUITE raw, fragments are not reassembled and it looks more like the "network card" end of things (but no ethernet headers anymore). So you'll have quite a bad time decoding the packets to filter with that solution.
There also is the firewall hook driver, as discussed in this codeproject article.
If you are on Vista or Server 2008 you'd better have a look at WFP (Windows Filtering Platform) instead, that seems to be the mandated API of the day for writing firewalls.
I don't know about it other than google turing it up some minutes ago when I googled for the filter hook driver.
Update: Forgot the debug tip:
Sysinternals DbgView shows kernel-mode DbgPrint output, and more important - it can also read them from the dump file your last blue screen produced. So sprinkle your code with dbgprint and if it bluescreens just load the dump into dbgview to see what happened before it died ... VERY useful. Using this I managed without having a kernel debugger.
I'm pretty sure you'd need to write a filter driver. http://en.wikipedia.org/wiki/Filter_driver I don't know much more than that :). It would definitely be a C/C++ Win32 app and you'd likely being doing some kernel side work. Start by downloading the DDK and finding some of the sample filter drivers.
If you just want to monitor what goes in and out of IIS, consider an ISAPI filter. Still C/C++ in Win32, but relatively easier than writing a device driver.
C# code to do this is here
I actually did this, several years ago. I'm hazy on the details at this point, but I had to develop a filter/pass-thru/intermediate driver using the Windows DDK. I got a lot of good information from pcausa. Here's a url which points to their product that does this: http://www.pcausa.com/pcasim/Default.htm
If you're doing this for practical reasons, and not just for fun, then you should take a look at Microsoft Network Monitor. The home page talks about the version 3.3 beta, but you can download version 3.2 from the Downloads page. There is also an SDK for NM, and the ability to write parsers for your own network protocols.
There's a question you need to ask which you don't know you need to ask; do you want to know which applications sockets belong to? or are you happy to be restricted to the IP:port quad for a connection?
If you want to know applications, you need to write a TDI filter driver, but that makes handling the receive almost impossible, since you can't block on the receive path.
If you're happy with IP:port, go in at the NDIS level, and I believe you can block on receive to your hearts content.
A word of warning; if you have no prior kernel experience, writing either of these drivers (although TDI is significantly harder) will take about two years, full time.
this:
TdiFw is a simple TDI-Based Open Source Personal Firewall for Windows NT4/2000/XP/2003
http://tdifw.sourceforge.net/
may help you
I'm looking for the concept to spawn a process such that:
it has only access to certain libraries/APIs
it cannot acess the file system or only specific parts
it can do least harm should malicious code run in it
This concept is known as sandbox or jail.
It is required to do this for each major Operating system (Windows, MacOSX and Linux) and the question is conceptual (as in what to do, which APIs to use and and what to observe) rather then language specific.
answer requirements
I really want to accept an answer and give you 20 points for that. I cannot accept my own answer, and I don't have it yet anyway. So if you really want your answer to be accepted, please observe:
The answer has to be specific and complete
With specific I mean that it is more then a pointer to some resource on the internet. It has to summarize what the resource says about the topic at least.
It may or may not contain example code, but if it does please write it in C
I cannot accept an answer that is 2/3 complete even if the 2/3 that are there are perfect.
this question FAQ
Is this homework? No.
Why do you ask this like a homework question? If you ask a specific question and you want to get a specific answer, and you know how that answer should look like, even though you don't know the answer, that's the style of question you get.
If you know how it should look like, why do you ask? 1) because I don't know all the answer 2) because on the internet there's no single place that contains all the details to this question in one place. Please also read the stackoverflow FAQ
Why is the main part of your question how to answer this question? Because nobody reads the FAQ.
Mac OS X has a sandbox facility code-named Seatbelt. The public API for it is documented in the sandbox(7), sandbox_init(3), and related manual pages. The public API is somewhat limited, but the facility itself is very powerful. While the public API only lets you choose from some pre-defined sandboxes (e.g. “All sockets-based networking is prohibited”), you can also use the more powerful underlying implementation which allows you to specify exactly what operating system resources are available via a Scheme-like language. For example, here is an excerpt of the sandbox used for portmap:
(allow process-exec (regex #"^/usr/sbin/portmap$"))
(allow file-read-data file-read-metadata (regex
#"^/etc"
#"^/usr/lib/.*\.dylib$"
#"^/var"
#"^/private/var/db/dyld/"
#"^/dev/urandom$"))
(allow file-write-data (regex
#"^/dev/dtracehelper$"))
You can see many sandboxes used by the system in /usr/share/sandbox. It is easy to experiment with sandboxes by using the sandbox-exec(1) command.
For Windows, you may want to have a look at David LeBlanc’s “Practical Sandboxing” talk given at Black Hat USA 2007. Windows has no built-in sandboxing technology per se, so the techniques described leverage an incomplete mechanism introduced with Windows 2000 called SAFER. By using restricted tokens, one can create a process that has limited access to operating system resources.
For Linux, you might investigate the complicated SELinux mechanism:
SELinux home,
a HOWTO. It is used by Red Hat, for example, to harden some system services in some of their products.
For Windows there is a sandbox in Google Chrome. You may want to investigate it. It uses liberal BSD-like license.
For Linux there would be good old chroot or more sophisticated http://plash.beasts.org/wiki/.
OS X since Leopard has some SELinux-like protection available.
The site codepad.prg has a good "About" page on how they safely allow the execution of any code snippets..
Code execution is handled by a supervisor based on geordi. The strategy is to run everything under ptrace, with many system calls disallowed or ignored. Compilers and final executables are both executed in a chroot jail, with strict resource limits. The supervisor is written in Haskell.
When your app is remote code execution, you have to expect security problems. Rather than rely on just the chroot and ptrace supervisor, I've taken some additional precautions:
The supervisor processes run on virtual machines, which are firewalled such that they are incapable of making outgoing connections.
The machines that run the virtual machines are also heavily firewalled, and restored from their source images periodically.
FreeBSD has specific concepts of jails, and Solaris has containers. Depending on what you're looking for, these may help.
chroot jails can help to limit what an application can do (though any app with root privileges can escape a jail), and they're available on most UNIXen, including OS X.
As for Windows, I'm not sure. If there was an easy way to sandbox a Windows app, most of them would be a lot more secure by now, I'm sure.
On windows (2000 and later) you can use Job objects to restrict processes.
If you really want a technique that will work with all these platforms, as opposed to a separate solution for each platform, then I think your only answer is to set up a virtual machine for each testing environment. You can restore back to a snapshot at any time.
Another big advantage of using virtualization is that you can have all of the testing environments with their guest operating systems all on the same box.
For Linux, there is AppArmor. Unfortunately, the project is somewhat on hiatus.
Another sandboxing-alternative is VServer, which uses virtualization.
Generally any virtual private server will do:
Linux VServer
http://linux-vserver.org/Welcome_to_Linux-VServer.org
Parallels Virtuozzo Containers
http://www.parallels.com/products/pvc/
and as was mentioned FreeBSD and Solaris has own implementations.
Oh. actually I've noticed you're asking it to work on ANY OS. Well, that might be complicated a bit as the I think less effort is just to reuse some VM that can support some level of sandboxing like:
Java
.NET
I'm not an expert on the topic, but i think the standard answer for linux is to define a SeLinux policy with the right capabilities for the process.
Despite primarily being a windows user, I am a huge fan of rsync. Now, I don't want to argue the virtues of rsync vs any other tool...this is not my point.
The only way I've ever found of running rsync on windows is via a version that is built to run on top of Cygwin, and as Cygwin has issues with Unicode, so does rsync.
Is anyone familiar enough with the workings of rsync to say if there are any real technical programming hurdles to porting rsync to a native Win32 binary?
Or is it maybe that there has just never been enough interest from windows users to care to port it over?
Partly I ask because I'm am considering trying to take on the task of starting a port, but I want to make sure there's not something I'm missing in terms of why it may not be possible.
The way that windows locks open files might cause an issue requiring you to hook into the Volume Shadowcopy Service.
About two years ago this fellow ported the algorithm to C#. I haven't taken a look at the code (or the provided binary), but it might be a place to start looking or someone to try contacting.
http://www.russiantequila.com/wordpress/?p=8
I've been evaluating an effort to undertake a win32 port as well. I don't believe anything major would block it, but evidence from both the rsync mailing list and another discussion points to a heavy reliance on unix fork() system calls. Using threads appears the way to go for win32.
Threads vs. Fork discussion
(disclaimer: i promise, i don't google myself, but google analytics brought me here)
i went through porting rsync to .net (sig11's link is my blog). there are no technical hurdles, just practical ones. as was already said, the code is rather... dense. difficult to follow, and complete lack of comments. i'm more than happy to make my work available, but unfortunately, since it was part of a commercial effort, it's not in significantly better shape.
i have, on a number of occasions, messed around with the idea of reverse-engineering the protocol and doing a ground-up implementation that's wire-compatible with the existing one, but ... a bit cleaner to work with. i've even started a wiki to that effect, but... as you can see from the lack of contents there, other item have taken priority. if anyone would like to work with me on this, that may be the impetus i need to get going.
the concept of the tool is great, as is the functionality it offers, however it's rather limited outside the *ix space, and could definitely benefit from an api.
wiki link for reference:
http://www.russiantequila.com/wiki/index.php?title=Main_Page
Have you seen this:
http://www.itefix.no/i2/taxonomy/term/39
I have used cwrsync without any problem (and with the much of the usual cygwin misery), but I haven't had any need for unicode filenames, so I've not seen that problem.
I don't really know why there isn't a native Win32 port, but I did look at the source a while back because I implemented a similar delta-copy system in C#. As one would expect from the world of brilliant *nix hackers, the source is largely single-character variable names and a total absence of comments, which isn't terrible helpful and might be rather off-putting to would-be porters.
I would really appreciate a port of rsync to MS-Windows such that it can be built using Visual Studio. I am encountering various protocol errors at random, somewhat intermittently. I am using rsync to distribute sw to a grid of around 200 machines and typically get around a a dozen failures. I am using GCC 4.4.2 and the latest cygwin to build rsync v3.0.7. It would help me alot if I could experiment with a version that does not require cygwin. This is because the machines in the grid already have another cygwin-based app running that is a different version to the one I have.
Having spent some time on the rsynv mailing list opinion seems to be divided as to cause of protocol errors on MS-Windows. Some say it is a bug in rsync where it failed to do a clean socket shutdown, a bug that was fixed a while ago. Others say that it is a fundamental protocol error in rsync where the client does not tell the server that it is finished, it just shuts down, causing MW-windows servers to get a RST signal on the socket, something that does not happen on Unix.