I'm developing a network-redirector like SMB.
I want to test various file I/O to compare NTFS or SMB implementation.
What I want to test are,
CreateFile
Read, WriteFile
DeleteFile
RenameFile
Set, GetFileInformationByHandle
etc.
And it' would be better if it can measure each I/Os duration.
Is there a program I can use?
If you are developing a file system driver or use some redirector driver (either our Callback File System or alternatives), you can use IFSTest tool to check your implementation for correctness.
XPerf will answer all of these questions, allowing you to see perf at both the file level and the block level. Check out the PDC09 video on the topic at http://www.microsoftpdc.com/2009/CL16
You can use the File Server Capacity Tool (FSCT) provided by Microsoft. It will let you simulate a typical user workload against a file server that supports SMB. The tool can simulate multiple users from a single client and aggregates the results into text files in the end.
You can get more information, including links to download and detailed presentations at http://blogs.technet.com/b/josebda/archive/2009/09/16/file-server-capacity-tool-fsct-1-0-available-for-download.aspx
Related
Is there any way to hook all disk writes going thru the system, and receive the file names of whatever's being modified, using the Win32 API? Or is this something that would require writing a driver?
You can't do this in user mode, it needs to be kernel mode and so that means a driver. You need a File System Filter Driver.
If you don't care about intercepting the actual data, and only want to know which files are being modified/created/deleted then you can use the ReadDirectoryChangesW API to get that info from userland. Note however that it's one of the hardest functions to use effectively and efficiently, and you should be familiar with IOCP to use it correctly.
Usually, when an application writes to one of it's files on disk, the file modified timestamp changes.
Sometimes, and in my case it is an application written in ProvideX (a Business Basic derivative i believe) doing the writing, the modified timestamp does not change after a write. A program like MyTrigger will not pick up on the write operation either, but Sysinternals ProcessMonitor does log the disk activity.
It seems obvious that there are different ways to ask windows to perform write operations, and the request could then be hooked or logged in various different ways as well.
I need to be able to hook the write operations coming from the ProvideX application. Any pointers on the different ways windows writes to disk, and the type of hooks available for them would be greatly appreciated.
Thanks
User-mode process can write to the file either using WriteFile API function or using MMF, memory-mapped file API (CreateFileMapping/MapViewOfFile/Write to memory block). Maybe your application goes MMF way. MMF writes to files very differently from WriteFile API, but they both lead to the same end point - IRP sent to file system driver. File system filter driver (such as the one used by Sysinternals stuff) can track write requests on that IRP level. It is technically possible to distinguish between write operations initiated by MMF and WriteFile as different IRPs are sent (cached and non-cached writing is involved). It seems that directory change monitoring function in windows tracks only one IRP type, and this causes MyTrigger to miss the change.
Is there a utility for Windows that allows you to test different aspects of file transfer operations across a Lan or a Wan.
Example...
How long does it take to move a file of a known size (500 MB or 1 GB) from Server A (on site) to Server B (on site) or to Server C (off site-Satellite location)?
D-ITG will allow you to test many aspects of your links. It does not necessarily allow you transfer a file directly, but it allows you to control almost all aspects of the transmission of data across the wire.
If all you are interested in is bulk transfer time (and not all the nitty-gritty details) you could just use a basic FTP application and time the transfer.
Probably nothing you've not already figured out. You could get some coarse grain metrics using a batch file to coordinate:
start monitoring
copy file
stop monitoring
Copy file might just be initiating a file copy between two nodes on the LAN, or it might initiate a FTP copy between two nodes on the WAN.
Monitoring could be as basic as writing the current time to output or file, or it could be as complex as adding performance counter metrics from the network adapter on the two machines.
A commercial WAN emulator would also give you the information your looking for. I've used the Shunra Appliance successfully in the past. Its pretty expensive, so I'd really only recommend it if critical business success is riding on understanding how application behavior could change based on network conditions and is something you could incorporate into regular testing activities.
I am doing a project on a centralized LAN management system. I need to know how many CPU cycles is each process of a remote PC consuming(as in a Task Manager )so that the network admin can close few processes,in case the CPU utilization of a system in network goes beyond acceptable rates..
I would like to know if there is a Win32 API for this requirement of mine n if so ,i request you to give me information about it..
thank you in advance..
Win32 API has lots of functions to find all kinds of information about currently running processes and threads, here's a link to the full list of them: http://msdn.microsoft.com/en-us/library/ms683223(VS.85).aspx
Explore the list and you should be able to find the function(s) there that meet your requirements, for example GetProcessTimes() returns structures that contain the amounts of time the process has executed in kernel mode, in user mode, etc.
You need to look at the performance monitor system. You can get the stats from there (in the Process counter).
Here's a (delphi) explanation of it, that's pretty good and simple to understand.
When you understand how it all works, you then need the Performance Counters API to read the data counters.
I'm trying to make file I/O over a network drive (likely over a WAN or VPN) as reliable as possible for a native C++ Windows app...
What are the possible error conditions that I need to be able to handle?
How can I simulate these error conditions in testing?
How do I get detailed information on a particular error? For example, if fopen() fails, does errno tell me everything I need to know, or do I need to get at the GetLastError() value?
How do I reliably distinguish between "network drive access fully functional but the file doesn't exist" and various problems with the network or server?
One particular error condition that I've noticed on my desktop (not specific to the app we're developing) is that sometimes the first attempt to access a file on a network drive will fail, but it presumably causes the drive to be reconnected in the background, because subsequent connections work. I don't know what causes this. This is an example of the kind of error condition that I want to properly handle.
EDIT: This is for a legacy distributed application that uses files on network shares for communication between nodes. Some nodes may be unattended, so passing the error on to the end user may not be an option. The long term goal is to switch to a better protocol, but in the short term I'd like to make the file I/O as reliable as possible.
I believe you're approaching this from a wrong perspective. There's little one can do in the application itself to improve what is essentially a network filesystem driver problem, perhaps except implementing the networked I/O itself. That being said, you should be better off choosing a suitable networked filesystem for your needs. Look at this on Wikipedia.
Generally, your application should behave like the file is locally-stored. Don't try too hard to handle network problems. But if your choice of a network filesystem is good, then these problems can be automatically mitigated.
So I'd say you should settle with checking errno in case of errors. Perhaps fallback on local storage in case writing a remote file fails (if the networked file system doesn't handle this itself).