We are running a Windows .EXE file via a Cygwin script and are encountering performance issues. I have seen various posts about Cygwin performance issues, including this one, one of whose answers delves enough into Cygwin internals to make me think there could be a problem. However, these posts do seem to be more about startup time, tab completion, etc. Before we launch on a benchmarking witch hunt, I was hoping to ask: is there any intrinsic reason why a Windows .EXE could run slower if kicked off from Cygwin vs. BAT?
Not the actual program, no.
Housekeeping and stuff before running the program may vary. Cmd probably calls CreateProcess directly. Cygwin's bash may first do argument parsing, wildcard expansion, fork via Cygwin's slow implementation and call exec with the parsed arguments, which Cygwin has to piece together into a string again to pass to CreateProcess. But in the end, a new process is created which has no ties to its parent anymore. So how fast your program runs entirely depends on that program, not on who launched it.
Related
I have searched the web for long time without success. How can I add a new command to the kernel command line? e.g. a command like ver without any arguments, that simply prints Hello World message when executed.
The Linux kernel does not handle any commands (but the GRUB bootloader can pass some arguments to the booted kernel). It is just handling system calls. You could add some new one, but learn about advanced linux programming & syscalls(2) before hacking the kernel.
Perhaps you want to add a new command available to your shell. This is quite simple, put a new executable file (either an ELF executable, e.g. compiled from C code, or some script) in a directory known to your PATH. You might also make a new shell function.
The system call to create a new process is fork(2) (sometimes emulated thru some appropriate call to clone(2)...). fork is difficult to understand, so take several hours to read about it. Almost every process (with a few exceptions, notably /sbin/init) is created by fork. The system call to execute a new program is execve(2). Your shell will often use fork & execve to run commands (except builtin ones, like cd). Use strace(1) to understand what syscalls are done (by some program).
PS. Don't dare hacking your kernel before understanding quite well user-land Linux system programming....
I decided to write a program in RUBY in which the following things should be done:
1 - this program must run a specific program (for example utorrent) at a specific time (for example 1 pm).
2 - this program must turn off my computer at another specific time.
I don't have any idea about the algorithm and manner of writing such program.
One of the easiest ways to do this is to simply send kill signals to the processes, requesting the app shut down normally (Linux), or in Windows use taskkill.
To shutdown a machine in Windows, you can use shutdown /s /f which forcibly closes any programs that are running, and turns the computer off.
No matter which way you do it, you'll basically be running the enter link description heresystem() command in Ruby, which runs command line commands. To make your app portable, you simply look up how to do these tasks in each target OS, and you're done.
Two more alternatives that work the same as your Ruby proposal, but which are not as easily portable:
Write a batch file in Windows that calls taskkill, or a bash script on Linux. Unless the program in question provides a specific way to shut it down via its own command-line parameters, this should work for any/all applications.
You can also use Task Scheduler in Windows, or cron in Linux to do the same thing.
my goal: to create a suite of scripts that do some common system tasks, which include these
copy/move/list/search/grep files
watch/start/stop processes
run queries against Oracle via sqlplus
i grew accustomed to using Cygwin/bash to ease my life at work, and frankly speaking, i don't want to move away from bash language and start learning PowerShell, for example - so i started searching for a way to run bash scripts on Windows, ... preferably something alternative to Cygwin.
the truth is that i am still not pleased with Cygwin installation, and the fact that there is no simple way around it, that it is targeting more or less expert users, and there are a number of things that might pop up during the installation. i mean. what i am trying to do now is to write a suite of scripts that will target someone less expert than me (and i am in no way a real expert) - in most cases some kind of an administrator who doesn't want to know the script details.
i am thinking that this user will also want to be able to run these scripts on another machine, and i want to be able to explain him/her how to do it, without just saying, call the support, and, me, eventually (so that we can install cygwin on another machine etc etc.)
i tried MinGW(msys) but it also needs manual steps to set things up - i mean, these manual steps have become something of a de facto standard in these Windows ports (sorry, maybe i have a mood for bragging). win-bash looked like it could be a solution, but i ended up trashing it too, because of the old bash version, and its inability to do things i was able to do in cygwin - specifically
here documents
things like "cmd /C dir *" (don't know why) - and yes, i do cmd /C dir in cases i am in some kind of shared network folder with thousands of files, and ls is significantly slower than dir
my questions at last:
am i doomed to use PowerShell? i guess i will, reluctantly, if i have to
is there a simple pre-packaged "slim" cygwin installation.. or, portable cygwin, even better? there is a cygwin-portable project on sourceforge, but it's not that doesn't need those manual steps, again, apparently - is there a way to automate those steps, perhaps? and if there is, i wonder why somebody hasn't done it already? - and then, would it be possible to call bash scripts from Windows command prompt using a simple command like "bash somescript.sh"?
thanks for your attention.
As mentioned here, the Cygwin installation can totally be scripted and parametrized to ran in a silently and automatic mode.
If you define the minimal list of cygwin packages you need, just use a little .bat script that call the cygmin setup executable like this
setup.exe --packages=list_of_packages_you_need --quiet-mode
If you wrap the cygwin install process, it should be tolerable for a less technical user.
The cygwin install can be streamlined using command-line args;
http://sources.redhat.com/ml/cygwin-apps/2003-03/msg00526.html
You can also automate the install of most cygwin packages through cyg-apt.
I haven't verified this but I suspect that msys implements a *nix look alike by creating windows executable versions of system commands. All of the common commands have an executable on my install of msys. If that is true then it should be possible to use them separate from a complete install.
Try copying "bash.exe", "cp.exe", etc. from the msys bin directory to a machine/vm that does not have an msys install and see if it works. You may need to copy some dll's or shared libraries as well. A windows dependency checker program would show which dll's an executable is using.
You could package up the stuff you use and make a simple installer or just copy the files with your scripts.
Take a look at MKS Toolkit. Unlike Cygwin, it can live within the Windows world. Files end in CR/LF like Windows files, and you don't have that /cygdrive/c stuff. Naked drive letters work fine in MKS Toolkit.
A few caveats:
I haven't used MKS Toolkit in a long time. See following reason.
MKS Toolkit is (sit down for this) $600 per license. Ouch! That's why I use Cygwin even though I don't think it's as good or works as well.
It's Kornshell based and not Bash (although this may be a bit different). Kornshell and BASH are 95% alike. However, that last 5% gets you. I actually like Kornshell better than BASH in many respects. Kornshell has the print statement which is way superior than the echo statement. Variable names don't disappear in blocks. You can easily do double loops because almost all the commands can take unit numbers of input and output. However, Kornshell doesn't have those neat escape characters in the prompt, and it's hard to find the exit status of a command in the middle of the pipeline.
I have been running drush scripts (for Drupal) with Cygwin on my relatively fast windows machine, but I still have to wait about a minute for any drush command (specifically drush cache clear to execute).
I'm quite sure it has something to do with the speed of Cygwin since my fellow developers (who are running Linux) can run these scripts in about 5 seconds.
Is there a way to make Cygwin use more memory and/or CPU per terminal?
The problem you're running into is not some arbitrary limit in Cygwin that you can make go away with a settings change. It's an inherent aspect of the way Cygwin has to work to get the POSIX semantics programs built under it expect.
The POSIX fork() system call has no native equivalent on Windows, so Cygwin is forced to emulate it in a very inefficient way. Shell scripts cause a call to fork() every time they execute an external process, which happens quite a lot since the shell script languages are so impoverished relative to what we'd normally call a programming language. External programs are how shell scripts get anything of consequence done.
There are other inefficiencies in Cygwin, though if you profiled it, you'd probably find that that's the number one speed hit. In most places, the Cygwin layer between a program built using it and the underlying OS is pretty thin. The developers of Cygwin take a lot of pains to keep the layer as thin as possible while still providing correct POSIX semantics. The current uncommon thickness in the fork() call emulation is unavoidable short of Microsoft adding a native fork() type facility to their OS. Their incentives to do that aren't very good.
The solutions posted above as comments aren't bad.
Another possibility is to go through the drush script and see if there are calls to external programs you can replace with shell intrinsics or more efficient constructs. I wouldn't expect a huge speed improvement by doing that, but it has the nice property that you'll speed things up on the Linux side as well. (fork() is efficient on Linux, but starting external programs is still a big speed hit that you may not have to pay as often as you currently do.) For instance:
numlines=`grep somepattern $somefile | wc -l`
if [ $numlines -gt 0 ] ; then ...
would run faster as:
if grep -q somepattern $somefile ; then ...
The first version is arguably clearer, but it requires at least three external program invocations, and with primitive shells, four. (Do you see all of them?) The replacement requires only one external program invocation.
Also look at things that slow down Cygwin startup:
Trim down your Windows PATH (to the bare bones like %SystemRoot%\system32;%SystemRoot%)
Remove things you don't need from bashrc and bash_profile
Move things you only need in your terminal window from bashrc to bash_profile
One surprisingly large time suck in Cygwin is Bash completion. If you are using it (and you should because it's great), only source completion for the commands you need (rather than all of them which used to be the default). And, as mentioned above, source them from bash_profile, not bashrc.
You can give Cygwin a higher priority.
Write a new batch file, for example, "cygstart.bat" with the following content:
start "Cygwin" /high C:\cygwin\Cygwin.bat
The /high switch gives the shell a higher process priority.
In Windows -- and probably Unix for that matter -- using the chdir() function in a (32-bit) program doesn't change the directory when the program exits. (It does in a 16-bit Windows program.)
Does anybody know how to do that in a Windows 32-bit program?
Uhm... IMHO it's exactly one of the things that the OS must guarantee not to happen. The current dir is a per-process property, a child process usually inherits it from the parent process, but the reverse should not happen (and it doesn't).
To obtain what you want, the parent could actively watch some information (message, file, shared memory...) in which the child process stores the new directory, and then call chdir() with the new value.
As far as I know, Windows' cmd.exe doesn't have any mechanism like that.
Actually, by using code injection techniques (e.g. CreateRemoteThread) on the parent process it could be possible to force it to do something unexpected, but it's a very dirty trick, not at all good neither general.
Win16 was different: there was a single "msdos" state for all the programs, but it was a limitation, not a feature.
It sounds like you're asking one process (your Win32 program) to change the CWD of another process (your shell). As far as I know, this is impossible without the latter process providing an API for such a purpose. The nearest I can come to any sort of reference for this assertion, however, is the following quote from MSDN:
A parent process can directly alter the environment variables of a child process during process creation. This is the only situation when a process can directly change the environment settings of another process.
Well yeah it's true the popular API calls to change directory change it for the process. ... BUT ...
(1.) 16-bit windows programs can change the global directory; probably because they run in the same process as the command.com thing. That's what I've been happily using for years; I assume XP somehow emulates this? ... But now Windows 7 64-bit won't run 16-bit programs anymore! (?)
(2.) Both Windows and Unix "cd" commands can of course change directories for the calling process -- presumably because they are built-in commands of the command shell. But successor Windows shells manage to accomplish this, or at least I hope PowerShell can do that. All built-ins?
(3.) The way I've wound-up doing it is modifying my programs that used to call the API to simply emit "cd \dst\directory" to stdout, then in a procedure do
chdirprogram >t~.bat
call T~.bat
Which works great. And of course the usual point of a change-directory program is to provide the functionality in a batch procedure with a computed destination. Which of course you can do in Unix with Bash etc. variables, but not in Windows batch files, although maybe (?) in the numerous successor Windows procedure things, which I don't want to use. ... Since this functionality is obviously useful, I was hoping someone knew of a sneaky Windows call what'd do it. The explanation that it's somehow wrong for a process to change the directory for a calling process is one of those bogus, "you're not supposed to do that and I won't tell you why" excuses. ... But I guess I'll just stick to my pitiful little batch files.
Are you talking about the SetCurrentDirectory function of Windows API? The article says that the function "changes the current directory for the current process". In for instance Delphi, there is a function ChDir that actually calls this API function.