Just tests a very simple command like:
while true; do bash -c "echo hello"; done
You will find how much slow the bash in Cygwin is. Does anybody know why?
It's a fresh install of cygwin 1.7 on win7.
thanks to Jared's testing idea, I modified the command to this(adds bash -c):
time for i in {1..10} ; do bash -c "echo Hello" ; done
Hello
...
real 0m7.711s //it's the problem
user 0m0.091s
sys 0m0.476s
How about excluding Cygwin paths from your antivirus software ?
Check your path. Referring a non-existant path or a very slow network share can cause the symptoms you are describing.
Below are 3 possible causes which I encountered. Of course, other problems not mentioned can also cause slow shell commands in Cygwin.
If you have the option "Automatically detect settings" in "LAN Settings", Windows will use the WPAD protocol to discover the local HTTP proxy. First it will send a DHCP "Inform" request with option 252, then it will try a DNS lookup on "wpad". These 2 operations can take a few seconds to time-out.
If the shell accesses some paths like /cygdrive/... , a NetBIOS name query will be executed, which can also take some time to time out.
If the shell accesses some paths like //mypath/... , a NetBIOS name query will be executed, which can also take some time to time out.
Solutions :
Disable "Automatically detect settings" in "LAN Settings" in the Windows "Internet Options" control panel.
Add the following entry in %SystemRoot%\system32\drivers\etc\hosts :
127.0.0.1 localhost cygdrive wpad
Make sure to avoid double slashes at the beginning of all paths.
Finally, I found the source - a service named "QQPCMgr RTP Service" running on my office computer, it's the real time protection service of "QQ PC Manager".
By disabling it, the time of the script in the question falls back to:
real 0m0.943s
user 0m0.105s
sys 0m0.231s
I have told the developers of QQPCMgr about this, hope they will find the reason.
This still much slower than Linux, but gets the same "real time" of other cygwin computers.
Thank you all!
This is how this problem manifested for me. For the first say 20 commands entered at the cygwin prompt it is fast, then it abruptly becomes painfully slow.
I checked that every item in my path was a valid directory.
I tried an authentication fix suggested elsewhere -
https://superuser.com/questions/877051/cygwin-from-windows-very-slow-laggy/1247517 .
It didn't work for me.
I replaced my exisitng virus scanner (System Center Endpoint Protection) with Sophos. That fixed the problem for me. Hope this helps.
I voted up on James McLeod, because starting a bash process takes some time, but it doesn't mean it will run commands slower than in UNIX.
Invoking bash -c from within a bash script is near to senseless, and Makefiles can call a lot of bash subprocesses, unless you append ; \ at the end of the commands.
For example, if a Makefile has the following:
echo Hello World
echo Good Bye
It will call two bash processes. To make it faster and call just one bash process:
echo Hello World; \
echo Good Bye
Debian has adopted dash instead of bash as the main shell, because starting many init scripts using bash will make the system take much longer to boot (each script call its own bash process).
Let's put some numbers to it. I ran the following:
time for i in {1..1000} ; do echo "Hello" ; done
The result I get from a standard Cygwin bash window is:
...
Hello
Hello
real 0m0.584s
user 0m0.031s
sys 0m0.000s
And from a xterm bash window on the same system I get:
...
Hello
Hello
real 0m0.037s
user 0m0.016s
sys 0m0.000s
I think this pretty much answers the question for you. The problem is you're going through a "Windows" "cmd" like window, which is inherently slow. Cygwin itself isn't the problem, it's the display trying to keep up that is slowing things down (for this test).
I was able to fix this problem by uninstalling bash-completion and switching from Kaspersky to Windows Security Essentials. (I diagnosed the Kaspersky interference using Process Explorer while running the "echo Hello" benchmark.) Brought my benchmark time from 7 seconds to 0.2.
On Comodo Internet Security Premium 10, i added "C:\Users\\Documents\MobaXterm\slash\bin" to the list of trusted files. (Settings -> File rating -> File list). Local terminal is fast again now.
For Windows Defender:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender]
"DisableAntiSpyware"=dword:00000001
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection]
"DisableBehaviorMonitoring"=dword:00000001
"DisableOnAccessProtection"=dword:00000001
"DisableScanOnRealtimeEnable"=dword:00000001
or less dramatically:
powershell -Command Add-MpPreference -ExclusionPath "C:\opt\cygwin64"
powershell -Command Add-MpPreference -ExclusionPath "C:\home"
powershell -Command Add-MpPreference -ExclusionProcess "git.exe"
powershell -Command Add-MpPreference -ExclusionExtension ".c"
powershell -Command Add-MpPreference -ExclusionExtension ".cpp"
powershell -Command Add-MpPreference -ExclusionExtension ".cxx"
For Sophos:
REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Scanning\Config" /v "OnAccessExcludeFilePaths" /t REG_MULTI_SZ /d "C:\Program Files (x86)\Sophos\Sophos Anti-Virus\\0C:\home\\0D:\Video\\" /f
REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Scanning\Config" /v "OnAccessExcludeProcessPaths" /t REG_MULTI_SZ /d "C:\opt\cygwin64\\" /f
Invoking bash from a shell script can't be fast. Is it faster to just use
while true; echo hello; done
?
Related
I want to send commands in the ADB shell itself as if i had done the following in cmd.
>adb shell
shell#:/ <command>
I am using python 3.4 on a windows 7 OS 64bit machine. I can send one-line shell commands simply using subprocess.getoutput such as:
subprocess.getoutput ('adb pull /storage/sdcard0/file.txt')
as long as the adb commands themselves are recognized by ADB specifically, such as pull and push, however there are other commands such as grep that need to be run IN the shell, like above, since they are not recognized by adb. for example, the following line will not work:
subprocess.getoutput ('adb shell ls -l | grep ...')
To enter the commands in the shell I thought I needed some kind of expect library as that is what 'everyone' suggests, however pexpect, wexpect, and winexpect all failed to work. they were written for python 2 and after being ported to python 3 and my going through the .py files by hand, even those tweaked for windows, nothing was working - each of them for different reasons.
how can i send the input i want to the adb shell directly?
If none of the already recommended shortcuts work for you you can still go the 'regular' way using 'subprocess.Popen' for entering commands in the adb shell with Popen:
cmd1 = 'adb shell'
cmd2 = 'ls -l | grep ...'
p = subprocess.Popen(cmd1.split(), stdin=PIPE)
time.sleep(1)
p.stdin.write(cmd2.encode('utf-8'))
p.stdin.write('\n'.encode('utf-8'))
p.stdin.flush()
time.sleep(3)
p.kill()
Some things to remember:
even though you import subprocess you still need to invoke subprocess.Popen
sending cmd1 as a string or as items in a list should work too but '.split()' does the trick and is easier on the eyes
since you only specidfied you want to enter input to the shell you only need stdin=PIPE. stdout would only be necessary if you wanted to receive output from the shell
time.sleep(1) isn't really necessary, however since many complained about input issues being faster or slower in python 2 vs 3 consider maybe using it. 'they' might have been using versions of 'expect' that need the shell's reply first. this code also worked when i tested it with simply swapping out and in the process with time.sleep(0)
stdin.write will return an error if the input is not encoded properly. python's default is unicode. entering by binary did not work for me in my tests like this "b\ls ..." but .encode() worked. dont forget the endline!
if you use .encode() there is a worry that the line might not get sent properly, so to be sure it might be good to include a flush().
time.sleep(3) is completely uneccesary, but if your command takes a long time to execute (eg a regressive search through the entire device piped out to a txt file on the memory card) maybe give it some extra time before killing anyhting.
remember to kill. if you didnt kill it, the pipe may remain open, and even after exiting the test app on the console the next commend still went to the shell even though the prompt appearsed to be my regular cmd prompt.
Amichai, I have to start with pointing out that your own "solution" is pretty awful. And your explanation makes it even worse. Doing all those unnecessary things just because you do not understand how shell (here I mean your PC's OS shell, not adb) command parsing works.
When all you needed was just this one command:
subprocess.check_output(['adb', 'shell', 'ls /storage/sdcard0 | grep ...']).decode('utf-8')
With the cmd.exe, i call a programm that I have not write myself. I cannot modify it. I would like to disable the message printed in the shell by this programm.
My OS is Windows 7 enterprise 32 bits.
Thank you
If it's outputting to standard error and output, you should be just able to redirect them:
dodgy_prog >nul: 2>nul:
(or use /dev/null under UNIXy operating systems - it's unclear which variant you want since you talk about the Windows cmd.exe but your question is tagged bash).
A small request: I read Stack Overflow's Perl questions every day, and answer/contribute where I can; today I need the community's help!
Perl setup: I'm running Active Perl 5.8.8 on Windows. The installation is on our department server's local drive, which is also shared to the network. All department users run Perl on their own PCs by pointing to this network-installed Perl. This has worked for years, and isn't causing a problem, but it's a piece of info needed to understand the problem.
The server in question is also our "cron" (Scheduled Task) server, handling a variety of automation tasks. Suddenly last week, system calls in Perl scripts (on the server) started failing (details below). At first, I suspected a corrupt Perl installation, but all of the client PCs can still run the same Perl scripts without any issues, leading me to think it's a server issue. I've rebooted the server twice, and the problem is persistent, thus I need help!
Here are some examples of the various way that system calls are failing, boiled down to Perl one-liners:
% perl -e "system('dir')"
That should print a "dir" listing, but instead it opens a sub-shell. If I type "exit", I can exit the sub-shell, and I'm back in the original shell (confirmed by examining the shell history using the UP arrow key).
% perl -e "print `dir`"
This actually hangs. Nothing happens at all. If I Ctrl-C to kill the process, I get the message "Terminating on signal SIGINT(2)" and the DOS prompt comes back. But, any future commands in the DOS prompt (even just hitting ENTER) cause the error "The process tried to write to a nonexistent pipe.". You have to exit the DOS prompt as it's effectively useless.
Last example:
% perl -e "system('Z:/Scripts/rebuild.pl')"
'ebuild.pl' is not recognized as an internal or external command,
operable program or batch file.
In this case, Perl switches the forward-slashes (/) to DOS/Windows back-slashes (), which it has done just fine for years. But, Perl is interpreting the "\r" at the beginning of the "rebuild.pl" filename as a carriage-return (I think) and looking for the remaining "ebuild.pl". Calls to other scriptnames whose characters can't be misinterpreted like that result in the above hangs (if you use backticks) of sub-shells being opened (for system() calls).
I'm not just puzzled by this - I'm desperate! Our department server's "cron" jobs are useless right now since we use a lot of system calls.
Again, I don't think this is a corrupt Perl install, since the network users can run fine. So, what could happen on an individual machine (not tied to the Perl install itself) that could cause Perl's system calls to fail like this?
Environment settings, as requested:
ALLUSERSPROFILE=C:\Documents and Settings\All Users
APPDATA=C:\Documents and Settings\engmodem\Application Data
CDSROOT=Z:\Cadence\SPB_16.5
CDS_CONCEPT_NOSPLASH=TRUE
CDS_LIC_ONLY=1
CDS_SITE=Z:\Cadence\Sites\16.5
CHDL_LIB_INST_DIR=%CDSROOT%
CLIENTNAME=USENTUTTLJL3C
ClusterLog=C:\WINDOWS\Cluster\cluster.log
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=CORPUSAPP5
ComSpec=C:\WINDOWS\system32\cmd.exe
CONCEPT_INST_DIR=%CDSROOT%
FP_NO_HOST_CHECK=NO
HOMEDRIVE=H:
HOMEPATH=\
HOMESHARE=\\PF1\HOME
ICMHOME=Z:\Software\PTC\INTERC~1
INSTDIR=%CDSROOT%
LOGONSERVER=\\ENGMAHO5
LSF_BINDIR=Z:\Software\LSF\bin
LSF_ENVDIR=\\hwc151\LSF_6.2\etc
MESSAGE=BROADCAST
NUMBER_OF_PROCESSORS=2
OA_PLUGIN_PATH=%CDSROOT%\Share\oaPlugIns
OS=Windows_NT
Path=C:\Program Files\Legato\nsr\bin;Z:\oracle\ora92\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Windows Resource Kits\Tools\;Z:\Software\Perl\5.8.8\bin;C:\Program Files\Oracle\jre\1.3.1\bin;C:\Program Files\Oracle\jre\1.1.8\bin;C:\Program Files\Support Tools\;Z:\Software\LSF\bin;C:\Program Files\PHP\;C:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\EMC RepliStor;C:\GitStack\python;C:\GitStack\python\Scripts;C:\GitStack\git\cmd;Z:\Scripts;Z:\bin;Z:\Cadence\SPB_16.5\tools\bin;Z:\Cadence\SPB_16.5\tools\fet\bin;Z:\Cadence\SPB_16.5\tools\pcb\bin;Z:\Cadence\SPB_16.5\OpenAccess\bin\win32\opt
PATHEXT=.COM;.EXE;.BAT;.PL;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.VBS
PCB_LIBRARY=16
PERL5SHELL=cmd
PHPRC=C:\Program Files\PHP\
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 29 Stepping 1, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=1d01
ProgramFiles=C:\Program Files
PROMPT=$P$G
PULLUP_DIFF_PAIRS=TRUE
SESSIONNAME=RDP-Tcp#1
SystemDrive=C:
SystemRoot=C:\WINDOWS
TZ=EST5EDT
VISUALSVN_SERVER=C:\Program Files\VisualSVN Server\
WF_RESOURCES=Z:\oracle\ora92\WF\RES\WFus.RES
windir=C:\WINDOWS
It turned out the reason of this weird behavior was incorrectly defined PERL5SHELL variable: cmd.exe (the shell interpreter in Windows) should be called with some parameters for proper processing - there parameters went missing after some updates. )
By the way, in The Doc it's said that Perl usually assumes the 'cmd.exe /x /c' line as a shell executable anyway if PERL5SHELL environment variable is not defined at all.
P.S. I really like this thread: it clearly shows the purpose of comments. )
I have some long running scripts with breaks requiring input/interaction to continue but when I switch to another window I'd like to be notified (by sound) that a task is complete and now awaiting input.
I would prefer to be able to play an audio clip (*.mp3, *.ogg, etc.) but wouldn't care if the only solution is to make the PC Speaker beep noise.
Any ideas? I'm open to any CLI utilities I can install that play sounds that in turn I can execute when needed.
FYI: My System is running WinXP Pro.
UPDATE: Doh! My Windows > Control Panel > Sounds > Default Beep: was set to (none). Grrr...
Problem solved.
This will make a beep from within bash
echo -en "\007"
Try this:
echo ^G
(^G is obtained by ctrl+G).
Note: you can't copy and paste this code in a batch file, it won't work. To obtain a ^G character in a file, type in a cmd window:
echo ^G > beep.txt
(again, ^G is obtained by ctrl+G).
Then you'll have a file named beep.txt, open it with notepad, there will be a square character. This is our ^G once it is saved in a file.
You can then copy and paste it in a batch file to make a sound (don't forget to put "echo" in front of it).
spd-say
sleep 2; spd-say 'get back to work'
Infinite loop with -w if you need extra motivation:
sleep 2; while true; do spd-say -w 'get back to work'; done
or if you prefer the carrot:
sleep 2; while true; do spd-say -t female1 -w "I'm done, come back to me, darling"; done
Pre-installed on Ubuntu 14.04 via the package speech-dispatcher: http://releases.ubuntu.com/trusty/ubuntu-14.04.4-desktop-amd64.manifest for blind people I suppose?
See also: https://askubuntu.com/questions/277215/how-to-make-a-sound-once-a-process-is-complete
Also add a popup
This combo is a life saver, b stands for beep:
b() ( spd-say 'done'; zenity --info --text "$(date);$(pwd)" & )
and then:
super-slow-command;b
If I'm somewhere in the room, I'll hear it and know that the long job is done.
Otherwise, I'll see the popup when I get back to my computer.
Related:
How to show a GUI message box from a bash script in linux?
https://superuser.com/questions/345447/how-can-i-trigger-a-notification-when-a-job-process-ends
https://askubuntu.com/questions/409611/desktop-notification-when-long-running-commands-complete
Listen to your cooler
I'm joking of course, but for compilation I noticed that I use often use this queue subconsciously. When the cooler stops humming for a while, it means that the compilation is over!
I know your question was for Window but just putting this here for any Mac OSX users who come across this article. OSX 10+ comes with the say command:
say "I'm done"
For example:
sleep 5 && say "I'm done waiting 5 seconds"
By setting this variable as follows
PROMPT_COMMAND="echo -en '\a'"
then bash will beep every time it shows the prompt. When you do not need it anymore,
unset PROMPT_COMMAND
To play the system sound from Windows command line you can run:
rundll32 user32.dll,MessageBeep
It should work on all version of Windows.
copy con beep.bat [Enter]
#echo off [Enter]
echo [Ctrl+G] [Enter]
[Ctrl+Z] [Enter]
beep.bat [Enter]
Simple answer without ^G
echo -en "\007"
In my bash profile I've added a BEEP to the script using #GregReynolds solution above then added this to PS1:
GREEN="\[\033[0;32m\]"
BEEP=$(echo -en "\007")
export PS1="$GREEN : ${BEEP}"
source ~/.bash_profile - you should hear the beep after the command prompt returns
I have git-autocomplete on usually so I've provided a much simplified version above
I have a PowerShell script that connects to a web site, and parses its returned data (It's about importing a previously uploaded SQL file into the web site's data base). The PowerShell script uses wget, something I may replace by a native function later.
The import process is embedded in a script that is executed by a 3rd party program called scriptFTP.
The script runs fine when I call it from a single .bat file like so:
powershell "& "C:\data\etc\run_import_script.ps1"
exit %ERRORLEVEL%
However, when I call this .bat file from within the greater ScriptFTP context, the following happens:
The PowerShell script is executed. I confirmed this my sending myself an E-Mail each time the remote import script got called.
PowerShell doesn't seem to exit, and script execution gets stuck. I can still cancel the whole thing using Ctrl+C but the following commands never get executed.
When I change the batch file to the following:
start powershell "& "C:\data\etc\run_import_script.ps1"
exit %ERRORLEVEL%
it works, running the PowerShell script in a new console, but I can't grab the error level that PowerShell returns.
I have tried calling PowerShell from ScriptFTP directly, bypassing the batch file, but with the same result: It just gets stuck.
Any output I have the PowerShell script do using Write-Output or Write-Host is not displayed.
All programs run under the same user, me.
Does anybody have any ideas what to do?
This variant might work for you.
powershell -NoProfile -Command "C:\data\etc\run_import_script.ps1; exit $LASTEXITCODE" < NUL
Taken from http://thepowershellguy.com/blogs/posh/archive/2008/05/20/hey-powershell-guy-how-can-i-run-a-powershell-script-from-cmd-exe-and-return-an-errorlevel.aspx
From my experience, PowerShell.exe can easily run scripts from within a batch file or shell script in a predictable way using the -File switch. One does not need to use the Start command.
The important thing to do is to append
< nul
to the command line from within a batch file. My research has shown that PowerShell runs the commands in the script indicated through the -File switch and then waits for additional PowerShell commands from the standard input (my brief experimentation with the -Command switch demonstrated similar behavior). By redirecting the standard input to nul, once PowerShell finishes executing the script and "reads end-of-file" from the standard input, PowerShell exits.
When invoking PowerShell from Cygwin, use
< /dev/null
For example, I've run PowerShell scripts from Cygwin using shell variables, like this:
PowerShell.exe -ExecutionPolicy RemoteSigned -File $_powershellscriptpath $_firstscriptparameter < /dev/null
Please post a comment if your experience varied from mine.
- Gordon
Try adding the /WAIT parameter. It will keep the .bat waiting until the PowerShell script completes.
START /WAIT powershell "& "C:\data\etc\run_import_script.ps1"
We had a similar issue. We wanted to call a powershell app from a piece of software that just had a box to enter "Command" and "Parameters" but although the powershell ran successfully (I could see the affected file updated.)
Finally my coworker helped me figure it out
Command needs to be:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
And Parameters:
-ExecutionPolicy unrestricted -Command "& {C:\scripts\apps\EDI\Test.ps1; [Environment]::Exit(1)}"
In our case it was important to use [Environment]::Exit(1) rather than Exit 1. I believe Exit was simply terminating the script, not closing Powershell itself.
If you want to capture the output of the powershell.exe commands then you can also use the /B parameter to force the process to run in the same command shell.
We've just seen a very odd instance of this problem. A batch file containing the call powershell.exe -command ... ran fine locally but stalled as described above when the batch file was run in the context of an msdeploy -postSync command. After experimenting with process.Kill() to force PowerShell to quit, we lit on the use of START /WAIT /B PowerShell.exe ..
No idea why this should work, but it does ...
IF that is exactly what's in your file it's because you have mismatched quotes. Powershell is waiting for the last quote.
PowerShell has, at least what I consider, strange behavior when being invoked in this manner. In short, it doesn't treat the command line arguments being passed to powershell.exe as scripts to run. Instead, it treats them as command to run. This is because the default for powershell.exe is -command - see powershell.exe /? for additional info.
C:\>powershell "'Hello'"
Hello
What you will need to do, is construct a clever input string to "source" the script you wish to run:
C:\>powershell ". .\test.ps1"
Hello, World!
As for the output, once you get the script running correctly, it should just be a matter of capturing STDOUT or whatever ends up fitting with your script.
Full Example
test.bat
#echo off
powershell.exe ". .\test.ps1"
test.ps1
"Hello, World!"
Invoke the command:
test.bat > test.txt
And verify the output was captured:
C:\>type test.txt
Hello, World!
The problem I bet is that the Powershell process continutes to run after executing the script. This means it has not exited and therefore there is no exit code. I'm having a similar problem when trying to run a Powershell script from C# that does stuff when finished, except it never finishes...
Hacky, but the only solution I've found is to put Stop-Process -processname powershell at the end of the .ps1 script. This kills off the PowerShell process (and all PowerShell windows you have open unfortunately) but seems to work. Might work for your script as well.
I had the exact issue, we were trying to integrate power shell scripts into another system and it kept giving a timeout error. Probably because of the issue mentioned by Gordon Smith, his solution might work. But to me I prefer to have complete control of my script from within the script itself and not depend on the way in which it is called.
$pid is built in variable with the PID. The below will end the current powershell process and leave the others intact. This way anyone can call your script as normal and it will work as expected.
Stop-Process $pid
you can simply add an 'exit' command to the end of the .ps1 script (or any variant of process termination that you wish). Powershell will continue to run at the end of the script as it has not been told to terminate (when run in PS or ISE it will auto-terminate at the end of the script but not when run through the DOS shell).