I built a toolchain script to prepare a Linux build environment. The script can be found here: https://github.com/mynameismevin/prometheus/blob/toolchain/ptool-make.sh
The script runs perfectly until after the Perl section around line 416. After Perl is done, when it goes to unzip sed, it complains with this error:
tar (child): sed-4.2.2.tar.bz2: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
However, if I run the sections by hand, it completes without errors. If I split the script into ptool-make1.sh (which ends at Perl) and ptool-make2.sh (which starts at sed) and run them sequentially then they both complete without any issues at all. At this point, I assert the issue isn't with the script, and I would like to debug the shell to see if it's an issue with the shell.
Here are some useful configurations:
user#ubuntu:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
user#ubuntu:~$ bash --version
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
user#ubuntu:~$ ls -lh $(which bash)
-rwxr-xr-x 1 root root 998K Oct 7 2014 /bin/bash
I don't think Ubuntu bash comes with debugging symbols, so I would assume I would have to recompile to include those?
How would I use gdb to debug the shell when I run a script? Or how would I have the shell log to a file while the script runs so I can open it with gdb after it's done? I know how to debug a shell script, I don't want to debug the script, I want to debug the shell.
Edit: It doesn't look like Ubuntu bash comes with debugging symbols out of the box. Reading symbols from bash...(no debugging symbols found)...done.
Edit: $PROMETHEUS is set in my root shell. At the end of Perl cd .. results in the same results a cd $PROMETHEUS/sources/.
Shells come with their own debugging tools. The most simple is running them with -x (bash -x ...script...) which will print each command after variable expansion but before it is executed. That's usually enough to determine the problem.
For more ideas, see How to debug a bash script?
You should also consider to write helper functions to reduce the size of the script to just a few lines. You could move special options to configure or post-build steps into extra files and run them from the helper function if they exist.
Looking at the code, it seems that this line is the culrit:
cd $PROMETHEUS/sources/
everywhere else, you just use cd ... If PROMETHEUS isn't defined (the script doesn't define it, that becomes cd /sources/ which should also fail but doesn't abort your script. Try
cd $PROMETHEUS/sources/ || exit 1
instead. Also #!/bin/bash -u might be useful (abort on undefined variables).
Lastly, you can use pushd and popd to navigate the folders.
Related
I found a difference of behaviour between GNU Make 4.1 and 3.81 and wonder whether my code is not POSIX compliant which 4 is enforcing more strictly, or whether something else is going on.
I distilled the failure case to this Makefile
.POSIX:
all: test-b
test-a:
cat a.txt b.txt c.txt >results.txt
test-b:
cat {a,b,c}.txt >results.txt
Assuming those files have been created with cat {a,b,c}.txt, target test-a always works, yet test-b works on Make 3.81 but fails on 4.1.
The output for 3.81:
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0
$ make
cat {a,b,c}.txt >results.txt
$ echo $?
0
The output for 4.1:
$ make --version
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ make
cat {a,b,c}.txt >results.txt
cat: {a,b,c}.txt: No such file or directory
Makefile:9: recipe for target 'test-b' failed
make: *** [test-b] Error 1
$ echo $?
2
It's possible the cat command is actually failing in 3.81 and it just isn't pointing it out, as later versions of GNU Make mention passing the -e flag to the shell when invoking target commands to make it more POSIX compliant, but I can't see how that command could be failing.
I assume the wildcards are handled solely by the shell, so I can't see how invoking the shell via a make target command should be any different.
Which is these behaviours are correct? If wildcards like that don't work in Makefiles, which other wildcards can I assume to work?
test-b still fails in 4.1 even if .POSIX: is removed from the file.
Recipes are sent to the shell. They are not interpreted by make. So your question is really, are curly-brace expansions supported by the shell?
That depends on which shell make uses. They are not supported by POSIX standard sh. They are supported by bash (and many other shells).
Make always invokes /bin/sh, regardless of what shell you personally use, unless you specifically set the make SHELL variable to something else. On some systems, /bin/sh is a symlink to /bin/bash so they are the same thing (bash runs in a "POSIX emulation" mode when invoked as /bin/sh but most bash features are still available). Other systems use different shells, such as dash, as /bin/sh which do not have extra bash features.
So, you can either (a) not have a portable makefile and assume /bin/sh is the same as /bin/bash, (b) set SHELL := /bin/bash in your makefile to force it to use bash always (but fail on systems that don't have bash installed), or (c) write your makefile recipes to use only POSIX sh features so it works regardless of which shell is used for /bin/sh.
I have an odd problem with tab completion under cygwin's bash on my Win 7 installation. I get fork failures ('permission denied') under very specific circumstances..
Tab completion works fine for all commands.
Tab completion works fine for arguments of builtins and any cygwin-supplied program in /usr/bin, /usr/sbin, etc. (including .exe files as well as shell scripts)
Tab completion works fine for commands in my personal ~/bin directory.
However, whenever I attempt tab completion for an argument to one of my own shell scripts, I get a fork/permission denied error.
I've compared file permissions and ownerships of my shell scripts to those of cygwin-supplied shell scripts, and they are equivalent.
I've created a directory outside of the /cygwin/C hierarchy (named it /mybin) and copied my scripts into that directory. Then invoked the scripts directly (e.g., /mybin/gv n[TAB]) and the failure still occurs.
I've made sure the scripts have mode 775 and owned by me. I've made sure the directory in which the scripts exists is mode 775 and is also owned by me.
I haven't created anything in /usr/share/bash-completion (afaik, I shouldn't have to for argument-based filename completion). But I've not had issues with cygwin scripts without entries in /usr/share/bash-completion (e.g., /bin/zegrep is a shell script but has nothing under /usr/share/bash-completion)
I'm stumped at this point, but would like to resolve it since tab completion is such a vital feature.
Here's an example...
% gv n[TAB] 0 [main] bash 9308 fork: child -1 -CreateProcessW
failed for 'C:\Users\binaryb\cygwin\bin\bash.exe', errno 13 bash:
fork: Permission denied
.. in this example I was attempting to complete a filename (in current directory) by typing 'n[TAB]'. There is only one file starting with 'n' in my current directory.
I am running this under Win 7 (64 bit), with the following cygwin bash.exe version...
% uname -a
CYGWIN_NT-6.1 binaryb 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
% bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
I think I found the problem, but I'm not 100% sure. It seems to occur for scripts whose filenames are only 1 or 2 characters long. If I rename my script to, say, gvx, the problem goes away.
% gv n[TAB] 0 [main] bash 4556 fork: child -1 - CreateProcessW
failed for 'C:\Users\binaryb\cygwin\bin\bash.exe',
errno 13 bash: fork: Permission denied
% mv ~/bin/gv ~/bin/gvx
% gvx n[TAB]
% gvx newfile.txt
I've installed Git Bash 2.5.1 64-bit on a fresh Windows 10 machine:
mark#Foo MINGW64 ~/Documents/FsCheck (master)
$ git --version
git version 2.5.1.windows.1
When I list the contents of a directory, I can see a shell script:
mark#Foo MINGW64 ~/Documents/FsCheck (master)
$ ls
appveyor.yml build.cmd build.sh* Docs/
bin/ build.fsx Contributors.txt examples/ FsCheck.sln
(Some files and folders removed for clarity.)
Notice that the shell script build.sh has an asterisk after the extension. In the console, it's also coloured green, where other files are light grey. I haven't seen this before, so I don't know if it means anything.
When I try to run the script, I get an error message:
mark#Foo MINGW64 ~/Documents/FsCheck (master)
$ build.sh
bash: build.sh: command not found
Also, there's no tab completion for any of the build files.
How can I run the shell script from Git Bash?
You probably have ls aliased to ls -F. So when an trailing asterisk appears that means that the file is executable.
In POSIX systems, you can't directly execute files in current directory (for safety reasons). If you want to, you can use this trick:
./build.sh
You can, of course, edit your PATH by adding ";." at the END of the current value. In mingw64_shell.bat do
SET PATH=%PATH%;.
Note: I'm looking at my MSYS2 installation, so you may need to look for the ".bat" file that launches bash. The biggest security problem with the WINDOWS default path that had "." in front of the path is that it allows a user to over-ride a system executable -- that would make your scripts do some weird stuff, especially if you import them from someone else.
I am unfortunately having to use windows in work, and so I have installed win-bash to have a unix shell running. all going well but I am having an issue running the following .sh file:
bash $ ./qf.sh
.\qf.sh: option not available on this NT BASH release
.\qf.sh: fork: Bad file descriptor
qf.sh is:
#!/bin/bash
cat test.csv | while read line
do
echo "${line//,/ }" | xargs ./adder
done
I find it hard to believe someone would create a bash emulator incapable of running a bash file. curious that the error message writes .\qf as opposed to ./qf
Can anyone shed some light on this?
use MinGW or Cygwin
MinGW: http://sourceforge.net/projects/mingw/files/?source=navbar
Cygwin: http://www.cygwin.com/
Using Cygwin absolutely killed this error for me. I do wonder however, how this should work with MinGW. I don't see any unix command executable directory to include in the path (like cygwin64\bin with Cygwin)
I have been trying to use GNU parallel for some time, but I have never been able to get it to function at all!
For example, running (in a non-empty directory!):
ls | parallel echo # Outputs single new line
ls | parallel echo echo echo # Outputs three new lines.
ls | parallel echo {} # /bin/bash: {}: command not found
ls | parallel echo '{}' # /bin/bash: {}: command not found
ls | parallel 'echo {}' # Outputs: {}
ls | parallel -IMM 'echo MM' # Outputs: MM
It seems that it is simply executing each argument as a command, which makes no sense.
I have tried bash, zsh, tcsh, csh, and sh, to no avail.
As I was about to complete writing this question, I ran parallel --version to report the version, only to find:
WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.
It is not clear to me why that flag is set by default. Needless to say, using --gnu worked!
Thought I would post this to save someone hours of frustration and confusion.
EDIT:
To fix this permanently (in Ubuntu at least), delete the --tollef flag in /etc/parallel/config
Depending on your operating system, you should check whether you're actually running the GNU version.
$ parallel --version
parallel: invalid option -- '-'
parallel [OPTIONS] command -- arguments
for each argument, run command with argument, in parallel
parallel [OPTIONS] -- commands
run specified commands in parallel
If this is the case, you're not running the GNU version. Ubuntu 12.04 is like this, and you'll need to manually install GNU parallel to get the functionality you expect.
Had issues running parallel as an external command from FREEMAT (MATLAB lookalike); the argumentFile was not fed to the command properly solved it by:
Adding --gnu to options
Not using cmdString syntax involving ["]
Code:
cmdString = 'parallel --gnu command ::: ';
while j<=jLength
cmdString = [cmdString argumentFilePath(j,:) ' '];
j=j+1;
end
system(cmdString)
Thank you for that :)
Im on Ubuntu 12.04 as well.
For me it was same issue but different problem. Just running parallel command was exiting silently. Also parallel --version was saying invalid option error. In my Path there was just one parallel executable binary but still it was not detecting.
I was able to fix it as below:
Run whereis parallel. This gives all the paths where executables named parallel is present. For my case there was just one path /usr/local/bin/parallel. Running using this path works just fine.
You can add an alias for this in ~/.bashrc or ~/.zshrc file like alias parallel='/usr/local/bin/parallel'
And now parallel works like charm.
dev-dsk % parallel --version
GNU parallel 20190322
Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.