Why can t the noclobber variable protect you from overwriting an existing file with cp or mv?
Because noclobber is functionality internal to the shell, whereas cp and mv are programs that are external to the shell, and wouldn't (and shouldn't) know about what goes on in the shell.
To make an analogy, your question is a bit like asking why the default font settings in Excel don't affect Word.
To be specific, what noclobber actually does do is it instructs the shell to not overwrite files as a result of output redirection, as when you run a command such as ls >files. The redirection of the output to files is a function carried out by the shell itself, and therefore it makes sense for it to be configurable in the shell.
On the other hand, cp and mv have their own functionality to do the same thing: You can call them with the -n switch to make them fail instead of clobber files. (Or with the -i switch to ask interactively before clobbering.)
Also, even tee won't honor the noclobber option, if set. For e.g.
[ankur#server1 ~]$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
[ankur#server1 ~]$ cat /etc/centos-release
CentOS Linux release 7.8.2003 (Core)
[ankur#server1 ~]$ set -o | grep noclobber
noclobber on
[ankur#server1 ~]$ echo hi | tee myfile
hi
[ankur#server1 ~]$ ls -i myfile
19731252 myfile
[ankur#server1 ~]$ cat myfile
hi
[ankur#server1 ~]$ echo hiii | tee myfile
hiii
[ankur#server1 ~]$ ls -i myfile
19731252 myfile
[ankur#server1 ~]$ cat myfile
hiii
Related
In bash, in Terminal on my Mac, (but not in Linux), grep -v -f behaves differently depending on whether it's executed at the command line or in a script. From the command line:
$ touch empty-file #create an empty file
$ printf 'foo' | grep -v -f empty-file
foo
That's as expected. But when that line is in a script, it outputs nothing. Here's the script:
$ cat grep-v-in-script.sh
#!/usr/bin/env bash
printf 'foo\n' | grep -v -f empty-file
printf 'end of script\n'
When I execute that script:
$ ./grep-v-in-script.sh
end of script
If I run that same script in Linux it works as expected:
herdrick#some-linux-server:~$ ./grep-v-in-script.sh
foo
end of script
FWIW on my Mac if I change the 'grep -v -f' to 'grep -f', then it again outputs nothing, but this time that is expected.
Here's my bash version:
$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.
The issue is simply an incompatibility between GNU grep and BSD grep. See the comment on the post by #that-other-guy.
My confusion was due to my having an alias set to use GNU grep. There is, otherwise, no difference between doing this at the command line and in a script.
The man page for echo says:
-n Do not print the trailing newline character. This may also be
achieved by appending `\c' to the end of the string, as is done by
iBCS2 compatible systems. Note that this option as well as the
effect of `\c' are implementation-defined in IEEE Std 1003.1-2001
(``POSIX.1'') as amended by Cor. 1-2002. Applications aiming for
maximum portability are strongly encouraged to use printf(1) to
suppress the newline character.
However this doesn't seem to work in sh on Mac:
sh-3.2$ echo $0
/bin/sh
sh-3.2$ which echo
/bin/echo
sh-3.2$ echo -n foo
-n foo
It works properly in bash:
bash-3.2$ echo $0
bash
bash-3.2$ which echo
/bin/echo
bash-3.2$ echo -n foo
foobash-3.2
FWIW this only seems to happen on Mac, on Linux it works properly:
$ echo $0
sh
$ echo -n foo
foo$
-n is a bash extension to echo. In version 3.2 (which ships with macOS), bash does not support the extension when invoked as sh. Starting with version 4.0 (some version of which is likely on your Linux box), bash does honor -n when invoked as sh.
Update: the xpg_echo option determines if bash's built-in echo is POSIX-compliant or not. In bash 3.2 (or at least the macOS build of 3.2), this option defaults to on; in bash 4.x, it defaults to off.
% sh -c 'shopt xpg_echo'
xpg_echo on
% ./sh -c 'shopt xpg_echo'
xpg_echo off
(./sh is a symlink to /usr/local/bin/bash, a local installation of bash 4.4 on my machine.)
I'm trying to write a script for editing an executable (e.g. /bin/bash) in this way:
Search for a string -
Replace old string with new string -
Save changes to original file (obviously I'm trying on a copy of /bin/bash, for security reason)
My string is a single word. How can I do it?
You can easily modify a string in a binary file with sed but as
noted in the comments it will most often only work when old and new string have the same length. With bash for example:
$ cp /bin/bash .
$ strings bash | grep such
describe_pid: %ld: no such pid
it creates, on systems that allow such control.
such as cd which change the current directory.
%s: no such job
$ sed 's,no such job,no such boj,' -i bash
$ ./bash --noprofile --norc
bash-4.3$ kill %3333
bash: kill: %3333: no such boj
-i for sed is not in
POSIX
so if you want to achieve maximum portability:
$ cp /bin/bash .
$ sed 's,no such job,no such boj,' bash > bash.bak
$ mv bash.bak bash
$ chmod +x ./bash
$ ./bash --noprofile --norc
bash-4.3$ kill %3333
bash: kill: %3333: no such boj
So I've made a script which is collecting data from many different files:
#!/bin/bash
mkdir DATAPOOL"$1"
grep achi *out>runner
grep treat *out>>runner
cat runner | grep Primitive *gout | grep '= '|awk '{print $1,$6}' > CellVolume"$1".txt
cat runner | grep ' c ' *gout | grep 'Angstrom '|awk '{print $1,$3}' > Cellc"$1".txt
cat runner | grep 'Final energy ' *gout |awk '{print $1,$5}' > CellEnergy"$1".txt
etc etc
cat runner |awk '{print "~/xtlanal",$1," > ",$1}' >runner2
vi runner2
:1,$s/gout:/xtl/
:1,$s/gout:/dat/
:wq
source runner2
grep Summary *dat | grep 'CAT-O ' |awk '{print $1,$6}' > AVE_NaO_"$1".txt
mv *txt DATAPOOL"$1"
So I end up with all the required text files when run without the vi part and so I know it all works. Furthermore when I run it with the vi commands, it just stops running at the vi command and then i can manually enter the 3 commands and I end up with the correct results. What I'm struggling with is I cant get vi to run the commands on its own so I can just execute the file multiple times within different directories and not have to manually enter commands time and time again.
Any help would be greatly appreciated.
Cheers
something like this as a bash script:
#!/bin/bash
vi filename.txt -c ':g/^/m0' -c ':wq'
where -c execute a command. Here the command is to reverse the lines in a textfile. After done, :wq to save and exit. (man vi to get more about -c)
If you don't want to type -c twice, you can do it this way:
vi -c "g/^/m0 | wq" filename.txt
For scripted editing tasks, you can use ed instead of vi:
ed runner2 <<'END'
1,$s/gout:/xtl/
1,$s/gout:/dat/
w
q
END
For global line-oriented search and replace, sed is a good choice:
sed -i 's/gout:/xtl/; s/gout:/dat/' runner2
Tested on VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Apr 10 2018 21:31:58)
The vi -c "g/^/m0 | wq" filename.txt may appear to work, but it does not actually!
Typing vi -c "g/^/m0 | wq" filename.txt will result in vi writing and quitting before any major changes are made to the file. (using the pipe in this situation will attempt to execute the wq line by line forcing it to quit before the intended operation)
In order to see a demonstration try typing it without the q and see how slow it works writing line by line:
vi -c "g/^/m0 | w" filename.txt
The more efficient way is using -c as B. Kocis states, or use +.
As B. Kocis stated:
#!/bin/bash
vi filename.txt -c ':g/^/m0' -c ':wq'
or
vi filename.txt +g/^/m0 +wq
starting bash with -v option produces a long output to the console
$ bash -v
source ~/Dropbox/bin/tim_functions.sh
\#!/bin/bash
...several hundred more lines
I would like to capture the output to a file to make it easier to browse through, but I have tried bash -v 2>&1 > out_bash.txt and bash -v | tee out_bash.txt and cannot capture the information on the terminal screen within a file. It is as if the verbose output is neither stderr or stdout. How can this be?
Can anyone suggest a way to capture the output of bash -v ?
bash -v 2>&1 > out_bash.txt
is not what you want, it should be
bash -v >out_bash.txt 2>&1
I poked around and found this http://www.commandlinefu.com/commands/view/3310/run-a-bash-script-in-debug-mode-show-output-and-save-it-on-a-file
On the website they use
bash -x test.sh 2>&1 | tee out.test, but I tested it with
bash -v test.sh 2>&1 | tee out.test and it worked fine.
you can also use the exec command in the script to redirect all output:
#!/bin/bash
exec >> out.txt 2>> out.txt
set -x
set -v
echo "testing debug of shell scripts"
ls
After reading other helpful answers, I believe this issue has to do with how bash is sending the verbose information to tty--which is somehow different than stderr or stdout. It can be caught with the following work around:
$ screen -L
$ bash -v
$ exit #from the bash session
$ exit #from the screen session
This results in a screenlog.0 file being generated containing the output.
The bash -v output of interest was on a mac running 10.7.3 (Lion) with
$ bash --version
GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin11)
Copyright (C) 2007 Free Software Foundation, Inc.)
Another 10.6.8 mac I tried had a less (interesting/verbose) output, despite a similar .bashrc file.
You can use,
bash -v 2>&1 | tee file.txt
or
bash -v 2>&1 | grep search_string
Have you tried wrapping your child bash in a subshell?
( bash -v ) 2>&1 > out_bash.txt