Running vi within a bash script and executing vi commands to edit another file - bash

So I've made a script which is collecting data from many different files:
#!/bin/bash
mkdir DATAPOOL"$1"
grep achi *out>runner
grep treat *out>>runner
cat runner | grep Primitive *gout | grep '= '|awk '{print $1,$6}' > CellVolume"$1".txt
cat runner | grep ' c ' *gout | grep 'Angstrom '|awk '{print $1,$3}' > Cellc"$1".txt
cat runner | grep 'Final energy ' *gout |awk '{print $1,$5}' > CellEnergy"$1".txt
etc etc
cat runner |awk '{print "~/xtlanal",$1," > ",$1}' >runner2
vi runner2
:1,$s/gout:/xtl/
:1,$s/gout:/dat/
:wq
source runner2
grep Summary *dat | grep 'CAT-O ' |awk '{print $1,$6}' > AVE_NaO_"$1".txt
mv *txt DATAPOOL"$1"
So I end up with all the required text files when run without the vi part and so I know it all works. Furthermore when I run it with the vi commands, it just stops running at the vi command and then i can manually enter the 3 commands and I end up with the correct results. What I'm struggling with is I cant get vi to run the commands on its own so I can just execute the file multiple times within different directories and not have to manually enter commands time and time again.
Any help would be greatly appreciated.
Cheers

something like this as a bash script:
#!/bin/bash
vi filename.txt -c ':g/^/m0' -c ':wq'
where -c execute a command. Here the command is to reverse the lines in a textfile. After done, :wq to save and exit. (man vi to get more about -c)
If you don't want to type -c twice, you can do it this way:
vi -c "g/^/m0 | wq" filename.txt

For scripted editing tasks, you can use ed instead of vi:
ed runner2 <<'END'
1,$s/gout:/xtl/
1,$s/gout:/dat/
w
q
END
For global line-oriented search and replace, sed is a good choice:
sed -i 's/gout:/xtl/; s/gout:/dat/' runner2

Tested on VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Apr 10 2018 21:31:58)
The vi -c "g/^/m0 | wq" filename.txt may appear to work, but it does not actually!
Typing vi -c "g/^/m0 | wq" filename.txt will result in vi writing and quitting before any major changes are made to the file. (using the pipe in this situation will attempt to execute the wq line by line forcing it to quit before the intended operation)
In order to see a demonstration try typing it without the q and see how slow it works writing line by line:
vi -c "g/^/m0 | w" filename.txt
The more efficient way is using -c as B. Kocis states, or use +.
As B. Kocis stated:
#!/bin/bash
vi filename.txt -c ':g/^/m0' -c ':wq'
or
vi filename.txt +g/^/m0 +wq

Related

Run commands in text file in bash

How can I run a command from a line inside a text file, one line of my text file looks like this
echo "RouterPing;`ping -c4 -w4 -q DeviceIP| tail -2 |awk '{print}' ORS=' '`;$(date)" >> somefile.txt &
I have a file that has thousands of lines that being generated by external program and want to execute every line in it. I need each line to be run exactly as if I am running it from bash shell
You can just run:
bash file.txt
you can use below , but i would highly not recommend executing 1000 commands from a file ,
#!/usr/bin/bash
filename="$1"
while read -r line
do
$line
done < "$filename"
How to use
./this_file_name.sh file_with_commands
you$ bash somefile.txt
Just make sure your file is executable (chmod 744 somefile.txt)
with dot ('.') (I removed the & because if the command runs in background it wont work)
echo "RouterPing;`ping -c4 -w4 -q DeviceIP| tail -2 |awk '{print}' ORS=' '`;$(date)" >> somefile.txt
. somefile.txt

xargs: exec command with prompt

I'm trying to do the following with xargs
pacman -Q | grep xf86-video | awk '{print $1}' | xargs pacman -R to remove all xf86-video-* driver on my machine. To make the question more clear, here is the output of pacman -Q | grep xf86-video | awk '{print $1}':
xf86-video-ark
xf86-video-ati
xf86-video-dummy
xf86-video-fbdev
xf86-video-glint
xf86-video-i128
xf86-video-intel
xf86-video-mach64
xf86-video-neomagic
xf86-video-nouveau
....
when I redirect the result to xargs, the output looks like this:
The point is, the command which xargs is about to execute need user to do some additional input(as you can see it needs a Yes/No), but xargs automatically add a unknown symbol #, and exit, which causes my purpose UNACHIEVED.
WHY xargs would do this or, what can I do to use xargs for command with prompt?
You can use
xargs -a <(pacman -Q | awk '/xf86-video/{print $1}') pacman -R
Explanation:
Without further arguments xargs does not work with interactive (command line) applications.
The reason for that is, that by defaultxargs gets its input from stdin but interactive applications also expect input from stdin. To prevent the applications from grabbing input that is intended for xargs, xargs redirects stdin from /dev/null for the applications it runs. This leads to the application just receiving an EOF. (Running just pacman -R SOMEPACKAGE and pressing Ctrl+D has the same effect).
To get xargs to work with interactive commands you have to use the --arg-file=FILE argument (short -a FILE). This tells xargs to get the arguments from FILE. This also leaves stdin unchanged.
So you could either put your package list into a temporary file
pacman -Q | awk '/xf86-video/{print $1}' > /tmp/packagelist
xargs -a /tmp/packagelist pacman -R
rm /tmp/packagelist
or you can use zsh's process substitution mechanism <(list). When executing a line with <(list), <(list) is replaced by a filename from where the output of list can be read.
xargs -a <(pacman -Q | awk '/xf86-video/{print $1}') pacman -R
The single # you get is not from xargs but from zsh itself. If the shell options PROMPT_CR and PROMPT_SP are set (which both are by default) zsh tries to preserve partial lines, that is lines that did not end with a newline. To signify that such a line has been preserved zsh prints an inverse+bold character at the end of that line, by default % for normal users and # for root.
You can eliminate the use of xargs with a single hyphen
Additionally, if stdin is not from a terminal and a single hyphen (-) is passed as an argument, targets will be read from stdin.
— https://archlinux.org/pacman/pacman.8.html
pacman -Q | awk '/xf86-video/{print $1}' | pacman -R -
You can use xargs, though ...
If you wish to use xargs, use it's -o, --open-tty option:
--open-tty
-o
Reopen stdin as /dev/tty in the child process before executing the command, thus allowing that command to be associated to the terminal while xargs reads from a different stream, e.g. from a pipe. This is useful if you want xargs to run an interactive application.
grep -lz PATTERN * | xargs -0o vi ①
— https://www.gnu.org/software/findutils/manual/html_node/find_html/xargs-options.html
① That should be an uppercase Z grep option (bug filed).
For your particular case:
pacman -Q | awk '/xf86-video/{print $1}' | xargs -o pacman -R
You need to run pacman the second time with the --noconfirm option:
pacman -Q | grep xf86-video | awk '{print $1}' | xargs pacman -R --noconfirm
This will disable 'are you sure' messages, and do things without requiring input.

How to refer to redirection file from within a bash script?

I'd like to write a bash script myscript such that issuing this command:
myscript > filename.txt
would return the name of the filename that it's output is being redirected to, filename.txt. Is this possible?
If you are running on Linux, check where /proc/self/fd/1 links to.
For example, the script can do the following:
#!/bin/bash
readlink /proc/self/fd/1
And then run it:
$ ./myscript > filename.txt
$ cat filename.txt
/tmp/filename.txt
Note that if you want to save the value of the output file to a variable or something, you can't use /proc/self since it will be different in the subshell, but you can still use $$:
outputfile=$(readlink /proc/$$/fd/1)
Using lsof:
outfile=$(lsof -p $$ | awk '/1w/{print $NF}')
echo $outfile

How execute bash script line by line?

If I enter bash -x option, it will show all the line. But the script will execute normaly.
How can I execute line by line? Than I can see if it do the correct thing, or I abort and fix the bug. The same effect is put a read in every line.
You don't need to put a read in everyline, just add a trap like the following into your bash script, it has the effect you want, eg.
#!/usr/bin/env bash
set -x
trap read debug
< YOUR CODE HERE >
Works, just tested it with bash v4.2.8 and v3.2.25.
IMPROVED VERSION
If your script is reading content from files, the above listed will not work. A workaround could look like the following example.
#!/usr/bin/env bash
echo "Press CTRL+C to proceed."
trap "pkill -f 'sleep 1h'" INT
trap "set +x ; sleep 1h ; set -x" DEBUG
< YOUR CODE HERE >
To stop the script you would have to kill it from another shell in this case.
ALTERNATIVE1
If you simply want to wait a few seconds before proceeding to the next command in your script the following example could work for you.
#!/usr/bin/env bash
trap "set +x; sleep 5; set -x" DEBUG
< YOUR CODE HERE >
I'm adding set +x and set -x within the trap command to make the output more readable.
The BASH Debugger Project is "a source-code debugger for bash that follows the gdb command syntax."
If your bash script is really a bunch of one off commands that you want to run one by one, you could do something like this, which runs each command one by one when you increment a variable LN, corresponding to the line number you want to run. This allows you to just run the last command again super easy, and then you just increment the variable to go to the next command.
Assuming your commands are in a file "it.sh", run the following, one by one.
$ cat it.sh
echo "hi there"
date
ls -la /etc/passwd
$ $(LN=1 && cat it.sh | head -n$LN | tail -n1)
"hi there"
$ $(LN=2 && cat it.sh | head -n$LN | tail -n1)
Wed Feb 28 10:58:52 AST 2018
$ $(LN=3 && cat it.sh | head -n$LN | tail -n1)
-rw-r--r-- 1 root wheel 6774 Oct 2 21:29 /etc/passwd
Have a look at bash-stepping-xtrace.
It allows stepping xtrace.
xargs: can filter lines
cat .bashrc | xargs -0 -l -d \\n bash
-0 Treat as raw input (no escaping)
-l Separate each line (Not by default for performances)
-d \\n The line separator

bash output redirect prob

I want to count the number of lines output from a command in a bash script. i.e.
COUNT=ls | wc -l
But I also want the script to output the original output from ls. How to get this done? (My actual command is not ls and it has side effects. So I can't run it twice.)
The tee(1) utility may be helpful:
$ ls | tee /dev/tty | wc -l
CHANGES
qpi.doc
qpi.lib
qpi.s
4
info coreutils "tee invocation" includes this following example, which might be more instructive of tee(1)'s power:
wget -O - http://example.com/dvd.iso \
| tee >(sha1sum > dvd.sha1) \
>(md5sum > dvd.md5) \
> dvd.iso
That downloads the file once, sends output through two child processes (as started via bash(1) process substitution) and also tee(1)'s stdout, which is redirected to a file.
ls | tee tmpfile | first command
cat tmpfile | second command
Tee is a good way to do that, but you can make something simpler:
ls > __tmpfile
cat __tmpfile | wc -l
cat __tmpfile
rm __tmpfile

Resources