I am making a shell script that will automate the install process of Arch Linux AUR packages. I need to list all dependencies of the package (to check if they are installed), they appear like this in install script:
depends=('sdl' 'libvorbis' 'openal')
The easiest way (or the only idea) that I could come up with is something like this:
grep "depends" PKGBUILD | awk -F"'" '{print $2 "\n" $4 "\n" $6;}'
But the dependency count varies from package to package. So, how I output the names in quotes if the word count is varying?
Thanks in advance,
-skazhy
If the depends is just one line, one thing you may try is to evaluate the line in bash itself... This will lead to an array called "depends" that holds all the values. Seems tricky, but not with dynamic languages:
depend_line=`grep depends $PKGBUILD`
eval "${depend_line}"
echo ${depend[0]} # Will print sdl in your example
You can avoid the security issues of using eval or sourcing a temporary file by using declare:
declare -a "$(grep "depends" PKGBUILD)"
This will create an array called "depends" containing "sdl", "libvorbis" and "openal" based on the example data.
Try this on for size:
grep "depends" PKGBUILD > /tmp/depends
. /tmp/depends
echo ${depends[#]}
Hey look, is that an array? Yes it is.
for d in "${depends[#]}" ; do
printf '"%s"' "$d"
done
Note: In a real script you'd want to be more careful with the naming of the temporary file.
You could do something like:
grep "depends" PKGBUILD | perl -e "while(<>){print \"\$1\\n\" while m/'.{-}'/g;}"
awk -F"'" '{for(i=2;i<=NF;i+=2) print($i)}'
Related
I made a bash script to insert the result of nmap command to an array. The script is working on bash 4.3.30, but it does not work when I try to run it on bash 4.4.12. It looks like the array is empty or it just have the first value.
Here is my code:
#!/bin/bash
declare -a IP_ARRAY
NMAP_OUTPUT=`nmap -sL $1 | grep "Nmap scan report" | awk '{print $NF}'`
read -a IP_ARRAY <<< $NMAP_OUTPUT
printf '%s\n' "${IP_ARRAY[#]}"
With bash 4.3, the values of the string NMAP_OUTPUT are well copied to the array IP_ARRAY. The the other version not and I don't find the error.
The string NMAP_OUTPUT looks like:
10.0.0.0 10.0.0.1 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9 10.0.0.10
Instead of using my code above, this code works:
IP_ARRAY=(${NMAP_OUTPUT})
I would like to understand with my previous code is working on one version and not in the other one.
Thank you very much!!!
Your script has multiple issues which could be fixed. It could be done very simply minimizing a number of steps.
You are using NMAP_OUTPUT as a variable. The bash shell does support arrays which you can use to store a list. Also independent entries present in a variable's context undergo Word-Splitting done by the shell. The consequence of that is, if a entry has spaces in-between, it will be tough to identify if it is a separate word or part of a whole word.
Storing the command output to a variable and later parsing to an array is round about way. You can directly pass the output to an array
Using grep and awk together is not needed, awk can do whatever grep can
Always quote the shell variable and array expansions. Never use unquoted expansion in your results (like in <<< $NMAP_OUTPUT). It could have adverse affects in case of words containing spaces.
Always use lower case variable names for user-defined functions/variables and array names.
Use mapfile built-in
Version of bash v4.0 on-wards provides options mapfile/readarray to directly read from a file or output of command.
All your script needs is
mapfile -t nmapOutput < <(nmap -sL "$1" | awk '/Nmap scan report/{print $NF}')
printf '%s\n' "${nmapOutput[#]}"
There is nothing I could infer why your script didn't work between the versions of bash you've indicated. I was able to run your script on the given input on bash 4.4.12
But the crux of the problem seems to be using variables and arrays interchangeably in the wrong way.
it seems you're trying to do this the hard way.
why not simply:
IP_ARRAY=( `nmap -sL 127.1/29 | grep "Nmap scan report" | awk '{print $NF}'` )
might be an easy question, I'm new in bash and haven't been able to find the solution to my question.
I'm writing the following script:
for file in `ls *.map`; do
ID=${file%.map}
convertf -p ${ID}_par #this is a program that I use, no problem
NAME=head -n 1 ${ID}.ind | cut -f1 -d":" #Now: This step is the problem: don't seem to be able to make a proper NAME function. I just want to take the first column of the first line of the file ${ID}.ind
It gives me the return
line 5: bad substitution
any help?
Thanks!
There are a couple of issues in your code:
for file in `ls *.map` does not do what you want. It will fail e.g. if any of the filenames contains a space or *, but there's more. See http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29 for details.
You should just use for file in *.map instead.
ALL_UPPERCASE names are generally used for system variables and built-in shell variables. Use lowercase for your own names.
That said,
for file in *.map; do
id="${file%.map}"
convertf -p "${id}_par"
name="$(head -n 1 "${id}.ind" | cut -f1 -d":")"
...
looks like it would work. We just use $( cmd ) to capture the output of a command in a string.
I'm trying to write a BASH script for patching my CentOS servers. My goal is that when I run the script, it outputs the package name, the currently installed version, and new package version. Something like this:
nspr.x86_64 / 4.8.8-3.el6 / 4.8.9-3.el6_2
This way, if I ever need to downgrade because a package broke something, I have a record of this.
The command yum check-update command gives me the 1st piece of information (the package name) and the 3rd piece of information (the new version) listed above, and I can use awk to separate the two. It's easy after that; just run rpm -q {package} to get the 2nd piece of information, then concatenate them.
However, I'm running into trouble with the for loop in my code. I need to pass multiple arguments into the loop (the package name and newer version) so I can echo them later.
Here's my code:
for package in `/usr/bin/yum --cacheonly --noplugins check-update | awk '{print $1, $2}'`;
do
OLD_VER=`rpm -q ${package}` # ${package} should actually be $1 from the awk statement above
NEW_VER=${2} # This is $2 from the awk statement above
echo "${package} / ${OLD_VER} / ${NEW_VER}"
done
Pardon the obvious errors in the code; I'm new to BASH scripting. My confusion mostly stems from awk; I'm not very familiar with it, nor how it interacts with a BASH script. However, I think it's clear what I'm trying to do. Does package get passed as an array? Is there a better way to do this?
you want to read the whole line:
/usr/bin/yum --cacheonly --noplugins check-update |
while read line; do
set - $line;
echo first: $1 second: $2
done
Try this:
/usr/bin/yum --cacheonly --noplugins check-update \
| awk '{package=$1; newVer=$2; cmd = "rpm -q " package; cmd | getline oldVer; close(cmd); print package " / " oldVer " / " newVer}'
The issue with your script was that although both bash and awk use $ to reference variables, they are different vars and you can't reference awk's $2 from bash or vice versa. Writing everything in either awk or bash should solve the issue.
my question is actually rather easy, but I suck at bash scripting and google was no help either. So here is the problem:
I have an executable that writes me a few variables to stdout. Something like that:
MrFoo:~$ ./someExec
Variable1=5
Another_Weird_Variable=12
VARIABLENAME=42
What I want to do now is to read in a specific one of these variables (I already know its name), store the value and use it to give it as an argument to another executable.
So, a simple call like
./function2 5 // which comes from ./function2 Variable1 from above
I hope you understand the problem and can help me with it
With awk you can do something like this (this is for passing value of 1st variable)
./someExec | awk -F= 'NR==1{system("./function2 " $2)}'
or
awk -F= 'NR==1{system("./function2 " $2)}' <(./someExec)
Easiest way to go is probably to use a combination of shell and perl or ruby. I'll go with perl since it's what I cut my teeth on. :)
someExec.sh
#!/bin/bash
echo Variable1=5
echo Another_Weird_Variable=12
echo VARIABLENAME=42
my_shell_script.sh
#!/bin/bash
myVariable=`./someExec | perl -wlne 'print $1 if /Variable1=(.*)/'`
echo "Now call ./function2 $myVariable"
[EDIT]
Or awk, as Jaypal pointed out 58 seconds before I posted my answer. :) Basically, there are a lot of good solutions. Most importantly, though, make sure you handle both security and error cases properly. In both of the solutions so far, we're assuming that someExec will provide guaranteed well-formed and innocuous output. But, consider if someExec were compromised and instead provided output like:
./someExec
5 ; rm -rf / # Uh oh...
You can use awk like this:
./function2 $(./someExec | awk -F "=" '/Variable1/{print $2}')
which is equivalent to:
./function2 5
If you can make sure someExec's output is safe you can use eval.
eval $(./someExec)
./function2 $Variable1
You can use this very simple and straight forward way:
./exp1.sh | grep "Variable1" | awk -F "=" '{print $2}'
If you want to use only one variable from the file use the below
eval $(grep 'Variable1' ./someExec )
./function2 $Variable1
And, if you want to use all the variables of a file, use
eval $(./someExec)
./function2 $<FILE_VARIBALE_NAME>
Let's say that during your workday you repeatedly encounter the following form of columnized output from some command in bash (in my case from executing svn st in my Rails working directory):
? changes.patch
M app/models/superman.rb
A app/models/superwoman.rb
in order to work with the output of your command - in this case the filenames - some sort of parsing is required so that the second column can be used as input for the next command.
What I've been doing is to use awk to get at the second column, e.g. when I want to remove all files (not that that's a typical usecase :), I would do:
svn st | awk '{print $2}' | xargs rm
Since I type this a lot, a natural question is: is there a shorter (thus cooler) way of accomplishing this in bash?
NOTE:
What I am asking is essentially a shell command question even though my concrete example is on my svn workflow. If you feel that workflow is silly and suggest an alternative approach, I probably won't vote you down, but others might, since the question here is really how to get the n-th column command output in bash, in the shortest manner possible. Thanks :)
You can use cut to access the second field:
cut -f2
Edit:
Sorry, didn't realise that SVN doesn't use tabs in its output, so that's a bit useless. You can tailor cut to the output but it's a bit fragile - something like cut -c 10- would work, but the exact value will depend on your setup.
Another option is something like: sed 's/.\s\+//'
To accomplish the same thing as:
svn st | awk '{print $2}' | xargs rm
using only bash you can use:
svn st | while read a b; do rm "$b"; done
Granted, it's not shorter, but it's a bit more efficient and it handles whitespace in your filenames correctly.
I found myself in the same situation and ended up adding these aliases to my .profile file:
alias c1="awk '{print \$1}'"
alias c2="awk '{print \$2}'"
alias c3="awk '{print \$3}'"
alias c4="awk '{print \$4}'"
alias c5="awk '{print \$5}'"
alias c6="awk '{print \$6}'"
alias c7="awk '{print \$7}'"
alias c8="awk '{print \$8}'"
alias c9="awk '{print \$9}'"
Which allows me to write things like this:
svn st | c2 | xargs rm
Try the zsh. It supports suffix alias, so you can define X in your .zshrc to be
alias -g X="| cut -d' ' -f2"
then you can do:
cat file X
You can take it one step further and define it for the nth column:
alias -g X2="| cut -d' ' -f2"
alias -g X1="| cut -d' ' -f1"
alias -g X3="| cut -d' ' -f3"
which will output the nth column of file "file". You can do this for grep output or less output, too. This is very handy and a killer feature of the zsh.
You can go one step further and define D to be:
alias -g D="|xargs rm"
Now you can type:
cat file X1 D
to delete all files mentioned in the first column of file "file".
If you know the bash, the zsh is not much of a change except for some new features.
HTH Chris
Because you seem to be unfamiliar with scripts, here is an example.
#!/bin/sh
# usage: svn st | x 2 | xargs rm
col=$1
shift
awk -v col="$col" '{print $col}' "${#--}"
If you save this in ~/bin/x and make sure ~/bin is in your PATH (now that is something you can and should put in your .bashrc) you have the shortest possible command for generally extracting column n; x n.
The script should do proper error checking and bail if invoked with a non-numeric argument or the incorrect number of arguments, etc; but expanding on this bare-bones essential version will be in unit 102.
Maybe you will want to extend the script to allow a different column delimiter. Awk by default parses input into fields on whitespace; to use a different delimiter, use -F ':' where : is the new delimiter. Implementing this as an option to the script makes it slightly longer, so I'm leaving that as an exercise for the reader.
Usage
Given a file file:
1 2 3
4 5 6
You can either pass it via stdin (using a useless cat merely as a placeholder for something more useful);
$ cat file | sh script.sh 2
2
5
Or provide it as an argument to the script:
$ sh script.sh 2 file
2
5
Here, sh script.sh is assuming that the script is saved as script.sh in the current directory; if you save it with a more useful name somewhere in your PATH and mark it executable, as in the instructions above, obviously use the useful name instead (and no sh).
It looks like you already have a solution. To make things easier, why not just put your command in a bash script (with a short name) and just run that instead of typing out that 'long' command every time?
If you are ok with manually selecting the column, you could be very fast using pick:
svn st | pick | xargs rm
Just go to any cell of the 2nd column, press c and then hit enter
Note, that file path does not have to be in second column of svn st output. For example if you modify file, and modify it's property, it will be 3rd column.
See possible output examples in:
svn help st
Example output:
M wc/bar.c
A + wc/qax.c
I suggest to cut first 8 characters by:
svn st | cut -c8- | while read FILE; do echo whatever with "$FILE"; done
If you want to be 100% sure, and deal with fancy filenames with white space at the end for example, you need to parse xml output:
svn st --xml | grep -o 'path=".*"' | sed 's/^path="//; s/"$//'
Of course you may want to use some real XML parser instead of grep/sed.