I've been trying to create a variable that holds the value of other values and also commands to be issued. I've found individual answers to both those sitatuons, but I'm struggling to put them together.
$DID and $SECTOR below are two variables that are already established.
DATAPARMS=$($DID,$SECTOR,date,uptime | sed 's/^.* up \+\(.\+\), \+[0-9] user.*$/\1/',ls -I README.txt /var/www/html | wc -l,grep "VERSION" /root/config | grep -o '".*"' | sed 's/"//g')
The individual commands (grep, sed, etc) can be ignore.
I would then like to call this with echo:
echo "$DATAPARMS"
Any pointers on how best to accomplish this? Thank you very much!
As described in BashFAQ #50, variables should not be used to store code. Use a function instead, as follows:
getDataParams() {
echo "$DID,$SECTOR,date,uptime" | sed 's/^.* up \+\(.\+\), \+[0-9] user.*$/\1/'
ls -I README.txt /var/www/html | wc -l
grep "VERSION" /root/config | grep -o '".*"' | sed 's/"//g'
}
getDataParams
Related
H!
So I am trying to run a script which looks for a string pattern.
For example, from a file I want to find 2 words, located separately
"I like toast, toast is amazing. Bread is just toast before it was toasted."
I want to invoke it from the command line using something like this:
./myscript.sh myfile.txt "toast bread"
My code so far:
text_file=$1
keyword_first=$2
keyword_second=$3
find_keyword=$(cat $text_file | grep -w "$keyword_first""$keyword_second" )
echo $find_keyword
i have tried a few different ways. Directly from the command line I can make it run using:
cat myfile.txt | grep -E 'toast|bread'
I'm trying to put the user input into variables and use the variables to grep the file
You seem to be looking simply for
grep -E "$2|$3" "$1"
What works on the command line will also work in a script, though you will need to switch to double quotes for the shell to replace variables inside the quotes.
In this case, the -E option can be replaced with multiple -e options, too.
grep -e "$2" -e "$3" "$1"
You can pipe to grep twice:
find_keyword=$(cat $text_file | grep -w "$keyword_first" | grep -w "$keyword_second")
Note that your search word "bread" is not found because the string contains the uppercase "Bread". If you want to find the words regardless of this, you should use the case-insensitive option -i for grep:
find_keyword=$(cat $text_file | grep -w -i "$keyword_first" | grep -w -i "$keyword_second")
In a full script:
#!/bin/bash
#
# usage: ./myscript.sh myfile.txt "toast" "bread"
text_file=$1
keyword_first=$2
keyword_second=$3
find_keyword=$(cat $text_file | grep -w -i "$keyword_first" | grep -w -i "$keyword_second")
echo $find_keyword
I have defined two variables as follows:
var1=$(unzip -c ./*.zip | grep -n "Channel8"| cut -f1 -d":")
var2=$(unzip -c ./*.zip | grep -n "Channel10"| cut -f1 -d":")
I have a very big file and I would like to extract the range of lines between $var1 and $var2 using sed. I am trying the following
sed -n '/"$var1","$var"2p' $(unzip -c ./*.zip)
But with no success. Could you give an explanation why and how to fix it? Thanks.
You can use:
unzip -c ./*.zip | sed -n "$var1,$var2 p"
Fixes are:
Not using single quotes around shell variable
Removal of leading / from sed command
Use of pipeline instead of command substitution
Variables aren't expanded inside single quotes. Also, you need to pipe the output of unzip to sed, not use it as command-line arguments.
unzip -c ./*.zip | sed -n "${var1},${var2}p"
But it seems like you're doing this the hard way, reading the zip file 3 times. Just use the pattern you want to match as the range:
unzip -c ./*.zip | sed -n '/^extracting:.*Channel8/,/^extracting:.*Channel10/p'
Use double quotes to expand the vars:
sed -n "${var1},${var2}p" $(unzip -c ./*.zip)
I want to process an old database where password are plain text (comma separated ; passwd is the 5th field in the csv file where the database has been exported) to crypt them for further use by dokuwiki. Here is my bash command (grep and sed are there to extract the crypted passwd from curl output) :
cat users.csv | awk 'FS="," { print $4 }' | xargs -l bash -c 'curl -s --data-binary "pass1=$0&pass2=$0" "https://sprhost.com/tools/SMD5.php" -o - ' | xargs | grep -o '<tt.*tt>' | sed -e 's/tt//g' | sed -e 's/<[^>]*>//g'
I get the following comment from xargs
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
And only the first line of the file is processed, and nothing appends then.
Using the -0 option, and playing around with quotes, doesn't solve anything. Where am I wrong in the command line ? May be a more advanced language will be more adequate to do this.
Thank for help, LM
In general, if you have such a long pipe of commands, it is better to split them if things go wrong. Going through your pipe:
cat users.csv |
Nothing unexpected there.
awk 'FS="," { print $4 }' |
You probably wanted to do awk 'BEGIN {FS=","} { print $4 }'. Try the first two commands in the pipe and see if they produce the correct answer.
xargs -l bash -c 'curl -s --data-binary "pass1=$0&pass2=$0" "https://sprhost.com/tools/SMD5.php" -o - ' |
Nothing wrong there, although there might be better ways to do an MD5 hash.
xargs |
What is this xargs doing in the pipe? It should be removed.
grep -o '<tt.*tt>' |
Note that this will produce two lines:
<tt>$1$17ab075e$0VQMuM3cr5CtElvMxrPcE0</tt>
<tt><your_docuwiki_root>/conf/users.auth.php</tt>
which is probably not what you expected.
sed -e 's/tt//g' |
sed -e 's/<[^>]*>//g'
which will remove the html-tags, though
sed 's/<tt>//;s/<.tt>//'
will do the same.
So I'd say a wrong awk and an xargs too many.
I'm trying to get the output from a grep and sed pipe to go to the terminal and a text file.
Neither
grep -Filr "string1" * 2>&1 | tee ~/outputfile.txt | sed -i "s|string1|string2|g"
nor
grep -Filr "string1" * | sed -i "s|string1|string2|g" 2>&1 | tee ~/outputfile.txt
work. I get "sed: no input files" going to the terminal so sed is not getting the correct input. I just want to see and write out to a text file which files are modified from the search and replace. I know using find instead of grep would be more efficient since the search wouldn't be done twice, but I'm not sure how to output the file name using find and sed when there is a search hit.
EDIT:
Oops I forgot to include xargs in the code. It should have been:
grep -Filr "string1" * 2>&1 | tee ~/outputfile.txt | xargs sed -i "s|string1|string2|g"
and
grep -Filr "string1" * | xargs sed -i "s|string1|string2|g" 2>&1 | tee ~/outputfile.txt
To be clear, I'm looking for a solution that modifies the matched files with the search and replace, and then outputs the modified files' file names to the terminal and a log file.
The -i option to sed is only useful when sed operates on a file, not on standard input. Drop it, and your first option is correct.
I'd use a loop:
for i in `grep -lr string1 *`; do sed -i . 's/string1/string2/g' $i; echo $i >> ~/outputfile.txt; done
I'd advise against using the 'i' option for grep, because it would match files which the sed command won't actually modify.
You can do the same with find and exec, but that's a dangerous tool.
I almost forgot about this. I eventually went with a for loop in a bash script:
#!/bin/bash
for i in $( grep -Flr "string1" * ); do
sed -i "s|string1|string2|g" $i
echo $i
echo $i >> ~/outputfile.txt
done
I'm using the vertical pipe | as the separator, because I'm replacing URL paths with lots of forward slashes.
Thank you both for your help.
I have to run many python script which differ just with one parameter. I name them as runv1.py, runv2.py, runv20.py. I have the original script, say runv1.py. Then I make all copies that I need by
cat runv1.py | tee runv{2..20..1}.py
So I have runv1.py,.., runv20.py. But still the parameter v=1 in all of them.
Q: how can I also replace v parameter to read it from the file name? so e.g in runv4.py then v=4. I would like to know if there is any one-line shell command or combination of commands. Thank you!
PS: direct editing each file is not a proper solution when there are too many files.
Below for loop will serve your purpose I think
for i in `ls | grep "runv[0-9][0-9]*.py"`
do
l=`echo $i | tr -d [a-z.]`
sed -i 's/v/'"$l"'/g' runv$l.py
done
Below command was to pass the parameter to script extracted from the filename itself
ls | grep "runv[0-9][0-9]*.py" | tr -d [a-z.] | awk '{print "./runv"$0".py "$0}' | xargs sh
in the end instead of sh you can use python or bash or ksh.