I have a list like below:
list="-list mail-username:123 --list mail-password:xyz --list url:https://www.google.com --list mail_username:123 --list mail_password:xyz --list mail_username:123 --list user_password:xyz"
I want to extract the values of all passwords and store it in a variable.
I tried using sed but couldn't get it.
You could try:
pwds="$(echo "$list" | sed 's/[[:space:]]\{1,\}/\n/g' | sed '/password/!d;s/^[^:]*password:\(.*\)$/\1/g')"
This will not work for passphrases with spaces, though.
This will include the trailing newline in the final password, but since you seem to be playing fast and loose with whitespace, that shouldn't be a problem. (Or, at least, it's not more of a problem than the other issues that will crop up from being cavalier about whitespace!)
echo "$list" | awk '/password/{print $2}' RS=' ' FS=:
Related
As part of a tiny automation script that I am trying to create, I have one section which filters out the response from an API call and outputs a number such as 12345 with the following:
tidy -xml -i - responseformatting/response.txt > response_tidied.txt | grep externalId response_tidied.txt | awk -F">" '{print $2}' | awk -F"<" '{print $1}' > response_tidied.txt
Is there a way I can pass the output obtained from this into sed, below? If I run this on its own, it works just fine. I just cannot seem to figure out how to use the value from the above tidying process into sed.
sed -i -e 's/\(<td>\).*\(<\/td>\)/<td>OUTPUT_FROM_ABOVE<\/td>/g' new.xml
Any help would be much appreciated :)
Please refer to sed - replace string with file contents, use
sed -i "s/<td>.*<\/td>/<td>$(sed 's:/:\\/:g' response_tidied.txt)<\/td>/g" new.xml
Bear in mind you need to use double quotes round the sed command. The point is that you need to escape all slashes in the replacement string.
I run the command
df -gP /data1 /data2 | grep -v File | awk '{print $1}' |
awk -F/dev/ '$0=$2' | tr '\n' '
on the AIX shell (ksh) and it prints the output below:
lv_data01 lv_data02 root#testhost:/
However, I would like the output to be printed this way. Could someone help?
lv_data01 lv_data02
Using grep … | awk … | awk … is not necessary; a single awk could do the whole job. So could sed and it might even be easier. I'd be tempted to deal with the spacing by using:
x=$(df … | sed …); echo $x
The tr command, once corrected, replaces newlines with spaces, so the prompt follows without a newline before it. The ; echo suggestion adds the missing newline; the echo $x suggestion (note no double quotes) does too.
As for the sed command:
sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'
Don't print anything by default
If the line doesn't match File (doing the work of grep -v):
remove the first space (blank or tab) and everything after it (doing the work of awk '{print $1}')
replace everything up to /dev/ with nothing and print (doing the work of awk -F/dev/ '{$0=$2}')
The command substitution and capture, followed by echo, deals with spaces and newlines.
So, my suggested solution is:
x=$(df -gP /data1 /data2 | sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'); echo $x
You could add unset x after the echo if you are going to be using this directly in the shell and not in a shell script. If it'll be encapsulated in a shell script, you don't have to worry about it.
I'm blithely assuming the output from df -gP won't contain a path such as this, with two occurrences of /dev:
/who/knows/dev/lv_data01/dev/bin
If that's a real problem, you can fix the sed script, but I don't think it will be. It's one thing the second awk script in the question handles differently.
i'm trying to extract a specific string from a grep output
uci show minidlna
produces a large list
.
.
.
minidlna.config.enabled='1'
minidlna.config.db_dir='/mnt/sda1/usb/db'
minidlna.config.enable_tivo='1'
minidlna.config.wide_links='1'
.
.
.
so i tried to narrow down what i wanted by running
uci show minidlna | grep -oE '\bdb_dir=\S+'
this narrows the output to
db_dir='/mnt/sda1/usb/db'
what i want is to output only
/mnt/sda1/usb/db
without the quotes and without the starting "db_dir" so i can run rm /mnt/sda1/usb/db/file.db
i've used the answers found here
How to extract string following a pattern with grep, regex or perl
and that's as close as i got.
EDIT: after using Ed Morton's awk command i needed to pass the output to rm command.
i used:
| ( read DB; (rm $DB/files.db) .
read DB passes the output into the vairable DB.
(...) combines commands.
rm $DB/files.db deletes the the file files.db.
Is this what you're trying to do?
$ awk -F"'" '/db_dir/{print $2}' file
/mnt/sda1/usb/db
That will work in any awk in any shell on every UNIX box.
If that's not what you want then edit your question to clarify your requirements and post more truly representative sample input/output.
Using sed with some effort to avoid single quotes:
sed -n 's/^minidlna.config.db_dir=\s*\S\(\S*\)\S\s*$/\1/p' input
Well, so you end up having a string like db_dir='/mnt/sda1/usb/db'.
I would first remove the quotes by piping this to
.... | tr -d "'"
Now you end up with a string like db_dir=/mnt/sda1/usb/db.
Say you have this string stored in a variable named confstr, then
${confstr##*=}
gives you just /mnt/sda1/usb/db, since *= denotes everything from the start to the equal sign, and ## denotes removal.
I would do this:
Once you either extracted your line about into file.txt (or pipe it into this command), split the fields using the quote character. Use printf to generate the rm command and pass this into bash to execute.
$ awk -F"'" '{printf "rm %s.db/file.db\n", $2}' file.txt | bash
rm: /mnt/sda1/usb/db.db/file.db: No such file or directory
With your original command:
$ uci show minidlna | grep -oE '\bdb_dir=\S+' | \
awk -F"'" '{printf "rm %s.db/file.db\n", $2}' | bash
I have an env.txt file in the following format:
lDRIVER={ODBC Driver 13 for SQL Server};
PORT=1433;
SERVER=serveename;
DATABASE=db;
UID=username;
PWD=password!
I have a git bash script (.sh) that requires the UID and PWD from that file. I was thinking about getting it by the last/second last line number. How do I do this/ is there a better way (say looking for UID and PWD and assigning the git bash variable that way)
There's lots of ways to do this. You could use awk which I would personally use since it's sort of like an x-acto knife for this type of thing:
uid=$(awk -F"[=;]" '/UID/{print $2}' env.txt)
pwd=$(awk -F"[=;]" '/PWD/{print $2}' env.txt)
Or grep and sed. sed is nice because it allows you to get very specific about the piece of info you want to cut from the line, but it's regex which has its learning curve:
uid=$(grep "UID" env.txt | sed -r 's/^.*=(.*)(;|$)/\1/g' )
pwd=$(grep "PWD" env.txt | sed -r 's/^.*=(.*)(;|$)/\1/g' )
As #JamesK noted in the comments you can use sed and have it do the search instead of grep. This is super nice and I would definitely choose this instead of the grep | sed.
uid=$(sed -nr '/UID/s/^.*=(.*)(;|$)/\1/gp' )
pwd=$(sed -nr '/PWD/s/^.*=(.*)(;|$)/\1/gp' )
Or grep and cut. Bleh... we can all do better, but sometimes we just want to grep and cut and not have to think about it:
uid=$(grep "UID" env.txt | cut -d"=" -f2 | cut -d";" -f1)
pwd=$(grep "PWD" env.txt | cut -d"=" -f2 | cut -d";" -f1)
I definitely wouldn't go by line number though. That looks like and odbc.ini file and the order in which the parameters are listed in each odbc entry are irrelevant.
First rename PWD to something like PASSWORD. PWD is a special variable used by the shell. Even better is to use lowercase variable names for all your own variables.
When the password is without special characters (spaces, $, ), you can
source env.txt
When the password has something special, consider editing the env.txt:
lDRIVER="{ODBC Driver 13 for SQL Server}"
PORT="1433"
SERVER="serveename"
DATABASE="db"
UID="username"
PASSWORD="password!"
When you are only interested in lowercase uid and passwd, consider selecting only the interesting fields and change the keywords to lowercase
source <(sed -rn '/^(UID|PWD)=/ s/([^=]*)/\L\1/p' env.txt)
I'm writing a small shell script that needs to reverse the lines of a text file. Is there a standard filter command to do this sort of thing?
My specific application is that I'm getting a list of Git commit identifiers, and I want to process them in reverse order:
git log --pretty=oneline work...master | grep -v DEBUG: | cut -d' ' -f1 | reverse
The best I've come up with is to implement reverse like this:
... | cat -b | sort -rn | cut -f2-
This uses cat to number every line, then sort to sort them in descending numeric order (which ends up reversing the whole file), then cut to remove the unneeded line number.
The above works for my application, but may fail in the general case because cat -b only numbers nonblank lines.
Is there a better, more general way to do this?
In GNU coreutils, there's tac(1)
There is a command for your purpose:
tail -r file.txt
Prints the lines of file.txt in reverse order!
The -r flag is non-standard, may not work on all systems, works e.g. on macOS.
Beware: Amount of lines limited. Works mostly, but when working with huge files be careful and check.
Answer is not 42 but tac.
Edit: Slower but more memory consuming using sed
sed 'x;1!H;$!d;x'
and even longer
perl -e'print reverse<>'
Similar to the sed example above, using perl - maybe more memorable (depending on how your brain is wired):
perl -e 'print reverse <>'
cat -b only numbers nonblank lines"
If that's the only issue you want to avoid, then why not use "cat -n" to number all the lines?
: "#(#)$Id: reverse.sh,v 1.2 1997/06/02 21:45:00 johnl Exp $"
#
# Reverse the order of the lines in each file
awk ' { printf("%d:%s\n", NR, $0);}' $* |
sort -t: +0nr -1 |
sed 's/^[0-9][0-9]*://'
Works like a charm for me...
In this case, just use --reverse:
$ git log --reverse --pretty=oneline work...master | grep -v DEBUG: | cut -d' ' -f1
rev <name of your text file.txt>
You can even do this:
echo <whatever you want to type>|rev
awk '{a[i++]=$0}END{for(;i-->0;)print a[i]}'
More faster than sed and compatible for embed devices like openwrt.