I'm reading from a file and want to
Get the text for value where key = "key".
i.e.
key = value
Trim any white space around the value or the key
However, value can also contain an = because it's a base64 encoded field.
I've been using this previously:
key=`egrep 'key' myfile | cut -f2 -d'=' | sed 's/ //g'`
But, cut works globally. Is there a way of making it work on just the first item?
Or perhaps there more efficient way of doing this.
e.g. if myfile = ceph.client.keyring
[client.admin]
key = AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==
auid = 0
caps mds = "allow"
caps mon = "allow *"
caps osd = "allow *"
I'm wanting to read into a variable the value for key (AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==)
Using sed
Your your example myfile:
$ sed -rn '/^[[:space:]]*key[ =]/ s/[^=]*=[[:space:]]*//p' myfile
AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==
How it works:
sed -rn
This tells sed to use extended regular expressions, -r, and not to print unless we explicitly ask it to, -n.
/^[[:space:]]*key[ =]/
This selects lines that optionally start with whitespace, followed by key, followed by either a blank or an equal sign, =.
s/[^=]*=[[:space:]]*//p
For those selected lines, this strips out everything before the first equal sign and any whitespace after the equal sign. The p option tells sed to print the resulting line.
Using awk
This assumes that, as in your example, the first equal sign is surrounded by spaces and that the value for key contains no whitespace:
$ awk '$1=="key"{print $3}' myfile
AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==
How it works:
$1=="key"
This selects lines whose first field is key.
{print $3}
For those selected lines, this prints the third field.
Alternate sed solution
Doing the selecting and substituting in one step:
$ sed -rn 's/^[[:space:]]*key[[:space:]]=[[:space:]]*//p' myfile
AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==
Using grep with Perl regex:
grep -oP "(?<=\skey\s=\s)[^\s]*" file
Output:
AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==
Note: It assumes there's only one white space between key and =, & = and value
Or using sed:
sed -nr 's/.*\skey\s*=\s*([^\s]*).*/\1/p' file
AQAa6HRVaDKxLxAANulnamD/5x2SBly7kPPatg==
Related
I have the following file:
APP_VERSION.ts
export const APP_VERSION = 1;
This is the only content of that file, and the APP_VERSION variable will be incremented as needed.
So, the APP_VERSION could be a single digit number or multiple digit number, like 15 or 999, etc.
I need to use that value in one of my bash scripts.
use-app-version.sh
APP_VERSION=`cat src/constants/APP_VERSION.ts`
echo $APP_VERSION
I know I can read it with cat. But how can I parse that string so I can get exactly the APP_VERSION value, whether it's 1 or 999, for example.
sed -En 's/(^.*APP_VERSION.*)([[:digit:]]+.*)(\;.*$)/\2/p' src/constants/APP_VERSION
Using sed, split the line into three sections defined by opening and closing brackets. Substitute the line for second section on ( the version value) and print.
You may use this awk:
app_ver=$(awk -F '[[:blank:];=]+' '$(NF-2) == "APP_VERSION" {print $(NF-1)}' src/constants/APP_VERSION.ts)
echo "$app_ver"
1
You can concat some commands to remove everything else:
APP_VERSION=`cat src/constants/APP_VERSION.ts | awk -F '=' '{print $2}' | tr -d ' ' | tr -d ';'`
1 - Cat get all file content
2 - AWK gets all content after '='
3 - Remove space
4 - Remove ;
A simple
APP_VERSION=$(grep --text -Eo '[0-9]+' src/constants/APP_VERSION.ts)
should be enough
With bash only:
APP_VERSION=$(cat src/constants/APP_VERSION.ts)
APP_VERSION=${APP_VERSION%;}
APP_VERSION=${APP_VERSION/*= }
Line 2 removes the trailing ';', line 3 removes everything before "= ".
Alternatively, you could set APP_VERSION as an array, take 5th element, and remove trailing ';'.
Or, another solution, using IFS:
IFS='=;' read a APP_VERSION < src/constants/APP_VERSION.ts
In this version, the space will remain before version number.
Assuming that the task can be rephrased to "extract the digits from a file", there are a few options:
Delete all characters that aren't digits with tr:
version=$(tr -cd '[:digit:]' < infile)
Use grep to match all digits and retain nothing but the match:
version=$(grep -Eo '[[:digit:]]+' infile)
Read file into string and delete all non-digits with just Bash:
contents=$(< infile)
version=${contents//[![:digit:]]}
I have a string like this:
/var/cpanel/users/joebloggs:DNS9=domain.example
I need to extract the username (joebloggs) from this string and store it in a variable.
The format of the string will always be the same with exception of joebloggs and domain.example so I am thinking the string can be split twice using cut?
The first split would split by : and we would store the first part in a variable to pass to the second split function.
The second split would split by / and store the last word (joebloggs) into a variable
I know how to do this in PHP using arrays and splits but I am a bit lost in bash.
To extract joebloggs from this string in bash using parameter expansion without any extra processes...
MYVAR="/var/cpanel/users/joebloggs:DNS9=domain.example"
NAME=${MYVAR%:*} # retain the part before the colon
NAME=${NAME##*/} # retain the part after the last slash
echo $NAME
Doesn't depend on joebloggs being at a particular depth in the path.
Summary
An overview of a few parameter expansion modes, for reference...
${MYVAR#pattern} # delete shortest match of pattern from the beginning
${MYVAR##pattern} # delete longest match of pattern from the beginning
${MYVAR%pattern} # delete shortest match of pattern from the end
${MYVAR%%pattern} # delete longest match of pattern from the end
So # means match from the beginning (think of a comment line) and % means from the end. One instance means shortest and two instances means longest.
You can get substrings based on position using numbers:
${MYVAR:3} # Remove the first three chars (leaving 4..end)
${MYVAR::3} # Return the first three characters
${MYVAR:3:5} # The next five characters after removing the first 3 (chars 4-9)
You can also replace particular strings or patterns using:
${MYVAR/search/replace}
The pattern is in the same format as file-name matching, so * (any characters) is common, often followed by a particular symbol like / or .
Examples:
Given a variable like
MYVAR="users/joebloggs/domain.example"
Remove the path leaving file name (all characters up to a slash):
echo ${MYVAR##*/}
domain.example
Remove the file name, leaving the path (delete shortest match after last /):
echo ${MYVAR%/*}
users/joebloggs
Get just the file extension (remove all before last period):
echo ${MYVAR##*.}
example
NOTE: To do two operations, you can't combine them, but have to assign to an intermediate variable. So to get the file name without path or extension:
NAME=${MYVAR##*/} # remove part before last slash
echo ${NAME%.*} # from the new var remove the part after the last period
domain
Define a function like this:
getUserName() {
echo $1 | cut -d : -f 1 | xargs basename
}
And pass the string as a parameter:
userName=$(getUserName "/var/cpanel/users/joebloggs:DNS9=domain.example")
echo $userName
What about sed? That will work in a single command:
sed 's#.*/\([^:]*\).*#\1#' <<<$string
The # are being used for regex dividers instead of / since the string has / in it.
.*/ grabs the string up to the last backslash.
\( .. \) marks a capture group. This is \([^:]*\).
The [^:] says any character _except a colon, and the * means zero or more.
.* means the rest of the line.
\1 means substitute what was found in the first (and only) capture group. This is the name.
Here's the breakdown matching the string with the regular expression:
/var/cpanel/users/ joebloggs :DNS9=domain.example joebloggs
sed 's#.*/ \([^:]*\) .* #\1 #'
Using a single Awk:
... | awk -F '[/:]' '{print $5}'
That is, using as field separator either / or :, the username is always in field 5.
To store it in a variable:
username=$(... | awk -F '[/:]' '{print $5}')
A more flexible implementation with sed that doesn't require username to be field 5:
... | sed -e s/:.*// -e s?.*/??
That is, delete everything from : and beyond, and then delete everything up until the last /. sed is probably faster too than awk, so this alternative is definitely better.
Using a single sed
echo "/var/cpanel/users/joebloggs:DNS9=domain.example" | sed 's/.*\/\(.*\):.*/\1/'
I like to chain together awk using different delimitators set with the -F argument. First, split the string on /users/ and then on :
txt="/var/cpanel/users/joebloggs:DNS9=domain.com"
echo $txt | awk -F"/users/" '{print$2}' | awk -F: '{print $1}'
$2 gives the text after the delim, $1 the text before it.
I know I'm a little late to the party and there's already good answers, but here's my method of doing something like this.
DIR="/var/cpanel/users/joebloggs:DNS9=domain.example"
echo ${DIR} | rev | cut -d'/' -f 1 | rev | cut -d':' -f1
I have a really bad habit of abusing tr.
I need to find another way, a different style.
all I want to so is print the list horizontally instead of vertically - so I can cut and past it into an email. Check out the use of the TR command. Just terrible.
$ cat /tmp/wig
update PTMM_ARCHIVE.FASTTRACK_USER set user_name = 'monohajoxx' where user_name = 'monohajo'
update PTMM_ARCHIVE.FASTTRACK_USER set user_name = 'wuemxx' where user_name = 'wuem'
update PTMM_ARCHIVE.FASTTRACK_USER set user_name = 'taraziemxx' where user_name = 'taraziem'
update PTMM_ARCHIVE.FASTTRACK_USER set user_name = 'mullankexx' where user_name = 'mullanke'
update PTMM_ARCHIVE.FASTTRACK_USER set user_name = 'fernanjaxx' where user_name = 'fernanja'
$ awk '{print $NF}' /tmp/wig | tr -d "'" | tr "\n" ", \s" ; echo "\n"
monohajo,wuem,taraziem,mullanke,fernanja,\n
Using awk
Here is one way to do it entirely with awk:
$ awk '{gsub(/'\''/,"",$NF); printf "%s%s",(NR>1?",":""),$NF} END{print "\\n"}' wig
monohajo,wuem,taraziem,mullanke,fernanja\n
The gsub command removes the single-quotes from the last field. The printf command prints the last field preceded by a comma if this isn't the first line. The final print statement finishes the line.
And, here is another:
$ awk '{printf "%s%s",(NR>1?",":""),substr($NF,2,length($NF)-2)} END{print "\\n"}' wig
monohajo,wuem,taraziem,mullanke,fernanja\n
This uses a similar printf statement but uses substr to remove the first and last characters of the last field.
Using sed
$ sed -nE "s/.*'([^']*)'/\1/"'; H; 1h; ${x; s/\n/,/g; s/$/\\n/; p}' wig
monohajo,wuem,taraziem,mullanke,fernanja\n
How it works:
-n tells sed not to print anything unless we explicitly ask it to.
-E tells sed to use extended regular expressions so that we don't have to type as many backslashes.
s/.*'([^']*)'/\1/
This removes everything from the line except for the last field single-quoted string (with the quotes are removed).
H; 1h;
H adds a newline to the hold space followed by a copy of the current pattern space (which now contains the last field, minus the quotes).
If this is the first line, however, the h command overwrites the hold space with just the current value of the pattern space (no newline).
${x; s/\n/,/g; s/$/\\n/; p}
On the last line, denoted by $, this does the following:
- `x` exchanges the hold and pattern spaces.
- `s/\n/,/g` converts all those newlines to commas.
- `s/$/\\n/` puts a `\n` at the end.
- `p` causes this pattern space to be printed.
I want to write a small script to generate the location of a file in an NGINX cache directory.
The format of the path is:
/path/to/nginx/cache/d8/40/32/13febd65d65112badd0aa90a15d84032
Note the last 6 characters: d8 40 32, are represented in the path.
As an input I give the md5 hash (13febd65d65112badd0aa90a15d84032) and I want to generate the output: d8/40/32/13febd65d65112badd0aa90a15d84032
I'm sure sed or awk will be handy, but I don't know yet how...
This awk can make it:
awk 'BEGIN{FS=""; OFS="/"}{print $(NF-5)$(NF-4), $(NF-3)$(NF-2), $(NF-1)$NF, $0}'
Explanation
BEGIN{FS=""; OFS="/"}. FS="" sets the input field separator to be "", so that every char will be a different field. OFS="/" sets the output field separator as /, for print matters.
print ... $(NF-1)$NF, $0 prints the penultimate field and the last one all together; then, the whole string. The comma is "filled" with the OFS, which is /.
Test
$ awk 'BEGIN{FS=""; OFS="/"}{print $(NF-5)$(NF-4), $(NF-3)$(NF-2), $(NF-1)$NF, $0}' <<< "13febd65d65112badd0aa90a15d84032"
d8/40/32/13febd65d65112badd0aa90a15d84032
Or with a file:
$ cat a
13febd65d65112badd0aa90a15d84032
13febd65d65112badd0aa90a15f1f2f3
$ awk 'BEGIN{FS=""; OFS="/"}{print $(NF-5)$(NF-4), $(NF-3)$(NF-2), $(NF-1)$NF, $0}' a
d8/40/32/13febd65d65112badd0aa90a15d84032
f1/f2/f3/13febd65d65112badd0aa90a15f1f2f3
With sed:
echo '13febd65d65112badd0aa90a15d84032' | \
sed -n 's/\(.*\([0-9a-f]\{2\}\)\([0-9a-f]\{2\}\)\([0-9a-f]\{2\}\)\)$/\2\/\3\/\4\/\1/p;'
Having GNU sed you can even simplify the pattern using the -r option. Now you won't need to escape {} and () any more. Using ~ as the regex delimiter allows to use the path separator / without need to escape it:
sed -nr 's~(.*([0-9a-f]{2})([0-9a-f]{2})([0-9a-f]{2}))$~\2/\3/\4/\1~p;'
Output:
d8/40/32/13febd65d65112badd0aa90a15d84032
Explained simple the pattern does the following: It matches:
(all (n-5 - n-4) (n-3 - n-2) (n-1 - n-0))
and replaces it by
/$1/$2/$3/$0
You can use a regular expression to separate each of the last 3 bytes from the rest of the hash.
hash=13febd65d65112badd0aa90a15d84032
[[ $hash =~ (..)(..)(..)$ ]]
new_path="/path/to/nginx/cache/${BASH_REMATCH[1]}/${BASH_REMATCH[2]}/${BASH_REMATCH[3]}/$hash"
Base="/path/to/nginx/cache/"
echo '13febd65d65112badd0aa90a15d84032' | \
sed "s|\(.*\(..\)\(..\)\(..\)\)|${Base}\2/\3/\4/\1|"
# or
# sed sed 's|.*\(..\)\(..\)\(..\)$|${Base}\1/\2/\3/&|'
Assuming info is a correct MD5 (and only) string
First of all - thanks to all of the responders - this was extremely quick!
I also did my own scripting meantime, and came up with this solution:
Run this script with a parameter of the URL you're looking for (www.example.com/article/76232?q=hello for example)
#!/bin/bash
path=$1
md5=$(echo -n "$path" | md5sum | cut -f1 -d' ')
p3=$(echo "${md5:0-2:2}")
p2=$(echo "${md5:0-4:2}")
p1=$(echo "${md5:0-6:2}")
echo "/path/to/nginx/cache/$p1/$p2/$p3/$md5"
This assumes the NGINX cache has a key structure of 2:2:2.
I have a string having this formats:
aa_bb_cc_dd
aa_bb_cc_dd_ee_ff
I want to obtain:
bb_cc
bb_cc_dd_ee
I've tried 'cut', but I didn't manage to obtain what I wanted.
when using bash you can use built-ins for this task:
strip_headtail() {
local s=$1
## strip the head
s=${s#*_}
## strip the tail
s=${s%_*}
echo ${s}
}
strip_headtail aa_bb_cc_dd
strip_headtail aa_bb_cc_dd_ee_ff
you might want to check the bash-manual (man bash) for more information on this.
search for Remove matching prefix pattern resp. Remove matching suffix pattern.
With awk:
$ echo "aa_bb_cc_dd
aa_bb_cc_dd_ee_ff" | awk -F_ '{for(i=1;i<NF;i++) $i=$(i+1); NF=NF-2}1' OFS=_
bb_cc
bb_cc_dd_ee
Explanation
-F_ and OFS=_ set input and output field separator as _.
{for(i=1;i<NF;i++) $i=$(i+1); NF=NF-2} set each field as the next one, so the nth will be the (n+1)th. Then, decrease number of fields in 2.
With sed:
$ echo "aa_bb_cc_dd
aa_bb_cc_dd_ee_ff" | sed -e 's/^[^_]*_//' -e 's/_[^_]*$//'
bb_cc
bb_cc_dd_ee
Explanation
sed -e is used to do multiple commands.
's/^[^_]*_//' delete from the beginning up to first _.
's/_[^_]*$//' delete from last _ up to the end of line.