I have a string that is sometimes
xxx.11_222_33_44_555.yyy
and sometimes
xxx.11_222_33_44.yyy
I would like to:
Check if has 4 occourances of _ (figured out how to do it).
If so - remove string's _33 (the 33 string changes, can be any number), so I am left with xxx.11_222_44.yyy.
Using sed :
sed 's/\(_[0-9]*\)_[0-9]*\(_[0-9]*_[0-9]*\)/\1\2/'
It matches the four underscores and replace the whole by the needed parts.
Test run :
$ echo "xxx.11_222_33_44_555.yyy" | sed 's/\(_[0-9]*\)_[0-9]*\(_[0-9]*_[0-9]*\)/\1\2/'
xxx.11_222_44_555.yyy
$ echo "xxx.11_222_33_44.yyy" | sed 's/\(_[0-9]*\)_[0-9]*\(_[0-9]*_[0-9]*\)/\1\2/'
xxx.11_222_33_44.yyy
perhaps something like this
echo "xxx.11_222_33_44.yyy" | sed -e's/\.\([0-9]\+\)_\([0-9]\+\)_\([0-9]\+\)_\([0-9]\+\)\./.\1_\2_\4./'
which checks if there are 4 groups of numbers separated by _ between the two dots and if yes, it leaves out the third group
try this;
echo "xxx.11_222_33_44_555.yyy" | awk -F'_' 'NF>4{print $1"_"$2"_"$4"_"$5};'
Solution using perl and Lookahead and Lookbehind
$ a="xxx.11_222_33_44_555.yyy"
$ perl -pe 's/\.\d+_\d+_\K\d+_(?=\d+_\d+\.)//' <<< "$a"
xxx.11_222_44_555.yyy
Related
How can I reverse a four length of letters with sed?
For example:
the year was 1815.
Reverse to:
the raey was 5181.
This is my attempt:
cat filename | sed's/\([a-z]*\) *\([a-z]*\)/\2, \1/'
But it does not work as I intended.
not sure it is possible to do it with GNU sed for all cases. If _ doesn't occur immediately before/after four letter words, you can use
sed -E 's/\b([a-z0-9])([a-z0-9])([a-z0-9])([a-z0-9])\b/\4\3\2\1/gi'
\b is word boundary, word definition being any alphabet or digit or underscore character. So \b will ensure to match only whole words not part of words
$ echo 'the year was 1815.' | sed -E 's/\b([a-z0-9])([a-z0-9])([a-z0-9])([a-z0-9])\b/\4\3\2\1/gi'
the raey was 5181.
$ echo 'two time five three six good' | sed -E 's/\b([a-z0-9])([a-z0-9])([a-z0-9])([a-z0-9])\b/\4\3\2\1/gi'
two emit evif three six doog
$ # but won't work if there are underscores around the words
$ echo '_good food' | sed -E 's/\b([a-z0-9])([a-z0-9])([a-z0-9])([a-z0-9])\b/\4\3\2\1/gi'
_good doof
tool with lookaround support would work for all cases
$ echo '_good food' | perl -pe 's/(?<![a-z0-9])([a-z0-9])([a-z0-9])([a-z0-9])([a-z0-9])(?!=[a-z0-9])/$4$3$2$1/gi'
_doog doof
(?<![a-z0-9]) and (?!=[a-z0-9]) are negative lookbehind and negative lookahead respectively
Can be shortened to
perl -pe 's/(?<![a-z0-9])[a-z0-9]{4}(?!=[a-z0-9])/reverse $&/gie'
which uses the e modifier to place Perl code in substitution section. This form is suitable to easily change length of words to be reversed
Possible shortest sed solution even if a four length of letters contains _s.
sed -r 's/\<(.)(.)(.)(.)\>/\4\3\2\1/g'
Following awk may help you in same. Tested this in GNU awk and only with provided sample Input_file
echo "the year was 1815." |
awk '
function reverse(val){
num=split(val, array,"");
i=array[num]=="."?num-1:num;
for(;i>q;i--){
var=var?var array[i]:array[i]
};
printf (array[num]=="."?var".":var);
var=""
}
{
for(j=1;j<=NF;j++){
printf("%s%s",j==NF||j==2?reverse($j):$j,j==NF?RS:FS)
}}'
This might work for you (GNU sed):
sed -r '/\<\w{4}\>/!b;s//\n&\n/g;s/^[^\n]/\n&/;:a;/\n\n/!s/(.*\n)([^\n])(.*\n)/\2\1\3/;ta;s/^([^\n]*)(.*)\n\n/\2\1/;ta;s/\n//' file
If there are no strings of the length required to reverse, bail out.
Prepend and append newlines to all required strings.
Insert a newline at the start of the pattern space (PS). The PS is divided into two parts, the first line will contain the current word being reversed. The remainder will contain the original line.
Each character of the word to be reversed is inserted at the front of the first line and removed from the original line. When all the characters in the word have been processed, the original word will have gone and only the bordering newlines will exist. These double newlines are then replaced by the word in the first line and the process is repeated until all words have been processed. Finally the newline introduced to separate the working line and the original is removed and the PS is printed.
N.B. This method may be used to reverse strings of varying string length i.e. by changing the first regexp strings of any number can be reversed. Also strings between two lengths may also be reversed e.g. /\<w{2,4}\>/ will change all words between 2 and 4 character length.
It's a recurrent problem so somebody created a bash command called "rev".
echo "$(echo the | rev) $(echo year | rev) $(echo was | rev) $(echo 1815 | rev)".
OR
echo "the year was 1815." | rev | tr ' ' '\n' | tac | tr '\n' ' '
How can I remove the set of characters from a specified length in a file using shell script.
Example:
Filename : abc.txt
helloshell
Now how can I remove characters starting from 8 to 10 (the ell at the end)?
I have tried sed -r command on Linux servers but it's not working on AIX servers.
Linux command:
sed -r 's/.(.{3}).*/\1/' filename.txt
With Bash (extract substring from 0 to 7th character):
str="helloshell"
echo ${str:0:7}
With sed (removes 3 characters starting from 7th position) :
str="helloshell"
startpos=7;
nbchar=3;
echo "$str" | sed "s/^\(.\{$startpos\}\).\{$nbchar\}\(.*\)/\1\2/"
$ sed -r 's/^(.{7}).{0,3}(.*)$/\1\2/g'
helloshell
hellosh
1234567890
1234567
$
{0,3} ensures 3 or less chars from 8th position (0,3 implies remove only if present and hence also removes 1/2/3 chars from 8th position) i.e match and remove minimum 0 chars(for no chars) and maximum 3 chars
If you want exactly only 3 chars to removed from 8th position use {3} but it wont remove chars from 8th position if there are less than 3 chars, eg:
$ sed -r 's/^(.{7}).{3}(.*)$/\1\2/g'
123456789
123456789
$
Edit1:
you can use this instead without the -r switch and some escaping: sed 's/^\(.\{7\}\).\{0,3\}\(.*\)$/\1\2/g'
for performing the above operation only lines starting with BH, you can add a restriction for the substitute command like this:
sed '/^BH/s/^\(.\{7\}\).\{0,3\}\(.*\)$/\1\2/g'
/^BH/s.. would ensure substitution is performed only on lines starting with BH
$ sed '/^BH/s/^\(.\{7\}\).\{0,3\}\(.*\)$/\1\2/g'
BHhelloshell
BHhelloll
BH123456789
BH123459
helloshell
helloshell
$
To exclude BH while counting you can use:
$ sed '/^BH/s/^BH\(.\{7\}\).\{0,3\}\(.*\)$/BH\1\2/g'
BHhelloshell
BHhellosh
BH123456789
BH1234567
helloshell
helloshell
$
Try this if you are always expecting the last 3 to be deleted.
echo helloshell | sed 's/...$//'
For 8-10 try this:
echo helloshell | sed 's/\(.\{7\}\).../\1/'
echo helloshellHowAreYou | sed 's/\(.\{7\}\).../\1/'
For AIX you may need to remove the \ from \{
sed 's/\(.{7}\).../\1/'
And if there is a pattern that you want in the search string you have to adjust the value within in the \( and \)
I want to write a small script to generate the location of a file in an NGINX cache directory.
The format of the path is:
/path/to/nginx/cache/d8/40/32/13febd65d65112badd0aa90a15d84032
Note the last 6 characters: d8 40 32, are represented in the path.
As an input I give the md5 hash (13febd65d65112badd0aa90a15d84032) and I want to generate the output: d8/40/32/13febd65d65112badd0aa90a15d84032
I'm sure sed or awk will be handy, but I don't know yet how...
This awk can make it:
awk 'BEGIN{FS=""; OFS="/"}{print $(NF-5)$(NF-4), $(NF-3)$(NF-2), $(NF-1)$NF, $0}'
Explanation
BEGIN{FS=""; OFS="/"}. FS="" sets the input field separator to be "", so that every char will be a different field. OFS="/" sets the output field separator as /, for print matters.
print ... $(NF-1)$NF, $0 prints the penultimate field and the last one all together; then, the whole string. The comma is "filled" with the OFS, which is /.
Test
$ awk 'BEGIN{FS=""; OFS="/"}{print $(NF-5)$(NF-4), $(NF-3)$(NF-2), $(NF-1)$NF, $0}' <<< "13febd65d65112badd0aa90a15d84032"
d8/40/32/13febd65d65112badd0aa90a15d84032
Or with a file:
$ cat a
13febd65d65112badd0aa90a15d84032
13febd65d65112badd0aa90a15f1f2f3
$ awk 'BEGIN{FS=""; OFS="/"}{print $(NF-5)$(NF-4), $(NF-3)$(NF-2), $(NF-1)$NF, $0}' a
d8/40/32/13febd65d65112badd0aa90a15d84032
f1/f2/f3/13febd65d65112badd0aa90a15f1f2f3
With sed:
echo '13febd65d65112badd0aa90a15d84032' | \
sed -n 's/\(.*\([0-9a-f]\{2\}\)\([0-9a-f]\{2\}\)\([0-9a-f]\{2\}\)\)$/\2\/\3\/\4\/\1/p;'
Having GNU sed you can even simplify the pattern using the -r option. Now you won't need to escape {} and () any more. Using ~ as the regex delimiter allows to use the path separator / without need to escape it:
sed -nr 's~(.*([0-9a-f]{2})([0-9a-f]{2})([0-9a-f]{2}))$~\2/\3/\4/\1~p;'
Output:
d8/40/32/13febd65d65112badd0aa90a15d84032
Explained simple the pattern does the following: It matches:
(all (n-5 - n-4) (n-3 - n-2) (n-1 - n-0))
and replaces it by
/$1/$2/$3/$0
You can use a regular expression to separate each of the last 3 bytes from the rest of the hash.
hash=13febd65d65112badd0aa90a15d84032
[[ $hash =~ (..)(..)(..)$ ]]
new_path="/path/to/nginx/cache/${BASH_REMATCH[1]}/${BASH_REMATCH[2]}/${BASH_REMATCH[3]}/$hash"
Base="/path/to/nginx/cache/"
echo '13febd65d65112badd0aa90a15d84032' | \
sed "s|\(.*\(..\)\(..\)\(..\)\)|${Base}\2/\3/\4/\1|"
# or
# sed sed 's|.*\(..\)\(..\)\(..\)$|${Base}\1/\2/\3/&|'
Assuming info is a correct MD5 (and only) string
First of all - thanks to all of the responders - this was extremely quick!
I also did my own scripting meantime, and came up with this solution:
Run this script with a parameter of the URL you're looking for (www.example.com/article/76232?q=hello for example)
#!/bin/bash
path=$1
md5=$(echo -n "$path" | md5sum | cut -f1 -d' ')
p3=$(echo "${md5:0-2:2}")
p2=$(echo "${md5:0-4:2}")
p1=$(echo "${md5:0-6:2}")
echo "/path/to/nginx/cache/$p1/$p2/$p3/$md5"
This assumes the NGINX cache has a key structure of 2:2:2.
I want to get the string between <sometag param=' and '>
I tried to use the method from Get any string between 2 string and assign a variable in bash to get the "x":
echo "<sometag param='x'><irrelevant stuff='nonsense'>" | tr "'" _ | sed -n 's/.*<sometag param=_\(.*\)_>.*/\1/p'
The problem (apart from low efficiency because I just cannot manage to escape the apostrophe correctly for sed) is that sed matches the maximum, i.e. the output is:
x_><irrelevant stuff=_nonsense
but the correct output would be the minimum-match, in this example just "x"
Thanks for your help
You are probably looking for something like this:
sed -n "s/.*<sometag param='\([^']*\)'>.*/\1/p"
Test:
echo "<sometag param='x'><irrelevant stuff='nonsense'>" | sed -n "s/.*<sometag param='\([^']*\)'>.*/\1/p"
Results:
x
Explanation:
Instead of a greedy capture, use a non-greedy capture like: [^']* which means match anything except ' any number of times. To make the pattern stick, this is followed by: '>.
You can also use double quotes so that you don't need to escape the single quotes. If you wanted to escape the single quotes, you'd do this:
-
... | sed -n 's/.*<sometag param='\''\([^'\'']*\)'\''>.*/\1/p'
Notice how that the single quotes aren't really escaped. The sed expression is stopped, an escaped single quote is inserted and the sed expression is re-opened. Think of it like a four character escape sequence.
Personally, I'd use GNU grep. It would make for a slightly shorter solution. Run like:
... | grep -oP "(?<=<sometag param=').*?(?='>)"
Test:
echo "<sometag param='x'><irrelevant stuff='nonsense'>" | grep -oP "(?<=<sometag param=').*?(?='>)"
Results:
x
You don't have to assemble regexes in those cases, you can just use ' as the field separator
in="<sometag param='x'><irrelevant stuff='nonsense'>"
IFS="'" read x whatiwant y <<< "$in" # bash
echo "$whatiwant"
awk -F\' '{print $2}' <<< "$in" # awk
How would I find the first letter of a word contained within a string using bash.
For example
Code:
str="my-custom-string'
I would want to find m,c,s. I know how to find the very first letter, but this is slightly more complicated.
Many thanks,
$ echo 'my-custom-string' | egrep -o '\b\w'
m
c
s
Pure Bash using parameter substitution. Remove minus, select first character of each word:
str="my-custom-string"
for word in ${str//-/ }; do
echo "${word:0:1}"
done
Result
m
c
s
Here's a sed version:
echo 'my-custom-string' | sed 's/\(^\|-\)\(.\)[^-]*/\2\n/g'
This might work for you (GNU sed);
echo 'my-custom-string' | sed 's/\B.//g;y/-/,/'
m,c,s
or:
echo 'my-custom-string' | sed 's/\B.//g;y/-/\n/'
m
c
s