Unmask data from matrix linux shell - shell

i have 2 file.
analizeddata.txt:
A001->A002->A003->A004
A001->A005->A007
A022->A033
[...]
and
matrix.txt:
A001|Scott
A002|Bob
A003|Mark
A004|Jane
A005|Elion
A007|Brooke
A022|Meggie
A023|Tif
[..]
How i can replace in analizeddata.txt, or obtain a new file, with the second column of matrix.txt?
The expected output file will be as:
Scott->Bob->Mark->Jane
Scott->Elion->Brooke
Meggie->Tif
[...]
Thanks

Just use sed to replace the string what you want.
sed 's/|/\//g' matrix.txt will generate the replace pattern likes A001/Scott which will be used as regexp/replacement of the second sed s/regexp/replacement/ command.
sed -i option will update directly analizeddata.txt file, back up it before exec this command.
for replace_mode in $(sed 's/|/\//g' matrix.txt); do sed -i 's/'$replace_mode'/g' analizeddata.txt; done

Suggesting awk script:
awk -F"|" 'FNR==NR{arr[$1]=$2;next}{for(i in arr)gsub(i,arr[i])}1' matrix.txt analizeddata.txt
with provided sample data, results:
Scott->Bob->Mark->Jane
Scott->Elion->Brooke
Meggie->A033

Related

Search Replace Sed using wild card

Hello I'm sort of new to scripting and have the following problem
I need to replace 20069.1216.0
HintPath..\packages\String.20069.1216.0\lib\net\Thoo.Tkc.dll/HintPath
This works fine at replacing 20069.1216.0 with whatever is provided in $2
`xargs sed -i 's/String.20.........0/String.'"${2}"'/g'`
I need a way for sed to search for **"String.*\lib\net\"**where anything in between **String.** and **\lib...** is wildcard
This what i have tried
sed -i 's/String.*\/String.'"${2}"'/g'
sed -i 's/String.*\\/String.'"${2}"'/g'
sed -i 's/String.\(.*\)\\/String.'"${2}"'/g'
I'll assume that you call sed inside a function.
So try this code:
#!/bin/bash
replace() {
echo 'HintPath..\packages\String.20069.1216.0\lib\net\Thoo.Tkc.dll/HintPath' |\
sed "s/String\.[0-9.]*/String\.${2}/"
}
replace dont_care filename
If your path may contain --SNAPSHOT this solution should work for you:
#!/bin/bash
replace() {
echo 'HintPath..\packages\String.20069.1216.0--SNAPSHOT\lib\net\Thoo.Tkc.dll/HintPath' |\
sed "s/String[0-9.]*\(--SNAPSHOT\)\{0,1\}/String\.${2}/"
}
replace dont_care filename

Replace string in place by another string when line matching exactly another string with sed

In the following example (example.txt)
name1 stringA1 stringB1 stringC1
name2 stringA2 stringB2 stringC2
I'd like to replace in file 'stringB1' by 'new_string' lines that are matching 'name1' using variable.
I succeed to print lines of interest using:
NAME="name1"
sed -n "/^\<${NAME}\>/p" example.txt
But I'm having problem to print lines when I'm trying to make the substitution using bash variables:
NAME="name1"
STRING_TO_REPLACE="stringB1"
NEW_STRING="new_string"
sed -n "/^\<${NAME}\>/s/\<${STRING_TO_REPLACE}>/\<${NEW_STRING}\>/" example.txt
The final idea is to replace the string in file directly using sed -i option
sed -i "/^\<${NAME}\>/s/\<${STRING_TO_REPLACE}>/\<${NEW_STRING}\>/" example.txt

Duplicate and modify words in bash with sed

I'm on Linux OS. I have a file to modify in my bash script.
My original file is like that:
...
ERIC-1898
HELENE-5456
THOMAS-54565
IRON-06516
...
And I'd like to modify this file with duplicate words (and -SYSTEM- word in second field), and add double quotes.
So, the result has to be like that:
...
"ERIC-1898" "ERIC-SYSTEM-1898"
"HELENE-5456" "HELENE-SYSTEM-5456"
"THOMAS-54565" "THOMAS-SYSTEM-54565"
"IRON-06516" "IRON-SYSTEM-06516"
...
How can I do that, for example with sed?
With sed and two capture groups:
$ sed 's/\(.*-\)\(.*\)/"&" "\1SYSTEM-\2"/' infile
"ERIC-1898" "ERIC-SYSTEM-1898"
"HELENE-5456" "HELENE-SYSTEM-5456"
"THOMAS-54565" "THOMAS-SYSTEM-54565"
"IRON-06516" "IRON-SYSTEM-06516"
Assuming that there is exactly one hyphen per input line.
awk solution:
awk -F'-' '{printf("\"%s\" \"%s-SYSTEM-%s\"\n", $1FS$2,$1,$2)}' file
The output would be like:
"ERIC-1898" "ERIC-SYSTEM-1898"
"HELENE-5456" "HELENE-SYSTEM-5456"
"THOMAS-54565" "THOMAS-SYSTEM-54565"
"IRON-06516" "IRON-SYSTEM-06516"
Not using external program:
#!/bin/bash
IFS=$'-'
while read -r first second;do
echo "\"$first-$second\" \"$first-SYSTEM-$second\""
done <infile
awk '{sub(/ /,"\" \"");print "\042" $0 "\042"}' file
"ERIC-1898" "ERIC-SYSTEM-1898"
"HELENE-5456" "HELENE-SYSTEM-5456"
"THOMAS-54565" "THOMAS-SYSTEM-54565"
"IRON-06516" "IRON-SYSTEM-06516"

How do I replace text using a variable in a shell script

I have a variable with a bunch of data.
text = "ABCDEFGHIJK"
file = garbage.txt //iiuhdsfiuhdsihf]sdiuhdfoidsoijsf
What I would like to do is replace the ] charachter in file with text. I've tried using sed but I keep getting odd errors.
output should be:
//iiuhdsfiuhdsihfABCDEFGHIJKsdiuhdfoidsoijsf
Just need to escape the ] character with a \ in regex:
text="ABCDEFGHIJK"
sed "s/\(.*\)\]\(.*\)/\1$text\2/" file > file.changed
or, for in-place editing:
sed -i "s/\(.*\)\]\(.*\)/\1$text\2/" file
Test:
sed "s/\(.*\)\]\(.*\)/\1$text\2/" <<< "iiuhdsfiuhdsihf]sdiuhdfoidsoijsf"
# output => iiuhdsfiuhdsihfABCDEFGHIJKsdiuhdfoidsoijsf
There is always the bash way that should work in your osx:
filevar=$(cat file)
echo "${filevar/]/$text}" #to replace first occurence
OR
echo "${filevar//]/$text}" #to replace all occurences
In my bash i don't even have to escape ].
By the way, the simple sed does not work?
$ a="AA"
$ echo "garbage.txt //iiuhdsfiuhdsihf]sdiuhdfoidsoijsf" |sed "s/]/$a/g"
garbage.txt //iiuhdsfiuhdsihfAAsdiuhdfoidsoijsf

sed multiple replacements with line range

I have a file with below records
user1,fuser1,luser1,user1#test.com,data,user1
user2,fuser2,luser2,user2#test.com,data,user2
user3,fuser3,luser3,user3#test.com,data,user3
I wanted to perform some text replacements from
user1,fuser1,luser1,user1#test.com,data,user1
to
New_user1,New_fuser1,New_luser1,New_user1#test.com,data,New_user1
so I wrote below sed script.
sed -i -e 's/user/New_user/g; s/fuser/New_fuser/g; s/luser/New_luser/g' file
This works perfect. Now I have a requirement that I want to replace in specific line range.
start=2
end=3
sed -i -e ''${start},${end}'s/user/New_user/g; s/fuser/New_fuser/g; s/luser/New_luser/g' file
but this command is replacing pattern in all lines. example output is,
user1,New_fuser1,New_luser1,user1#test.com,data,New_user1
user2,New_fuser2,New_luser2,user2#test.com,data,New_user2
user3,New_fuser3,New_luser3,user3#test.com,data,New_user3
Looks like range is getting applied only to first expression and remaining expressions are getting applied on whole file. How to apply this range to all expressions?
You can use awk variables to use for this functionality, controlling the row and column numbers used for replacing
awk -vFS="," -vOFS="," -v columnStart=2 -v columnEnd=3 -v rowStart=1 -v rowEnd=2 \
'NR>=rowStart&&NR<=rowEnd{for(i=columnStart; i<=columnEnd; i++) \
$i="New_"$i; print }' file
where the awk variables columnStart, columnEnd, rowStart and rowStart determine which columns and rows to replace with , as the de-limiter adopted.
For your input file:-
$ cat input-file
user1,fuser1,luser1,user1#test.com,data,user1
user2,fuser2,luser2,user2#test.com,data,user2
user3,fuser3,luser3,user3#test.com,data,user3
Assuming I want to do replacement in lines 2 and 3 from columns 3-4, I can set-up my awk as
awk -vFS="," -vOFS="," -v columnStart=3 -v columnEnd=4 -v rowStart=2 -v rowEnd=3 \
'NR>=rowStart&&NR<=rowEnd{for(i=columnStart; i<=columnEnd; i++) \
$i="New_"$i; print }' file
user2,fuser2,New_luser2,New_user2#test.com,data,user2
user3,fuser3,New_luser3,New_user3#test.com,data,user3
To apply on the say the last column, set the columnStart and columnEnd to the same value e.g. say on column 6 and on last line only.
awk -vFS="," -vOFS="," -v columnStart=6 -v columnEnd=6 -v rowStart=3 -v rowEnd=3 \
'NR>=rowStart&&NR<=rowEnd{for(i=columnStart; i<=columnEnd; i++) \
$i="New_"$i; print }' file
user3,fuser3,luser3,user3#test.com,data,New_user3
When using GNU Sed (present on Ubuntu, probably Debian, and probably others).
There is a feature which makes this easy:
https://www.gnu.org/software/sed/manual/sed.html#Common-Commands
A group of commands may be enclosed between { and } characters. This
is particularly useful when you want a group of commands to be
triggered by a single address (or address-range) match.
Example: perform substitution then print the second input line:
$ seq 3 | sed -n '2{s/2/X/ ; p}'
X
Given the original question, this should do the trick:
sed -i -e '2,3 {s/user/New_user/g; s/fuser/New_fuser/g; s/luser/New_luser/g}' file
The following works for me:
START=2
NUM=1
sed -i -e "$START,+${NUM} s/user/New_user/g; $START,+${NUM} s/fuser/New_fuser/g; $START,+${NUM} s/luser/New_luser/g" file
As you can see, there are several changes:
The line range has to be present at each expression
The range should be represented (in this case) as the start line number and number of lines (the number of affected lines is NUM+1)
You put extra apostrophe symbols.
Using a single s command:
start=1
end=2
sed -e "$start,$end s/\([fl]*\)user/New_\1user/g" file
[fl]*user will match user with optional f or l first letter
output:
New_user1,New_fuser1,New_luser1,New_user1#test.com,data,New_user1
New_user2,New_fuser2,New_luser2,New_user2#test.com,data,New_user2
user3,fuser3,luser3,user3#test.com,data,user3

Resources