awk script for copy/paste part of string [closed] - bash

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 21 days ago.
This post was edited and submitted for review 21 days ago.
Improve this question
Problem:
I have several config files (ProgramName.conf) that need to be edited with awk or sed. The first line contains several letters in place after ":" and end with "]" ProgramName. Or copy filename (it's same as ProgramName). I need to copy ProgramName to the 4th between "command=docker-compose run --rm --name" and "artisan". After conteiner name always word "artisan" in place.
Upd
Thanks for answers, problem solved!
input:
[program:ProgramName]
process_name=
directory=
command=docker-compose run --rm --name artisan some text
target output:
[program:ProgramName]
process_name=
directory=
command=docker-compose run --rm --name ProgramName artisan some text

If ed is available/acceptable, something like:
#!/bin/sh
ed -s file.txt <<-'EOF'
/^\[program:.*\]$/t/^command=/
s/^\[.*:\(.*\)\]$/ \1/
-;+j
,p
Q
EOF
In one-line
printf '%s\n' '/^\[program:.*\]$/t/^command=/' 's/^\[.*:\(.*\)\]$/ \1/' '-;+j' ,p Q | ed -s file.txt
Change Q to w if in-place editing is required.
Remove the ,p to silence the output.

gawk -i inplace '
BEGIN { RS = ORS = ""; FS = OFS = "\n" }
match($1, /^\[program:(.*)\]/, a) {
for (i = 2; i <= NF; ++i)
if ($i ~ /^command=docker-compose run --rm --name/)
$i = "command=docker-compose run --rm --name " a[1]
}
{ print $0 RT }' file
Study the GNU Awk manual for details.

I would harness GNU AWK for this task following way, let file.txt content be then
[program:ProgramName]
process_name=
directory=
command=docker-compose run --rm --name
then
awk 'BEGIN{FS="]|:|\\["}/^\[program:/{name=$3;n=NR}NR==n+3{$0=$0 " " name}{print}' file.txt
gives output
[program:ProgramName]
process_name=
directory=
command=docker-compose run --rm --name ProgramName
Explanation: I inform GNU AWK that field separator (FS) is ] or : or [ then for line starting with [program: I store third column value as name and number of row (NR) as n, then for line which is three lines after thtat happen i.e. number row equals n plus three I append space and name to line and set that as line value, for every line I print it. Be warned that it does modify
to the third line
to comply with your requirement, therefore make sure it is right place do change for all your cases.
(tested in GNU Awk 5.0.1)

Related

Store output of lsblk command key=value into associative array in bash [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
> lsblk -Po mountpoint,label,uuid /dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Output from the lsblk example command:
MOUNTPOINT="/media/user/GParted Live" LABEL="GParted Live" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
I will be using above command in bash and I want to store the key values in partition associative array. The above lsblk output therefore needs to be processed and placed in an associative array
Like -
partition[MOUNTPOINT] should have /media/user/GParted Live
partition[LABEL] should have GParted Live
partition[UUID] should have xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Can someone please help by giving me a readable bash script?
Assumptions:
the lsblk output does not contain double quotes (") embedded in the 'value' strings
for this answer I'm going to place OP's lsblk output into a file (lsblk.out) and use said file as input to the proposed answer.
Sample input data:
$ cat lsblk.out
MOUNTPOINT="/media/user/GParted Live" LABEL="GParted Live" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
We'll start with a sed solution that uses a capture group to break the input into separate lines:
$ sed -E 's/([^ ][A-Z]*=\"[^\"]*\")[ $]/\1\n/g' lsblk.out
MOUNTPOINT="/media/user/GParted Live"
LABEL="GParted Live"
UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
We'll then feed this to a while loop to populate our array:
unset partition
declare -A partition
while IFS="=" read -r idx data
do
partition[${idx}]="${data}"
done < <(sed -E 's/([^ ][A-Z]*=\"[^\"]*\")[ $]/\1\n/g' lsblk.out)
We can then check the results of the operation :
$ typeset -p partition
declare -A partition=([UUID]="\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\"" [LABEL]="\"GParted Live\"" [MOUNTPOINT]="\"/media/user/GParted Live\"" )
Alternatively:
for idx in "${!partition[#]}"
do
echo "${idx} : ${partition[${idx}]}"
done
Which generates:
UUID : "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
LABEL : "GParted Live"
MOUNTPOINT : "/media/user/GParted Live"
If OP wants to strip out the double quotes ("):
while IFS="=" read -r idx data
do
partition[${idx}]="${data//\"/}" # strip out double quotes
done < <(sed -E 's/([^ ][A-Z]*=\"[^\"]*\")[ $]/\1\n/g' lsblk.out)
Then verify:
typeset -p partition
for idx in "${!partition[#]}"
do
echo "${idx} : ${partition[${idx}]}"
done
Which generates:
declare -A partition=([UUID]="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" [LABEL]="GParted Live" [MOUNTPOINT]="/media/user/GParted Live" )
UUID : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
LABEL : GParted Live
MOUNTPOINT : /media/user/GParted Live
OP should be able to feed the lsblk command into the while loop like such:
while IFS="=" read -r idx data
do
partition[${idx}]="${data}"
# partition[${idx}]="${data//\"/}"
done < <(lsblk -Po mountpoint,label,uuid /dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx | sed -E 's/([^ ][A-Z]*=\"[^\"]*\")[ $]/\1\n/g')
NOTE: to remove double quotes (") move the comment (#) up one line

Best way to alter a file in a bash script [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
i have a file that have variables and values
objective: open the file and replace all id by input id
[FILE]
var1 = 2
id = 3
var3 = 5
id = 12
var4 = 5
and i can't replace the id values to new ones.
here's my code, any help or something will help. thanks
#!/bin/bash
filename=$1
uuid=$2
input="./$filename"
# awk -v find="id " -v field="5" -v newval="abcd" 'BEGIN {FS=OFS="="} {if ($1 == find) $field=newval; print $1}' $input
while IFS= read -r line
do
awk -v find="id " -v field="5" -v newval="abcd" 'BEGIN {FS=OFS="="} {if ($1 == find) $field=newval;}' $input
echo $line
done < "$input"
expected output
execute
./myscript.sh file.cnf 77
expected output:
[FILE]
var1 = 2
id = 77
var3 = 5
id = 77
var4 = 5
I think sed is the right tool for this. You can even use its -i switch and update the file in-place.
$ cat file.txt
var1 = 2
id = 3
var3 = 5
id = 12
var4 = 5
$ NEW_ID=1234
$ sed -E "s/(id\s*=\s*)(.+)/\1${NEW_ID}/g" file.txt
var1 = 2
id = 1234
var3 = 5
id = 1234
var4 = 5
The string inside the quotes is a sed script for substituting some text with different text, and its general form is s/regexp/replacement/flags where "regexp" stands for "regular expression".
In the above example, the script looks for the string "id = ..." with any number of spaces or tabs around the "=" character. I divided the regexp into 2 groups (using parentheses) because we only want to replace the part to the right of the "=" character, and I don't think sed allows partial substitutions, so as a workaround I used \1 in the "replacement", which inserts the contents of the 1st group. The ${NEW_ID} actually gets evaluated by the shell so the value of the variable ("1234") is already part of the string by the time sed processes it. The g at the end stands for "global" and is probably redundant in this case. It makes sure that all occurrences of the regex on every line will get replaced; otherwise sed would only replace the first occurrence on each line.
Not sure. Bash scripts are extremely sensitive. I'm guessing your touch is what is causing this issue for a couple of reasons.
First whenever you touch a file name is should not consist of an operand or prefix unliss it is part of the shell script and $filename is shell or inline block quote. Touch is usually used for binaries or high priority data objects.
Second I'd try changing input and adjusted to $done and instead of echoing the $line echo the entire script using esac or end if instead of a do while loop.

Update version number in property file using bash

I am new in bash scripting and I need help with awk. So the thing is that I have a property file with version inside and I want to update it.
version=1.1.1.0
and I use awk to do that
file="version.properties"
awk -F'["]' -v OFS='"' '/version=/{
split($4,a,".");
$4=a[1]"."a[2]"."a[3]"."a[4]+1
}
;1' $file > newFile && mv newFile $file
but I am getting strange result version="1.1.1.0""...1
Could someone help me please with this.
You mentioned in your comment you want to update the file in place. You can do that in a one-liner with perl:
perl -pe '/^version=/ and s/(\d+\.\d+\.\d+\.)(\d+)/$1 . ($2+1)/e' -i version.properties
Explanation
-e is followed by a script to run. With -p and -i, the effect is to run that script on each line, and modify the file in place if the script changes anything.
The script itself, broken down for explanation, is:
/^version=/ and # Do the following on lines starting with `version=`
s/ # Make a replacement on those lines
(\d+\.\d+\.\d+\.)(\d+)/ # Match x.y.z.w, and set $1 = `x.y.z.` and $2 = `w`
$1 . ($2+1)/ # Replace x.y.z.w with a copy of $1, followed by w+1
e # This tells Perl the replacement is Perl code rather
# than a text string.
Example run
$ cat foo.txt
version=1.1.1.2
$ perl -pe '/^version=/ and s/(\d+\.\d+\.\d+\.)(\d+)/$1 . ($2+1)/e' -i foo.txt
$ cat foo.txt
version=1.1.1.3
This is not the best way, but here's one fix.
Test case
I am assuming the input file has at least one line that is exactly version=1.1.1.0.
$ awk -F'["]' -v OFS='"' '/version=/{
> split($4,a,".");
> $4=a[1]"."a[2]"."a[3]"."a[4]+1
> }
> ;1' <<<'version=1.1.1.0'
Output:
version=1.1.1.0"""...1
The """ is because you are assigning to field 4 ($4). When you do that, awk adds field separators (OFS) between fields 1 and 2, 2 and 3, and 3 and 4. Three OFS => """, in your example.
Minimal change
$ awk -F'["]' -v OFS='"' '/version=/{
split($1,a,".");
$1=a[1]"."a[2]"."a[3]"."a[4]+1;
print
}
' <<<'version=1.1.1.0'
version=1.1.1.1
Two changes:
Change $4 to $1
Since the input field separator (-F) is ["], $4 is whatever would be after the third " (if there were any in the input). Therefore, split($4, ...) splits an empty field. The contents of the line, before the first " (if any), are in $1.
print at the end instead of ;1
The 1 after the closing curly brace is the next condition, and there is no action specified. The default action is to print the current line, as modified, so the 1 triggers printing. Instead, just print within your action when you are done processing. That way your action is self-contained. (Of course, if you needed to do other processing, you might want to print later, after that processing.)
You can use the = as the delimiter, like this:
awk -F= -v v=1.0.1 '$1=="version"{printf "version=\"%s\"\n", v}' file.properties

sed multiple replacements with line range

I have a file with below records
user1,fuser1,luser1,user1#test.com,data,user1
user2,fuser2,luser2,user2#test.com,data,user2
user3,fuser3,luser3,user3#test.com,data,user3
I wanted to perform some text replacements from
user1,fuser1,luser1,user1#test.com,data,user1
to
New_user1,New_fuser1,New_luser1,New_user1#test.com,data,New_user1
so I wrote below sed script.
sed -i -e 's/user/New_user/g; s/fuser/New_fuser/g; s/luser/New_luser/g' file
This works perfect. Now I have a requirement that I want to replace in specific line range.
start=2
end=3
sed -i -e ''${start},${end}'s/user/New_user/g; s/fuser/New_fuser/g; s/luser/New_luser/g' file
but this command is replacing pattern in all lines. example output is,
user1,New_fuser1,New_luser1,user1#test.com,data,New_user1
user2,New_fuser2,New_luser2,user2#test.com,data,New_user2
user3,New_fuser3,New_luser3,user3#test.com,data,New_user3
Looks like range is getting applied only to first expression and remaining expressions are getting applied on whole file. How to apply this range to all expressions?
You can use awk variables to use for this functionality, controlling the row and column numbers used for replacing
awk -vFS="," -vOFS="," -v columnStart=2 -v columnEnd=3 -v rowStart=1 -v rowEnd=2 \
'NR>=rowStart&&NR<=rowEnd{for(i=columnStart; i<=columnEnd; i++) \
$i="New_"$i; print }' file
where the awk variables columnStart, columnEnd, rowStart and rowStart determine which columns and rows to replace with , as the de-limiter adopted.
For your input file:-
$ cat input-file
user1,fuser1,luser1,user1#test.com,data,user1
user2,fuser2,luser2,user2#test.com,data,user2
user3,fuser3,luser3,user3#test.com,data,user3
Assuming I want to do replacement in lines 2 and 3 from columns 3-4, I can set-up my awk as
awk -vFS="," -vOFS="," -v columnStart=3 -v columnEnd=4 -v rowStart=2 -v rowEnd=3 \
'NR>=rowStart&&NR<=rowEnd{for(i=columnStart; i<=columnEnd; i++) \
$i="New_"$i; print }' file
user2,fuser2,New_luser2,New_user2#test.com,data,user2
user3,fuser3,New_luser3,New_user3#test.com,data,user3
To apply on the say the last column, set the columnStart and columnEnd to the same value e.g. say on column 6 and on last line only.
awk -vFS="," -vOFS="," -v columnStart=6 -v columnEnd=6 -v rowStart=3 -v rowEnd=3 \
'NR>=rowStart&&NR<=rowEnd{for(i=columnStart; i<=columnEnd; i++) \
$i="New_"$i; print }' file
user3,fuser3,luser3,user3#test.com,data,New_user3
When using GNU Sed (present on Ubuntu, probably Debian, and probably others).
There is a feature which makes this easy:
https://www.gnu.org/software/sed/manual/sed.html#Common-Commands
A group of commands may be enclosed between { and } characters. This
is particularly useful when you want a group of commands to be
triggered by a single address (or address-range) match.
Example: perform substitution then print the second input line:
$ seq 3 | sed -n '2{s/2/X/ ; p}'
X
Given the original question, this should do the trick:
sed -i -e '2,3 {s/user/New_user/g; s/fuser/New_fuser/g; s/luser/New_luser/g}' file
The following works for me:
START=2
NUM=1
sed -i -e "$START,+${NUM} s/user/New_user/g; $START,+${NUM} s/fuser/New_fuser/g; $START,+${NUM} s/luser/New_luser/g" file
As you can see, there are several changes:
The line range has to be present at each expression
The range should be represented (in this case) as the start line number and number of lines (the number of affected lines is NUM+1)
You put extra apostrophe symbols.
Using a single s command:
start=1
end=2
sed -e "$start,$end s/\([fl]*\)user/New_\1user/g" file
[fl]*user will match user with optional f or l first letter
output:
New_user1,New_fuser1,New_luser1,New_user1#test.com,data,New_user1
New_user2,New_fuser2,New_luser2,New_user2#test.com,data,New_user2
user3,fuser3,luser3,user3#test.com,data,user3

Bash how to get text in file [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I file /tmp/txt
contents of the file: aaa aaa aaa _bbb bbb bbb
I need to save the file /tmp/txt_left: aaa aaa aaa
I need to save the file /tmp/txt_right: bbb bbb bbb
!!! attention seeking solutions without the use of variables !!!
awk -F '_' '{print $1> "/tmp/txt_left"; print $2 > "/tmp/txt_right" }' /tmp/txt
You could try cutting the line, slitting on the underscore
Cat /tmp/txt | cut -d_ -f 1 > txt_left
A sed way:
Shorter and quicker:
sed -ne $'h;s/_.*$//;w /tmp/txt_left\n;g;s/^.*_//;w /tmp/txt_right' /tmp/txt
Explained: It could be written:
sed -ne '
h; # hold (copy current line in hold space)
s/_.*$//; # replace from _ to end of line by nothing
w /tmp/txt_left
# Write current line to file
# (filename have to be terminated by a newline)
g; # get (copy hold space to current line buffer)
s/^.*_//; # replace from begin of line to _ by nothing
w /tmp/txt_right
# write
' /tmp/txt
Bash as bash
This is not a real variable, I use first argument element for doing the job and restore argument list once finish:
set -- "$(</tmp/txt)" "$#"
echo >>/tmp/txt_right ${1#*_}
echo >>/tmp/txt_left ${1%_*}
shift
I unshift the string at first place in argument line,
do operation on $1, than shift the argument line so no variable is used and in fine, the argument line return in his original state
... and this is a pure bash solution ;-)
Using bash process substitution, tee, and cut:
tee -a >(cut -d _ -f 0 > /tmp/txt_left) >(cut -d _ -f 1 >/tmp/txt_right) < /tmp/txt

Resources