When running perl -n or perl -p, each command line argument is taken as a file to be opened and processed line by line. If you want to pass command line switches to that script, how can I do that?
There are three primary ways of passing information to Perl without using STDIN or external storage.
Arguments
When using -n or -p, extract the arguments in the BEGIN block.
perl -ne'BEGIN { ($x,$y)=splice(#ARGV,0,2) } f($x,$y)' -- "$x" "$y" ...
Command-line options
In a full program, you'd use Getopt::Long, but perl -s will do fine here.
perl -sne'f($x,$y)' -- -x="$x" -y="$y" -- ...
Environment variables
X="$x" Y="$y" perl -ne'f($ENV{X},$ENV{Y})' -- ...
Here is a short example program (name it t.pl), how you can do it:
#!/bin/perl
use Getopt::Std;
BEGIN {
my %opts;
getopts('p', \%opts);
$prefix = defined($opts{'p'}) ? 'prefix -> ' : '';
}
print $prefix, $_;
Call it like that:
perl -n t.pl file1 file2 file3
or (will add a prefix to every line):
perl -n t.pl -p file1 file2 file3
Related
I've got a whitelist file of env variables that looks like this:
ENV_A
ENV_B
ENV_C
I have another file filetoreplace.txt that (let's say) looks like this:
asdfasdfasdfasdf{{ENV_A}}asdfasdfasdfasdfasdf
adsfasdf
asdf{{ENV_B}}
asdf{{ENV_A}}
adsfasdfdsfdf{{ENV_C}}
I want to come up with a (one line) sed command that reads the variable names from the file and does the replacement.
For this environment
ENV_A=1
ENV_B=2
ENV_C=3
This would be the output given the above file
asdfasdfasdfasdf1asdfasdfasdfasdfasdf
adsfasdf
asdf2
asdf1
adsfasdfdsfdf3
I think it's kind of similar to this:
< whitelist | while read var; do
sed -i 's/{{'"$var"'}}/'${!var}'/g' filetoreplace.txt
done
Looking for a bit of code golf to do this in a one liner here
Got 2 impractical variants based on your idea with $var\${!var}
First, obvy
vars=( $(cat whitelist) ); for var in ${vars[#]}; { sed "s|{{$var}}|${!var}|g" -i filetoreplace.txt; }
Second, monstrous)
vars=( $(cat filetoreplace.txt) ); for var in ${vars[#]}; { env=${var//[!ENV_ABC]/}; [[ $env ]] && val=${!env} || var=; echo ${var//"{{$env}}"/$val}; }
This is a relatively simple alternative solution :
#!/usr/bin/env bash
bash -c "source whitelist; cat << EOF
$(perl -pe 's/\{(\{.*?})}/\$$1/' filetoreplace.txt)
EOF
"
The aim of the perl script is to transform {{ENV_A}} to ${ENV_A}
I am new in bash scripting and I need help with awk. So the thing is that I have a property file with version inside and I want to update it.
version=1.1.1.0
and I use awk to do that
file="version.properties"
awk -F'["]' -v OFS='"' '/version=/{
split($4,a,".");
$4=a[1]"."a[2]"."a[3]"."a[4]+1
}
;1' $file > newFile && mv newFile $file
but I am getting strange result version="1.1.1.0""...1
Could someone help me please with this.
You mentioned in your comment you want to update the file in place. You can do that in a one-liner with perl:
perl -pe '/^version=/ and s/(\d+\.\d+\.\d+\.)(\d+)/$1 . ($2+1)/e' -i version.properties
Explanation
-e is followed by a script to run. With -p and -i, the effect is to run that script on each line, and modify the file in place if the script changes anything.
The script itself, broken down for explanation, is:
/^version=/ and # Do the following on lines starting with `version=`
s/ # Make a replacement on those lines
(\d+\.\d+\.\d+\.)(\d+)/ # Match x.y.z.w, and set $1 = `x.y.z.` and $2 = `w`
$1 . ($2+1)/ # Replace x.y.z.w with a copy of $1, followed by w+1
e # This tells Perl the replacement is Perl code rather
# than a text string.
Example run
$ cat foo.txt
version=1.1.1.2
$ perl -pe '/^version=/ and s/(\d+\.\d+\.\d+\.)(\d+)/$1 . ($2+1)/e' -i foo.txt
$ cat foo.txt
version=1.1.1.3
This is not the best way, but here's one fix.
Test case
I am assuming the input file has at least one line that is exactly version=1.1.1.0.
$ awk -F'["]' -v OFS='"' '/version=/{
> split($4,a,".");
> $4=a[1]"."a[2]"."a[3]"."a[4]+1
> }
> ;1' <<<'version=1.1.1.0'
Output:
version=1.1.1.0"""...1
The """ is because you are assigning to field 4 ($4). When you do that, awk adds field separators (OFS) between fields 1 and 2, 2 and 3, and 3 and 4. Three OFS => """, in your example.
Minimal change
$ awk -F'["]' -v OFS='"' '/version=/{
split($1,a,".");
$1=a[1]"."a[2]"."a[3]"."a[4]+1;
print
}
' <<<'version=1.1.1.0'
version=1.1.1.1
Two changes:
Change $4 to $1
Since the input field separator (-F) is ["], $4 is whatever would be after the third " (if there were any in the input). Therefore, split($4, ...) splits an empty field. The contents of the line, before the first " (if any), are in $1.
print at the end instead of ;1
The 1 after the closing curly brace is the next condition, and there is no action specified. The default action is to print the current line, as modified, so the 1 triggers printing. Instead, just print within your action when you are done processing. That way your action is self-contained. (Of course, if you needed to do other processing, you might want to print later, after that processing.)
You can use the = as the delimiter, like this:
awk -F= -v v=1.0.1 '$1=="version"{printf "version=\"%s\"\n", v}' file.properties
I want to be able to modify db001 with a string I pass into the command via CLI. At any given time db001 could be a different value so I can't just look for that value.
./myscript modify_db <new value>
myfile.txt
./myscript modify_db mynewdbname002
Before: database_node=db001.mydomain.local
After: database_node=mynewdbname002.mydomain.local
./myscript modify_db db003
Before: database_node=mynewdbname002.mydomain.local
After: database_node=db003.mydomain.local
You can use this sed command inside your script:
sed "s/^\(database_node=\)[^.]*/\1$1/" file
Example:
s='database_node=db001.mydomain.local'
repl() {
sed "s/^\(database_node=\)[^.]*/\1$1/" <<< "$s";
}
and call it as:
repl mynewdbname002
database_node=mynewdbname002.mydomain.local
repl db003
database_node=db003.mydomain.local
You could have a script like, just like below taking an input argument having the replacement value,
#!/bin/bash
perl -lpe "s/database_node=(\w+)/database_node=$1/g" file
and just do
./script.sh newdbname
Use the -i flag for in-place replacement and -i.bak for in-place replacement with a backup of your original file
perl -lpe -i.bak "s/database_node=(\w+)/database_node=$1/g" file
(or) with a simple bash function
function replaceFile() {
perl -lpe -i.bak "s/database_node=(\w+)/database_node=$1/g" file
}
I would avoid trying to produce the new state from the previous state and rather just use a template :
function modify_db() {
echo "database_node=$1.mydomain.local"
}
I use echo here for illustration but you should obviously do whatever you want to do with the "database_node=$1.mydomain.local".
Supposing it should modify the only line starting with database_node from a file db_conf after having printed the old value :
function modify_db() {
echo "Before: $(grep '^database_node=' db_conf)"
sed -i "s/^database_node=.*\.mydomain\.local/database_node=$1.mydomain.local/" db_conf
echo "After: $(grep '^database_node=' db_conf)"
}
I am creating a bash script to modify and summarize information with grep and sed. But it gets stuck.
#!/bin/bash
# This script extracts some basic information
# from text files and prints it to screen.
#
# Usage: ./myscript.sh </path/to/text-file>
#Extract lines starting with ">#HWI"
ONLY=`grep -v ^\>#HWI`
#replaces A and G with R in lines
ONLYR=`sed -e s/A/R/g -e s/G/R/g $ONLY`
grep R $ONLYR | wc -l
The correct way to write a shell script to do what you seem to be trying to do is:
awk '
!/^>#HWI/ {
gsub(/[AG]/,"R")
if (/R/) {
++cnt
}
END { print cnt+0 }
' "$#"
Just put that in the file myscript.sh and execute it as you do today.
To be clear - the bulk of the above code is an awk script, the shell script part is the first and last lines where the shell just calls awk and passes it the input file names.
If you WANT to have intermediate variables then you can create/print them with:
awk '
!/^>#HWI/ {
only = $0
onlyR = only
gsub(/[AG]/,"R",onlyR)
print "only:", only
print "onlyR:", onlyR
if (/R/) {
++cnt
}
END { print cnt+0 }
' "$#"
The above will work robustly, portably, and efficiently on all UNIX systems.
First of all, and as #fedorqui commented - you're not providing grep with a source of input, against which it will perform line matching.
Second, there are some problems in your script, which will result in unwanted behavior in the future, when you decide to manipulate some data:
Store matching lines in an array, or a file from which you'll later read values. The variable ONLY is not the right data structure for the task.
By convention, environment variables (PATH, EDITOR, SHELL, ...) and internal shell variables (BASH_VERSION, RANDOM, ...) are fully capitalized. All other variable names should be lowercase. Since
variable names are case-sensitive, this convention avoids accidentally overriding environmental and internal variables.
Here's a better version of your script, considering these points, but with an open question regarding what you were trying to do in the last line : grep R $ONLYR | wc -l :
#!/bin/bash
# This script extracts some basic information
# from text files and prints it to screen.
#
# Usage: ./myscript.sh </path/to/text-file>
input_file=$1
# Read lines not matching the provided regex, from $input_file
mapfile -t only < <(grep -v '^\>#HWI' "$input_file")
#replaces A and G with R in lines
for((i=0;i<${#only[#]};i++)); do
only[i]="${only[i]//[AG]/R}"
done
# DEBUG
printf '%s\n' "Here are the lines, after relpace:"
printf '%s\n' "${only[#]}"
# I'm not sure what you were trying to do here. Am I gueesing right that you wanted
# to count the number of R's in ALL lines ?
# grep R $ONLYR | wc -l
I have a script chk.awk to which I want to pass some command line arguments. It has awk statements, sed command etc. Just for example I have taken a small program below to which I want to pass command line arguments.
#!/bin/bash
var1=$1
gawk '
BEGIN {
printf "argc = %d\n argv0=%s\n argv1=%s\n var1=%s\n",ARGC,ARGV[0],ARGV[1],$var1
}'
But when I try :
$ sh chk.awk 10 20
argc = 1
argv0=gawk
argv1=
var1=
Above I tried to display the command line arguments by both ways i.e. argv & $1, but none of them work. Can anyone let me know where I am going wrong here? What is the correct way to do that?
The problem is that you give arguments to the shell script, but not to the awk script.
You must add "$#" to the call of gawk.
#!/bin/bash
var1=$1
gawk '
BEGIN {
printf "argc = %d\n argv0=%s\n argv1=%s\n var1=%s\n",ARGC,ARGV[0],ARGV[1],$var1
}' "$#"
Otherwise you will your arguments in the shell-script and they will be not passed to gawk.
Update 1
If you have additional args (e.g. filenames that are to be processed),
you must remove the first portition of args first (in the BEGIN section):
#!/bin/bash
var1=$1
gawk '
BEGIN {
printf "argc = %d\n argv0=%s\n argv1=%s\n var1=%s\n",ARGC,ARGV[0],ARGV[1],$var1;
delete ARGV[1]
}' "$#" filename