I have a trivial problem with regular expression in bash.
#!/bin/bash
FNAME=$1
echo ${FNAME//.*\/tests\//}
I want to remove everything before /test/ including the /test/ as well. Because of some reasons ".*" doesn't work.
$ ./eclipse/unittest.sh /foo/tests/bar
/foo/tests/bar
How do I select anything in bash reg exp?
You can use # followed by a pattern to remove everything up to and including the pattern. It will use the shortest match:
function f {
echo ${1#*/tests/}
}
$ f /foo/tests/bar
bar
$ f /foo/tests/bar/tests/last
bar/tests/last
If you want to use the longest match, you can use ##:
function f {
echo ${1##*/tests/}
}
$ f /foo/tests/bar
bar
$ f /foo/tests/bar/tests/last
last
Related
Sometimes I have a one-liner that I am repeating many times for a particular task, but will likely never use again in the exact same form. It includes a file name that I am pasting in from a directory listing. Somewhere in between and creating a bash script I thought maybe I could just create a one-liner function at the command line like:
numresults(){ ls "$1"/RealignerTargetCreator | wc -l }
I've tried a few things like using eval, using numresults=function..., but haven't stumbled on the right syntax, and haven't found anything on the web so far. (Everything coming up is just tutorials on bash functions).
Quoting my answer for a similar question on Ask Ubuntu:
Functions in bash are essentially named compound commands (or code
blocks). From man bash:
Compound Commands
A compound command is one of the following:
...
{ list; }
list is simply executed in the current shell environment. list
must be terminated with a newline or semicolon. This is known
as a group command.
...
Shell Function Definitions
A shell function is an object that is called like a simple command and
executes a compound command with a new set of positional parameters.
... [C]ommand is usually a list of commands between { and }, but
may be any command listed under Compound Commands above.
There's no reason given, it's just the syntax.
Try with a semicolon after wc -l:
numresults(){ ls "$1"/RealignerTargetCreator | wc -l; }
Don't use ls | wc -l as it may give you wrong results if file names have newlines in it. You can use this function instead:
numresults() { find "$1" -mindepth 1 -printf '.' | wc -c; }
You can also count files without find. Using arrays,
numresults () { local files=( "$1"/* ); echo "${#files[#]}"; }
or using positional parameters
numresults () { set -- "$1"/*; echo "$#"; }
To match hidden files as well,
numresults () { local files=( "$1"/* "$1"/.* ); echo $(("${#files[#]}" - 2)); }
numresults () { set -- "$1"/* "$1"/.*; echo $(("$#" - 2)); }
(Subtracting 2 from the result compensates for . and ...)
You can get a
bash: syntax error near unexpected token `('
error if you already have an alias with the same name as the function you're trying to define.
The easiest way maybe is echoing what you want to get back.
function myfunc()
{
local myresult='some value'
echo "$myresult"
}
result=$(myfunc) # or result=`myfunc`
echo $result
Anyway here you can find a good how-to for more advanced purposes
I have few log files that look like:
#LOGa# 180.149.126.169 ## 85 with value 350.00000000000000000000 due brand: 350.00000000000000000000 country: 0 {2020-11-26_11-01-00}
#DETAILS_hits# 180.149.126.169 ## hits=([brand/17]="1" [brand/18]="1" [no_brand]="1" ) {2020-11-26_11-01-00}
#LOG_brand# 180.149.126.169 ## BRANDS=([anyBrand]="1" ) {2020-11-26_11-01-00}
#LOG_country# 180.149.126.169 ## COUNTRY=([anyCountry/17]="1" [anyContinent/18]="1" ) {2020-11-26_11-01-00}
and I want to extract dome values of some especific Log lines
sure I can go with
grep -HiRE "(#LOGa#)(.+)(## )(.+)" --include \myFile.log | while read _ ip _ rank _ value _ _ valueBrand _ _ valueCountry _ ; do printf "%.0f %.0f\n" $valueBrand $valueCountry; done
but isn't a more elegant way,
somethign like
cleanME myFile.log "(#LOGa#)($ip)(## )($rank)(with value)($value)(due brand:)($valueBrand)(country:)($valueCountry)(.*)" "$valueBrand.0f $valueCountry.0f"
sure I can go building a function like this, but I short of remember that it was better ways to do that than grep + while + printf
If Perl happens to be your option, would you please try:
perl -ne '/^#LOGa#\s+([\d.]+)\s+##\s+([\d.]+)\s+with value\s+([\d.]+)\s+due brand:\s+([\d.]+)\s+country:\s+([\d.]+)/ && printf "%.0f %.0f\n", $4, $5' myFile.log
Output for the provided input:
350 0
The option -n tells Perl to process the input file line by line as sed.
The option -e enables a one-liner.
The syntax /regex/ && printf ... prints the arguments only if the line
matches the regex as grep.
The parens within the regex create capture group and the matched substrings
can be referred with $1, $2, ... in order.
I'm not sure this is any better, but consider this:
find . -type f -name myFile.log -print | xargs sed -En 's/^#LOGa# .+ ## .+ with value [0-9.-]+ due brand: ([0-9.-]+) country: ([0-9]+).*$/\1 \2/1;Tx;p;:x'
Explanation:
find -- recursively find in the current directory (.) all files (-type f) with the name myFile.log (-name myFile.log) and -print them. (sed does not have a -R option like grep.) Pipe this to xargs which will, for each one, invoke the stream editor sed, with extended regexp syntax (-E) and no automatic printing of lines (-n). Substitute (s//) the given regexp, using grouping operators to capture the valueBrand and valueCountry, and substituting for the whole line these two values captured in parentheses (\1 \2) in the first occurrence (/1). Then if this substitution does not happen, jump to label x (;Tx); otherwise print the line. Then the label x (;:x) and end (just exits).
I'm not sure if you are intentionally truncating the decimal places in the output or not; to do that you'd have to pass it through a bash printf statement (while read a b; do printf "%.0f %.0f" a b; done) or some other program, or do it another way. Or if you really want to truncate (i.e. not round) to zero decimal places, you could use brand: ([0-9-]+)\.[0-9]* instead of brand: ([0-9.-]+) . This just excludes the decimal point and mantissa from the string and drops it.
Let's suppose I have a directory with this Structure:
I am making a script that removes the letter "x" in those cases when there is one at the end in order to have this:
So, I was thinking to do a For Loop that goes to each directory and then take the last part of the folder path
for d in Fruits_and_Vegetables/*/
do
(cd "$d" && Fruit_or_Vegetable=`basename "$PWD"`)
done
The problem is that I am not sure how to tell to go and take only the last Directory
And then, I was thinking in modifying the string
echo $Fruit_or_Vegetable | awk '{print substr($1, 1, length($1)-1)}' # substr(s, a, b) : it returns b number of chars from string s, starting at position a.
# The parameter b is optional, in which case it means up to the end of the string.
The problem is that I don't know how to tell AWK to consider "Moscato Giallox" as just one word, because when I execute the command it returns "Moscat Giallox" instead of "Moscato Giallo" .. also I guess I have to place an if statement to see if the last letter is x, and only execute the command in those cases.
Could you give me some suggestions, thanks.
shopt and Parameter Expansion parsing can make that pretty easy. Starting with your directory structure above:
$: find #(Fruits|Vegetables)
Fruits
Fruits/Grapes
Fruits/Grapes/Muskat Grape
Fruits/Grapes/Muskat Grape/Muscat Ottonel
Fruits/Grapes/Muskat Grape/Muscato Gallox
Fruits/Mangoes
Fruits/Mangoes/Ataulfo Mango
Fruits/Mangoes/Tommy Atkins Mangox
Vegetables
Vegetables/Potatoes
Vegetables/Potatoes/Ratte Potatox
Vegetables/Potatoes/Yukon Gold Potato
Code for the changes you want.
shopt -s extglob # allow fancy #(...) construct to specify dirs
shopt -s globstar # add double-asterisk for flexible depth
for d in #(Fruits|Vegetables)/**/*x/ # *EDITED* - added trailing / for dirs only
do echo "mv \"$d\" \"${d%x/}/\" " # show the command first
mv "$d" "${d%x/}/" # rename the dirs
done
Output from the echo statements:
mv "Fruits/Grapes/Muskat Grape/Muscato Gallox/" "Fruits/Grapes/Muskat Grape/Muscato Gallo/"
mv "Fruits/Mangoes/Tommy Atkins Mangox/" "Fruits/Mangoes/Tommy Atkins Mango/"
mv "Vegetables/Potatoes/Ratte Potatox/" "Vegetables/Potatoes/Ratte Potato/"
Result:
$: find #(Fruits|Vegetables)
Fruits
Fruits/Grapes
Fruits/Grapes/Muskat Grape
Fruits/Grapes/Muskat Grape/Muscat Ottonel
Fruits/Grapes/Muskat Grape/Muscato Gallo
Fruits/Mangoes
Fruits/Mangoes/Ataulfo Mango
Fruits/Mangoes/Tommy Atkins Mango
Vegetables
Vegetables/Potatoes
Vegetables/Potatoes/Ratte Potato
Vegetables/Potatoes/Yukon Gold Potato
I'm trying to search for a specific string in a file. The string includes a tilde- I am trying to isolate the line that contains the string "~ ca_cert".
This is my script:
#!/bin/bash
LIST=("~ ca_cert" "backup_window")
FILE=./test
for x in "${LIST[#]}"; do
grep $x $FILE
done
When I run it, it returns other lines that contain tildes. For example, in a file that contains the following, it return all of the lines, when my intention is for it to only return the bottom line that contains ~ ca_cert:
./test:./terraform.tfplan: ~ update in-place
./test:./terraform.tfplan: ~ resource "aws_db_instance" "rds_instance" {
./test:./terraform.tfplan: ~ ca_cert_identifier = "rds-ca-2019" -> "rds-ca-2015"
Problem is not quoting pattern i.e. $x in your grep command. That basically runs your command as grep '~' ca_cert ./test and finds all the lines matching ~ with an error.
However you don't really need to run a loop here. Just use grep -f with process substitution:
grep -Ff <(printf '%s\n' "${LIST[#]}") ./test
./terraform.tfplan: ~ ca_cert_identifier = "rds-ca-2019" -> "rds-ca-2015"
I need a command that will help me accomplish what I am trying to do. At the moment, I am looking for all the ".html" files in a given directory, and seeing which ones contain the string "jacketprice" in any of them.
Is there a way to do this? And also, for the second (but separate) command, I will need a way to replace every instance of "jacketprice" with "coatprice", all in one command or script. If this is feasible feel free to let me know. Thanks
find . -name "*.html" -exec grep -l jacketprice {} \;
for i in `find . -name "*.html"`
do
sed -i "s/jacketprice/coatprice/g" $i
done
As for the second question,
find . -name "*.html" -exec sed -i "s/jacketprice/coatprice/g" {} \;
Use recursive grep to search through your files:
grep -r --include="*.html" jacketprice /my/dir
Alternatively turn on bash's globstar feature (if you haven't already), which allows you to use **/ to match directories and sub-directories.
$ shopt -s globstar
$ cd /my/dir
$ grep jacketprice **/*.html
$ sed -i 's/jacketprice/coatprice/g' **/*.html
Depending on whether you want this recursively or not, perl is a good option:
Find, non-recursive:
perl -nwe 'print "Found $_ in file $ARGV\n" if /jacketprice/' *.html
Will print the line where the match is found, followed by the file name. Can get a bit verbose.
Replace, non-recursive:
perl -pi.bak -we 's/jacketprice/coatprice/g' *.html
Will store original with .bak extension tacked on.
Find, recursive:
perl -MFile::Find -nwE '
BEGIN { find(sub { /\.html$/i && push #ARGV, $File::Find::name }, '/dir'); };
say $ARGV if /jacketprice/'
It will print the file name for each match. Somewhat less verbose might be:
perl -MFile::Find -nwE '
BEGIN { find(sub { /\.html$/i && push #ARGV, $File::Find::name }, '/dir'); };
$found{$ARGV}++ if /jacketprice/; END { say for keys %found }'
Replace, recursive:
perl -MFile::Find -pi.bak -we '
BEGIN { find(sub { /\.html$/i && push #ARGV, $File::Find::name }, '/dir'); };
s/jacketprice/coatprice/g'
Note: In all recursive versions, /dir is the bottom level directory you wish to search. Also, if your perl version is less than 5.10, say can be replaced with print followed by newline, e.g. print "$_\n" for keys %found.