I am using Unix Shell. How to remove newline character between two specific strings.
For example, input is:
CASE when a in ('abcd','bdcdf') then
Shng,
END as xyz
Output should be:
CASE when a in ('abcd','bdcdf') then Shng END as xyz,
Parse the file line for line and remember when you see a CASE or END.
The code beneath uses a short syntax for an if-statement and an echo that suppresses \n by the -n parameter.
incase=0
cat x.sql | while read -r line; do
[[ ${line} = CASE* ]] && incase=1;
[[ ${line} = END* ]] && incase=0
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} = 1 ]] && echo -n "${line} "
done
EDIT:
When you have nested CASEs (like CASE ... CASE ... END ... END) and all
CASEs start on different lines you can count how deep your nested.
incase=0
cat x.sql | while read -r line; do
[[ ${line} = CASE* ]] && (( incase = incase + 1)) ;
[[ ${line} = END* ]] && (( incase = incase - 1))
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} > 0 ]] && echo -n "${line} "
done
# You might want an extra echo here so your last line will finish with a \n
echo
EDIT 2: Often you can avoid cat (look for UUOC). Here the code is better as
incase=0
cat x.sql | while read -r line; do
[[ ${line} = CASE* ]] && incase=1;
[[ ${line} = END* ]] && incase=0
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} = 1 ]] && echo -n "${line} "
done
EDIT:
When you have nested CASEs (like CASE ... CASE ... END ... END) and all
CASEs start on different lines you can count how deep your nested.
incase=0
while read -r line; do
[[ ${line} = CASE* ]] && (( incase = incase + 1)) ;
[[ ${line} = END* ]] && (( incase = incase - 1))
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} > 0 ]] && echo -n "${line} "
done < x.sql
# You might want an extra echo here so your last line will finish with a \n
echo
Related
I have a file named my.txt:
abc_default_flow
#abc_default_flow -p sam
abc_default_flow -p sam
# abc_default_flow -p david
abc_default_flow -p david -z what_is_it
I want to match a particular line which has multiple strings and want to match the exact line which contains all the strings.
I tried the below piece of code, but as soon as it matches the partial string it comes out of the loop, rather than actual line which contains all the strings.
#!/bin/bash -f
f_name=abc_default_flow
p_name=sam
file_content=./my.txt
#echo "file_content: $file_content"
while IFS= read -r file_line
do
echo $file_line
if [[ $file_line != *"#"* ]] && [[ $file_line != "" ]] && echo $file_line | grep -E "${f_name}|${p_name}"; then
#if [[ $file_line != *"#"* ]] && [[ $file_line != "" ]] && echo $file_line | grep -v "${f_name}\|${p_name}"; then
#if [[ $file_line != *"#"* ]] && [[ $file_line != "" ]] && [[ $file_line =~ $f_name ]] && [[ $file_line =~ $p_name ]]; then
if [[ -z "$p_name" ]]; then
f_name=${f_name}_${p_name}
fi
echo "f_name: ${f_name}"
break
fi
done < $file_content
What would be the right way to grep or use any other process to find the right line within the file?
Update: With the below code I am able to get the output, but is there any simple way with grep or sed or awk to find the result in single line instead of nested if loops.
#!/bin/bash -f
f_name=abc_default_flow
p_name=david
file_content=./my.txt
echo "f_name: $f_name, p_name: $p_name"
while IFS= read -r file_line
do
if [[ $file_line != *"#"* ]] && [[ $file_line != "" ]]; then
echo "l1"
if [[ ! -z "$f_name" ]] && [[ $file_line =~ "$f_name" ]]; then
echo "l2, $file_line, $f_name, $p_name"
if [[ ! -z "$p_name" ]] && [[ $file_line =~ "$p_name" ]]; then
f_name=${f_name}_${p_name}
echo "f_name: ${f_name}"
break
elif [[ ! -z "$p_name" ]] && [[ ! $file_line =~ "$p_name" ]]; then
continue
else
break
fi
fi
fi
done < $file_content
You could use the same [[ ]] style checking for the two strings you're looking for:
if [[ $file_line != *"#"* ]] && [[ $file_line == *"$f_name"* ]] && [[ $file_line == *"$p_name"* ]]; then
...
fi
I removed the empty string check since an empty line won't contain $f_name and $p_name anyways.
If you expect sam will always come after abc_default_flow then you could combine the two checks into a single test:
if [[ $file_line != *"#"* ]] && [[ $file_line == *"$f_name"*"$p_name"* ]]; then
...
fi
If we look at the script as a whole, it'd be nice to get away from the explicit line-by-line loop. Scripts are more idiomatic when they chain together tools that process entire files. Something like:
sed -r '/^\s*#/d' my.txt | grep "$f_name" | grep "$p_name"
i have this function that accepts 3 parameters , ech one contain of 4 numbers and a capital letter for example : "1234A"
and i want to print 1 if the second parameter is bigger than the third one and smaller than the first one ,
i wrote this function that i cutted the 4 numbers in a parameter for each parameter and the letter in diffrent paramater for each one and i began to compare
but the problem it print nothing !!
anyone know how to do things in one if statement rather than two if statements ??
what i did :
function check {
curr_letter=`echo "$1" | cut -c5`
min_letter=`echo "$3" | cut -c5`
sm_letter=`echo "$2" | cut -c5`
curr_nums=`echo "$1" | cut -c1-4`
min_nums=`echo "$3" | cut -c1-4`
sm_nums=`echo "$2" | cut -c1-4`
if [[ sm_nums -eq curr_nums && sm_letter < curr_letter ]] ; then
if [[ sm_nums -eq min_nums && sm_letter > min_letter ]] ; then
echo 1
fi
if [[ sm_nums > min_nums ]] ; then
echo 1
fi
fi
if [[ sm_nums < curr_nums ]] ; then
if [[ sm_nums -eq min_nums && sm_letter > min_letter ]] ; then
echo 1
fi
if [[ sm_nums > min_nums ]] ; then
echo 1
fi
fi
}
i get nothing when i test this in bash , i get an empty line..
this is how i tested it :
p=`check "1617B" "1617A" "0000A"` echo $p
You can omit the $ in variable names within arithmetic context ((...)).
Within [[ ... ]] you cannot omit it.
Instead of calling echo ... | cut -c..., you can easily extract substrings using Bash's very own syntax {var:start:length}.
Within a [[ ... ]] or ((...)),
use == instead of -eq.
Note however that < and > operators sort lexicographically within a [[ ... ]], but numerically in arithmetic context ((...)).
Therefore the string-valued variables (named *_letter in your example)
should be compared within [[ ... ]], the numeric variables (named *_nums in your example) should be compared within ((...)).
Like this:
function check() {
curr_letter=${1:4:1}
min_letter=${3:4:1}
sm_letter=${2:4:1}
curr_nums=${1:0:4}
min_nums=${3:0:4}
sm_nums=${2:0:4}
if (( sm_nums == curr_nums )) && [[ $sm_letter < $curr_letter ]]; then
if (( sm_nums == min_nums )) && [[ $sm_letter > $min_letter ]] ; then
echo 1
fi
if (( sm_nums > min_nums )) ; then
echo 1
fi
fi
if (( sm_nums < curr_nums )) ; then
if (( sm_nums == min_nums )) && [[ $sm_letter > $min_letter ]] ; then
echo 1
fi
if (( sm_nums > min_nums )) ; then
echo 1
fi
fi
}
Lastly, instead of p=`check "1617B" "1617A" "0000A"`; echo $p,
better write like this:
echo $(check "1617B" "1617A" "0000A")
why not just
awk '$3 <= $2 && $2 <= $1 {print 1}'
or if you need a function
check() { awk '$3 <= $2 && $2 <= $1 {print 1}' <<< "$#"; }
or
check() { awk "BEGIN{if($3 <= $2 && $2 <= $1) print 1}"; }
I have been beating my head up about this.
I wanted to loop over a multiline string character by character in bash but was loosing all newlines. First thing I did when i didn't find any obvious error was to run shellcheck on it, it seemed fine with the program.
script.sh:
#!/usr/bin/env bash
transform_single() {
if [[ $# -ne 1 ]]; then
echo 'Error: illegal number of args' 1>&2
fi
equation=''
delim0=0
delim="$1"
while IFS= read -rn1 c; do
if [[ $delim0 -eq 0 ]] && [[ "$c" == "$delim" ]]; then
delim0=1
equation=''
elif [[ $delim0 -ne 0 ]] && [[ "$c" == "$delim" ]]; then
delim0=0
echo -n "$equation" | texmath
elif [[ $delim0 -ne 0 ]]; then
equation="$equation$c"
else
echo -n "$c"
fi
done
}
transform_single '$'
input.txt:
<newlines>
<newlines>
# Hello world!
<newlines>
This is a test string.
<newlines>
invocation:
bash script.sh < input.txt
output:
# Hello world!This is a test string.
excepted output:
The same as in the input file.
Working script
#!/bin/bash
transform_single() {
if (($# != 1)); then echo 'Error: illegal number of args' 1>&2; fi
equation=''
delim0=0
delim="$1"
while IFS= read -r -d $'\0' -n 1 c; do
if ((delim0 == 0)) && [[ "$c" == "$delim" ]]; then
delim0=1
equation=''
elif ((delim0 != 0)) && [[ "$c" == "$delim" ]]; then
delim0=0
echo -n "$equation" | texmath
elif ((delim0 != 0)); then
equation="$equation$c"
else
echo -n "$c"
fi
done
}
transform_single '$'
The issue is that you must set a delimiter to read, a null character, to preserve line feed.
I've been playing with bash scripting for 40'ish days with 0 experience so forgive me if my code looks like crap. I have a script that will take the configured NTP servers out of the /etc/ntp.conf file (/root/ntp.conf for testing)
NTPSRVCounter=1
echo "--- NTP Configuration ---"
echo " "
while read -r line; do
if [ $NTPSRVCounter == 1 ] ; then
echo "Primary NTP: $line"
SEDConfiguredNTP1="$(echo $line | sed 's/\./\\./g')"
((NTPSRVCounter++))
echo " "
else
SEDConfiguredNTP2="$(echo $line | sed 's/\./\\./g')"
echo "Secondary NTP: $line"
echo ""
fi
done < <(grep -o -P '(?<=server ).*(?= iburst)' /root/ntp.conf)
And asks you if you want to change it with a case statement:
echo "Do you wish to change it? [Y/n]"
NTPSRVCounter2=1
read opt
case $opt in
Y|y) read -p "Enter in your primary NTP server: " -e -i '0.debian.pool.ntp.org' UserInputNTP1
read -p "Enter in your secondary NTP serer: " -e -i '1.debian.pool.ntp.org' UserInputNTP2
for NTP in "$UserInputNTP1" "$UserInputNTP2" ; do
is_fqdn "$NTP"
if [[ $? == 0 && $NTPSRVCounter2 == 1 ]] ; then
SEDUserInput1=$(echo $UserInputNTP1 | sed 's/\./\\./g')
((NTPSRVCounter2++))
elif [[ $? == 0 && $NTPSRVCounter2 == 2 ]] ; then
SEDUserInput2=$(echo $UserInputNTP2 | sed 's/\./\\./g')
sudo sed -i "s/$SEDConfiguredNTP1/$SEDUserInput1/g" /root/ntp.conf
sudo sed -i "s/$SEDConfiguredNTP2/$SEDUserInput2/g" /root/ntp.conf
else
echo "Fail!!! :-( "
fi
done
;;
N|n) return 0
;;
*) echo "I don't know what happened, but, eh, you're not supposed to be here."
;;
esac
The problem is with the "elif" statement and the function "is_fqdn" on the second run of the function. If I put "bash -x" on the script and run it, I see "is_fqdn" returning 0 on both runs of the function, but the elif statement "$?" is coming up as 1 instead of 0.
The two functions used are below. Have to validate NTP addresses as either valid domain names or I.P. addresses, right? :)
is_fqdn() {
hostname=$1
if [[ "$hostname" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
valid_ip "$hostname"
elif [[ "$hostname" == *"."* && "$hostname" != "localhost." && "$hostname" != "localhost" ]] ; then
return 0
else
return 1
fi
host $hostname > /dev/null 2>&1 || return 1
}
valid_ip(){
local stat=1
local ip=$1
if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
OIFS=$IFS
IFS="."
ip=($ip)
IFS=$OIFS
[[ ${ip[0]} -le 255 && ${ip[1]} -le 255 && ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
stat=$?
fi
return "$stat"
}
The condition in your if sets the value of $?, and that is what's used by the condition in the elif part, not the return value of is_fqdn. You need to save the value if you want to use it in multiple places:
is_fqdn "$NTP"
is_fqdn_rv=$?
if [[ $is_fqdn_rv == 0 && $NTPSRVCounter2 == 1 ]] ; then
SEDUserInput1=$(echo $UserInputNTP1 | sed 's/\./\\./g')
((NTPSRVCounter2++))
elif [[ $is_fqdn_rv == 0 && $NTPSRVCounter2 == 2 ]] ; then
SEDUserInput2=$(echo $UserInputNTP2 | sed 's/\./\\./g')
sudo sed -i "s/$SEDConfiguredNTP1/$SEDUserInput1/g" /root/ntp.conf
sudo sed -i "s/$SEDConfiguredNTP2/$SEDUserInput2/g" /root/ntp.conf
else
echo "Fail!!! :-( "
fi
I have been using a combination of tail, head and grep to print out code that is relavant to me. It helps me find functions and get straight to the point. I do this about 30 to 40 times a day so I thought i'd ask for some assistance on this since I am not good with sh scripting.
The command format I'd eventually like is something like giveme "searchstring" -nn << where nn is the number to give to head. err.. moving on
Below is what I am currently doing
grep -rl searchitem ./
cat ./filename | grep -n searchitem
tail -n +line ./filename | head -numberoflinestodisplay
Example:
jezzy#forum:/var/www/sgs# grep -rl insert_into_selectbox ./
./bin/ext/js/functions_edit.js
./src/ext/js/functions_edit.js
jezzy#forum:/var/www/sgs# cat ./bin/ext/js/functions_edit.js | grep -n insert_into_selectbox
195: for (key in data) insert_into_selectbox(selectbox, data[key], key, 0);
232: insert_into_selectbox(selectbox, data[key], key,1);
273: for (key in data) if (key!="_overload_") insert_into_selectbox(selectbox, data[key], key, 0);
289:function insert_into_selectbox(selectbox,text,value,selected) {
323: insert_into_selectbox(id,right.value,right.value,1);
334: insert_into_selectbox(id,options_right[i].text,options_right[i].value,1);
Then I choose one of the greps that works for me. I do not mind if the bash script does it for all occurrences of grep. It'll teach me to be more specific with my searches.
jezzy#forum:/var/www/sgs# tail -n +289 ./bin/ext/js/functions_edit.js | head -30
function insert_into_selectbox(selectbox,text,value,selected) {
if (text=="" || value=="") return;
var obj = getObj(selectbox);
var index = _selectbox_find(obj, value);
if (index == -1) {
index = obj.options.length;
obj.options[index] = new Option(text,value);
}
if (!obj.multiple && selected) obj.options[index].selected = true;
}
I think I could figure it out if I knew how to get all instances of the items found.
Merci and Thanks
Try this, it should be do what you look for, if I understand it correctly:
# Usage: giveme searchstring [numlines]
find . -type f -exec awk -v n=${2:-10} '
/'"$1"'/ {
printf("==== %s ====\n",FILENAME);
for(i=0;i<n;i++)
{
printf("%4d: %s\n",FNR,$0);
getline
}
}' {} +
You can try this script:
#!/bin/bash
shopt -s extglob
SEARCHDIR_DEFAULT="./"
read -ep "Enter search directory [$SEARCHDIR_DEFAULT]: " SEARCHDIR
[[ $SEARCHDIR =~ ^[[:space:]]*$ ]] && SEARCHDIR=$SEARCHDIR_DEFAULT
for (( ;; )); do
read -p "Enter search item: " SEARCHITEM
read -p "Enter number of lines to display: " NUMLINES
readarray -t MATCHEDFILES < <(exec grep -rl "$SEARCHITEM" "$SEARCHDIR")
echo "Please select a file to search into. Press C to cancel, or X to exit."
select FILE in "${MATCHEDFILES[#]}"; do
if [[ -n $FILE ]]; then
readarray -t MATCHEDLINES < <(exec grep -n "$SEARCHITEM" "$FILE")
echo "Please select starting line."
select LINE in "${MATCHEDLINES[#]}"; do
if [[ -n $LINE ]]; then
LINENUM=${LINE%%:*}
tail -n "+$LINENUM" "$FILE" | head -n "$NUMLINES"
elif [[ $REPLY == [cC] ]]; then
break
elif [[ $REPLY == [xX] ]]; then
break 3
fi
done
elif [[ $REPLY == [cC] ]]; then
break
elif [[ $REPLY == [xX] ]]; then
break 2
fi
done
done
New version:
#!/bin/bash
shopt -s extglob
SEARCHDIR_DEFAULT="./"
NUMLINES_DEFAULT=10
for (( ;; )); do
until
read -ep "Enter search directory [$SEARCHDIR_DEFAULT]: " SEARCHDIR
[[ $SEARCHDIR =~ ^[[:space:]]*$ ]] && SEARCHDIR=$SEARCHDIR_DEFAULT
[[ -d $SEARCHDIR ]]
do
echo "Please enter a valid directory."
done
SEARCHDIR_DEFAULT=$SEARCHDIR
until
read -p "Enter search item: " SEARCHITEM
[[ ! $SEARCHITEM =~ ^[[:blank:]]*$ ]]
do
echo "Please enter a valid search item."
done
until
read -p "Enter number of lines to display [$NUMLINES_DEFAULT]: " NUMLINES
[[ $NUMLINES =~ ^[[:space:]]*$ ]] && NUMLINES=$NUMLINES_DEFAULT
[[ $NUMLINES =~ ^[[:digit:]]+$ && NUMLINES -gt 0 ]]
do
echo "Please enter a valid number of lines."
done
NUMLINES_DEFAULT=$NUMLINES
readarray -t MATCHEDFILES < <(exec grep -rle "$SEARCHITEM" "$SEARCHDIR")
P1="Please select a file to search into. [ENTER = show list again, C = cancel, X = exit]: "
P2="Please select starting line. [ENTER = show list again, C = cancel, X = exit]: "
PS3=$P1
select FILE in "${MATCHEDFILES[#]}"; do
if [[ -n $FILE ]]; then
readarray -t MATCHEDLINES < <(exec grep -ne "$SEARCHITEM" "$FILE")
if [[ ${#MATCHEDLINES[#]} -eq 0 || ! ${MATCHEDLINES[0]} =~ ^[[:digit:]]+: ]]; then
echo "No matching lines were extracted. Perhaps file is a binary."
elif [[ ${#MATCHEDLINES[#]} -eq 1 ]]; then
LINENUM=${MATCHEDLINES[0]%%:*}
echo "----------------------------------------"
tail -n "+$LINENUM" "$FILE" | head -n "$NUMLINES"
echo "----------------------------------------"
else
PS3=$P2
select LINE in "${MATCHEDLINES[#]}"; do
if [[ -n $LINE ]]; then
LINENUM=${LINE%%:*}
echo "----------------------------------------"
tail -n "+$LINENUM" "$FILE" | head -n "$NUMLINES"
echo "----------------------------------------"
elif [[ $REPLY == [cC] ]]; then
break
elif [[ $REPLY == [xX] ]]; then
break 3
fi
done
PS3=$P1
fi
elif [[ $REPLY == [cC] ]]; then
break
elif [[ $REPLY == [xX] ]]; then
break 2
fi
done
done