Delete <feff> from text file, UTF 8 - bash

I use this BASH script to filter text from one text file to another. Text is encoded in UTF 8.
#!/bin/bash
mid=$1
infile="/var/www/tmp/textgrid_uploads/${mid}.txt"
outfile="/home/var/www/vids/$mid/${mid}_textgrid.mlf"
tmpfile="/home/var/www/vids/$mid/${mid}.tmp"
i=1
touch $tmpfile
cat $infile | grep "text =" | cut -d '"' -f2 | tr -d ',' | tr -d '.' | tr -d ':' | tr -d ';' | tr -d '!' | tr -d '?' > $tmpfile
#| awk '{ print tolower($0) }'
#cat $infile | grep -v "<" | egrep -v '^[[:space:]]*$' | tr -d '.' | tr -d "," | tr -d ";" | tr -d ":" | tr -d "^" | tr -d '#' | tr -d '?' | tr -d '!' | tr -d '%' | tr -d '#' | tr -d '*' | tr -d '~' | grep -v '((xxxxx))' | awk '{ print tolower($0) }' > $tmpfile
#cat $infile | grep -v 'WEBVTT' | grep -v "\--" | grep -v '^$' | sed 's/?/./g' | sed 's/!/./g' | tr -d '.' | tr -d "," | tr -d ";" | tr -d ":" | awk '{ print tolower($0) }' > $tmpfile
nlines=$(cat $tmpfile | wc -l)
echo "#!MLF!#" >> $outfile
echo "\"*/dummyfile.lab\"" >> $outfile
while [ $i -le $nlines ]
do
line=$(cat $tmpfile | sed $i'q;d') #zobrazi konkretny riadok
printf '%s\n' $line | sed '/^\s*$/d' | cut -d "/" -f1 | egrep -v '^[[:space:]]*$' >> $outfile #zapis po riadkoch s odstranenim znaciek
i=$[$i+1] #pocitadlo
done
echo "." >> $outfile
rm $tmpfile
Output of this script is:
#!MLF!#
"*/dummyfile.lab"
<feff> V utorok o devätnástej bude vo fejs
I want to remove first word feff with both brackets. Please give me whole example how to do it. I am new in Bash and I just use this script I didn't create it. Thanks guys.

Related

i dont know why it writed me 11:not found

it writes me "file name" 18:11 not found
line 18 is where the if statement starts
#!/bin/bash
westfunc() {
echo "please select 2 airports(example:jfk):\n"
read place
read place2
echo "\e[32m--------------------------------------------------------------------------------"
echo
echo " $place $place2"
de=`weather-util -m $place | grep "Temperature"| cut -d "T" -f2 | cut -d "e" -f4| cut -d ":" -f2`
we=`weather-util -m $place | grep "Wind" | cut -d "W" -f3 | cut -d ")" -f2`
sc=`weather-util -m $place | grep "Sky conditions" | cut -d "n" -f3 | cut -d ":" -f2`
var=`weather-util -m $place | grep "Temperature"| cut -d " " -f5 | cut -d "." -f1`
if (( $var > 5 ))
then
echo "hello"
fi
}
i tried to change $ the ( and stuff like that

How to fix an unexpected end of file error in bash?

The code saying that there is "unexpected end of file" error in line 16. Could someone please tell me my mistake?
#!/bin/bash
total=0
for i in `grep 01/Oct/2006 log.txt | cut -d' ' -f1 | sort | uniq -c | sort -n | tail`;
do if [[ $i =~ ^[0-9]+$ ]]; then
total=$(( $total + $i )); fi
for i in `grep 01/Oct/2006 log.txt | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -10 | tr -s ' ' | cut -d' ' -f2,3 | sed -E 's/(\S*) (\S*)/\2 - \1/' | nl -s'. '`;
do
if ! [[ $i =~ ^[0-9]+$ ]];
then
printf $i;
printf " ";
else
printf " $i - $(echo "scale=0; $i * 100 / $total" | bc )%% \n" ;
fi
done
Your first for loop lacks a done.
Here's a working version with improved formatting (but with all original flaws and bugs left inside, I just fixed the one issue asked for here):
#!/bin/bash
$total;
for i in $(
grep 01/Oct/2006 log.txt |
cut -d' ' -f1 |
sort |
uniq -c |
tail);
do
if [[ $i =~ ^[0-9]+$ ]]
then
$total += $i
fi
done
for i in $(
grep 01/Oct/2006 log.txt |
cut -d' ' -f1 |
sort |
uniq -c |
sort -rn |
head -10 |
tr -s ' ' |
cut -d' ' -f2,3 |
sed -E 's/(\S*) (\S*)/\2 - \1/' |
nl -s'. ')
do
if ! [[ $i =~ ^[0-9]+$ ]];
then
printf " $i - ";
else
printf " $i - 0$(echo "scale=0; $i / $total" | bc)%% " ;
fi
done

Shell script result output to a file

I just faced a problem about shell script output problem. Here is the code
read_each_user_rating(){
TOTAL_RATING_NUMBER="$(grep -c '<Author>' $1)" #Find how many rating in each file
ALL_AUTHOR="$(grep '<Author>' $1 | sed -e 's/<Author>//'| tr -d '\r')"
ALL_COMMENT="$(grep '<Content>' $1 | sed -e 's/<Content>//'| tr -d '\r')"
ALL_DATE="$(grep '<Date>' $1 | sed -e 's/<Date>//'| tr -d '\r')"
ALL_RATING_FILE="$(grep '<Overall>' $1 | sed -e 's/<Overall>//'| tr -d '\r')"
ALL_VALUE="$(grep '<Value>' $1 | sed -e 's/<Value>//'| tr -d '\r')"
ALL_ROOMS="$(grep '<Rooms>' $1 | sed -e 's/<Rooms>//'| tr -d '\r')"
ALL_LOCATION="$(grep '<Location>' $1 | sed -e 's/<Location>//'| tr -d '\r')"
ALL_CLEANLINESS="$(grep '<Cleanliness>' $1 | sed -e 's/<Cleanliness>//'| tr -d '\r')"
ALL_CHECKIN="$(grep '<Check in / front desk>' $1 | sed -e 's/<Check in / front desk>//'| tr -d '\r')"
ALL_SERVICE="$(grep '<Service>' $1 | sed -e 's/<Service>//'| tr -d '\r')"
ALL_BUSSINESS="$(grep '<Bussiness service>' $1 | sed -e 's/<Bussiness service>//'| tr -d '\r')"
for ((COUNTER_A=1;COUNTER_A<=$TOTAL_RATING_NUMBER;COUNTER_A++))
do
echo "INSERT INTO UserRating (Author,Comment,Date,Overall,Value,Rooms,Locations,Cleanliness,Checkin,Service,Bussiness)" >> hotelreviews.sql
echo $($ALL_AUTHOR | sed "${COUNTER_A}q;d") >> hotelreviews.sql
done
}
read_each_user_rating $1
I can output
echo "INSERT INTO UserRating (Author,Comment,Date,Overall,Value,Rooms,Locations,Cleanliness,Checkin,Service,Bussiness)" >> hotelreviews.sql to the file. But why i can output "echo $($ALL_AUTHOR | sed "${COUNTER_A}q;d") >> hotelreviews.sql" part to file too?
echo $($ALL_AUTHOR | sed "${COUNTER_A}q;d") >> hotelreviews.sql
is malformed. The expression inside $() must be a valid command (of which $ALL_AUTHOR is almost certainly not).
More likely you need something such as:
echo "$ALL_AUTHOR" | sed "${COUNTER_A}q;d"
inside the $().
In this case however, it's almost certainly not necessary to invoke a sub-command at all when you can simply do:
echo "$ALL_AUTHOR" | sed "${COUNTER_A}q;d" >>hotelreviews.sql

how to read words from a file using shell script

i have a file /ws/$1-rcd/temp.txt which has only one line as follows
198|/vob/ccm_tpl/repository/open_source/commons_collections/3_2_2/...
i have a script to get the value repository/open_source/commons_collections and 3_2_2 by reading the file and looping through it using for loop
i have my code as follows
grep -n "$4" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f1,2 | sed -e 's/\:element/|/g' | sed -e 's/ //g' > /ws/$1-rcd/temp.txt
for i in `cat /ws/$1-rcd/temp.txt`
do
line=`echo $i | cut -d"|" -f1`
path=`echo $i | cut -d"|" -f2`
whoami
directory_temp=`echo $path | awk -F "/" '{ print $(NF-2)}'`
if [ "$directory_temp" == "$4" ]
then
OLD_VERSION=`sed -n "${line}p" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f2 | awk -F "/" '{ print $(NF-1)}'`
total_fields=`sed -n "${line}p" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f2 | awk -F "/" '{ print NF }'`
dir_path=`expr ${total_fields} - 2`
loc=`sed -n "${line}p" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f2 | cut -d"/" -f1-"${dir_path}"`
location=`echo $loc | cut -d"/" -f4,5,6`
fi
done
but when i run this code it gives me an error as
-bash: line 45: syntax error near unexpected token |'
-bash: line 45:for i in 198|/vob/ccm_tpl/repository/open_source/commons_collections/3_2_2/...'
can anyone please suggest what am i doing wrong
If you want to iterate through each line of a file, use while loop like below
while read -r line ;do
echo $line
done <file.txt
so, your code can be rewritten as
grep -n "$4" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f1,2 | sed -e 's/\:element/|/g' | sed -e 's/ //g' > /ws/$1-rcd/temp.txt
while read i ; do
line=`echo $i | cut -d"|" -f1`
path=`echo $i | cut -d"|" -f2`
whoami
directory_temp=`echo $path | awk -F "/" '{ print $(NF-2)}'`
if [ "$directory_temp" == "$4" ]
then
OLD_VERSION=`sed -n "${line}p" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f2 | awk -F "/" '{ print $(NF-1)}'`
total_fields=`sed -n "${line}p" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f2 | awk -F "/" '{ print NF }'`
dir_path=`expr ${total_fields} - 2`
loc=`sed -n "${line}p" /ws/$1-rcd/raw-vobs-config-spec | cut -d " " -f2 | cut -d"/" -f1-"${dir_path}"`
location=`echo $loc | cut -d"/" -f4,5,6`
fi
done < /ws/$1-rcd/temp.txt
You may be better served relying on parameter expansion and substring removal. For example:
#!/bin/sh
a=$(<dat/lline.txt) ## read file into a
a=${a##*ccm_tpl/} ## remove from left to ccm_tpl/
num=${a##*collections/} ## remove from left to collections/
num=${num%%/*} ## remove from right to /
a=${a%%${num}*} ## remove from right to $num
Input File
$ cat dat/lline.txt
198|/vob/ccm_tpl/repository/open_source/commons_collections/3_2_2/..
Output
$ sh getvals.sh
a : repository/open_source/commons_collections/
num : 3_2_2
If you need to trim in some other way, just let me know and I'm happy to help further.

Using bash command on a variable that will be used as reference for an array

Short and direct, basically I want to use the value of $command on a variable, instead using it inside the while loop as a command itself. So:
This Works, but I think it's ugly:
#!/bin/bash
IFS=$'\n'
lsof=`which lsof`
whoami=`whoami`
while true ; do
execution_array=($(${lsof} -iTCP -P 2> /dev/null | grep ':' | grep ${whoami} | awk '{print $9}' | cut -f2 -d'>' | sort | uniq ))
for i in ${execution_array[*]}; do
echo $i
done
sleep 1
done
unset IFS
This doesn't work ( no output happens ), but i think is less ugly:
#!/bin/bash
IFS=$'\n'
lsof=`which lsof`
whoami=`whoami`
command="${lsof} -iTCP -P 2> /dev/null | grep ':' | grep ${whoami} | awk '{print $9}' | cut -f2 -d'>' | sort | uniq"
while true ; do
execution_array=($(command))
for i in ${execution_array[*]}; do
echo $i
done
sleep 1
done
unset IFS
This solved my problem:
#!/bin/bash
IFS=$'\n'
lsof=$(which lsof)
list_connections() {
${lsof} -iTCP -P 2> /dev/null | grep ':' | grep $(whoami) | awk '{print $9}' | cut -f2 -d'>' | sort | uniq
}
while true ; do
execution_array=($(list_connections))
for i in ${execution_array[*]}; do
echo $i
done
sleep 1
done
unset IFS

Resources