I just faced a problem about shell script output problem. Here is the code
read_each_user_rating(){
TOTAL_RATING_NUMBER="$(grep -c '<Author>' $1)" #Find how many rating in each file
ALL_AUTHOR="$(grep '<Author>' $1 | sed -e 's/<Author>//'| tr -d '\r')"
ALL_COMMENT="$(grep '<Content>' $1 | sed -e 's/<Content>//'| tr -d '\r')"
ALL_DATE="$(grep '<Date>' $1 | sed -e 's/<Date>//'| tr -d '\r')"
ALL_RATING_FILE="$(grep '<Overall>' $1 | sed -e 's/<Overall>//'| tr -d '\r')"
ALL_VALUE="$(grep '<Value>' $1 | sed -e 's/<Value>//'| tr -d '\r')"
ALL_ROOMS="$(grep '<Rooms>' $1 | sed -e 's/<Rooms>//'| tr -d '\r')"
ALL_LOCATION="$(grep '<Location>' $1 | sed -e 's/<Location>//'| tr -d '\r')"
ALL_CLEANLINESS="$(grep '<Cleanliness>' $1 | sed -e 's/<Cleanliness>//'| tr -d '\r')"
ALL_CHECKIN="$(grep '<Check in / front desk>' $1 | sed -e 's/<Check in / front desk>//'| tr -d '\r')"
ALL_SERVICE="$(grep '<Service>' $1 | sed -e 's/<Service>//'| tr -d '\r')"
ALL_BUSSINESS="$(grep '<Bussiness service>' $1 | sed -e 's/<Bussiness service>//'| tr -d '\r')"
for ((COUNTER_A=1;COUNTER_A<=$TOTAL_RATING_NUMBER;COUNTER_A++))
do
echo "INSERT INTO UserRating (Author,Comment,Date,Overall,Value,Rooms,Locations,Cleanliness,Checkin,Service,Bussiness)" >> hotelreviews.sql
echo $($ALL_AUTHOR | sed "${COUNTER_A}q;d") >> hotelreviews.sql
done
}
read_each_user_rating $1
I can output
echo "INSERT INTO UserRating (Author,Comment,Date,Overall,Value,Rooms,Locations,Cleanliness,Checkin,Service,Bussiness)" >> hotelreviews.sql to the file. But why i can output "echo $($ALL_AUTHOR | sed "${COUNTER_A}q;d") >> hotelreviews.sql" part to file too?
echo $($ALL_AUTHOR | sed "${COUNTER_A}q;d") >> hotelreviews.sql
is malformed. The expression inside $() must be a valid command (of which $ALL_AUTHOR is almost certainly not).
More likely you need something such as:
echo "$ALL_AUTHOR" | sed "${COUNTER_A}q;d"
inside the $().
In this case however, it's almost certainly not necessary to invoke a sub-command at all when you can simply do:
echo "$ALL_AUTHOR" | sed "${COUNTER_A}q;d" >>hotelreviews.sql
Related
I am confused with this make.sh file. I read previous posts about shell scripts structure but I could not find out about this file. what is the function of this file? ....
Can anyone explain it step by step?
#!/bin/sh
rm out/*
example_number=0
for name in `ls in`
do
out=`cat in/$name | grep ".o " | tr -s \ | cut -d\ -f2`
inp=`cat in/$name | grep ".i " | tr -s \ | cut -d\ -f2`
echo -n "${name} (i=${inp}, o=${out}) "
if [ $inp -le 12 ]
then
cat in/$name \
| sed '/.i/d' \
| sed '/.o/d' \
| sed '/.p/d' \
| sed '/.e/d' \
| sed 's/|/ /g' \
| tr -s \ \
| sed 's/^[ \t]*//;s/[ \t]*$//' \
> out/${name}.in
tst=`cat out/${name}.in | cut -d\ -f2 | grep - -c`
if [ $tst -ne 0 ]
then
echo "remove file"
rm out/${name}.in
else
echo processing...
./unix2dos.exe -q out/${name}.in
example_number=`expr $example_number + 1`
fi
else
echo " skip"
fi
done
for name in `grep 2 -l out/*`
do
echo Remove $name
rm $name
example_number=`expr $example_number - 1`
done
echo Number of examples is $example_number
# bad files
# apla ( 222? )
# tms
# mainpa...
It is not a Makefile, but a shell script. You can see this from the file extension .sh and from the header
#!/bin/sh
Which is the instruction to use the shell to execute this file.
I am trying to write a simple shell script that takes a text file as input and checks all non-punctuated words against a dictionary (english.txt). It should return all non-matching (misspelled) words. I am using grep but it does not seem to successfully match all the lines in english.txt. I have included my code below.
#!/bin/bash
cat $1 |
tr ' \t' '\n\n' |
sed -e "/'/d" |
tr -d '[:punct:]' |
tr -cd '[:alpha:]\n' |
sed -e "/^$/d" |
grep -v -i -w -f english.txt
I have bash script that would merge a huge list of text files and filter it. However I'll encounter 'argument line too long' error due to the huge list.
echo -e "`cat $dir/*.txt`" | sed '/^$/d' | grep -v "\-\-\-" | sed '/</d' | tr -d \' | tr -d '\\\/<>(){}!?~;.:+`*-_ͱ' | tr -s ' ' | sed 's/^[ \t]*//' | sort -us -o $output
I have seen some similar answers here and i know i could rectify it using find and cat the files 1st. However, i would i like to know what is the best way to run a one liner code using echo -e and cat without breaking the code and to avoid the argument line too long error. Thanks.
First, with respect to the most immediate problem: Using find ... -exec cat -- {} + or find ... -print0 | xargs -0 cat -- will prevent more arguments from being put on the command line to cat than it can handle.
The more portable (POSIX-specified) alternative to echo -e is printf '%b\n'; this is available even in configurations of bash where echo -e prints -e on output (as when the xpg_echo and posix flags are set).
However, if you use read without the -r argument, the backslashes in your input string are removed, so neither echo -e nor printf %b will be able to process them later.
Fixing this can look like:
while IFS= read -r line; do
printf '%b\n' "$line"
done \
< <(find "$dir" -name '*.txt' -exec cat -- '{}' +) \
| sed [...]
grep -v '^$' $dir/*.txt | grep -v "\-\-\-" | sed '/</d' | tr -d \' \
| tr -d '\\\/<>(){}!?~;.:+`*-_ͱ' | tr -s ' ' | sed 's/^[ \t]*//' \
| sort -us -o $output
If you think about it some more you can probably get rid of a lot more stuff and turn it into a single sed and sort, roughly:
sed -e '/^$/d' -e '/\-\-\-/d' -e '/</d' -e 's/\'\\\/<>(){}!?~;.:+`*-_ͱ//g' \
-e 's/ / /g' -e 's/^[ \t]*//' $dir/*.txt | sort -us -o $output
I have problem with echo command I need export data to csv but its empty file
#!/bin/bash
while read domain
do
ownname= whois $domain | grep -A 1 -i "Administrative Contact:" |cut -d ":" -f 2 | sed 's/ //' | sed -e :a -e '$!N;s/ \n/,/;ta'
echo -e "$ownname" >> test.csv
done < dom.txt
You need to use command substitution to store command's output in a shell variable:
#!/bin/bash
while read domain; do
ownname=$(whois $domain | grep -A 1 -i "Administrative Contact:" |cut -d ":" -f 2 | sed 's/ //' | sed -e :a -e '$!N;s/ \n/,/;ta')
echo -e "$ownname" >> test.csv
done
PS: I haven't tested all the piped commands.
I have a folder with a bunch of files, the files only have a url in it i.e
http://itunes.apple.com/us/app/keynote/id361285480?mt=8
Here is my code. How can I get it to do this for each url in each file?
var='{"object":"App","action":"scrape","args":{"itunes_url":"!!!!HERE!!!!"}}'
string=$(echo "$var" | sed -e 's/"/\\"/g')
string='{"request":"'"$string"'"}'
api="http://api.lewis.com"
output=$(curl -s -d "request=$string" "$api")
code=$(echo "$output" | tr '{', '\n' | sed -n "2p" | sed -e 's/:/ /' | awk '{print $2}')
if [ "${code:0:1}" -ne "2" ]; then
# :(
echo "Error: response code $code was returned, "
else
string=$(echo "$output" | tr '{', '\n' | sed -e '/"signature":\(.*\)/d;/"data":\(.*\)/d;/"signature":\(.*\)/d;/"code":\(.*\)/d' |sed -e 's/\\"//g;s/\\\\\\\//\//g;s/\\//g' | tr '}', '\n' | sed -e 's/"//' | sed '/^$/d')
echo "$string"
fi
use a for loop
for filename in folder/*; do
-- your code where you do something using $filename --
done
og if you prefer to give the filenames as arguments to the script then:
for filename do
-- your code where you do something using $filename --
done
then run your script followed by the files
./script.sh folder/*
You could do:
for file in *; do
for line in $(cat $file); do
# Stuff goes here
done
done
Or even just:
for line in $(cat *); do
# Stuff goes here
done