This question already has answers here:
here-document gives 'unexpected end of file' error
(10 answers)
Closed 6 years ago.
I have the following function;
function getdetails {
if ! "${PSQL_PATH}" -d "$DB" -U "$DB_USER" -h localhost -p "$DB_PORT" -t -c | while read -a Record ; do
taskid="${Record[0]}"
clientname="${Record[1]}"
backup_pass="${Record[2]}"
backup_dir="${Record[3]}"
done; then
echo "Could not fech next task metadata from database"
exit 1
fi <<EOF
WITH firsttask AS (SELECT taskid from tasks
WHERE status = 'PENDING'
ORDER BY date_started ASC
LIMIT 1)
SELECT taskid, username, storage_password AS backup_password, location AS backup_dir
FROM firsttask
INNER JOIN users USING (userid)
INNER JOIN storage USING (userid)
WHERE (username = '$1');
EOF
}
For some reason, bash does not detect the last EOF and reports:
./processor.sh: line 138: warning: here-document at line 41 delimited by end-of-file (wanted `EOF')
./processor.sh: line 139: syntax error: unexpected end of file
Any ideas why the EOF is not picked up? Thank you!
It fails because the closing EOF word has trailing whitespace. It must be on a line by itself, with no leading or trailing whitespace.
An exception is when using the <<- syntax, in which case the closing word may be preceded by one or more TABs (but never spaces).
Related
This question already has answers here:
Why does my Bash code fail when I run it with 'sh'?
(2 answers)
Closed 10 months ago.
I am using the below script to read the values of the 3rd column of the CSV file. However, I am getting a Syntax error.
tr -d '\r' < p_test.csv > fixed_p_test.csv
while IFS="," read -r rec1 rec2
do
echo "Displaying Record-$rec1"
echo "Price: $rec2"
done < <(cut -d "," -f1,3 fixed_p_test.csv | tail -n +2)
While running the above script I am getting the below error:
pScanFinal.sh: line 9: syntax error near unexpected token `<'
pScanFinal.sh: line 9: `done < <(cut -d "," -f1,3 fixed_p_test.csv | tail -n +2)'
My p_test.csv look like:
A
B
C
D
E
192.158
True
12
HT
Open
254.658
False
58
SM
Closed
How can I resolve this error?
You don't actually need cut, tail, or process substitution here, which means you can make your script POSIX compliant to match how you are running it.
{
read # Skip the first line
while IFS=, read -r rec1 _ rec2 _; do
...
done
} < fixed_p_test.csv
As jq is tagged (yet I don't know why), here's how to do it with jq.
Input and output should be read (-R) and written (-r) as raw text. Then split (/) each line by comma (,), and select column [2] (0-based).
jq -Rr '(./",")[2]' file.csv
12
58
Demo
Not asked/tagged, but imho awk would be more appropriate:
awk -F, '{print $3}' file.csv
This question already has answers here:
How to cat <<EOF >> a file containing code?
(5 answers)
Closed 3 years ago.
I am attempting to create a simple function during the startup of a postgres instance. I am doing this by mapping some .sh files to the /docker-entrypoint-initdb.d dir.
I continuously hit a problem with this simple function.
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE OR REPLACE FUNCTION hi() RETURNS TEXT AS $$
BEGIN
RETURN 'hi'
END;
$$;
EOSQL
2020-01-29 05:12:30.817 UTC [62] ERROR: syntax error at or near "1" at character 49
2020-01-29 05:12:30.817 UTC [62] STATEMENT: CREATE OR REPLACE FUNCTION hi() RETURNS TEXT AS 1
BEGIN
RETURN 'hi'
END;
ERROR: syntax error at or near "1"
LINE 1: CREATE OR REPLACE FUNCTION hi() RETURNS TEXT AS 1
If I change it to something with actual content:
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE OR REPLACE FUNCTION hi() RETURNS TEXT AS $func$
BEGIN
RETURN 'hi'
END;
$func$;
EOSQL
I get another error at the same place:
2020-01-29 05:17:36.161 UTC [62] ERROR: syntax error at or near "$" at character 49
2020-01-29 05:17:36.161 UTC [62] STATEMENT: CREATE OR REPLACE FUNCTION hi() RETURNS TEXT AS $
BEGIN
RETURN 'hi'
END;
ERROR: syntax error at or near "$"
LINE 1: CREATE OR REPLACE FUNCTION hi() RETURNS TEXT AS $
Running with the latest postgres version: 12.1
What precisely is the problem with this function definition and why am I getting this error?
The part of your script from <<-EOSQL to EOSQL is called a "here document", and that functionality is documented at the Bash Reference Manual, §3.6.6 "Here Documents". Per that section:
If any part of word [in your case EOSQL] is quoted, […] the lines in the here-document are not expanded. If word is unquoted, all lines of the here-document are subjected to parameter expansion, command substitution, and arithmetic expansion, […]
In other words — because you haven't quoted any part of EOSQL, Bash is performing parameter expansion (and other similar substitutions) on the contents of the here-document, which includes replacing $$ with the process-ID, and $func with the empty string, before PostgreSQL sees them.
If you just change <<-EOSQL to <<-'EOSQL', it won't do that.
$func and $$ are variables in bash. The first one is the process ID of the shell instance, which explains the number in your error output; the second is just a normal user-defined variable, presumably empty, so $test$ becomes just $ (as $test gets replaced by an empty string).
You can either escape the dollar sign that bash finds significant, using \$\$ and \$test\$, or you can use <<-'EOSQL' to use single-quote (no variable substitution) semantics on the heredoc.
None of this has to do with PostgreSQL, it's bash doing what it thinks you want.
the below script gives me an error. Basically I am trying to delete the records that I got from the first query. I have put them in a text file, formatted them and used them in the delete operation.
After executing the script I am getting the below error:-
: line 5: syntax error at line 27: `<<' unmatched
Can't tell because the code you dumped is unformatted, but my first guess would be you have leading spaces in front of the EOF in your here document.
This should work (note that there are no leading spaces in front of the EOF.:
sqlplus -s $dbcreds << EOF > output.txt
SET SERVEROUTPUT OFF
select empname from emp where dept_no=123;
EOF
if [ -s "output.txt" ]
then
echo " Found the below employees....Deleting them from Database ..............!!!! \n"
cat output.txt
sed "s/(.*)/'\1'/" output.txt| tr '\n' ','|sed 's/.$//' >final_employees.txt
while read line
do
sqlplus -s $dbcreds <<EOF
SET SERVEROUTPUT OFF
Delete from emp where empname in ($line);
EOF
done < final_employees.txt
else
echo " No employees found....!!!"
fi
while executing the following code
list="$(Result "$VARDB" "$VAR2" - <<-EOT
SELECT d.day || '#' || t.mo
FROM daa a,dab b;
EOT)"
I get the following error:
warning: here-document at line 5 delimited by end-of-file (wanted `EOT')
The code works well under bash version 3.2.25 but it triggers the error under version 4.1.2
Try this instead :
list="$(Result "$VARDB" "$VAR2" - <<EOT
SELECT d.day || '#' || t.mo
FROM daa a,dab b;
EOT
)"
It's because the here-doc starts literally with any string. So the start is EOT and the end of file is found instead of EOT.
The shell just see EOT) not EOT
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 9 months ago.
I have a simple bash script that downloads stock prices and appends them to a variable, which is then outputted:
#!/bin/bash
output=" "
for stock in GOOG AAPL
do
price=$(curl -s "http://download.finance.yahoo.com/d/quotes.csv?s=$stock&f=l1")
output+="$stock: $price "
done
echo "$output"
This script only displays AAPL: 524.896, the last piece of data fetched. According to whatswrongwithmyscript, there isn't anything wrong with the script, and I thought I was following this answer properly. This answer discussed a similar problem (appending to a string variable inside a loop) and suggested a different method which I used like this:
#!/bin/bash
output=" "
for stock in GOOG AAPL
do
price=$(curl -s "http://download.finance.yahoo.com/d/quotes.csv?s=$stock&f=l1")
output="$output$stock: $price "
done
echo "$output"
The output is still the same. I'm using bash 4.2.45 on debian jessie x64.
More info
I echoed the result in a loop to debug, and from the first script, this is what I get:
GOOG: 1030.42
AAPL: 524.896
AAPL: 524.896
And the second script gives the same thing:
GOOG: 1030.42
AAPL: 524.896
AAPL: 524.896
When I run your script and pipe the output to od -c, the result is illuminating:
0000000 G O O G : 1 0 3 0 . 4 2 \r
0000020 A A P L : 5 2 4 . 8 9 6 \r \n
0000040
So you can see that it IS in fact getting all the entries and concatenating them, but its ALSO getting CR characters (the \r in the od output), which causes them to print over the top of each other when you print the string.
You can pipe the output of curl to tr -d '\r' to strip off the problematic CRs:
price=$(curl -s "...." | tr -d '\r')
I'm pretty sure that the problem is that curl is returning a carriage return and this is messing with the printing of both values. If you redirect the output of the curl command to a file and view it in vi, you'll see it's created a DOS file.
This seems to work:
#!/bin/bash
output=""
for stock in GOOG AAPL
do
price=$(curl -s "http://download.finance.yahoo.com/d/quotes.csv?s=$stock&f=l1" | tr -d '\r')
output+="$stock $price\n"
done
echo -e "$output"