I have read literally every answer on the net that I could find. Nothing is similar to my problem, so here it is:
I have a bash script with curl and I get a variable back. I want to update my database with this variable, however it doesn't work.
My variable is $stream and no matter what I do, I always get the word "$stream" into the database instead of the result of the curl.
My script is:
#!/bin/bash
stream=$(curl --silent "https://player.mediaklikk.hu/playernew/player.php?video=mtv1live&noflash=yes&output=vast&unviewed_position_start=1&env=vp&impl=s&correlator=&osfamily=Windows&osversion=10&browsername=Chrome&browserversion=97.0.4692.99&title=M1&contentid=mtv1live&embedded=0" | grep -Po 'file": "\K(.*?)(?=")' | sed 's/\\\/\\\//https:\\\/\\\//g')
echo $stream
mysql
use mydatabase;
UPDATE my_table SET my_url = "$stream" WHERE my_name = 'stream_name';
You can use the -e option to execute a query. Put this in double quotes and variables will be expanded.
#!/bin/bash
stream=$(curl --silent "https://player.mediaklikk.hu/playernew/player.php?video=mtv1live&noflash=yes&output=vast&unviewed_position_start=1&env=vp&impl=s&correlator=&osfamily=Windows&osversion=10&browsername=Chrome&browserversion=97.0.4692.99&title=M1&contentid=mtv1live&embedded=0" | grep -Po 'file": "\K(.*?)(?=")' | sed 's/\\\/\\\//https:\\\/\\\//g')
echo $stream
mysql mydatabase -e "UPDATE my_table SET my_url = '$stream' WHERE my_name = 'stream_name';"
Related
I need the $table var to expand but while keeping the quotes that are required by MSSQL for the table_name parameter. I don't know if this is possible as I have been searching for a while. The common answer I see is if there are any quotes then variables won't be expanded. Is it simply not possible to do what I need here?
Code
cat <<EOF | isql $host sa 'password' -d, | sed '-e 1,10d;$d' | sort > mssql_table_${table}_column_info
use $database;
select column_name from information_schema.columns where table_name = '$table';
EOF
Desired Output
select column_name from information_schema.columns where table_name = 'mytable_name';
Notice that the output has single quotes still around the table name. This is necessary for MSSQL to select the appropriate table.
Are you sure the variable expansion in the here document is the problem though? If you just inspect the output of the cat command (using bash):
$ database=database123 table=mytable_name cat <<EOF >/dev/stdout
use $database;
select column_name from information_schema.columns where table_name = '$table';
EOF
use database123;
select column_name from information_schema.columns where table_name = 'mytable_name';
You might want to break down what the other commands are doing to make sure where the error is.
On a side note, your mention of turning off the variable expansion via quotation marks (' or ") apparently conflates the syntax of the here document with the unrelated syntax of other commands in the pipeline.
For example this is correct:
### works, prints 'hello hello'
MYVAR='hello' cat <<EOF | grep 'hello hello'
hello $MYVAR
EOF
As opposed to:
### WRONG, response empty
MYVAR='hello' cat <<'EOF' | grep 'hello hello'
hello $MYVAR
EOF
Does not perform the variable substitution, because the word 'EOF' is quoted in the here document, turning off variable expansion. This is regardless of whether other commands in the pipeline, that is grep 'hello hello', have quotes or not.
When I'm trying to execute script below and getting error: "curl: (3) URL using bad/illegal format or missing URL"
#!/bin/bash
stage="develop"
branch="branch_name"
getDefinition=$(curl -u user#example.com:password -X GET "https://dev.azure.com/organization/project/_apis/build/definitions?api-version=5.1")
for def in $(echo "$getDefinition" | jq '.value[] | select (.path=="\\Some_path\\'$stage'") | .id'); do
getBuildInfo=$(curl -u user#example.com:password -X GET "https://dev.azure.com/organization/project/_apis/build/definitions/${def}\?api-version=5.1")
# echo $def
body=$(echo "${getBuildInfo}" | jq '.repository.defaultBranch = "refs/heads/release/'"${branch}"'"' | jq '.options[].inputs.branchFilters = "[\"+refs/heads/release/'"${branch}"'\"]"' | jq '.triggers[].branchFilters[] = "+refs/heads/release/'"${branch}"'"')
echo ${body} > data.json
done
It happens when I'm trying to pass variable ${def} into a line:
curl -u user#example.com:password -X GET "https://dev.azure.com/organization/project/_apis/build/definitions/${def}\?api-version=5.1"
But when I declare an array, curl works as expected.
Example:
declare -a def
def=(1 2 3 4)
curl -u user#example.com:password -X GET "https://dev.azure.com/organization/project/_apis/build/definitions/${def}\?api-version=5.1"
Could you please suggest how can I pass variable into URL properly?
Do you need to call curl for 4 times? If so.
for def in 1 2 3 4; do curl -u user#example.com:password -X GET "https://dev.azure.com/organization/project/_apis/build/definitions/${def}\?api-version=5.1"; done
Set IFS=$' \t\r\n' at the top of your script.
IFS is the Interactive Field Separator.
In UNIX, IFS is space, tab, newline, or $' \t\n', but on Windows this needs to be $' \t\r\n'.
The ^M character is \r.
I have used the method to assign the SQL output to a variable as below.
dbRole=$(${SQLPLUSPGM} -s / as sysdba <<-EOF
set head off
set verify off
set feedback off
select trim(translate(database_role,' ','_')) from v\$database;
exit;
EOF
)
But the variable o/p appending a "\n" character i.e \nPHYSICAL_STANDBY
However, when I use the below method it is working fine
${SQLPLUSPGM} -s / as sysdba <<-EOF | grep -v '^$' | read dbRole
set head off
set verify off
set feedback off
select trim(translate(database_role,' ','_')) from v\$database;
exit;
EOF
Any suggestion why it is appending `\n' and how we can get rid of it.
Appreciate your suggestions.
Your second method, with the grep -v, removes the additional line.
You can use some filter inside your first method with additional parentheses.
dbRole=$((cat | grep -v "^$") <<-EOF
1
2
3
5
EOF
)
Alternative filters with some differences are grep ., head -1, sed '$d'.
I have a SQL query which contains:
join (&max_dt1) t3
I want to replace &max_dt1 so:
MAX_DT1="'2017-11-03'"
REPLACE="sed -i 's/&max_dt1/select ($MAX_DT1) as dt/g' $FINAL_QUERY"
cat $FINAL_QUERY | eval $REPLACE
Result:
join (select (2017-05-09) as dt) t3
Why is it deleting the quote marks of my variable MAX_DT1?
My real code:
MAX_DT1="'$MAX_DT1'"
echo $MAX_DT1
REPLACE="sed -i 's/&max_dt1/select ($MAX_DT1) as dt/g' $FINAL_QUERY"
echo sed -i "s/&max_dt1/select ($MAX_DT1) as dt/g $FINAL_QUERY
cat $FINAL_QUERY | eval $REPLACE
MAX_DT1 first of all contains just: 2017-11-03
follow this
$ query="join (&max_dt1) t3"
$ MAX_DT1="'2017-11-03'" # <--- you have to quote the single quotes!!
$ sed "s/&max_dt1/select ($MAX_DT1) as dt/g" <<< $query
join (select ('2017-11-03') as dt) t3
Quotes indicates that you affect a string to MAX_DT1 variable. It's not part of your string.
If your variable value has to contains quotes then you have to add them.
For instance:
MAX_DT1="'2017-11-03'"
Unfortunately your Next REPLACE variable won't have what you're expecting and the eval command called later won't work since its sed command would be bad formatted (if you keep your code this way).
Fortunately, your problem is solved if you call directly the sed command this way:
MAX_DT1="'2017-11-03'"
sed -i "s/&max_dt1/select ($MAX_DT1) as dt/g" $FINAL_QUERY
I have a command that I'd like to run and get output from. Here's the command:
sh execute_odi_script.sh Audit_ExecutionLogStart.sql heelo
It should return
Begin Informatica Script 18 Redshift Delta copy completed at: 04/10/17 18:46:20 END
Now I need to grab just the output from the SQL file, which is 18 in this case.
Someone was able to help me achieve that, but only if I do it in 2 steps like so:
> logStart=$( sh execute_odi_script.sh Audit_ExecutionLogStart.sql heelo )
> echo $logStart | sed -r s/.*Begin\ Informatica\ Script\ \(\.*\)\ Redshift\ Delta.*/\\1/ ); echo $logStart
18
I've been trying to play around with how I'm doing the command substitution but am having trouble understanding what to even change. How do I do all this in one line?
First -- if you only care about one line, grep for that line:
logStart=$( sh execute_odi_script.sh Audit_ExecutionLogStart.sql heelo \
| grep -e "Begin Informatica Script" \
| sed -r 's/.*Begin Informatica Script (.*) Redshift Delta.*/\1/' )
echo "$logStart"
Alternately, you the entire pipeline inside the command substitution:
logStart=$( sh execute_odi_script.sh Audit_ExecutionLogStart.sql heelo \
| tr '\n' ' ' \
| sed -r 's/.*Begin Informatica Script (.*) Redshift Delta.*/\1/' )
echo "$logStart"
If you want to avoid the echo as a separate step, you might add a | tee /dev/stderr to the end of the pipeline (again, inside $()).