I'm trying to script a query from a bash file but the select query is in a file. For compatibility and test, I'd like to keep it this way.
In short I'd like to write something like:
psql -c data_base "\copy (<file.sql>) To './test.csv' With CSV"
Would you know how to do that ?
EDIT
The "file.sql" contains a query :
$cat file.sql
$SELECT country FROM world;
You can use bash substitution as,
psql -c data_base "\copy ($(<file.sql)) To './test.csv' With CSV"
$(<file) expands the contents of the file
Related
I'm having hard time executing using -e option with column name is in quotes.
I want to execute some thing like below using unix level. Trying to run from shell script. When i try to put my values in quotes, its taking away the quotes for my column.
select * from keyspace.cf where "columnname"=''
Tried this:
cqlsh hostname -e "select * from keyspace.cf where "columnname"=''"
It is executing as cqlsh hostname -e 'select * from keyspace.cf where columnname='
stdin>:1:InvalidRequest: Error from server: code=2200 [Invalid query] message="Undefined name columnaname in where clause ('columnname= 'value'')"
You don't need to put quotes around columnname, you just need to set it and prefix it with a $.
cqlsh -u cassandra -p cassandra -e "SELECT $COLUMN FROm $KEYSPACE.$TABLE;"
That's an excerpt from a script I wrote called getVersion.sh.
#!/bin/bash
KEYSPACE="system"
TABLE="local"
COLUMN="release_version"
~/local/apache-cassandra-3.10/bin/cqlsh -u cassandra -p cassandra -e "SELECT $COLUMN FROm $KEYSPACE.$TABLE;"
aploetz#dockingBay94:~/scripts$ ./getVersion.sh
release_version
-----------------
3.10
(1 rows)
The same will work if your column names contain quotes. Just be sure to escape them in your variable definition. This is a similar script, but it queries the "columnName" TEXT column:
#!/bin/bash
KEYSPACE="stackoverflow"
TABLE="stuff_with_quotes"
COLUMN="\"columnName\""
~/local/apache-cassandra-3.10/bin/cqlsh -u cassandra -p cassandra -e "SELECT $COLUMN FROm $KEYSPACE.$TABLE;"
I am running a impala query in while loop and for that I have created one separate query file and I am calling it from my shell script.
My question is: Can we pass shell variable matching with impala query in query file?
A="INSERT_SBP_ME_VS_ME_INCOME_LAST_THIRTY_DAYS_Q"${Count}
echo "value of A is $A"
source ${SBP2_MNY_IN_LAST_THIRTY_DAYS_QF}
${IMPALA_CON} -q "${${A}}"
'A' value is like INSERT_SBP_ME_VS_ME_INCOME_LAST_THIRTY_DAYS_Q1 (as count is 1)
I am doing this in this way but getting bad substitution error and I also tried
${IMPALA_CON} -q "${A}"
but not getting a successful result.
You seem to be looking for --var (IMPALA-2179). To substitute from the command line, you can do:
impala-shell -f test.q --var=L=2;
where test.q is:
select * from p_test limit ${VAR:L};
Your query should be :
impala-shell -q "$A"
Refer:
impala-shell Configuration Options
similar post
impala-shell -i node.domain:port -B --var"table=metadata" --var="db=retail" -f "file.sql"
file.sql:
SELECT * FROM ${var:db}.${var:table}"
I have a string in my bash shell like:
out=$(su - user -c "someCommand -f 'string text "problemString"'")
The problem here is that it's getting parsed as so:
out=\$(su - user -c \"someCommand -f 'string text \"problemString\"'\")
I don't want "problemString" to be parsed out -- i.e., it needs to stay exactly as-is, including the quotes. How can I do that?
Update: I've attempted to escape the inner " with:
out=$(su - user -c "someCommand -f 'string text \"problemString\"'"),
but when the command is executed on the host machine, it returns an error from someCommand:
Unknown command '\p'
Update 2:
Real example:
OUTPUT=$(su - mysql -c "mysql --skip-column-names --raw --host=localhost --port=3306 --user=user--password=pass -e 'show variables where variable_name = \"max_connections\"'")
I'm passing this bash script via fabric in Python:
# probably not relevant, but just in case..
def ParseShellScripts(runPath, commands):
for i in range(len(commands)):
if commands[i].startswith('{shell}'):
# todo: add validation/logging for directory `sh` and that scripts actually exist
with open(os.path.join(runPath, 'sh', commands[i][7:]),"r") as shellFile:
commands[i] = shellFile.read()
print commands[i]
return commands
This prints:
OUTPUT=$(su - mysql -c "mysql --skip-column-names --raw --host=localhost --port=3306 --user=pluto_user --password=pluto_user -e 'show variables where variable_name = \"max_connections\"'")
which then gets executed on some remote box via fabric, which results in ERROR at line 1: Unknown command '\m'.
You can write:
out=$(su - user -c "someCommand -f 'string text \"problemString\"'")
Simply use single quotes. Strings in single quotes don't get parsed or interpreted. For instance:
echo 'a"b'
outputs:
a"b
Because no parsing occurs.
For reference: bash manual on quoting.
I am running some commands to mongodb in a bash script but need to insert a string into a heredoc text. I am unable to get the value inserted correctly. How would this be done?
today=`date -d "00:00:00" +%s`
todaytime=$(($today*1000))
mongo <<EOF > test
use log
db.translogs.remove("{Dt: {$lt: new Date($todaytime)}}")
exit
EOF
I don't know mongodb, but it looks like $lt is part of a query and not a shell variable. But in your code, the shell is trying to expand it before passing the here-doc to mongo. So I think all you need to do is escape the $ in $lt:
today=`date -d "00:00:00" +%s`
todaytime=$(($today*1000))
mongo test
use log
db.translogs.remove("{Dt: {\$lt: new Date($todaytime)}}")
exit
EOF
If I replace mongo with cat in the above code, we see the commands-to-be-sent to mongo in the test file as:
use log
db.translogs.remove("{Dt: {$lt: new Date(1393315200000)}}")
exit
I am querying a database from a bash script using following query:-
Output = echo "$QUERY_STR" | mysql -h $DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME
it gives me the required output which I save in a Variable
However when I echo $output I do not get proper formatted output like in command line of mysql query.
Read one of the post to use -t in the query however for large data set it does not give proper output.
To work around it, I am saving the output in a .csv file.
To maintain all the whitespace that is indeed kept in the variable's value, it is crucial to double-quote the variable:
echo "$output"
Also, you cannot have whitespace around the equal sign in a variable assignment:
output=$(mysql ... <<< "$QUERY_STR")