This question already has answers here:
Insert into ... values ( SELECT ... FROM ... )
(27 answers)
Closed 7 years ago.
I want to create a script to automate some processes with Redshift. Specifically, I want find an attribute of one of my tables with a SELECT and then use it in a INSERT. My script looks like this:
psql -h ... -c "SELECT id_process FROM process WHERE de_process = 'EMR'"
psql -h ... -c "INSERT INTO execution (id_process) values (X);"
In the first sentence I get a unique value, the ID I'm looking for, in a format like this:
id_proceso
------------
2
(1 row)
Then I would like to use it as the value to insert in the second sentence, substituting the "X, but I don't know how to save into a variable and then reuse the output of the first sentence.
Any suggestion?
P.D. In other question it shows how to do it in a unique sentence, but I need to save the value for a future use.
Check psql options, but sample script can be the following:
psql -h localhost -d testdb <<EOF
\out sample.txt
\pset border 1
WITH test_data AS ( SELECT 2 AS id_process)
SELECT id_process FROM test_data;
\out
EOF
Result for cat sample.txt will be:
id_process
------------
2
(1 row)
If you want to get just pure value from SELECT statement, consider following params in the example above:
\t:
Toggles the display of output column name headings and row count
footer. This command is equivalent to \pset tuples_only and is
provided for convenience.
\pset format unaligned:
unaligned format writes all columns of a row on one line, separated by
the currently active field separator. This is useful for creating
output that might be intended to be read in by other programs (for
example, tab-separated or comma-separated format).
Related
This question already has answers here:
Shell script to properly align columns
(1 answer)
How to printf align my output like a table?
(1 answer)
Closed 9 months ago.
Im trying to have some symple output, the issue is that I can't seem to set the spaces. I have to manually calculate what each variable will have. Is there a way to format table outputs better with echo?
This code
echo """
Datasets available to download are:
Dataset - File Format
${dataset_one[6]} - ${dataset_one[5]}
${dataset_two[6]} - ${dataset_two[5]}
${dataset_three[6]} - ${dataset_three[5]}
You currently have downloaded to hdfs the the following datasets:
"""
Returns:
Dataset - File Format
dataset1 - csv
ds2 - csv
datasnytaxi - parquet
I think column -t is what you are looking for:
-t Determine the number of columns the input contains and create a table. Columns are delimited with whitespace, by default, or with the characters supplied using the -s option. Useful for pretty-printing displays.
Try using '\t' with echo command. you can also refer to Echo tab characters in bash script for how to use '\t' with echo in bash.
alternative you can also try printf;
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
while IFS=, read -r col1 col2 col3
do
email="$col3"
email+='#gmail.com'
echo $email
done < ~/Desktop/Names.csv
I don't have to do anything special with Column 1 or 2, but adding #gmail.com to column 3 just outputs #gmail.com or if the input string is longer #gmail.com+longer bit of input string.
ie.
If column 3 was Dekkars, I get #gmail.com. If it is aaaaaaaaaaa (one length longer than #gmail.com) I get #gmail.coma
I'm sure this is something to do with the # sign, but I've tried using \ to escape it, single quotes, etc. Any ideas?
I've already read concatenating bash strings, and I'm doing what it suggests with different outcomes than are expected.
Here is input Data
Test Name,8,aaaaaaaaaaa
John Doe,8,bbbbbbbbbbbb
Name,Grade,ID
(Note, I have columns at bottom because otherwise my while loop won't read the bottom row)
Output
#gmail.coma
#gmail.combb
dos2unix fixed the problem. Incorporated it into my script so I don't have to remember to call it every time I change the .csv...
Thank you all!
When I execute my query in PostgreSQL:
SELECT names
from user;
I obtain the following result:
names
--------------
Anna
Julius
Perico
(3 rows)
What I want to get is the following output:
Anna Julius Perico
I need this because is a bash script ant I need to save it in a variable.
The string_agg function might be what you want here:
select string_agg(names, ' ' order by names) from user;
I said "might" because this assumes that the names column itself could be used to generate the order in the single string output. If you want a different order, then you would need another column.
If you want to use it in a shell script, this would be the best way:
myvar=`psql --no-align --quiet --tuples-only --command='SELECT name FROM users'`
No need to have them in one line; the shell accepts a line feed as field separator as well:
for n in $myvar; do
echo "name: $n"
done
name: Anna
name: Julius
name: Perico
In our environment we do not have access to Hive meta store to directly query.
I have a requirement to generate tablename , columnname pairs for a set of tables dynamically.
I was trying to achieve this by running "describe extended $tablename" to a file for all tables and pick up tablename and column name pairs from the file.
is there any easier way it is done/it can be done other than this way .
The desired output is like
table1|col1
table1|col2
table1|col3
table2|col1
table2|col2
table3|col1
This script will print columns in desired format for single table. AWK parses strings from describe command, takes only column_name, concatenates with "|" and table_name variable, each string printed with \n as a delimiter between them.
#!/bin/bash
#Set table name here
TABLE_NAME=your_schema.your_table
TABLE_COLUMNS=$(hive -S -e "set hive.cli.print.header=false; describe ${TABLE_NAME};" | awk -v table_name="${TABLE_NAME}" -F " " 'f&&!NF{exit}{f=1}f{printf c table_name "|" toupper($1)}{c="\n"}')
You can easily modify it for generating output for all tables using show tables command for example.
The easier way is to access metadata database directly.
So I've searched quite a bit for this and I'm pretty new to Shell,
I want to iterate over Resultset rows in SHELL script and for each row I want to execute some code using each column of the current row.
Lets assume the resultset look like this.
Query Priority Size
---------------------------------------------------
this is a sentence to execute high 124400
this is another example low 15000000
...
So how do I manage to iterate over this Resultset and storing each column into his own variable? Here is an exemple for the first line:
var1="this is a sentence to execute"
var2="high"
var3=124400
#repeat process for next line
Here is what you could do:
Extract your data into a delimited file (say, csv), if possible, avoid having headers. If you do have headers, make sure to exclude them while reading.
Read data from the file into an array or a set of variables
like so:
# using an array - works great if we have a variable number of columns
while IFS=, read -r -a row; do
# ${row[0]} => column1, ${row[1]} => column2 and so on
# process data
done < input.dat
# using variables - works great if we have a fixed set of variables
while IFS=, read -r column1 column2 column3; do
# process data
done < input.dat
See also: How do I split a string on a delimiter in Bash?