I am fetching the data from psql in the shell script and assign to the global variable but the global variable is not updating below i have tried:
#!/bin/bash
res_count=0
psql -h $db_host -U $db_user -d $db_name -At -c "select count(id) as dCount from abc" --no-password --field-separator ' ' | \
while read dCount ; do
res_count=$dCount
done;
echo $res_count
$res_count is not updating, it has still value 0, please correct me where i am wrong thanks
Your while loop executes in a subshell because it is executed as part of the pipeline. You can avoid it by using lastpipe or placing the psql command inside process substitution.
#/bin/bash
shopt -s lastpipe
...
Or
res_count=0
while read dCount ; do
res_count=$dCount
done < <(psql -h "$db_host" -U "$db_user" -d "$db_name" -At \
-c "select count(id) as dCount from abc"
--no-password --field-separator ' ')
echo "$res_count"
As a side note, quote your variables properly.
Related
I'm trying to make the private key SSH connection with LDAP.
/etc/ssh/sshd_config
AuthorizedKeysCommand /etc/ldap_ssh_authorized_keys.sh
AuthorizedKeysCommandUser nobody
Script to get public keys from LDAP server
/etc/ldap_ssh_authorized_keys.sh
#!/bin/bash
USERSLIST=$( ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" -s sub '(objectClass=posixAccount)' -u 'uid' \
grep '^uid:' | sed -n '/^ /{H;d};/uid:/x;$g;s/\n *//g;s/uid: //gp' \
)
while IFS= read -r line; do
exists=$(ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" \
-s sub "(&(objectClass=posixGroup)(cn=sysadmin)(memberUid=${line}))" | grep "^# numEntries:")
if [[ ! -z $exists ]]
then
ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" \
-s sub "(&(objectClass=posixAccount)(uid=${line}))" \
-u 'sshPublicKey' \
| sed -n '/^ /{H;d};/sshPublicKey:/x;$g;s/\n *//g;s/sshPublicKey: //gp'
echo -e "";
fi;
done <<< "$USERSLIST"
When I'm running script with /bin/bash it's working well and return my public keys.
All environment variables defined normally.
LDAP_URI
LDAP_BASEDN
LDAP_USER
LDAP_PASSWORD
The script also running normally when trying to make an SSH connection. But environment variables not available.
I'm trying also with AuthorizedKeysCommandUser as root. But nothing changed.
I solved this problem by getting the environment variables from /proc/1/environ.
Reference
I have a psql command where I am able to get result but i unable to remove footer
my command
sshpass -p 'password' ssh mptios#xx.xx.xxx.xxx "PGPASSWORD=xxxxx psql -a -h 11.11.111.11 -d TGM_bb_les -U bi_it -t -c \"select count(1) from dwpub.td_bank \" "
my result for above command :-
select count(1) from dwpub.td_bank
29
but I need output as
29
I have tried this command but still unable to get expected output
sshpass -p 'password' ssh mptios#xx.xx.xxx.xxx "PGPASSWORD=xxxxx psql -a -h 11.11.111.11 -d TGM_bb_les -U bi_it --pset\"footer=off\" -c \"select count(1) from dwpub.td_bank \" "
can anyone help me with this
Use grep or egrep to match digits only.
sshpass -p 'password' ssh mptios#xx.xx.xxx.xxx "PGPASSWORD=xxxxx psql -a -h 11.11.111.11 -d TGM_bb_les -U bi_it --pset\"footer=off\" -c \"select count(1) from dwpub.td_bank \" " | grep -Eo '[0-9]+$'
Try with command:
sshpass -p 'password' ssh mptios#xx.xx.xxx.xxx "PGPASSWORD=xxxxx psql -a -h 11.11.111.11 -d TGM_bb_les -U bi_it -t -c \"select count(1) from dwpub.td_bank \" "|tail -1
This will show only the last line
I'm having difficulty getting the correct variable substitution to work within this function, especially the use of echo "$(...) string.
outputOFF ()
{
host='mydevreporting.com'
_pw='123456foobar'
_dt=$(date +'%m-%d-%y')
exp="SELECT * FROM metrics.account_use where account='foo' and profile='bar' order by date desc;";
echo "$(grep real < <({ time mysql -u admin -p${_pw} -h ${host} -N -e "$exp"} 2>&1)):localhost:${_dt}"
#echo "$(grep real < <(\{ time mysql -u admin -p${_pw} -h ${host} -N -e "$exp"\} 2>&1)):localhost:${_dt}"
}
From the command line, it will work:
echo "$(grep real < <({ time mysql -u admin -p123456foobar -h mydevreporting.com -N -e "SELECT * FROM metrics.account_use where account='foo' and profile='bar' order by date desc;"; } 2>&1)):localhost:$(date +'%m-%d-%y')"
As I'm seeing error messages:
./tst.sh: command substitution: line 40: syntax error near unexpected token `)'
./tst.sh: command substitution: line 40: `{ time mysql -u admin -p${_pw} -h ${host} -N -e "$exp"} 2>&1)'
As you can see the echo "$(...) is inside this function outputOFF().
I've also tried escaping the braces \{, \}, which allows the variables to substitute, but somehow that command isn't working as it should.
echo "$(grep real < <(\{ time mysql -u admin -p${_pw} -h ${host} -N -e "$exp"\} 2>&1)):localhost:${_dt}"
So, i'm stuck.
You are missing a ; in the {...} group expression, after "$exp" (see this documentation for why). Here is the corrected version:
echo "$(grep real < <({ time mysql -u admin -p${_pw} -h ${host} -N -e "$exp"; } 2>&1)):localhost:${_dt}"
Less braces makes it more readable, at least for me.
result=$( { time mysql -u admin -p${_pw} -h ${host} -N -e "$exp"; } 2>&1 | grep real );
echo "${result}:localhost:${_dt}"
time is reporting on stderr. Thus the {} are neceessary to capture the output.
Or discarding the result, and only capture the result of time.
result=$( { time mysql -u admin -p${_pw} -h ${host} -N -e "$exp" >/dev/null; } 2>&1 )
echo ${result}":localhost:${_dt}"
The unquoted ${result} is printed without the newlines. Thus you can keep all information from time with the additional timestamp.
is there any way, by which I can generate a DDL of GP Functions
My actual requirement is I want to create/generate DDL of each objects available in particular schema in separate SQL file
I am able to do it for tables and views via below script
tbl_list=`psql -At -c "select tablename from pg_tables where schemaname ='${PGSCHEMA}' and tablename not like '%_prt_%' order by 1;"` # Fetch all table Name
for fname in $tbl_list
do
ddl=`PGPASSWORD='passwd' pg_dump -h 10.128.19.297 -U gpadmin jiodata -s -t "${PGSCHEMA}.$fname" >${script_dir}/${output_dir}/$fname.sql` # Fetch ddl for all tables
#pg_dump -h 10.128.19.297 -U gpadmin jiodata -s -t "${PGSCHEMA}.$fname" >${script_dir}/${output_dir}/$fname.sql
echo "Table DDL generated : "${PGSCHEMA}.$fname | tee -a ${log_file}
done
Did any one tried it for Functions
Please help if anyone know
Thanks
Create a sql file named "get_functions.sql" with this code:
select sub.function_name || '(' || array_to_string(array_agg(typname), ',') || ')'
from (
select n.nspname as schema_name, proname as function_name, unnest(proallargtypes) as parm_type, unnest(proargmodes) as parm_direction
from pg_proc p
join pg_namespace n on p.pronamespace = n.oid
where n.nspname = :schema_name
) as sub
join pg_type t on sub.parm_type = t.oid
where sub.parm_direction = 'i'
group by sub.function_name;
Next, create a bash script with this:
#!/bin/bash
set -e
schema_name="$1"
if [ "$schema_name" == "" ]; then
echo "ERROR: You must provide the schema name."
echo "Example usage: ./runme.sh gp_toolkit"
exit 1
fi
echo "pg_dump -Fc -s -n $schema_name > $schema_name.sql"
pg_dump -Fc -s -n $schema_name > $schema_name.sql
for i in $(psql -t -A -v schema_name="'$schema_name'" -f get_functions.sql); do
filename=$(echo $i | tr \( _ | tr \) _)
filename+=".sql"
echo "cat $schema_name.sql | pg_restore -P '$i' > $filename"
cat $schema_name.sql | pg_restore -P ''$i'' > $filename
done
Fix the permissions on the script.
chmod 755 runme.sh
And execute the script.
./runme.sh gp_toolkit
This will create a file per function and it allows for overloaded functions too.
Is there some way to make psql separate the fields and records by \0, aka NUL? It's the only way to be able to pass arbitrary data to Bash scripts.
Based on Matthew Wood's answer, I would expect this to print more that 1 on a newly initialized database:
declare -i count=0
echo "\pset recordsep '\000'
\f '\000'
select typname from pg_type" | \
sudo -iu postgres psql --no-align --quiet --tuples-only -d dbname -U username | while IFS= read -r -d ''
do
#echo "$REPLY"
let count++
done
if [ -n "$REPLY" ]
then
#echo "$REPLY"
let count++
fi
echo $count
Workaround: Iff the SELECT results are unique, you can use this workaround to handle one at a time:
next_record() {
psql --no-align --quiet --tuples-only -d dbname -U username <<SQL
SELECT colname
FROM tablename
WHERE colname > '${1}'
ORDER BY colname
LIMIT 1
SQL
}
last_col=
while true
do
colx="$(next_record "$last_col"; printf x)"
if [ "$colx" = x ]
then
exit
fi
col="${colx%$'\nx'}" # The extra \n character is from psql
# Do your thing here
col_escaped="${col//"'"/''}" # Double single quotes
col_escaped="${col_escaped//\\/\\\\}" # Double backslashes
last_col="$col_escaped"
done
This is not supported. psql uses C print functions to print out the result tables, and printing a zero byte just doesn't work there.
Update: This is now supported in PostgreSQL 9.2-to-be (git).
Try this:
psql --field-separator '\000' --no-align -c '<your query>'
Edit: Maybe not. However, it appear to work in psql using these commands:
\f '\000'
\a
Newer versions of psql support the --field-separator-zero flag.