I'm writing a bash one liner.
This works (prints date to the console or tries to execute it):
-bash-4.1$ DATE=`$(date --date="2 days ago" +%F)` echo "${DATE}"
But this:
`DATE=$(date --date="2 days ago" +%F)` psql -d some_db -c "select row from table where started >= '${DATE}' and started < ('${DATE}'::date + '1 day'::interval);"
Gives:
ERROR: invalid input syntax for type timestamp: ""
LINE 1: ... table where started >= '' ...
Which means that ${DATE} is empty there.
What should I modify to make it work?
The first command is broken for a couple of reasons. Firstly, you are mixing backticks and $() in such a way that you are executing the output of the date command as if it were the name of another command.
To save the output of a command to a variable, use this syntax:
output=$(command)
Secondly, the variable would be expanded by the shell before it had been assigned a value, so you need to split up the command into two statements:
date=$(date_command); psql -c "select ... '$date'"
...or use a command substitution directly:
psql -c "select ... '$(date_command)"
Why do the date math in bash? PG can do it directly:
SELECT ... WHERE started >= (datefield - interval '2 day')
^^^^^^^^^^^^^^^^^
Related
I have a file with the following entries:
MySql-DataBase-2020-09-22_183748.zip
MySql-DataBase-2020-09-22_184023.zip
MySql-DataBase-2020-09-23_205331.zip
MySql-DataBase-2020-09-23_205606.zip
MySql-DataBase-2020-09-24_200123.zip
MySql-DataBase-2020-09-24_200358.zip
MySql-DataBase-2020-09-25_115839.zip
MySql-DataBase-2020-09-25_120115.zip
MySql-DataBase-2020-09-26_094608.zip
MySql-DataBase-2020-09-26_094843.zip
MySql-DataBase-2020-09-27_122523.zip
MySql-DataBase-2020-09-27_122758.zip
MySql-DataBase-2020-10-01_230024.zip
MySql-DataBase-2020-10-01_230300.zip
MySql-DataBase-2020-10-02_120944.zip
MySql-DataBase-2020-10-02_121219.zip
MySql-DataBase-2020-10-03_151414.zip
MySql-DataBase-2020-10-03_151649.zip
MySql-DataBase-2020-10-04_211059.zip
MySql-DataBase-2020-10-04_211334.zip
MySql-DataBase-2020-10-05_064049.zip
MySql-DataBase-2020-10-05_064324.zip
I want to extract Files which are between today's date & 3 days back.
For today's date 2020/10/05, 3 days back is 2020/10/02.
The output must be:
MySql-DataBase-2020-10-02_120944.zip
MySql-DataBase-2020-10-02_121219.zip
MySql-DataBase-2020-10-03_151414.zip
MySql-DataBase-2020-10-03_151649.zip
MySql-DataBase-2020-10-04_211059.zip
MySql-DataBase-2020-10-04_211334.zip
MySql-DataBase-2020-10-05_064049.zip
I tried using this command to gets 3days back date
date --date='-3 day' '+%Y/%m/%d'
And then used these command to get output between date range
sed -n '/3day=date --date='-3 day' '+%Y/%m/%d'/,/date/p' s.txt
I am getting this error:
sed: -e expression #1, char 20: unterminated address regex
Please Help me to fix this issue. I'll be using this in a bash script.
Using a process substitution to generate the dates and feed that to grep as the patterns to search for:
grep -F -f <(for d in {0..3}; do date -d "$d days ago" "+%F"; done) file
sed -n "/$(date --date='-3 day' '+%Y-%m-%d')/,/$(date +'%y-%m-%d')/p"
MySql-DataBase-2020-10-02_120944.zip
MySql-DataBase-2020-10-02_121219.zip
MySql-DataBase-2020-10-03_151414.zip
MySql-DataBase-2020-10-03_151649.zip
MySql-DataBase-2020-10-04_211059.zip
MySql-DataBase-2020-10-04_211334.zip
MySql-DataBase-2020-10-05_064049.zip
Notice, the use of double quotes at the outermost level. Also, notice the same format of date in both boundaries.
I'm trying to run SQL against one-or-many psql-compatible hosts in parallel, with SQL run in sequence on each host, using xargs.
The bash script, which I'm sourcing from another script:
# Define the count of hosts (also the number of parallel processes)
export pe_fpe_hosts_line_count=$(cat $pe_fpe_hosts_file_loc | wc -l)
# Define the function that runs SQL from a file
function func_pe_exec_sql {
while read pe_sql_file; do
psql -q -t -c "\"$pe_sql_file"\"
done <$pe_fpe_sql_file_loc
}
export -f func_pe_exec_sql
# Define the xargs parallel function
function func_pe_parallel {
while read pe_hosts_file; do
echo $pe_hosts_file | xargs -d '\n' -P $pe_fpe_hosts_line_count func_pe_exec_sql
done <$pe_fpe_hosts_file_loc
}
The error I get: xargs: func_pe_exec_sql: No such file or directory. This is weird - I've exported the function!
Example SQL file:
INSERT INTO public.psql_test SELECT 1 as myint, now() as mytime;
INSERT INTO public.psql_test SELECT 2 as myint, now() as mytime;
INSERT INTO public.psql_test SELECT 3 as myint, now() as mytime;
INSERT INTO public.psql_test SELECT 4 as myint, now() as mytime;
INSERT INTO public.psql_test SELECT 5 as myint, now() as mytime;
Example SQL Host file:
--host=myhost1 --port=5432 --dbname=postgres --username=cooluser
--host=myhost2 --port=5432 --dbname=postgres --username=cooluser
pe_fpe_sql_file_loc is the path to the SQL file, and pe_fpe_hosts_file_loc is the path to the SQL Host file.
The SQL must always be run in separate transactions, and each row in the SQL file needs to be inserted separately, one after another. 5 should be in the same row as the greatest of the mytime values.
I am using it as an ETL framework with functions defined in the database though, and not for simple inserts :)
I think your invocation of xargs is incorrect. You are not actually passing the line from pe_hosts_file to the function func_pe_exec_sql.
You need to pass the input from the pipe to the function, to do that; you need to have a place-holder which -I flag in xargs provides.
-I replace-str
Replace occurrences of replace-str in the initial-arguments with names
read from standard input. Also, unquoted blanks do not terminate input items;
instead the separator is the newline character. Implies -x and -L 1.
Using that something like below needs to be used.
| xargs -d '\n' -I {} -P "$pe_fpe_hosts_line_count" bash -c 'func_pe_exec_sql "{}"'
where the {} is the place-holder for the value piped and we are passing it to the sub-shell spawned by bash -c directly to the function func_pe_exec_sql. The special double quotes around {} is to ensure, the shell to expand the value before the function is invoked.
So I am trying to output a file with the name of like: lastlogin-"yyyymmdd" where it contains the current date.
I figured out the date should be : date +"%Y%m%d" and I tried to do a variable
now = date +"lastlogin-%Y%m%d.txt"
filename = ${now}
xxxxx > ${filename}
but nothing seems to work
Please help
Use command substitution:
lastlogin-"$(date '+%Y%m%d')".txt
To save in a variable:
filename="lastlogin-"$(date '+%Y%m%d')".txt"
and then do:
echo 'foobar' >"$filename"
You should use $() for command execution and storage of result:
now=$(date +"lastlogin-%Y%m%d.txt")
I 'm using bash
I have a date string now, for example:
2015111301
(yyyymmddHH)
how to get the date string of next hour?
that is:
2015111302
Try this:
in="2015111301"
out="$(date -d "${in:0:8} ${in:8:2}:00:00 +1hour" '+%Y%m%d%H')"
echo "$out"
Output:
2015111302
See: 3.5.3 Shell Parameter Expansion
This below script worked for me
#!/bin/bash
tempval=$(echo 2015111301 | sed 's/\(.\{8\}\)/\1 /g')
tempval=$(echo "$tempval 1 hour")
date -d "$tempval" +%Y%m%d%H
I have a problem with a simple subtraction but I don't understand what's wrong.
My code :
start= date +%s%N | cut -b1-13
#Treatment...
end= date +%s%N | cut -b1-13
delta=`expr $end - $start`
echo "delta $delta"
My console display :
1374652348283
...
1374652349207
delta -
My question is : Why do I got a - symbol returned ?
The command:
a= b
(note the space) will set a to an empty string while it runs the command b. It's a way to temporarily set environment variables for a single command, things like:
PATH=/path/to/somwhere gcc whatever # Here, PATH has the modified value.
echo $PATH # Here, PATH has its original value.
So the command line:
start= date +%s%N | cut -b1-13
sets start temporarily to nothing and runs the date command. Hence both start and end are still empty when you use them, which is why you only get the -, since expr - just gives you -.
If you want to get the results of the date command into a variable, use:
start=$(date +%s%N | cut -b1-13)
You didn't assign to the variables. You must not have spaces around the equals sign.
Also, you're doing it wrong.
start=$(date +%s%N | cut -b1-13)