I've a huge amount of file that are named like A2012178.1220.051.2012178233531.hdf There, from the 2nd character to the 8th is the date, year plus julian day. From the 13rd character to the extention of the file the name is rubbish to me... so I would like to rename the files and convert the date (for convenience).
So far I'm able to convert the date in bash
CONVERTED=$(date -d "${Year}-01-01 +${JulianDay} days -1 day" "+%Y%m%d")
But I have no idea of how to read the year and julian day from the file name and replace the name within the bash script....
Any idea?
Use parameter substitution to get the substrings:
name="A2012178.1220.051.2012178233531.hdf"
Year=${name:1:4}
JulianDay=${name:5:3}
The tricky part is that the julian day has a variable length, so you don't know ahead of time how to slice it out. So, you might do something like this:
read begin date end < <(echo $filename | sed -e 's/\(.\)\([[:digit:]]\+\)\(.*\)\
/\1 \2 \3/')
Year=${date::4}
JulianDay=${date:4}
newdate=$(date -d "${Year}-01-01 +${JulianDay} days -1 day" "+%Y%m%d")
mv $filename "${begin}${newdate}${end}"
This basically splits up the file name pulling out the date field and then grabbing the first 4 digits as the year with the rest being the julian day.
(This assumes the julian day isn't 0 padded, which isn't clear from your question. If it's zero padded, it's much easier).
Related
I am stuck in one situation where I am having a bunch of files and I need to pick the oldest one on the basis of time present in name only. Not on basis of the timestamp as I am doing SCP from one system to another so timestamp would be same for all the files once SCP runs
I have files like
UAT-2019-03-21-16-31.csv
UAT-2019-03-21-17-01.csv
AIT-2019-03-21-17-01.csv
Here, 2019 represents the year, 03 the month, 21 the day, 16 the hours in 24-hour format and 31 represent the minutes.
I need to pick the UAT-2019-03-21-16-31.csv file from the above files first.
How can I do in shell scripting.
I tried doing ls -1 but it will sort alphabetically, that means AIT-2019-03-21-17-01.csv will be picked first, but I need according to time mentioned in the file name
You can try this
ls -1 | sort -t"-" -k2 -k3 -k4 -k5 -k6 | head -n1
Output :
UAT-2019-03-21-16-31.csv
Curious about alternatives answer as I know that parsing ls output is not ideal.
The best and efficient way to do this would be to convert the filename time stamp to epoch time and find the oldest among them.
You need to write a script that does below in order:
Get all the filename timestamp into a variable.
Convert all filename timestamp to epoch time.
Find the oldest and get the filename.
Command to convert the filename timestamp to epoch time would be
date -d"2019-03-21T17:01" +%s
date -d"YYYY-MM-DDTHH:MM" +%s
You can try these steps in script
Hope so this helps you to start writing the script.
I have a .txt file with many lines of text on macOS. I would like to filter only dates and have them saved in order of appearance line by line in a new text file.
I am, however, not interested in all dates, only in those who are complete, looking like 02/03/2019, and those where the number of days is below 13, i. e. 01...12.
Then, I would like to have those dates removed where the number for the day and month are the same like 01/01/2019 and 02/02/2019 etc.
How can I achieve this with awk or similar software in bash?
If perl is a choice:
perl -ne 'print if m:(\d\d)/(\d\d)/(\d\d\d\d): && $1 < 13 && $1 != $2' dates.txt >newdates.txt
this assumes this format /dd/mm/yyyy
Note that I am using a m: : notation instead of the usual / / for regex matching. Thus I do not need to escape the / slashes in the date.
Deleting Dates Inside a Text File
The following command will delete all dates of the form✱ aa/bb/cccc where aa = bb < 13. The original file will be copied to yourFile.txt.bak as a backup and the new text with deleted dates will overwrite the old file.
sed -E -i.bak 's:\b(0[0-9]|1[0-2])/\1/[0-9]{4}\b::g' yourFile.txt
If you want to insert something instead of just deleting the dates you can do so by writing the replacement between the two ::. For instance sed … 's:…:deleted date:/g' … will replace each matching date with the text deleted date.
✱ Note that it doesn't matter for your criterion whether the date format is dd/mm/yyyy or mm/dd/yyyy since your are only interested in dates where dd and mm are equal.
Extracting Specific Dates From A Text File
If you do not want to delete, but only extract specific dates as mentioned in your comment, you can use the following command.
grep -Eo '\b([0-9]{2}/){2}[0-9]{4}\b' yourFile.txt | awk -F/ '$1<13 && $1!=$2'
This will extract all dates in dd/mm/yyyy (!) format where mm ≠ dd < 13. The dates are printed in order of appearance on stdin. If you want to save them to a file append > yourOutputFile.txt to the end of the command.
i try to delete all days of a csv file which not matched last days. But I find not the right solution.
date,price
2018-07-02,162.17
2018-06-29,161.94
2018-06-28,162.22
2018-06-27,162.32
2018-06-12,163.01
2018-06-11,163.53
2018-05-31,164.87
2018-05-30,165.59
2018-05-29,165.42
2018-05-25,165.96
2018-05-02,164.94
2018-04-30,166.16
2018-04-27,166.69
The output I want become
date,price
2018-06-29,161.94
2018-05-31,164.87
2018-04-30,166.16
I try it with cut + grep
cut -d, -f1 file.csv | grep -E "28|29|30"
Work but bring nothing when combine -f1,2.
I find csvkit which seem to me the right tool, but I find not the solution for multiple grep.
csvgrep -c 1 -m 30 file.csv
Bring me the right result but how can combine multiple search option? I try -m 28,29,30 and -m 28 -m 29 -m 30 all work not. Best it work with last day of every month.
Maybe one have here a idea.
Thank you and nice Sunday
Silvio
You want to get all records of the LAST day of the month. But months vary in length (28-29-30-31).
I don't see why you used cut to extract the first field (the date part), because the data in the second field does not look like dates at all (xx-xx).
I suggest to use grep directly to display the lines that matches the following pattern mm-dd; where mm is the month number, and dd is the last day of the month.
This command should do the trick:
grep -E "01-31|02-(28|29)|03-31|04-30|05-31|06-30|07-31|08-30|09-31|10-30|11-31|12-30" file.csv
This command will give the following output:
2018-05-31,164.87
2018-04-30,166.16
I want to store yesterday's date in BASH variable to search for yesterday's files with that variable in the file-name wildcard search. I am using the following format for New York City, NY, USA (EST) time zone, and wanted to know whether it is guaranteed to fetch yesterday's date from the system date; else I can make further changes to the variable.
yesterday=$(TZ=GMT+28 date +%Y%m%d)
...
for file in $HOME_DIR/*$yesterday*.txt;
...
The text filename in HOME_DIR would be as follows:
"ABC_20171011064612.txt"
update 1: Attempt for removing daylight savings related issues:
yesterday=$(echo -e "$(TZ=GMT+28 date +%Y%m%d)\n$(TZ=GMT+18 date +%Y%m%d)"|grep -v $(date +%Y%m%d)|sort|tail -1)
1) Convert two dates to string, 24 hours and 14 (picked arbitrarily to be less than 24 hours) hours before today's date
2) Filter for dates that are not today's date
3) Sort strings from 2) in ascending order
4) Assign yesterday variable to last tail -1 entry of the list
It may not be always right due to DST, although it will not be a big issue.
You could rather say:
yesterday=$(date -d yesterday +%Y%m%d)
You attempted
yesterday=$(echo -e "$(TZ=GMT+28 date +%Y%m%d)\n$(TZ=GMT+18 date +%Y%m%d)|
grep -v $(date +%Y%m%d)|sort|tail -1)
I think it worked.
I want to read a file which contain a date and rewrite this file with plus one day
I have this command line :
date --date=$(cat /parm/control.date) "+%Y-%m-%d" -d "tomorrow" > control.date
When I run many times this command it return tomorrow of the current date ..
Thank you :)
date --date="$(< /dz-ceibo/applis/db5/parm/control.date) + 1 day" "+%Y-%m-%d"
Using bash's $(< file) instead of cat