I'm writing a script that sends a user an email when their AWS access keys are too old.
Right now the output looks like this:
Hello tdunphy, \n Your access key: AKIAICDOHVTMEAB6RM5Q was created on Wed Feb 7 22:55:51 EST 2018 and needs to be replaced. All AWS Keys need to be replaced if they are older than 90 days. \n Regards, \n Cloud Ops
I want the message to look like this:
Hello tdunphy,
Your access key: AKIAICDOHVTMEAB6RM5Q was created on Wed Feb 7 22:55:51 EST 2018 and needs to be replaced. All AWS Keys need to be replaced if they are older than 90 days.
Regards,
Cloud Ops
This is the line that sets up the body of the message:
MAIL_TXT1="Hello $user_name, \\n Your access key: $user_access_key1 was created on $date1 and needs to be replaced. All AWS Keys need to be replaced if they are older than 90 days. \\n Regards, \\n Cloud Ops"
Why are the newlines not working in this example? How can I get this to work?
You can use:
template=$'Hello tdunphy, \n Your access key: AKIAICDOHVTMEAB6RM5Q was created on Wed Feb 7 22:55:51 EST 2018 and needs to be replaced. All AWS Keys need to be replaced if they are older than 90 days. \n Regards, \n Cloud Ops'
Then either echo works:
$ echo "$template"
Hello tdunphy,
Your access key: AKIAICDOHVTMEAB6RM5Q was created on Wed Feb 7 22:55:51 EST 2018 and needs to be replaced. All AWS Keys need to be replaced if they are older than 90 days.
Regards,
Cloud Ops
Or printf:
$ printf "%s" "$template"
Your access key: AKIAICDOHVTMEAB6RM5Q was created on Wed Feb 7 22:55:51 EST 2018 and needs to be replaced. All AWS Keys need to be replaced if they are older than 90 days.
Regards,
Cloud Ops
With printf you can assemble the template fields more easily as well:
$ t2=$'Fields:\n\tf1=%s\n\tf2=%s'
$ printf "$t2" 101 102
Fields:
f1=101
f2=102
Related
Basically, I want to take a time, and day of the week, in UTC+8, and adjust the datetime object to a given UTC offset, within bash, I don't have any code to show because I'm not sure how to start attempting this in the first place honestly
(I'm writing a custom script for a friend who lives in UTC+8 and want to make the input as easy as possible for them, basically they just give it a time in their timezone, and a day of the week, and it'll tell them what date and time that'll be in a different timezone, for an overarching purpose)
For a reference, look at the section 1 of the manual page for "date":
In your shell, just type: man 1 date
or see the online man page:
https://man7.org/linux/man-pages/man1/date.1.html
One way is to parse the date into the number of seconds since epoch (since 1970), and then convert that number of seconds into the format you want:
For example:
$ date +%s --date='2022-12-27 11:30:17 +008'
1672140137
$ date +%c --date='#1672140137'
Tue 27 Dec 2022 06:22:17 AM EST
or you could also convert to ISO format then back to local time
$ date -Iseconds --date='TZ="GMT" 2022-12-22 11:33:44 +08'
2022-12-21T22:33:44-05:00
$ date --date='2022-12-21T22:33:44-05:00'
Wed 21 Dec 2022 10:33:44 PM EST
I hope this helps you get started with some ideas for converting to/from different timezones.
Also, to help with user input, you can show the current month calendar using cal
$ cal
December 2022
Su Mo Tu We Th Fr Sa
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
$ date --date='Fri 08:30'
Fri 30 Dec 2022 08:30:00 AM EST
In the above example, I specified "Fri 08:30" which gets set to the next Friday at 08:30 in the morning for my local timezone.
I am using Elastcsearch 7.9 and elasticdump 6.7
I am trying to get a dump (.json) file from elastcsearch with all the documents of a index.
I am getting
Thu, 27 May 2021 06:26:35 GMT | starting dump
Thu, 27 May 2021 06:26:35 GMT | Error Emitted => URI malformed
Thu, 27 May 2021 06:26:35 GMT | Error Emitted => URI malformed
Thu, 27 May 2021 06:26:35 GMT | Total Writes: 0
Thu, 27 May 2021 06:26:35 GMT | dump ended with error (get phase) => URIError: URI malformed
My command
elasticdump \
--input=https://username:password#elasticsearchURL:9200/index \
--output=/home/ubuntu/dump.json \
--type=data
Here the problem is password have many special characters.
I cant change the password.
I tried
quotes for password.
Escape special character.
Encoding the url
for all cases I am getting same error
Please help me to send password with special characters(# & % ^ * : ) , $)
Thanks in advance.
actually, ı tried elastisdump but properly not working,you should use elastic snapshot,maybe working in less data but if data is big elasticdump wont work
loook at the elastic snapshot
snapshot
for example using snapshot
PUT /_snapshot/my_fs_backup?pretty
{
"type": "fs",
"settings": {
"location": "/home/zeus/Desktop/denesnapshot",
"compress": true
}
}
#chmod 777 /home/zeus/Desktop/denesnapshot
#create repo
PUT /_snapshot/my_fs_backup/deneme
#get all repo
GET /_snapshot/my_fs_backup/_all
#if you want delete repo
DELETE /_snapshot/my_fs_backup/deneme
#restore
POST /_snapshot/my_fs_backup/deneme/_restore
extra you should add
path.repo: /home/zeus/Desktop/denesnapshot
to elasticsearch.yml file
I imported 1.5 million data and that was 300GB
I made mistake that I have encoded the complete URL.Don't encode the complete URL.
Encode the password while adding password into url.
for example
Password - a#b)c#d%e*f.^gh
Encoded password - a%40b%29c%23d%25e*f.%5Egh
My script will be:
elasticdump \
--input=https://username:a%40b%29c%23d%25e*f.%5Egh#elasticsearchURL:9200/index \
--output=/home/ubuntu/dump.json \
--type=data
Please refer ASCII Encoding Reference for encoding the password
I have text file TEST.txt which has below data which is unformated:
0411 14:30:00 INF[baag.reporting.main.Logss.ExecuteLogsRunnable] Executing cron report Freigabe 14:30 for cron job Freigabe 14:30 for TRE_ClientServiceGroup#TEST.fs, Businesspartner#TEST.fs
0411 14:30:02 INF[baag.reporting.main.Logss.ExecuteLogsRunnable] Freigaben had no results
0411 14:30:02 INF[baag.reporting.main.Logss.ExecuteLogsRunnable] Freigabe 14:30 NOT sent to TRE_ClientServiceGroup#TEST.fs, Businesspartner#TEST.fs since all reports were empty and empty reports should not be send
0411 17:03:14 INF[baag.reporting.db.DataSourceMapFactory] Datasource [itraderdbint] has been added to datasource map
0411 17:03:14 INF[baag.reporting.db.DataSourceMapFactory] Datasource [otc_sv2599] has been added to datasource map
0411 17:03:14 INF[baag.reporting.db.DataSourceMapFactory] Datasource [qlp_devp] has been added to datasource map
0411 17:03:15 INF[baag.reporting.main.Logss.QuarzLogsManager] Added Trigger for QUARTZ that fires next on Tue Apr 13 08:00:00 CEST 2021 for Logs Compliance MAR Crossingprüfung/Frontrunning DI-FR
0411 17:03:15 INF[baag.reporting.main.Logss.QuarzLogsManager] Added Trigger for QUARTZ that fires next on Tue Apr 13 08:20:00 CEST 2021 for Logs Compliance OR Umsatzstatistik DI-FR
0411 17:03:15 INF[baag.reporting.main.Logss.QuarzLogsManager] Added Trigger for QUARTZ that fires next on Mon Apr 12 08:20:00 CEST 2021 for Logs Compliance OR Umsatzstatistik MO
Now i want to create Shell script which will prepare this unformated data into below format and create for example PrepardFile.txt. I want to separate every string with pipe operator. The first part is date format so i want this as complete string. The second part always start with INF[ and ends with ] or we can take the complete part without spaces starting from INF[ and this would be my second string separated as pipe operator. The third part will be the remaining part which would be my third string. I want to add header for better understanding of what does this field value indicate:
DATE_FORMAT|ROW_EXECUTE|ROW_VALUE
0411 14:30:00|INF[baag.reporting.main.Logss.ExecuteLogsRunnable]|Executing cron report Freigabe 14:30 for cron job Freigabe 14:30 for TRE_ClientServiceGroup#TEST.fs, Businesspartner#TEST.fs
0411 14:30:02|INF[baag.reporting.main.Logss.ExecuteLogsRunnable]|Freigaben had no results
0411 14:30:02|INF[baag.reporting.main.Logss.ExecuteLogsRunnable]|Freigabe 14:30 NOT sent to TRE_ClientServiceGroup#TEST.fs, Businesspartner#TEST.fs since all reports were empty and empty reports should not be send
0411 17:03:14|INF[baag.reporting.db.DataSourceMapFactory]|Datasource [itraderdbint] has been added to datasource map
0411 17:03:14|INF[baag.reporting.db.DataSourceMapFactory]|Datasource [otc_sv2599] has been added to datasource map
0411 17:03:14|INF[baag.reporting.db.DataSourceMapFactory]|Datasource [qlp_devp] has been added to datasource map
0411 17:03:15|INF[baag.reporting.main.Logss.QuarzLogsManager]|Added Trigger for QUARTZ that fires next on Tue Apr 13 08:00:00 CEST 2021 for Logs Compliance MAR Crossingprüfung/Frontrunning DI-FR
0411 17:03:15|INF[baag.reporting.main.Logss.QuarzLogsManager]|Added Trigger for QUARTZ that fires next on Tue Apr 13 08:20:00 CEST 2021 for Logs Compliance OR Umsatzstatistik DI-FR
0411 17:03:15|INF[baag.reporting.main.Logss.QuarzLogsManager]|Added Trigger for QUARTZ that fires next on Mon Apr 12 08:20:00 CEST 2021 for Logs Compliance OR Umsatzstatistik MO
I am very new to Shell script and dont know if this possbile to do with the help of shell script.
#Symonds
This response is regarding your comment asking for adding a header section and further explanation.
To add header section, you can use echo and create the PreparedFile.txt first. Then use >> operator to append to the file. You can copy the complete code to a file named Script.sh and then run it using bash Script.sh
#!/bin/bash
echo "DATE_FORMAT|ROW_EXECUTE|ROW_VALUE" > PreparedFile.txt
cat TEST.txt | sed 's/ /|/2' | sed 's/] /]|/1' >> PreparedFile.txt
As far as the explanation you have asked for, you can chain commands using the pipe symbol |. The sed command allows you to substitute occurrences of regular expressions you specify with a replacement. In my first pipeline following cat command, I use s/ /|/2. This means replace the second occurence of blank space with |. You can read more about the sed command usage here.
You can use the below Shell script and see if it helps. It uses sed command and combination of pipes to replace second occurrence of space first and then the closing square bracket.
cat TEST.txt | sed 's/ /|/2' | sed 's/] /]|/1' > PreparedFile.txt
I have extracted the current month using
currentMonth=`date +"%b"`
I want to get all previous five months in a file. Suppose its October now, i want Sep, Aug, Jul, Jun, May in a file. I tried using multiple utilities like cal, ncal, date but could not achieve this.
I want the months in the file as
Sep
Aug
Jul
Jun
May
Any help or suggestion is greatly appreciated.
Simple bash/shell solution (for Unix):
for i in 1 2 3 4 5; do date -d" - $i month" +%b; done
The output:
Sep
Aug
Jul
Jun
May
For MacOS:
for i in {1..5}; do date -v-"$i"m +%b; done
I would like to store my incoming e-mails as *.msg files and i try this with the following external delivery method; This is working well but i receive the mail as one large plain text string.
foofoofoo unix - n n - - pipe
flags=F user=www-data argv=/mle/scripts/parse_postfix.sh ${sender} ${recipient}
My parse_postfix.sh contains the following code;
#!/bin/bash
mail=$(cat);
ddate=`date +%Y%m%d_%H%M%S`
msg_file=${ddate}.msg
path=/mle/spool/${code}
echo $mail > ${path}/${msg_file};
The output of this method is one large string and i can't read it with a MIME:Parser;
(example 1)
From foo#gmail.com Sat Aug 23 19:22:27 2014 Received: from mail-yh0-f49.google.com (mail-yh0-f49.google.com [209.85.213.49]) by foo.testtesttest.com (Postfix) with ESMTPS id D83DC43C8 for <foo#testtesttest.com>; Sat, 23 Aug 2014 19:22:26 +0000 (UTC) Received: by mail-yh0-f49.google.com with SMTP id b6so9924529yha.36 for <foo#testtesttest.com>;
The example mail listed below is the way how i can read it properly with the MIME:Parser;
(example 2)
Delivered-To: foo#testtesttest.com
Received: by 10.140.108.135 with SMTP id j7csp222526qgf;
Thu, 3 Jul 2014 06:26:51 -0700 (PDT)
X-Received: by 10.194.82.106 with SMTP id h10mr3033237wjy.115.1404394010626;
Thu, 03 Jul 2014 06:26:50 -0700 (PDT)
Return-Path: <foo#gmail.com>
Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [2a00:1450:400c:c03::22c])
by mx.google.com with ESMTPS id e6si24028130wix.75.2014.07.03.06.26.50
for <foo#testtesttest.com>
(version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
Thu, 03 Jul 2014 06:26:50 -0700 (PDT)
Received-SPF: pass (google.com: domain of foo#gmail.com designates 2a00:1450:400c:c03::22c as permitted sender) client-ip=2a00:1450:400c:c03::22c;
What is the best way to convert the 1st example to the 2nd example? Or to save the e-mail properly like the 2nd example?
You are not quoting the variable properly.
echo "$mail"
The sane solution is to not use a variable at all, though.
ddate=$(date +%Y%m%d_%H%M%S)
msg_file=${ddate}.msg
path=/mle/spool/${code}
cat> ${path}/${msg_file}
This will still be problematic if there is more than one message arriving during the same second.
I cannot see why you are not using the built-in delivery mechanisms of Postfix itself, anyway. If you want more control over delivery, maybe hook in Procmail, which has a built-in delivery mode for running numbering of messages. Proper Maildir would still be better for most real-world scenarios, as it is more robust.