Converting CSV to Column separated values but placing the values most right - shell

Concept is a debian running computer with an SH script file in retrieving data.
I have searched on how to convert data from csv back to columns and rows. This was fairly easy.
for instance, i have this:
And using this command: column -t -s, > this will gives me this:
Date SkyT_Min SkyT_Min_Time SkyT_Max SkyT_Max_Time SkyT_Mean AmbT_Min AmbT_Min_Time AmbT_Max AmbT_Max_Time
2019-09-19 -22.8 07:29:48.00 -1.6 12:27:57.00 -16.77 5.9 05:15:11.00 23.4 14:28:49.00
2019-09-25 -15.8 11:49:40.00 9.1 20:17:11.00 1.07 14.7 02:47:10.00 21.7 11:46:38.00
2019-09-26 -9.6 02:59:29.00 10.8 11:09:18.00 5.66 16.4 20:58:37.00 23.4 14:08:58.00
So overall that is already great!
But how do i get it to get the values in each column to start on the far right, instead on the left?
So i would get something like this:
Thank you very much for your answer.
UPDATE:
I have found the answer to my question:
--R
But it gives me errors: "invalied option -- 'R'
So if anyone has any idea. :)

You can work with awk:
-F tells awk the column separator
"%20s " tells awk to reserve 20 characters for each column and to concat a whitespace to each entry
awk -F "," '{for(i=1;i<=NF;i++){printf "%20s ", $i}; printf "\n"}' inputfile.csv
Output:
Date SkyT_Min SkyT_Min_Time SkyT_Max SkyT_Max_Time SkyT_Mean AmbT_Min AmbT_Min_Time AmbT_Max AmbT_Max_Time
2019-09-19 -22.8 07:29:48.00 -1.6 12:27:57.00 -16.77 5.9 05:15:11.00 23.4 14:28:49.00
2019-09-25 -15.8 11:49:40.00 9.1 20:17:11.00 1.07 14.7 02:47:10.00 21.7 11:46:38.00
2019-09-26 -9.6 02:59:29.00 10.8 11:09:18.00 5.66 16.4 20:58:37.00 23.4 14:08:58.00
P.S.: Please avoid screenshots in the future because nobody likes them.

Related

Shell script to find difference between time stamps in milliseconds unix operating system

I am new to shell scripting.
This is the sample date and time that was saved in the UNIX system.
I'm going to use a shell script to cut this date part here.
20221010042234.671 - 20221010042234.491
20221010132747.826 - 20221010132747.712
20221010060016.904 - 20221010060016.903
Input
132747.826 - 132747.712
060016.904 - 060016.903
042234.671 - 042234.491
Expected output
0.18
0.11
0.001
Our UNIX system doesn't support mktime, use, my
date -d, most of the functions are missing.
I'm looking for assistance in writing a shell script to calculate the time difference.
awk -F, -v OFS=',' '{ print $10-$9 }' Simple subtraction is not working for the 00th and 59th minutes.

What is the way to read tables displayed after commands in bash

Is there is the best way to read results of some bash commands, which are displayed as a table, if I need some particular row and column there?
For example, when I run this line:
gcloud app instances describe $instance_name --service=postprocessing --version=$instance_version
I get this:
startTime: '2020-08-03T16:29:29.142Z'
vmDebugEnabled: true
vmIp: xx.xx.xxx.xxx
vmStatus: RUNNING
I need only IP from this table, how do I get it?
I found only one way, to save the whole output as an array and then get -3rd element of it. I was wondering if there is a better way to it?
Also for other types of output with more "columns" which have a header, like this one:
mst#cloudshell:~ (me)$ free -m
total used free shared buff/cache available
Mem: 1995 469 279 0 1247 1379
Swap: 767 0 767
How do I get free/Swap, for example?
I suggest using awk to get the value you want in the output.
To set $VMIP variable with the vmIP value (2nd field):
VMIP=$(gcloud app ... | awk '/vmIp:/ {print $2}' )
To set $SWAP_FREE variable with the Swap free value (4th field):
SWAP_FREE=$(free -m | awk '/Swap:/ {print $4}')

FreeBSD script to show active connections and append number remote file

I am using NetScaler FreeBSD, which recognizes many of the UNIX like commands, grep, awk, crontab… etc.
I run the following command to get the number of connected users that we have on the system
#> nsconmsg -g aaa_cur_ica_conn -d stats
OUTPUT (numbered lines):
Line1: Displaying current counter value information
Line2: NetScaler V20 Performance Data
Line3: NetScaler NS11.1: Build 63.9.nc, Date: Oct 11 2019, 06:17:35
Line4:
Line5: reltime:mili second between two records Sun Jun 28 23:12:15 2020
Line6: Index reltime counter-value symbol-name&device-no
Line7: 1 2675410 605 aaa_cur_ica_conn
…
…
From above output - I only need the number of connected users (represented in Line 7, 3rd column (605 to be precise), along with the Hostname and Time (of the running script)
Now, to extract this important 3rd column number i.e. 605, along with the hostname, and time of data collected - I wrote the following script:
printf '%s - %s - %s\n' "$(hostname)" "$(date '+%H:%M')" "$(nsconmsg -g aaa_cur_ica_conn -d stats | grep aaa_cur_ica_conn | awk '{print $3}')"
The result is perfect, showing hostname, time, and the number of connected users as follows:
Hostname - 09:00 – 605
Now can anyone please shed light on how I can:
Run this script every day - 5am to 5pm (12hours)?
Each time scripts runs - append a file on a remote Unix share with the output?
I appreciate this might be a bit if a challenge... however would be grateful for any bash scripts wizards out there that can create magic!
Thanks in advance!
I would suggest a quick look into the FreeBSD Handbook or For People New to Both FreeBSD and UNIX® so that you could get familiar with the operating system and tools that could help you achieve better what you want.
For example, there is a utility/command named cron
The software utility cron is a time-based job scheduler in Unix-like computer operating systems.
For example, to run something all days between 5am to 5pm every minute, you could use something like:
* 05-17 * * * command
Try more options here: https://crontab.guru/#*_05-17_*_*_*.
There are more tools for scheduling commands, for example at (https://en.wikipedia.org/wiki/At_(command)) but this something you need to evaluate and read more about it.
Now regarding the command, you are using to get the "number of connected users", you could avoid the grep and just used awk for example:
awk '/aaa_cur_ica_conn/ {print $3}'
This will print only column 3 if line contains aaa_cur_ica_conn, but as before I invite you to read more about the topic so that you could bet a better overview and better understand the commands.
Last but not least, check this link How do I ask a good question? the better you could format, and elaborate your question the easy for others to give an answer.

compare 2 csv files in shell using awk

Thanks.
I have 2 csv files which I need to compare them and report back if it's different. file format is same in both files and even first column data (column A) in both files have same content(it's header info).
Tried using awk command but have there is conditions which am not sure how to implement.
conditions :
a. Need to exclude first 2 rows (since those are not required for comparison). can this be achieved by doing :
NFR=NR > 2
b. If any of the value differ then in output need to report back with header info and it's respective servername along with values.
File1.csv :
Status Check
APP servers
Server name,abc,def,ghi,jkl,mno,
Summary,,,,,,
System Start Time,Nov/12/2016 20:12:24 GMT,Nov/12/2016 20:15:38 GMT,Nov/12/2016 20:15:37 GMT,Nov/12/2016 20:15:57 GMT,Nov/12/2016 20:11:42 GMT,
System Life Time,118day.14hr.15min.19sec,118day.14hr.12min.01sec,118day.14hr.12min.03sec,118day.14hr.11min.44sec,118day.14hr.16min.01sec,
OS Version,SunOS 5.10,SunOS 5.10,SunOS 5.10,SunOS 5.10,SunOS 5.10,
Service Pack Version,Generic_147148-26,Generic_147148-26,Generic_147148-26,Generic_147148-26,Generic_147148-26,
State,Up,Up,Up,Up,Up,
File2.csv :
Status Check
APP servers
Server name,abc,def,ghi,jkl,mno,
Summary,,,,,,
System Start Time,Nov/13/2016 20:12:24 GMT,Nov/13/2016 20:15:38 GMT,Nov/13/2016 20:15:37 GMT,Nov/13/2016 20:15:57 GMT,Nov/13/2016 20:11:42 GMT,
System Life Time,118day.14hr.15min.19sec,118day.14hr.12min.01sec,118day.14hr.12min.03sec,118day.14hr.11min.44sec,118day.14hr.16min.01sec,
OS Version,SunOS 5.10,SunOS 5.10,SunOS 5.11,SunOS 5.12,SunOS 5.10,
Service Pack Version,Generic_147148-26,Generic_147148-26,Generic_147148-26,Generic_147148-26,Generic_147148-26,
State,Down,Up,Down,Up,Down,
Result/Output:
OS Version value is different for server name ghi and jkl : 5.11,5.12
State value is different for server name abc, ghi and mno : Down,Down,Down
Is it possible to exclude 5/6 column for comparison as well since that will be date/time related so not required for comparison.
can just give key value(say column b/c) only those specific columns data gets compared b/w files ?
this may give you an idea how to approach the problem
$ paste -d, file{1,2} |
awk -F, 'NR<3 {next}
NR==3 {n=split($0,h); m=n/2}
NR!=5 && NR!=6 {for(i=2;i<=m-1;i++)
if($i!=$(i+m)) print $1,h[i],$i,$(i+m)}'
OS Version ghi SunOS 5.10 SunOS 5.11
OS Version jkl SunOS 5.10 SunOS 5.12
State abc Up Down
State ghi Up Down
State mno Up Down
your output formatting can be added but will complicate the code. Since your values contain space you may want to keep comma as output field separator as well.

Bash scripting: date -d won't accept my string format of hhmmss. I need a workaround

I need a Bash script to accept 1 argument representing a time in hhmmss format, and from that derive a second time 3 minutes before that.
I've been trying to use date -d:
#! /bin/bash
DATE=`date +%Y%m%d`
TIME=$1
NEWTIME=`date -d "$DATE $TIME - 3 minutes" +%H%M%S`
echo $NEWTIME
In action:
$ ./myscript.sh 123456
invalid date `20141022 123456 - 3 minutes'
It seems the problem is with the 6 character time format because 4 characters (eg 1234) works. The subtraction of the 3 minutes is not the problem because I get the same error when I remove it.
It has occurred to me I could parse the time into a more palatable format before sending it to date. I tried inserting delimiters by adding this line:
TIME=${TIME:0:2}:${TIME:2:2}:${TIME:4:2}
It accepted that format but the answer to the - 3 minutes part was inexplicably very wrong (it subtracted 2 hours and 1 minute):
$ ./myscript.sh 123456
103356
Vexing.
It has also occurred to me that I might be able to provide date with an input format, like strptime which I'm familiar with from Python. I've found references to strptime in the context of Bash but I've been unable to get it to do anything.
Does anyone have any suggestions on getting the hhmmss time-string to work? Any help is much appreciated.
FYI: I'm trying to avoid changing the 6 character input format because that would involve changing other scripts as well as getting certain human users to alter long-entrenched habits. I'm also trying to avoid outsourcing this task to another language. (I could easily do this in Python). I want a Bash solution to this problem, if there is one.
TIME=093000
TIME=${TIME:0:2}:${TIME:2:2}:${TIME:4:2} # your line
date -d "2014-10-20 $TIME 3 mins ago" +%H%M%S
Output:
092700

Resources