SED not adding a first line to .csv file - bash

I am doing a project for school and my head has gone through 3 walls with how many times I have bashed it. the project is to ask a name and color and assign each to a variable then make a directory from the color variable in the /tmp directory. create a .csv file with header, pull the information from a given .txt file out of order and add only select columns. I have gotten to the point of adding the columns but no matter what i do I cant get sed to add a header or import the information from the .txt file.
as you can see i have tried multiple ways to modify the file but I dont know enough yet to make it work
the input file format is as follows
1. 734-44-2041 James SMITH jsmith#beltec.us 360-555-4778 360-555-0158
and it should look like
james,smith,james.smith#beltec.us,734-44-2041-000
I am assuming that the 3 commas are intended to be 0's at the end
this is the code I have so far
#!/bin/bash
#interactive=
#variables
color=/tmp/$color
csvfile=/tmp/blue/midterm.csv
if [ "$1" == "" ]; then
echo "you should use the -c or -C flags
exit
fi
#adding the -c flag and setting the filename variable
while [ "$1" != "" ]; do
case $1 in
-c | -C ) shift
filename=$1
;;
* ) echo "you should use the -c flag"
exit 1
esac
shift
done
#get user's name
echo "what is your name"
read user_name
#get fav color from user
echo "what is your favorate color"
read color
#make the fov color directory
if [ ! -f /tmp/$color ]; then
mkdir /tmp/$color
else
echo "bad luck $user_name"
exit 1
fi
#cd into the directory
cd /tmp/$color
#make a csv file in /temp/$color
touch midterm.csv
akw '
BEGIN { FS=OFS=","; print "Firstname","lastname","Maildomain","Password" }
{ print $2,$3,$4,$1 }
' "$filename" > "/tmp/$color/midterm.csv"

sed by default outputs its results on the standard output.
In case you need to overwrite the old file use -i (or better -i.bak) to keep previous file version in <filename>.bak
Moreover in case you need to add something only at the beginning of the file use following syntax:
sed '1iYOUR_TEXT'

You never need sed when you're using awk. All you need to create a header + content is:
awk '
BEGIN { FS=OFS=","; print "Firstname", "Lastname", "Maildomain", "Password" }
{ print $3, $4, $5, $2 }
' "$filename" > "/tmp/$color/midterm.csv"
Or if your input file isn't a CSV as it seems not to be by your updated question:
awk '
BEGIN { OFS=","; print "Firstname", "Lastname", "Maildomain", "Password" }
{ print $3, $4, $5, $2 }
' "$filename" > "/tmp/$color/midterm.csv"

Related

How to send shell script output in a tablular form and send the mail

I am a shell script which will give few lines as a output. Below is the output I am getting from shell script. My script flow is like first it will check weather we are having that file, if I am having it should give me file name and modified date. If I am not having it should give me file name and not found in a tabular form and send email. Also it should add header to the output.
CMC_daily_File.xlsx Not Found
CareOneHMA.xlsx Jun 11
Output
File Name Modified Date
CMC_daily_File.xlsx Not Found
CareOneHMA.xlsx Jun 11
UPDATE
sample of script
#!/bin/bash
if [ -e /saddwsgnas/radsfftor/coffe/COE_daily_File.xlsx ]; then
cd /sasgnas/radstor/coe/
ls -la COE_daily_File.xlsx | awk '{print $9, $6"_"$7}'
else
echo "CMC_COE_daily_File.xlsx Not_Found"
fi
Output
CMC_COE_daily_File.xlsx Jun_11
I thought I might offer you some options with a slightly modified script. I use the stat command to obtain the file modification time in more expansive format, as well as specifying an arbitrary, pre-defined, spacer character to divide the column data. That way, you can focus on displaying the content in its original, untampered form. This would also allow the formatted reporting of filenames which contain spaces without affecting the logic for formatting/aligning columns. The column command is told about that spacer character and it will adjust the width of columns to the widest content in each column. (I only wish that it also allowed you to specify a column divider character to be printed, but that is not part of its features/functions.)
I also added the extra AWK action, on the chance that you might be interested in making the results stand out more.
#!/bin/sh
#QUESTION: https://stackoverflow.com/questions/74571967/how-to-send-shell-script-output-in-a-tablular-form-and-send-the-mail
SPACER="|"
SOURCE_DIR="/saddwsgnas/radsfftor/coe"
SOURCE_DIR="."
{
printf "File Name${SPACER}Modified Date\n"
#for file in COE_daily_File.xlsx
for file in test_55.sh awkReportXmlTagMissingPropertyFieldAssignment.sh test_54.sh
do
if [ -e "${SOURCE_DIR}/${file}" ]; then
cd "${SOURCE_DIR}"
#ls -la "${file}" | awk '{print $9, $6"_"$7}'
echo "${file}${SPACER}"$(stat --format "%y" "${file}" | cut -f1 -d\. | awk '{ print $1, $2 }' )
else
echo "${file}${SPACER}Not Found"
fi
done
} | column -x -t -s "|" |
awk '{
### Refer to:
# https://man7.org/linux/man-pages/man4/console_codes.4.html
# https://www.ecma-international.org/publications-and-standards/standards/ecma-48/
if( NR == 1 ){
printf("\033[93;3m%s\033[0m\n", $0) ;
}else{
print $0 ;
} ;
}'
Without that last awk command, the output session for that script was as follows:
ericthered#OasisMega1:/0__WORK$ ./test_55.sh
File Name Modified Date
test_55.sh 2022-11-27 14:07:15
awkReportXmlTagMissingPropertyFieldAssignment.sh 2022-11-05 21:28:00
test_54.sh 2022-11-27 00:11:34
ericthered#OasisMega1:/0__WORK$
With that last awk command, you get this:

Storing data in shell

I have a need to store a list of e-mails in a shell script. This script will get called and passed a customer number. depending on the customer number I want to populate a variable based on the passed in customer number.
I am not sure how to accomplish this and have been looking.
command example
gcb "type" "customernumber" "date"
I want to pull an e-mail associated with that customer number and populate a variable with it.
I would prefer this get stored in the script and not in a separate file if possible.
#shellter
So as you can see above my command has the customer number as $2, i am trying to get the email finder to work with that in mind. So I created a script to test the e-mail finder function with. It works fine as you have it below, but if i want it to look for $2 == cust_id it returns nothing. Here is my code below.
#!/bin/sh
#case $# in 0 ) echo "usage: myEmailFinder2 CustID" ; exit 1 ;; esac
cfgDir="/verification"
# given cust file like
# cust_id "\t" email_addr
fn_myEmailFinder() {
awk -F"\t" -v cust_id="$2" '{if ($2 == cust_id) {print $3}}' /verification/custlist.cfg
}
emailAddr=$( fn_myEmailFinder "$1")
echo $emailAddr
The command I run to test this is this
sh emailtest.sh test 90624
My config file is layed out like this, tab delimited
CustomerNumber CustomerName Email
I am going to store more data in this file to populate other variables, I'm sure once i get this figured out, I can sort out the other data.
I appreciate all of your help.
This script will get called and passed a customer number.
myEmailFinder "$CustID"
I want to populate a variable based on the passed in customer number.
emailAddr=$( myEmailFinder "$CustID")
I want to pull an e-mail associated with that customer number and populate a variable with it.
I would prefer this get stored in teh script and not in a separate file if possible.
Using a database is preferred, but .... per your written specification, try this
cat myEmailFinder
#!/bin/bash
case $# in 0 ) echo "usage: myEmailFinder CustID" ; exit 1 ;; esac
# given cust file like
# cust_id "\t" email_addr
fn_myEmailFinder() {
awk -F"\t" -v cust_id="$1" '{
if ($1 == cust_id) {
print $2
}
}' <<-EOF
1 user1#myCorp.com
2 user2#myCorp.com
5 user3#myCorp.com
EOF
#--^tabCh^---make sure you put a real tab char between custID and emailAddr
#tabCh-TabCh--- data indented with TabChars. EOS indented with **only** tabCh.
#send an email to cust in $1
emailAddr=$( fn_myEmailFinder "$1")
mailx -S "Test Email" "$emailAddr" <<-EOM
Here is the body of an email addressed to $emailAddr with CustID=$custID
EOM
#end of script
The block delimited by EOF is the place to store your custID and associated email Addresses. One per line, tab-delimited. The Indents on each line should be done with tab chars. The closing EOF line must be done ONLY with tab chars.
A preferable solution would be to store the "lookup table" in a separate file. That would look like
cat myEmailFinder2
#!/bin/bash
case $# in 0 ) echo "usage: myEmailFinder2 CustID" ; exit 1 ;; esac
cfgDir="/usr/local/data"
# given cust file like
# cust_id "\t" email_addr
fn_myEmailFinder() {
awk -F"\t" -v cust_id="$1" '{
if ($1 == cust_id) {
print $2
}
}' "$cfgDir"/emaillist.cfg
#send an email to cust in $1
emailAddr=$( fn_myEmailFinder "$1")
mailx -S "Test Email" "$emailAddr" <<-EOM
Here is the body of an email addressed to $emailAddr with CustID=$custID
EOM
where emaillist.cfg is laid out as above, tab-delimited.
IHTH
#!/bin/bash -
''''echo "Customer number: $1"
X=$(/bin/env python $0 $1)
echo $X
exit
'''
customers = {
42: 'customerA'
,43: 'customerB'
}
import sys
print customers.get(int(sys.argv[1]), '')
sys.exit(0)
:-|
if [ "$1" = "42" ]; then X="CustomerA" ; fi
if [ "$1" = "43" ]; then X="CustomerB" ; fi

appending text to specific line in file bash

So I have a file that contains some lines of text separated by ','. I want to create a script that counts how much parts a line has and if the line contains 16 parts i want to add a new one. So far its working great. The only thing that is not working is appending the ',' at the end. See my example below:
Original file:
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
Expected result:
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,xx
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,xx
This is my code:
while read p; do
if [[ $p == "HEA"* ]]
then
IFS=',' read -ra ADDR <<< "$p"
echo ${#ADDR[#]}
arrayCount=${#ADDR[#]}
if [ "${arrayCount}" -eq 16 ];
then
sed -i "/$p/ s/\$/,xx/g" $f
fi
fi
done <$f
Result:
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
,xx
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
,xx
What im doing wrong? I'm sure its something small but i cant find it..
It can be done using awk:
awk -F, 'NF==16{$0 = $0 FS "xx"} 1' file
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,xx
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a
b,b,b,b,b,b
a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,xx
-F, sets input field separator as comma
NF==16 is the condition that says execute block inside { and } if # of fields is 16
$0 = $0 FS "xx" appends xx at end of line
1 is the default awk action that means print the output
For using sed answer should be in the following:
Use ${line_number} s/..../..../ format - to target a specific line, you need to find out the line number first.
Use the special char & to denote the matched string
The sed statement should look like the following:
sed -i "${line_number}s/.*/&xx/"
I would prefer to leave it to you to play around with it but if you would prefer i can give you a full working sample.

Modify existing code for creating menu names in an interactive shell script

When i give the command awk NR==7 ABC.mod it gives me the title ('Add') I need which is present on the 7th line of the file Currently I am just able to read the title but I am not aware of how to append it to the output. Can somebody help me organize this so I can modify the code to get the expected menu output with minimal disruption (I hope) to the existing script?
Assuming you can pull out the "Add", "Delete" ... and other "titles" from the 7th line of each *.mod file, then you need to modify your script where it looks at the file a1.out somewhere before the line which seems to create the menu, namely: tr ' ' '\n' < ~/a1.out > ~/b.dat.
I say "assuming" because, even though you mention awk NR==7, I don't see where you are using it in the script. In any case, if you can get the "title" from the 7th line of a given *.mod file, then you can get the menu "name" from the file name (which seems to be the way you are constructing your menu) like this:
awk '{ln=length(ARGV[1]); if(NR==7) print substr(ARGV[1],0,ln-4)"..."$0}' ABC.mod
outputs:
ABC...Add
There's may be a shorter, easier way to do this using sed, but you mentioned awk.
For me at least, there's not really enough information to go on to help you much further. If you update your question someone may be able to give more concrete advice.
EDIT:
I'll post my work here in the hope that you find it useful. It is not a complete solution. I have to say, this is a strangely written application - with shell code and variables hardwired to temporary data to locations strewn about the file system. It's a bit hairy to try and set up a local version to try it out. Hopefully by experimenting and making modifications to the cod e you will learn more about how your application works and about shell programming in general. Extra advice: record your changes; sketch out how/where your application reads and writes its data; use comments in the source code to help you and others remember how the code works; make backups; use source control.
My assumptions:
pradee.sh looks like this (why does the file has a .sh extension - it seems more like a it defines some constants for your script)
% cat pradee.sh
HBKTM
ABC
HBKTM
CBC
HBKTM
DBC
HBKTM
IBC
HBKTM
MBCE
HBKTM
UBC
HBKTM
VBCM
Here's how I created my "test environment":
% for file in `grep -v HBKTM pradee.sh`; do touch $file.mod ; done
% ls
ABC.mod CBC.mod DBC.mod IBC.mod MBCE.mod UBC.mod VBCM.mod pradee.sh
% echo -e "_ctrl.jsp \n\n\n\n\n" > *.mod # mod files have required text+6 lines
% echo -e "_ctrl.jsp \n\n\n\n\n" > HBKTM.mod # this file seems special ?
% sed -i'' -e "7i\\[Ctrl-V Ctrl-J]
Add" ABC.mod
OR since the files now have 6 lines ... echo the menu title onto the last line:
% echo "Delete" >> DBC.mod
% echo "Insert" >> IBC.mod
... [continue inserting titles like "Add" "Delete" etc to the other *.mod files]
After that I think I have data files that mimic your set up. You tell me. Now, if I make a few small changes to your script (so the file locations don't remove overwrite my own files) and add the awk command I mentioned previously, here is what I end up with:
# menu_create.sh
# See http://stackoverflow.com/questions/17297671
rm -f *.dat
clear
cont="y"
while [ "$cont" = "y" ] # "$" is need for POSIX
do
echo -e "\n\nPlease Enter ONS Name : "
read ons
currpath=.
up=$(echo $ons|tr '[:lower:]' '[:upper:]')
#echo "\n ONS menu name \n"
#echo $up
if [ -f $up.mod ]; then
#in=$(grep -ri $up pradee.sh) # changed to following
# - how could this have worked ?
in=$(grep -v $up pradee.sh)
if [ -n "$in" ]; then
onsname=$(grep -ri "_ctrl.jsp" $up.mod)
#echo "onsname : $onsname"
if [ -n "$onsname" ]; then
echo -e "\n ONS menu name : $up "
echo $in > a1.dat
#echo "written to a1.dat\n"
#cat ~/a1.dat
#tr ' ' '\n' < ~/a1.dat > ~/a.dat
#cat ~/a.dat
sed "s/$up//g" a1.dat >a1.out
for i in `cat a1.dat`;
do
awk '{ln=length(ARGV[1]);if(NR==7) print substr(ARGV[1],0,ln-4)"..."$0}' $i.mod >> menu.dat ;
done
echo -e "\n FINUX Names \n"
#tr ' ' '\n' < a1.out > b.dat
tr ' ' '\n' < menu.dat > b.dat
cat b.dat
else
echo -e "ONS Name Not Valid !"
fi
else
echo -e "FINUX menu Name not found in our Repository"
fi
else
echo -e "\n Please Enter valid ONS name !!"
fi
echo -e "\n\n Press "y" to continue, Any other key to exit"
read cont
done
It gives me this output:
Please Enter ONS Name :
hbktm
ONS menu name : HBKTM
FINUX Names
ABC...Add
CBC...Cancel
DBC...Delete
IBC...Insert
MBCE...Modify
UBC...Undelete
VBCM...Verify
Press y to continue, Any other key to exit
q
I hope my response to your question helps you learn more about how to modify your application.

Use awk to parse source code

I'm looking to create documentation from source code that I have. I've been looking around and something like awk seems like it will work, but I've had no luck so far. The information is split in two files, file1.c and file2.c.
Note: I've set up an automatic build environment for the program. This detects changes in the source and builds it. I would like to generate a text file containing a list of any variables which have been modified since the last successful build. The script I'm looking for would be a post-build step, and would run after compilation
In file1.c I have a list of function calls (all the same function) that have a string name to identify them such as:
newFunction("THIS_IS_THE_STRING_I_WANT", otherVariables, 0, &iAlsoNeedThis);
newFunction("I_WANT_THIS_STRING_TOO", otherVariable, 0, &iAnotherOneINeed);
etc...
The fourth parameter in the function call contains the value of the string name in file2. For example:
iAlsoNeedThis = 25;
iAnotherOneINeed = 42;
etc...
I'm looking to output the list to a txt file in the following format:
THIS_IS_THE_STRING_I_WANT = 25
I_WANT_THIS_STRING_TOO = 42
Is there any way of do this?
Thanks
Here is a start:
NR==FNR { # Only true when we are reading the first file
split($1,s,"\"") # Get the string in quotes from the first field
gsub(/[^a-zA-Z]/,"",$4) # Remove the none alpha chars from the forth field
m[$4]=s[2] # Create array
next
}
$1 in m { # Match feild four from file1 with field one file2
sub(/;/,"") # Get rid of the ;
print m[$1],$2,$3 # Print output
}
Saving this script.awk and running it with your example produces:
$ awk -f script.awk file1 file2
THIS_IS_THE_STRING_I_WANT = 25
I_WANT_THIS_STRING_TOO = 42
Edit:
The modifications you require affects the first line of the script:
NR==FNR && $3=="0," && /start here/,/end here/ {
You can do it in the shell like so.
#!/bin/sh
eval $(sed 's/[^a-zA-Z0-9=]//g' file2)
while read -r line; do
case $line in
(newFunction*)
set -- $line
string=${1#*\"}
string=${string%%\"*}
while test $# -gt 1; do shift; done
x=${1#&}
x=${x%);}
eval x=\$$x
printf '%s = %s\n' $string $x
esac
done < file1.c
Assumptions: newFunction is at the start of the line. Nothing follows the );. Whitespace exactly as in your samples. Output
THIS_IS_THE_STRING_I_WANT = 25
I_WANT_THIS_STRING_TOO = 42
You can execute file file2.c so variables will be defined in bash. Then, you will just have to print $iAlsoNeedThis to get value from iAlsoNeedThis = 25;
It can be done with . file2.c.
Then, what you can do is:
while read line;
do
name=$(echo $line | cut -d"\"" -f2);
value=$(echo $line | cut -d"&" -f2 | cut -d")" -f1);
echo $name = ${!value};
done < file1.c
to get the THIS_IS_THE_STRING_I_WANT, I_WANT_THIS_STRING_TOO text.

Resources