I would like to split a text file with IFS my text file looks like :
name1
name2
name3
name4
I want to read this file and get name by name here is my code :
names=$(</text.txt)
IFS='\n' read -a -r names_list <<< "$names"
for name in "${names_list[#]}"
do
echo "$name"
done
it's always showing the first name and not the others, any solution ?
I am unsure about the output you are looking for but to go through a file using a loop, here is the syntax:
sylvainkalache#holbertonschool$ cat text.txt
name1
name2
name3
name4
sylvainkalache#holbertonschool$ while read name
> do
> echo "$name"
> done < /tmp/file
name1
name2
name3
name4
sylvainkalache#holbertonschool$
You actually do not need to change the IFS variable.
Related
Fairly new to bash, I have a few different files, all with thousands of lines each, I want to take each line from each file, and put them together on one line, in a new file,
for example, if file1 contains IPs
192.168.1.1
192.168.1.2
192.168.1.3
192.168.1.4
192.168.1.5
file2 contains ports
123
124
125
126
127
file3 contains timestamps, file4 contains a description
i'd like my output to be:
192.168.1.1 : 123 : 01/01/2012 : blah blah blah
192.168.1.2 : 124 : 01/02/2012 : blah blah
how could i go about putting them all together?
With paste(1) and sed(1).
paste file1 file2 file3 file4 | sed 's/\t/ : /g' > out
Open the four files as four separate file descriptors 3 through 6, then read one line from each in a loop. 0, 1, and 2 are already open as stdin, stdout, and stderr, respectively, so 3 is the first unused descriptor.
exec 3< file1
exec 4< file2
exec 5< file3
exec 6< file4
while IFS= read -r ip <&3 &&
IFS= read -r port <&4 &&
IFS= read -r timestamp <&5 &&
IFS= read -r description <&6
do
echo "$ip : $port : $timestamp : $description"
done
I am new to Unix and need help with a certain code.
My if condition acts weird.
Suppose my data is as follows -
name1 place1 date1
name2 place2 date2
name3 place3 date3
name4 place4 date4
name5 place5 date5
This is what I had originally done -
SRC_PATH="/app/Informatica9.5.1/server/infa_shared/0149icadd/SrcFiles"
rm -f temp_con_total
cat $SRC_PATH/Control_Total.txt > temp_con_total
echo $SRC_PATH
i=0
while read line
do
echo $line
if [[[ $i -ge 0 ]]];
then
file_name=`echo -e $line | cut -d ' ' -f 2`
file_size=`echo -e $line | cut -d ' ' -f 3`
file_size=$(( $file_size ))
actual_size=`cat $SRC_PATH/$file_name | wc -l`
actual_size=`echo -e $actual_size | cut -d ' ' -f 1`
actual_size=$(( $actual_size - 1 ))
echo $file_name
echo $file_size
echo $actual_size
fi
i=$(( i + 1 ))
done < temp_con_total
in this case, it gave me only first 2 records.
Later, I removed one set of brackets from if condition -
if [ $i -ge 0 ];
in this case, it gave me only the first row.
later i put three brackets, as follows -
if [[[ $i -ge 0 ]]];
now, it did give me all rows but after each row it also gave me error as follows:
name1 place1 date1
loop.sh[10]: [[[: not found [No such file or directory]
name2 place2 date2
loop.sh[10]: [[[: not found [No such file or directory]
name3 place3 date3
loop.sh[10]: [[[: not found [No such file or directory]
name4 place4 date4
loop.sh[10]: [[[: not found [No such file or directory]
name5 place5 date5
loop.sh[10]: [[[: not found [No such file or directory]
Can someone please help me with this? Why is this happening?
loop.sh is the name of my script
What I need as output is, all rows in the file as follows-
name1 place1 date1
name2 place2 date2
name3 place3 date3
name4 place4 date4
name5 place5 date5
I want to know why is there an error after it prints each row? loop.sh[10]: [[[: not found [No such file or directory]
And how does the number of brackets affect the code?
Bash does not use square brackets. Not as brackets anyway. It looks like it does, but it doesn't.
There is a command called [ (that's its name, an opening bracket) that is much the same as the test command. That command takes a series of command line parameters, just like any other command, and it expects that the last parameter will be a closing bracket, ], but no actual bracket processing is done. This is why you always follow the closing bracket with a semi-colon, ;, to finish the command. If you look in /bin (or /usr/bin?) you'll see that /bin/[ exists as a file.
Likewise, there's another command called [[ (again, that's its name, two opening brackets together). This is the same idea as the [ command, but has slightly different features. This one doesn't usually have a file in /bin.
Both commands are implemented as bash builtins, for efficiency in the case of [ and because [[ has to be, I think, but that doesn't affect the syntax.
There is no command named [[[, hence "No such file or directory".
As for your script, the if is completely pointless because you test if $i is greater than, or equal to zero when there's no way it can't be. I think this should do what you want:
SRC_PATH="/app/Informatica9.5.1/server/infa_shared/0149icadd/SrcFiles"
rm -f temp_con_total
cp $SRC_PATH/Control_Total.txt temp_con_total
echo $SRC_PATH
while read line
do
echo $line
file_name=`echo -e $line | cut -d ' ' -f 2`
file_size=`echo -e $line | cut -d ' ' -f 3`
actual_size=`cat $SRC_PATH/$file_name | wc -l | cut -d ' ' -f 1`
(( actual_size-- ))
echo $file_name
echo $file_size
echo $actual_size
done < temp_con_total
I wanted to see different type of answers I receive from you guys for the below problem. I am curious to see below problem being solved completely through array or any other matching (if there is any).
Below is the problem. Keeping Name as the key we need to print their various phone numbers in a line.
$cat input.txt
Name1, Phone1
Name2, Phone2
Name3, Phone1
Name4, Phone5
Name1, Phone2
Name2, Phone1
Name4, Phone1
O/P:
$cat output.txt
Name1,Phone1,Phone2
Name2,Phone2,Phone1
Name3,Phone1
Name4,Phone5,Phone1
I solved the above problem but I wanted to see a solving technique perhaps one that is more effective than me. I am not an expert in shell still at a beginner level. My code below:
$cat keyMatchingfunction.sh
while read LINE; do
var1=(echo "$LINE"|awk -F\, '{ print $1 }')
matching_line=(grep "$var1" output.txt|wc -l)
if [[ $matching_line -eq 0 ]]; then
echo "$LINE" >> output.txt
else
echo $LINE is already present in output.txt
grep -q -n "$var1" output.txt
line_no=(grep -n "$var1" output.txt|cut -d: -f1)
keymatching=(echo "$LINE"|awk -F\, '{ print $2 }')
sed -i "$line_no s/$/,$keymatching/" output.txt
fi
done
Try this:
awk -F', ' '{a[$1]=a[$1]","$2}END{for(i in a) print i a[i]}' input.txt
Output:
Name1,Phone1,Phone2
Name2,Phone2,Phone1
Name3,Phone1
Name4,Phone5,Phone1
With bash and sort:
#!/bin/bash
declare -A array # define associative array
# read file input.txt to array
while IFS=", " read -r line number; do
array["$line"]+=",$number"
done < input.txt
# print array
for i in "${!array[#]}"; do
echo "$i${array[$i]}"
done | sort
Output:
Name1,Phone1,Phone2
Name2,Phone2,Phone1
Name3,Phone1
Name4,Phone5,Phone1
#!/bin/bash
IFS=''
replace="XXX"
for line in `cat test.csv`;do
name=`echo $line|cut -d"|" -f1`
date=`echo $line|cut -d"|" -f3`
echo $name
echo $date
sed -n "s/$name/XXX/gpw" output' test.csv
done
I need to replace value of $name by XXX but it's not working.
CSV file contains:
38880|update|20121227|customerXXXX|CXXX|Credit|Comp any|channel:XXX|XXX|XXX|0|Active|N|N|2012-12-31 17:37:46|Y|2012-12-31 17:37:46
And error is:
sed: -e expression #1, char 8: unterminated `s' command
I think you are doing too many things just to change the first column.
This should make:
awk -v subs="XXX" 'BEGIN{FS=OFS="|"} {$1=subs}1' file
For your given input it returns:
XXX|update|20121227|customerXXXX|CXXX|Credit|Comp any|channel:XXX|XXX|XXX|0|Active|N|N|2012-12-31 17:37:46|Y|2012-12-31 17:37:46
To update your file, use:
awk -v subs="XXX" 'BEGIN{FS=OFS="|"} {$1=subs}1' file > new_file && mv new_file file
If you needed to use bash, you can make use of IFS and read like this:
while IFS="|" read -r name _ date _
do
echo $name
echo $date
sed -i.bak "s/$name/XXX/g" another_file #do the replacement
done < file
Note $name gets the value of the first field and $date the value of the third, based on | as delimiter.
First of all, you can simplify things greatly by using while instead of for. You also don't need the sed. Bash, and other shells, have some quite extensive string manipulation abilities. So, a working version of your script could be:
$ while IFS='|' read -r name rest; do
printf "XXX|%s\n" "$rest"
done < test.csv > new.csv
XXX|update|20121227|customerXXXX|CXXX|Credit|Comp any|channel:XXX|XXX|XXX|0|Active|N|N|2012-12-31 17:37:46|Y|2012-12-31 17:37:46
That will write the new lines into the new.csv file. If you also want to echo the values of $name and $date to the terminal but only save the changed file, use this instead:
$ > new.csv; while IFS='|' read -r name f2 date rest; do
printf "%s\n%s\n" "$date" "$name"
printf "XXX|%s|%s|%s\n" "$f2" "$date" "$rest" >> new.csv
done < test.csv
You seem to want to do more manipulations as well. If so, you can read each field into a variable:
while IFS='|' read -r f{1..10}; do ... ;done; done < test.csv
The fields will be available as $f1 through $f15. Alternatively, you could use arrays:
$ while IFS='|' read -r -a fields; do
for((i=0; i<${#fields[#]}; i++)); do
echo "Field $i : ${fields[$i]}"
done
done < test.csv
Field 0 : 38880
Field 1 : update
Field 2 : 20121227
Field 3 : customerXXXX
Field 4 : CXXX
Field 5 : Credit
Field 6 : Comp any
Field 7 : channel:XXX
Field 8 : XXX
Field 9 : XXX
Field 10 : 0
Field 11 : Active
Field 12 : N
Field 13 : N
Field 14 : 2012-12-31 17:37:46
Field 15 : Y
Field 16 : 2012-12-31 17:37:46
sed 'h;s/^\([^|]*\)|[^|]*|\([^|]*\)|.*/\1\
\2/p
g
:a
s/^\([^|]*\)\(|.*\)\1/\1\2XXX/
t a
s/^\([^|]*\)|/XXX|/' test.csv
try this.
extract and print name + date
recursively replace name in line
replace first occurence of name
I don't have a linux at hand and instead use compileonline.com to check out some code in bash, yet I'm new to bash. Could somebody give a hand?
for var
do echo $var > fniz
cat fniz
done
arguments are 123 abc xyz
My guess is, the output would be:
123
123
abc
123
abc
xyz
But I'm not sure, whether echo $var > fniz overwrites fniz or writes a new line to it. Does it overwrite the file?
> always overwrites.
Writing a new line would be achieved by using the “append” redirection operator >>.
It overwrites the file each time
$ cat script.sh
for var in 123 abv xyz
do
echo $var > fniz
cat fniz
done
$ ./script.sh
123
abv
xyz
If you want to append, use >>