I'm stuck with a strange behavior while reading a CSV file and selecting its lines with a specific column float value.
Here's an extract from the input file.
ben#truc:$ head summary.fasta.csv
scf7180000753635;170043549;XP_001849446.1;27.72;184;2e-13;74.7
scf7180000753636;340728919;XP_003402759.1;25.78;322;8e-19;93.6
scf7180000753642;328716306;XP_003245892.1;33.51;191;7e-27;119
scf7180000753642;512919417;XP_004929373.1;43.18;132;1e-23;108
scf7180000753642;512914080;XP_004928052.1;40.16;127;5e-21;94.7
scf7180000753664;328696819;XP_003240139.1;37.99;179;2e-23;107
scf7180000753664;328696819;XP_003240139.1;26.67;30;2e-23;25.4
scf7180000753664;328703138;XP_003242103.1;31.65;218;1e-20;99.4
scf7180000753669;383855900;XP_003703448.1;68.92;74;2e-23;102
scf7180000753669;380030611;XP_003698937.1;72.06;68;3e-22;99.8
Here's my shell script code:
#!/bin/sh
echo "extracting the values"
# prepare output files
echo "" > "40_sequence_identity.csv"
echo "" > "60_sequence_identity.csv"
echo "" > "80_sequence_identity.csv"
while read -r line; do
#debug: check if line is correclty read
echo $line
#attribute each CSV column value to a variable
query=`echo $line | cut -d ';' -f1`
gi=`echo $line | cut -d ';' -f2`
refseq=`echo $line | cut -d ';' -f3`
seq_identity=`echo $line | cut -d ';' -f4`
align_length=`echo $line | cut -d ';' -f5`
evalue=`echo $line | cut -d ';' -f6`
score=`echo $line | -d ';' -f7`
#debug: check if cut command is OK
echo "seqidentity:"$seq_identity
# test float value of column 4, if superior to a threshold, write the line in a specific line
if [ $( echo "$seq_identity >= 40" | bc ) ]; then
echo "$line" >> "40_sequence_identity.csv"
fi
if [ $( echo "$seq_identity >= 60" | bc ) ]; then
echo "$line" >> "60_sequence_identity.csv"
fi
if [ $( echo "$seq_identity >= 80" | bc ) ]; then
echo "$line" >> "80_sequence_identity.csv"
fi
done < "summary.fasta.csv"
echo "DONE!"
And here's the strange outputs.
extracting the values
scf7180000753635;170043549;XP_001849446.1;27.72;184;2e-13;74.7
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:27.72
scf7180000753636;340728919;XP_003402759.1;25.78;322;8e-19;93.6
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:25.78
scf7180000753642;328716306;XP_003245892.1;33.51;191;7e-27;119
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:33.51
scf7180000753642;512919417;XP_004929373.1;43.18;132;1e-23;108
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:43.18
scf7180000753642;512914080;XP_004928052.1;40.16;127;5e-21;94.7
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:40.16
scf7180000753664;328696819;XP_003240139.1;37.99;179;2e-23;107
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:37.99
scf7180000753664;328696819;XP_003240139.1;26.67;30;2e-23;25.4
./create_project_directories.sh: 1: ./create_project_directories.sh: -d: not found
seqidentity:26.67
First, the 3 output files (blast_summary_superior_40_sequence_identity.csv ...) contain all the lines, as if the tests didn't work.
Second, the file parsing seems OK, but this strange message: -d: not found , comes from nowhere.Though, it appears before the 'echo' displaying the value of $seqidentity and is probably related to the cut command.
Any idea why I have such output ?
When I manually execute the commands in the console, this works.
But not when executing the whole script.
Thanks for your help.
You are getting Error : -d: not found because on line number 17 command is incomplete
score=`echo $line | -d ';' -f7`
So it should be :
score=$(echo $line | cut -d ';' -f7)
Related
I'm still learning how to work with bash but my program is part of part 1 which is posted below but they are wanting us to use the file cs.rosters.txt to generate the new_users.txt file.
The cs.rosters.txt looks like this:
CMPSC 1513 03|doejan|Doe, Jane|0510|0350
CMPSC 1513 03|smijoh|Smith, John|0510
CMPSC 1133 01|cp2stu3| CPII, Student3 Neither|2222|0020
Below is what I created for part 1 which runs correctly:
#!/bin/bash
while read -r line || [[ -n "$line" ]]; do
username=$(echo "$line" | cut -d: -f1)
GECOS=$(echo "$line" | cut -d: -f5)
homedir=$(echo "$line" | cut -d: -f6)
echo "adduser -g '$GECOS' -d '$homedir' -s /bin/bash '$username'"
done < "new_users.txt"
Here's where I'm struggling
The code above pretty much displays the information in the new_users.txt file
I'm struggling on the second part, my code is posted below:
What I want to do is create a script that generates the cs.rosters.txt file like I did above.
#!/bin/bash
while read line; do
user=$(echo $line | cut -d'|' -f2)
pass=$(echo $line | cut -d'|' -f3)
home=$(echo $line | cut -d'|' -f4)
useradd -m -d $home -s /bin/bash $user
echo $pass | passwd --stdin $user
done < cs_roster.txt
I'm getting two errors: one is saying the Computer directory is invalid and the function --stdin is invalid. Can you help me?
i have this CSV file. how can i copy the first row and the next 2 values. server.2=cl1z2 server.3=cl1z3. i want to repeat this task with the other Values. (e.g. third row and server.2=cl2z2 server.3=cl2z3 )
z_name;z_hosts;z_clientPort;z_leaderPort;z_electionport;tickTime;initLimit;syncLimit;snapRetainCount;purgeInterval
cl1z1;server.1=cl1z1;2180;2890;3890;200;5;2;3;24
cl1z2;server.2=cl1z2;2181;2891;3891;200;5;2;3;24
cl1z3;server.3=cl1z3;2182;2892;3892;200;5;2;3;24
cl2z1;server.1=cl2z1;2183;2893;3893;200;5;2;3;24
cl2z2;server.2=cl2z2;2184;2894;3894;200;5;2;3;24
cl2z3;server.3=cl2z3;2185;2895;3895;200;5;2;3;24
Thanks for Help.
You can try this
result=""
while read line ; do
if echo "$line" | grep -q 'z_name'; then
result+="$line\n"
else
result+=$(echo "$line" | cut -d';' -f2 | cut -d';' -f1)"\n"
fi
done < your_csv_file
echo -e "$result"
I'm completely new to coding and this is my first script. I've been receiving this error from my code and can't seem to figure out what I'm doing wrong.
| LAST NAME | FIRST NAME | JOB | OLD | NEW | MESSAGE
: line 25: syntax error near unexpected token `|'
: line 25: ` else | '
The script is just supposed to read a text file. II've attempted to use gedit to see my mistake and all I've noticed is my JOB= line is discolored.
#!/bin/bash
printf "| % -15s | % -10s | % -6s | % -6s | % -6s | % -40s\n" "LAST NAME" "FIRST NAME" "JOB" "OLD" "NEW" "MESSAGE"
while read line
do
USER ID=` echo $line | cut -d '|' -f1 `
LAST NAME=` echo $line | cut -d '|' -f2 `
FIRST NAME=` echo $line | cut -d '|' -f3 `
JOB=` echo $line | cut -d '|' -f4 `
OLD NICE=` echo $line | cut -d '|' -f5 `
PREFERRED PASSWORD=` echo $line | cut -d '|' -f6 `
FULL NAME="$FIRST_NAME $LAST_NAME"
if [ "$JOB" == "P" ]
then
NEW_NICE=3
elif [ "$JOB" == "S" ]
then
NEW_NICE=6
else |
NEW_ICE=7
fi
test -e /home/$USER_ID
if [ "$?" == "0" ]
then
MESSAGE="$USER_ID already exists"
else
ENCRYPTED_PASSWORD=` echo "$PREFERRED_PASSWORD" | openssl passwd -1 -stdin `
useradd -m -c "$FULL_NAME" -p $ENCRYPTED_PASSWORD $USER_ID
mkdir "/home/$USER_ID/${LAST_NAME}_backup"
chown -r $USER_ID "/home/$USER_ID/${LAST_NAME}_backup"
echo "$USER_ID created"
fi
printf "| % -15s | % -10s | % -6s | % -6s | % -6s | % -40s\n" "$LAST NAME" "$FIRST NAME" "$JOB" "$OLD" "$NEW" "$MESSAGE"
done < test3_data.txt
I am trying to delete few lines (that are 1 day older) from a file using sed but it gives an error while executing the script.
What could be causing the above error? Could anyone please help?
~]# ./test.sh
Jan 20
36
sed: -e expression #1, char 3: unexpected `,'
Here is the script:
month=$(date --date="1 day ago" | cut -d " " -f2,3)
echo $month
line=$(grep -n "$month" test.log | cut -d : -f 1 | tail -1)
echo $line
if [ ! -z "$line" -a "$line" != " " ];
then
sed -i '1,"$line"d' test.log
#echo "sed -i '1,"$line"d' test.log"
else
exit
fi
I suggest you to change the sed line as,
sed -i '1,'"$line"'d' test.log
^ ^
| |
I have the following:
FILENAME=$1
cat $FILENAME | while read LINE
do
response="$LINE" | cut -c1-14
request="$LINE" | cut -c15-31
difference=($response - $request)/1000
echo "$difference"
done
When I run this script it returns blank lines. What am I doing wrong?
Might be simpler in awk:
awk '{print ($1 - $2)/1000}' "$1"
I'm assuming that the first 14 chars and the next 17 chars are the first two blank-separated fields.
You need to change it to:
response=`echo $LINE | cut -c1-14`
request=`echo $LINE | cut -c15-31`
difference=`expr $response - $request`
val=`expr $difference/1000`
You are basically doing everything wrong ;)
This should be better:
FILENAME="$1"
cat "$FILENAME" | while read LINE
do
response=$(echo "$LINE" | cut -c1-14) # or cut -c1-14 <<< "$line"
request=$(echo "$LINE" | cut -c15-31)
difference=$((($response - $request)/1000)
echo "$difference"
done