I have used the following a guide with these commands:
echo "accession\taccession.version\ttaxid\tgi" > reference_proteomes.taxid_map
zcat */*/*.idmapping.gz | grep "NCBI_TaxID" | awk '{print $1 "\t" $1 "\t" $3 "\t" 0}' >> reference_proteomes.taxid_map
I used echo "accession\taccession.version\ttaxid\tgi" > reference_proteomes.taxid_map & zcat *.idmapping.gz | grep "NCBI_TaxID" | awk '{print $1 "\t" $1 "\t" $3 "\t" 0}' >> reference_proteomes.taxid_map
but I received the messaged that the argument list is too long:/usr/bin/zcat: Argument list too long
so I have tried xargs in this: find /Volumes/My\ Passport\ for\ Mac/uniprot | xargs zcat *.idmapping.gz | grep "NCBI_TaxID" | awk '{print $1 "\t" $1 "\t" $3 "\t" 0}' >> reference_proteomes.taxid_map
but still received the long list argument message -bash: /usr/bin/xargs: Argument list too long, any suggestion?
Related
Egrep and Awk to output columns of a line , with a specific value for the first column
I am to tasked to write a shell program which when ran as such
./tool.sh -f file -id id OR ./tool.sh -id id -f file
must output the name surname and birthdate (3 columns of the file ) for that specific id.
So far my code is structured as such :
elif [ "$#" -eq 4 ];
then
while [ "$1" != "" ];
do
case $1 in
-f)
cat < "$2" | egrep '"$4"' | awk ' {print $3 "\t" $2 "\t" $5}'
shift 4
;;
-id)
cat < "$4" | egrep '"$2"' | awk ' {print $3 "\t" $2 "\t" $5}'
shift 4
esac
done
(Ignoring the opening elif cause there are more subtasks for later)
My output is nothing. The program just runs.
I've tested the cat < people.dat | egrep '125' | awk ' {print $3 "\t" $2 "\t" $5}'
and it runs just fine.
I also had an instance where i had an output from the program while it was run like so
cat < "$2" | egrep '["$4"]' | awk ' {print $3 "\t" $2 "\t" $5}'
but it wasnt only that specific ID.
`egrep "$4"` was correct instead of `egrep '["$4"]'` in
`cat < "$2" | egrep '["$4"]' | awk ' {print $3 "\t" $2 "\t" $5}'`
Double quotes allow variables, single quotes don't. No commands need
certain types of quotes, they are purely a shell feature that are not
passed to the command. mentioned by(#that other guy)
i have this script in bash:
#!/bin/bash
dir="/home/dortiz/Prueba"
for i in $dir/*
do
cat $i | awk '{print $1" " $2" " $3" " $4"\n " $5}' | \
awk '/gi/{print ">" $0; getline; print}' | \
awk '$3>20.00 {print $0; getline; print;}' \
> "${i}.outsel"
done
cd /home/dortiz/Prueba
mv *.outsel /home/dortiz/Prueba2
and i would like to set an argument to change the value after ""awk '$3>"" in an easy way from my main program that will call this script.
i have read something about getopts but i dont uderstand it at all
Thanks a lot in advance
The simplest way is to just pass an argument to your script:
yourscript.sh 20.0
Then in your script
#!/bin/bash
value=$1 # store the value passed in as the first parameter.
dir="/home/dortiz/Prueba"
for i in $dir/*; do
awk '{print $1" " $2" " $3" " $4"\n " $5}' "$i" |
awk '/gi/{print ">" $0; getline; print}' |
awk -v val="$value" '$3>val {print $0; getline; print;}' > "${i}.outsel"
# ^^^^^^^^^^^^^^^
done
...
and the cat|awk|awk|awk pipeline can probably be written like this:
awk -v val="$value" '
$3 > val {
prefix = /gi/ ? ">" : ""
print prefix $1 " " $2" " $3" " $4"\n " $5
}
' "$i" > "$i.outsel"
I'm new to programming with shell and want to ask what is wrong with my code?
#!/bin/bash
#DHT11
SCRIPT="/var/www/ErnestynoFailai/scripts/DHT 11 4"
#DHT22
#SCRIPT="/root/to/folder/DHT 22 4"
#AM2302
#SCRIPT="/root/to/folder/DHT 2302 4"
HUMIDITY=`$SCRIPT | grep "Temp" | awk -F " " '{print $7}'`
TEMPRATURE=`$SCRIPT | grep "Temp" | awk -F " " '{print $3}'`
#-a = AND = &&
while [ $HUMIDITY=="" -a $TEMPRATURE=="" ]
do
$HUMIDITY=`$SCRIPT | grep "Temp" | awk -F " " '{print $7}'`
$TEMPRATURE=`$SCRIPT | grep "Temp" | awk -F " " '{print $3}'`
done
echo "$HUMIDITY"
echo "$TEMPRATURE"
I'm getting:
line 14 or 15 =26: or =: command not found...
There are two problems:
These lines are not returning anything, or at least a string:
HUMIDITY=`$SCRIPT | grep "Temp" | awk -F " " '{print $7}'`
TEMPRATURE=`$SCRIPT | grep "Temp" | awk -F " " '{print $3}'`
This is causing =: command not found errors.
Your while condition needs to be
while [[ $HUMIDITY == "" && $TEMPRATURE == "" ]]
Finally, while not causing problems, TEMPERATURE is misspelled, which might cause you grief later on.
Variables have to be assigned without leading $:
HUMIDITY=`$SCRIPT | grep "Temp" | awk -F " " '{print $7}'`
TEMPRATURE=`$SCRIPT | grep "Temp" | awk -F " " '{print $3}'`
I'm running a command which basically parses some JSON and then extracts an ID using awk and sed.
When I run the command on its own it give the correct output eg
cat CustomThemeProfile.json | sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g'
2F13F732-4BCB-49DC-A0FB-C91B5DE58472
But when I want to assign the output to a variable I get nothing returned. eg
cat CustomThemeProfile.json | id=$(sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g'); echo $id
Any ideas. I really want this to be ran from a script but for the moment the script just does nothing, sits waiting for something?
Script i'm calling from.
First script just finds all json files and then calls this script. so the file is passed
#!/bin/bash
echo "running search and replace script ..."
id="$(sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g')"
filler="0-0000-0000-0000-000000000000"
echo $id
if [ $(#id) -ge 8]; then echo "New Profile ID in use"; exit
else idnew=$id$filler
fi
sed -i '"s/$id/$idnew/g"' $1
sed -i 's/ps_hpa/ps_hpa/g' $1
You need to rearrange your syntax a little bit:
id=$(sed -e 's/[{}]/''/g' CustomThemeProfile.json | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g')
Notice I am avoiding a useless use of cat and passing the file directly to sed. This is why your script does nothing - sed is waiting for some input. It would be possible to move cat inside the command substitution but there's no advantage to doing so. If a tool is capable of reading a file itself, then you should use that capability.
The better solution would be to parse your JSON properly, using jq for example. In order for us to show you how to do that, you should edit your question to show us a sample of your input.
I am storing date inside a variable and using that in the sed as below.
DateTime=`date "+%m/%d/%Y"`
Plc_hldr1=`head -$i place_holder.txt | tail -1 | awk -F ' ' '{ print $1 }'`
Plc_hldr2=`head -$i place_holder.txt | tail -1 | awk -F ' ' '{ print $2 }'`
sed "s/$Plc_hldr1/$DateTime/;s/$Plc_hldr2/$Total/" html_format.htm >> /u/raskar/test/html_final.htm
While running the sed command I am getting the below error.
sed: 0602-404 Function s/%%DDMS1RT%%/01/02/2014/;s/%%DDMS1C%%/1235/ cannot be parsed.
I suppose this is happening as the date contains the following output which includes slashes '/'
01/02/2014
I tried with different quotes around the date. How do I make it run?
Change the separator to something else that won't appear in your patterns, for example:
sed "s?$Plc_hldr1?$DateTime?;s?$Plc_hldr2?$Total?"
Not the direct quertion but replace
Plc_hldr1=`head -$i place_holder.txt | tail -1 | awk -F ' ' '{ print $1 }'`
Plc_hldr2=`head -$i place_holder.txt | tail -1 | awk -F ' ' '{ print $2 }'`
by
Plc_hldr1=`sed -n "$i {s/ .*//p;q}"`
Plc_hldr2=`sed -n "$i {s/[^ ]\{1,\} \{1,\}\([^ ]\{1,\}\) .*/\1/p;q}"`
and with aix/ksh
sed -n "$i {s/\([^ ]\{1,\} \{1,\}[^ ]\{1,\}\) .*/\1/p;q}" | read Plc_hldr1 Plc_hldr2