This is a GNU make question.
Maybe a simple one, but I did search some
textbooks before and didn't find the answer.
Short description of what I want to do:
copy a range of bytes from a file in a temporary file
calculate the checksum of this file with crc32 utility
just print the checksum in the build process for now
delete the temporary file
The problem I see is that every command is done in a separate shell,
but I need a way to get input from the previous command to execute the next one.
eg:
/opt/rtems-4.11/bin/arm-rtems4.11-nm -a px_pmc.elf | grep bsp_section_start_begin | awk '{print $$1}'
/opt/rtems-4.11/bin/arm-rtems4.11-nm -a px_pmc.elf | grep _Check_Sum | awk '{print $$1}'
These commands will print in shell the limits of the range of bytes I want,
but how do I store them in two variables, say low_limit/high_limit so I can copy that range
in the temp file in the next make command ?
dd if=px_pmc.bin skip=low_limit bs=(high_limit-low_limit) count=1 of=temp.bin
(in C you can do this with a simple variable, I'm looking for the equivalent here)
regards,
Catalin
You can chain all your shell commands such that they are all executed in the same shell:
cmd1; cmd2; cmd3
If you prefer one command per line you can also use the line continuation (\):
cmd1; \
cmd2; \
cmd3
(be careful, no spaces after the \). And you can assign the output of a shell command to a shell variable:
a="`cmd1`"
So, the only subtlety here is that make will expand the recipe before passing it to the shell and this will eat all $ signs. If you want to preserve them such that they are passed to the shell, you must double them ($$):
a="`cmd1`"; \
b="`cmd2`"; \
cmd3 "$$a" "$$b"
In your case you can try this (untested):
target:
low_limit="`/opt/rtems-4.11/bin/arm-rtems4.11-nm -a px_pmc.elf | grep bsp_section_start_begin | awk '{print $$1}'`"; \
high_limit="`/opt/rtems-4.11/bin/arm-rtems4.11-nm -a px_pmc.elf | grep _Check_Sum | awk '{print $$1}'`"; \
bs="`expr "$$high_limit" - "$$low_limit"`"; \
dd if=px_pmc.bin skip="$$low_limit" bs="$$bs" count=1 of=temp.bin
I added the computation of high_limit-low_limit using expr. This should be more or less compatible with the bourne shell which is the default shell make uses.
Related
I am not sure why i am getting the unexpected syntax '( err
#!/bin/bash
DirBogoDict=$1
BogoFilter=/home/nikhilkulkarni/Downloads/bogofilter-1.2.4/src/bogofilter
echo "spam.."
for i in 'cat full/index |fgrep spam |awk -F"/" '{if(NR>1000)print$2"/"$3}'|head -500'
do
cat $i |$BogoFilter -d $DirBogoDict -M -k 1024 -v
done
echo "ham.."
for i in 'cat full/index | fgrep ham | awk -F"/" '{if(NR>1000)print$2"/"$3}'|head -500'
do
cat $i |$BogoFilter -d $DirBogoDict -M -k 1024 -v
done
Error:
./score.bash: line 7: syntax error near unexpected token `('
./score.bash: line 7: `for i in 'cat full/index |fgrep spam |awk -F"/" '{if(NR>1000)print$2"/"$3}'|head -500''
Uh, because you have massive syntax errors.
The immediate problem is that you have an unpaired single quote before the cat which exposes the Awk script to the shell, which of course cannot parse it as shell script code.
Presumably you want to use backticks instead of single quotes, although you should actually not read input with for.
With a fair bit of refactoring, you might want something like
for type in spam ham; do
awk -F"/" -v type="$type" '$0 ~ type && NR>1000 && i++<500 {
print $2"/"$3 }' full/index |
xargs $BogoFilter -d $DirBogoDict -M -k 1024 -v
done
This refactors the useless cat | grep | awk | head into a single Awk script, and avoids the silly loop over each output line. I assume bogofilter can read file name arguments; if not, you will need to refactor the xargs slightly. If you can pipe all the files in one go, try
... xargs cat | $BogoFilter -d $DirBogoDict -M -k 1024 -v
or if you really need to pass in one at a time, maybe
... xargs sh -c 'for f; do $BogoFilter -d $DirBogoDict -M -k 1024 -v <"$f"; done' _
... in which case you will need to export the variables BogoFilter and DirBogoDict to expose them to the subshell (or just inline them -- why do you need them to be variables in the first place? Putting command names in variables is particularly weird; just update your PATH and then simply use the command's name).
In general, if you find yourself typing the same commands more than once, you should think about how to avoid that. This is called the DRY principle.
The syntax error is due to bad quoting. The expression whose output you want to loop over should be in command substitution syntax ($(...) or backticks), not single quotes.
I have to run many python script which differ just with one parameter. I name them as runv1.py, runv2.py, runv20.py. I have the original script, say runv1.py. Then I make all copies that I need by
cat runv1.py | tee runv{2..20..1}.py
So I have runv1.py,.., runv20.py. But still the parameter v=1 in all of them.
Q: how can I also replace v parameter to read it from the file name? so e.g in runv4.py then v=4. I would like to know if there is any one-line shell command or combination of commands. Thank you!
PS: direct editing each file is not a proper solution when there are too many files.
Below for loop will serve your purpose I think
for i in `ls | grep "runv[0-9][0-9]*.py"`
do
l=`echo $i | tr -d [a-z.]`
sed -i 's/v/'"$l"'/g' runv$l.py
done
Below command was to pass the parameter to script extracted from the filename itself
ls | grep "runv[0-9][0-9]*.py" | tr -d [a-z.] | awk '{print "./runv"$0".py "$0}' | xargs sh
in the end instead of sh you can use python or bash or ksh.
I currently try to figure out How exactly i can generate a Doskey Alias that allows Pipes.
I want to gather the foldersize in kB.
I have tried
alias dirsize=du -P -c -a -b $1 | grep total | awk '{print "Folder has: " $1 "kB"}'
But i simply get the Output
Folder has: kB
When I just use
dirsize=du -P -c -a -b $1 | grep total
It gets me
C:\>dirsize Temp
1364201 total
But how do I use the awk pipe now?
What am I doing wrong?
doskey dirsize=du -P -c -a -b $1 $b grep -E "total$" $b awk "{i=1;print \"Has \" $i \" bytes\"}"
Inside a doskey alias
the pipe character is $b
all instances of $1 will be replaced with the first argument in the call to the alias, so, the $1 inside the awk command will be replaced during the alias parse and at execution time it will not be a reference to the argument
The usual solution to this problem inside a doskey alias is to convert the $1 into something that will not interfere the command execution but avoids doskey parser to handle it. To do it, the usual solution is to use $^1, but in this case, this sequence is not handled by cmd, it is handled by awk and the ^ is not considered a escape character.
We need to solve it inside awk, replacing the direct $1 using a variable (i) to store the index to retrieve and replacing the $1 with $i that has no special meaning inside doskey aliases.
I found this solution to build hash-values:
echo -n wicked | shasum | awk '{print $1}'
But this works only with string input. I don't know how to hanlde input as hex, for example if i want to build sha1-value of sha1-value.
upd: I just found out there is option -b for shasum but it produces wrong output. Does it expect bytes with reversed endianness?
upd2: for example: I do the following input:
echo -n 9e38cc8bf3cb7c147302f3e620528002e9dcae82 | shasum -b | awk '{print $1}'
The output is bed846bb1621d915d08eb1df257c2274953b1ad9 but according to the hash calculator the ouput should be 9d371d148d9c13050057105296c32a1368821717
upd3: the -b option seems not to work at all. There is no difference whether I apply this parameter or not, i get the same result.
upd4: the whole script lookes as follows. It doesn't work because the null-byte gets removed as i either assign or concatenate .
password="wicked"
scrumble="4d~k|OS7T%YqMkR;pA6("
stage1_hash=$(echo -n $password| shasum | awk '{print $1}')
stage2_hash=$(echo $(echo -n $stage1_hash | xxd -r -p | shasum | awk '{print $1}') | xxd -r -p)
token=$(./xor.sh $(echo -n $scrumble$(echo 9d371d148d9c13050057105296c32a1368821717 | xxd -r -p) | shasum | awk '{print $1}') $stage1_hash)
echo $token
You can use xxd -r -p to convert hexadecimal to binary:
echo -n 9e38cc8bf3cb7c147302f3e620528002e9dcae82 | xxd -r -p | shasum -b | awk '{print $1}'
Note that the output of this is 9d371d148d9c13050057105296c32a1368821717; this matches what I get from hashing 9e38cc8bf3cb7c147302f3e620528002e9dcae82 using hash calculator. It appears that the value you got from bash calculator was a results of a copy-paste error, specifically leaving off the final "2" in the hex string.
UPDATE: I'm not sure exactly what the entire script is supposed to do, but I can point out several problems with it:
Shell variables, command arguments, and c strings in general cannot contain null bytes. There are also situations where trailing linefeeds get trimmed, and IIRC some early versions of bash couldn't handle delete characters (hex 7F)... Basically, don't try to store binary data (as in stage2_hash) or pass it as arguments (as in ./xor.sh) in the shell. Pipes, on the other hand, can pass raw binary just fine. So store it in hex, then convert to binary with xxd -r -p and pipe it directly to its destination.
When you expand a shell variable ($password) or use a command substitution ($(somecommand)) without wrapping it in double-quotes, the shell does some additional parsing on it (things like turning spaces into word breaks, expanding wildcards to lists of matching filenames, etc). This is almost never what you want, so always wrap things like variable references in double-quotes.
Don't use echo for anything nontrivial and expect it to behave consistently. Depending on which version of echo you have and/or what the password is, echo -n "$password" might print the password without a linefeed after it, or might print it with "-n " before it and a linefeed after, might do something with any backslash sequences in the password, or (if the password starts with "-") interpret the password itself as more options to the echo command. Use printf "%s" "$password" instead.
Don't use echo $(somecommand) (or even printf "%s" "$(somecommand)"). The echo and $() are mostly canceling each other here, but creating opportunities for problems in between. Just use the command directly.
Clean those up, and if it doesn't work after the cleanup try posting a separate question.
openssl command may help you. see HMAC-SHA1 in bash
like:
echo -n wicked | openssl dgst -sha1
I am supposed to make a script that prints all sizes and file-names in the current directory, ordered by size, using the "set" command.
#!/bin/bash
touch /tmp/unsorted
IFS='#'
export IFS
ls -l | tr -s " " "#" | sed '1d' > /tmp/tempLS
while read line
do
##set probably goes here##
echo $5 $9 >> /tmp/unsorted
done < /tmp/tempLS
sort -n /tmp/unsorted
rm -rf /tmp/unsorted
By logic, this is the script that should work, but it produces only blank lines.
After discussion with my classmates, we think that the "set" command must go first in the while loop. The problem is that we cant understand what the "set" command does, and how to use it. Please help. Thank you.
ls -l | while read line; do
set - $line
echo $5 $9
done | sort -n
or simply
ls -l | awk '{print $5, $9}' | sort -n
Set manipulates shell variables. This allows you to adjust your current environment for specific situations, for example, to adjust current globbing rules.
Sometimes it is necessary to adjust the environment in a script, so that it will have an option set correctly later on. Since the script runs in a subshell, the options you adjust will have no effect outside of the script.
This link has a vast amount of info on the various commands and options available.