Using bash shell in Mac OSX
I've been trying to write a short shell script to take care of part of my compilation process, however when I'm trying to run the script on every file, the wildcards seem to not be acting like I would expect them to.
The script is as follows:
rm *.cs
rm *.bin
protoc -I. *.proto -o*.bin
mono protogen -i:*.bin -o:*.cs
Now, when I hand-type this at the command line for each file, it works fine. However, when run through this shell script using wildcards, it winds up creating "*.bin" and "*.cs", rather than individual .bin and .cs files for each of my different inputs. Am I misusing wildcards, or is something just going weird?
Wildcards don't work like that. They just get replaced by the list of existing matching filenames. So if you rm *.bin, then *.bin matches nothing in the next statement. They do not magically "capture" the replaced value from another token in the same command line.
In bash, the result would be that *.bin remained unaltered, so it should create the file *.bin, not .bin. But you may be using a different shell.
You could accomplish this task easily with make, by the way.
try something like (not tested):
protofiles="*.proto"
rm *.cs
rm *.bin
for i in $protofiles; do
binfile=${i/.proto/.bin}
protc -I. $i -o$binfile
done
binfiles="*.bin"
for i in $binfiles; do
csfile="${i/.bin/.cs}
mono prtogen -i:$i -o:$csfile
done
Anyway... I suggest you to use Makefiles for this job. Try something like this (not tested):
CSFILES=a.cs b.cs d.cs e.cs
clean:
rm *.cs *.bin
%.bin: %.proto
protoc -I. $< -o $#
%.cs: %.bin
mono prtogen -i:$< -o:$#
all: $(CSFILES)
Related
My Makefile downloads a number of third-party files if they are not locally available.
CLOSURE_VERSION=20161024
CLOSURE_BASE_URL="http://dl.google.com/closure-compiler"
build/bin/closure-compiler.jar: build/src/hashes/closure-compiler-${CLOSURE_VERSION}.tar.gz.sha256
download-if-sha-matches <$< >$#.tar.gz \
${CLOSURE_BASE_URL}/compiler-${CLOSURE_VERSION}.tar.gz
tar -zxf $#.tar.gz closure-compiler-v${CLOSURE_VERSION}.jar
mv closure-compiler-v${CLOSURE_VERSION}.jar $#
rm $#.tar.gz
Here, build/src/hashes/closure-compiler-${CLOSURE_VERSION}.tar.gz.sha256 is the saved hash of the version of the file which we already know is correct.
download-if-sha-matches <hash >outfile url downloads the url and compares its hash to stdin, failing if they don't match.
This recipe works except that it always runs, even if build/bin/closure-compiler.jar already exists. Naturally, its timestamp is later than that of $< so I would expect this to not execute the recipe the second time I run make.
What have I gotten wrong?
Looks like tar -x preserves the timestamps of the contained files.
Add this to the recipe.
touch $#
EDIT2:
THIS POST IS SOLVED (if you would like to see the product that tis post resulted in please visit the site http://www.nasmagic.com )
here is my plan firstly
i am using Platypus (http://sveinbjorn.org/platypus) a program for OSX that lets you create little "droplets" that basically wrap a bash script in a fancy drag and drop GUI.
now im using this as a nice easy way of making myself a drag and drop Nasm assembler.
i have made a few of these "droplets" with simple bash scripts inside them, one example would be the folowing:-
#!/bin/bash
chmod u+x $1
this as you can see makes my scripts executable,... very handy.
The way they work is they take one variable only and that is the path to the file eg.
/Users/MiRAGE/Desktop/example.sh
now here is my conundrum for the day
i need to do the following command:-
/usr/local/bin/nasm -f macho example.asm && ld -macosx_version_min 10.7.0 -o example example.o
this is one command that works fine with the non variable filenames. however in the droplet context it has a problem.
it can execute each of these commands in two seperate droplets but without changing the command it will not find the outputed file of the first command as it is not 'cd'd into the directory where the file is outputed so it doesent know what to do.
now as i say at the moment i have successfully got it to compile with one droplet with this command:-
/usr/local/bin/nasm -f macho $1
which i drag the file into first and it spits out a ".o" file
then i drop that ".o" file into droplet2 which has this command inside:-
ld -macosx_version_min 10.7.0 -lSystem -o $1.5y $1
this command is much less elegant than the first.
the only way i could get it to compile the file is to append my made up extention otherwise it would just not work. the problem with this method is that while it does output the compiled binary it ends up looking like this "exampleFilename.o.5y".
now i could go in and delete that and i would, i guess be where i need to be. but its very messy. 2 droplets, renaming files..... not what i want i want a drag and drop hassle free assembler.
so heres the code i have in mind except this is not real and clearly doesn't work.
if [filename $1 == ".asm"] # if the file extension is ".asm"
then # then do next step
/usr/local/bin/nasm -f macho $1 # make mach-o file
fi # end
else if [filename $1 == ".o"] # else if the file extension is ".o"
then # then do this step
ld -macosx_version_min 10.7.0 -lSystem -o $1 $1.o # take the file ".o" and outfile with no extention but the same name.
fi #end
this way i can drag it in once, it will spit out the ".o" file, then i drag that in, and it then converts it to the executable binary. PERFECT
but i cant for the life of me find out how to write this properly if anyone can help i would be very very appreciative
many thanks in advance
MiRAGE
If my understanding is correct, you want to be able to drag the .asm file to the droplet, then drag the resulting .o file to the droplet to produce an executable, in two steps.
In which case, after downloading platypus and recent nasm, I find this script works for me:
#!/bin/bash
# Get the path to the input file, and enter that directory
pathname=$(dirname "$1")
cd $pathname
# attempt strip the .asm extension off the input filename (if there is one)
filestem=$(basename -s .asm "$1")
# If the input file was a .asm file, then assemble it:
if [ "${pathname}/${filestem}.asm" == "$1" ]; then
/usr/local/bin/nasm -f macho "$1"
fi
# attempt strip the .o extension off the input filename (if there is one)
filestem=$(basename -s .o "$1")
# If the input file was a .o file, then link it:
if [ "${pathname}/${filestem}.o" == "$1" ]; then
ld -macosx_version_min 10.7.0 -o "$filestem" "$1"
fi
This script takes care to make sure output files are placed into the same directory as the input files.
Conversely, if you want a script to assemble and link in one shot, this works for me:
#!/bin/bash
# Get the path to the input file, and enter that directory
pathname=$(dirname "$1")
cd $pathname
# attempt strip the .asm extension off the input filename (if there is one)
filestem=$(basename -s .asm "$1")
# If the input file was an .asm file, then assemble and link it:
if [ "${pathname}/${filestem}.asm" == "$1" ]; then
/usr/local/bin/nasm -f macho "$1" && ld -macosx_version_min 10.7.0 -o "${pathname}/$filestem" "${pathname}/${filestem}.o"
fi
Note that what you have here basically replicates a lot of make or similar build systems do for you.
Note also that third-party software like playtpus is not strictly needed for the drag-n-drop part. You can use the built-in automator application to create similar applications. e.g I created one that you just drag example.asm onto the automator application icon and it runs the same shell script for you.
Also, you can test the shell script at the command line, simply by calling:
./myscript.sh example.asm
or
./myscript.sh example.o
Try this...
#!/bin/bash
ext=${1/*./}
test $ext == "asm" && /usr/local/bin/nasm -f macho "$1"
output=${1/*\//}; output=${output/.*/}
test $ext == "o" && ld -macosx_version_min 10.7.0 -lSystem -o $output $1
If you want to try make to do this, the following will do it:
#!/bin/bash
target=$(dirname "$1")/$(basename -s ".${1/*./}" "$1")
export CC=ld
export LDFLAGS="-macosx_version_min 10.7.0 -lSystem"
make -f - "$target" <<EOF
%.o: %.asm
/usr/local/bin/nasm -f macho $<
.PRECIOUS: %.o
EOF
(Note there is a TAB character at the start of the nasm line.)
The beauty of using make is that targets are only rebuilt if they don't exist, or they are older than their prerequisites. Also make has built-in implicit rules for many things, including linking a .o to create an executable, which we make use of.
This script will accept a .asm file or .o file as input, from which it will derive a make target (the name of the expected executable) by stripping off the extension.
explicitly set the linker to be "ld" instead of the default of "cc"
set necessary linker flags in the LDFLAGS variable
call make with the derived target. Normally make will parse a Makefile for its rules, but in this case, we redirect a makefile using a bash here-document.
The redirected makefile has one implicit rule, which says to assemble x.asm into x.o
The redirected makefile also has a special .PRECIOUS rule, which prevents deletion of intermediate files (.o files in this case)
This is what I mean:
ls -l *.ec
For each result found, I would like to compile it one at a time.
e.g., for $something.ec:
esql $something.ec -o $something
$something.ec is original file
$something is target exe file.
My question is: how can I put that into a loop to go through all the files in my directory?
You can use a for-loop as follows:
for file in *.ec
do
esql "$file" -o "${file//.ec}"
done
I recommend you write a simple Makefile:
INPUTS = $(wildcard *.ec)
all: $(INPUTS:.ec=)
%: %.ec
esql $# -o $<
and then simply run make (or make -B to force execution) whenever you want to re-execute them.
I'm not sure if it's gmake or gcc that I don't understand here.
I'm using the -MM and -MD options to generate dependency rules for the Unit Testing framework I'm using. Specifically:
$(TEST_OBJ_DIR)/%.d: $(TEST_SRC_DIR)/%.cpp
#$(CPPC) -MM -MD $< -o $#
#sed -i -e 's|\(.*\)\.o:|$(OBJ_DIR)/\1.o $(TEST_OBJ_DIR)/\1.d $(TEST_OBJ_DIR)/\1.o:|' $#
-include $(TEST_DEP_FILES)
When I run make, after all binaries are linked (properly), I see the following extra (unexplained) line before make exits
rm test/obj/dice.d test/obj/regex.o test/obj/inventoryContainer.d test/obj/color-string.d test/obj/dice.o test/obj/inventoryContainer.o test/obj/color-string.o test/obj/regex.d
From whence is that rm command coming? The only place - anywhere - that I have an rm command in my makefile is in the clean directive
test-clean:
rm -f $(TEST_BIN_FILES)
rm -f $(TEST_OBJ_DIR)/*.{a,d,o}
Any ideas?
make will automatically create intermediate files if necessary to chain two rules together, but it will delete them at the end of the build. You can use the .PRECIOUS special target to prevent it from removing them
One helpful option for debugging these kind of problems is the -n switch:
make -n {TARGET}
It will show you the commands it would run but won't actually run them. This lets you see what rules are firing but doesn't give you all the extra output that makes it difficult to diagnose the problem.
The -d debug flag can also be useful but be sure to run it in a context where you can scroll around easily, you'll be getting a lot of output. I usually use emacs shell mode as it has good searching functionality and saves the buffer.
I'm wondering how I can avoid some echo in a Makefile :
clean:
rm -fr *.o
this rule will print:
$>make clean
rm -fr *.o
$>
How can I avoid that?
To start with: the actual command must be on the next line (or at least that is the case with GNU Make, it might be different with other Make's - I'm not sure of that)
clean:
rm -rf *.o
(note, you need a TAB before rm -rf *.o as in every rule)
Making it silent can be done by prefixing a #:
so your makefile becomes
clean:
#rm -rf *.o
If there are no *.o files to delete, you might still end up with an error message. To suppress these, add the following
clean:
-#rm -rf *.o 2>/dev/null || true
2>/dev/null pipes any error message to /dev/null - so you won't see any errors
the - in front of the command makes sure that make ignores a non-zero return code
In fact I was looking for something else, adding this line to the Makefile :
.SILENT:clean
while execute every step of the "clean" target silently.
Until someone point some drawback to this, I use this as my favourite solution!
I'm responding to this ancient topic because it comes up high in search and the answers are confusing. To do just what the user wants,all that is needed is:
clean:
#rm -f *.o
The # means that make will not echo that command.
The -f argument to rm tells rm to ignore any errors, like there being no *.o files, and to return success always.
I removed the -r from the OPs example, because it means recursive and here we are just rming .o files, nothing to recurse.
There's no need for the 2>&1 >/dev/null because with the -f there will be no errors printed.
.SILENT: clean
works in place of the #, but it isn't at the same place in the Makefile as the command that it affects, so someone maintaining the project later might be confused. That's why # is preferred. It is better locality of reference.
If you put an # in front of the command, it doesn't echo onto the shell. Try changing rm to #rm. (Reference)
From the manual: .SILENT is essentially obsolete since # is more flexible.
Much worse is that make prints far too much information. Warning/error/private messages are buried in the output. On the other hand -s (.SILENT) suppresses just anything. Especially the "nothing to be done" and "up to date" messages can be a pain. There is no option to suppress them. You have to filter them out actively or use something like colormake. Here is a solution for grep:
make | egrep -hiv 'nothing to be done|up to date'
But the output will have line numbers. The Perl solution is therefore better, because it suppresses line numbers and flushes stdout immediately:
make | perl -ne '$|=1; print unless /nothing to be done|up to date/i'
Make's a flawed tool. "What’s Wrong With GNU make?" explains this better than I can.
There's a great article on using .SILENT that explains how to conditionally activate it.
I have used that information to put this in my Makefile:
# Use `make V=1` to print commands.
$(V).SILENT:
# Example rule, only the #echo needs to be added to existing rules
*.o: %.c
#echo " [CC] $<"
gcc ...
What this does is if you run make normally, normal output is silenced and instead the echo commands work:
$ make
[CC] test.c
[CC] test2.c
But it allows you to debug problems by passing the V=1 parameter, which still shows the [CC] messages as it helps break up the output, but the traditional Makefile output is also visible:
$ make V=1
[CC] test.c
gcc ...
[CC] test2.c
gcc ...