I have a directory that contains a bunch of folders, all of which contain a .go file that I will want to build, and I'm trying to create a Makefile that can accomplish this. These are the requirements I'm looking to fill:
1.) I want this to be dynamic so that the number of folders can grow and change, and I won't need to update the Makefile every time.
2.) I also want the .go files to be able to have different names than the folder that holds them (for example, if the folder is lambda1 then the .go file shouldn't need to be named lambda1.go).
3.) Lastly, I want each binary to be output in the same folder that its source file is in.
This is what the folder structure looks like with the built binaries included:
lambdas
lambda1
--first.go
--first (binary)
lambda2
--second.go
-- second (binary)
lamba3
--third.go
--third (binary)
...
This is what the actual build commands look like:
GOOS=linux go build -o lambdas/lambda1/first lambdas/lambda1/first.go
GOOS=linux go build -o lambdas/lambda2/second lambdas/lambda2/second.go
and then in a later step I need to zip that binary like this:
zip lambdas/lambda1/function.zip lambdas/lambda1/first
zip lambdas/lambda2/function.zip lambdas/lambda2/second
It seems like what I need is to loop through the lambdas directory, and for each directory, get the name of the .go file, and then using those dir paths and file names, I could create each build command.
I'm brand new to Makefiles and have been trying to figure this out for about a day now and am not having luck. Thank you for any help.
What about something like this:
# Find all the .go files under lambdas
GOFILES := $(shell find lambdas -name \*.go)
GOPROGS := $(GOFILES:%.go=%)
GOZIPS := $(GOPROGS:%=%.zip)
# We want to build all the progs and zips
all: $(GOPROGS) $(GOZIPS)
# How to build each prog from a .go file
$(GOPROGS): % : %.go
GOOS=linux go build -o $# $<
# How to build each zip from a prog file
$(GOZIPS) : %.zip : %
zip $# $<
I'm using static pattern rules here but you could use normal pattern rules as well.
Oh I didn't notice you want the zip files to have the same name. This of course assumes that there will be only one .go file, and hence one program, in every directory? Seems limited.
This is not so simple because the names don't match. Although it could be done via separate rules if you really wanted to, the simplest thing to do is just put both the compile and zip into the same rule:
# Find all the .go files under lambdas
GOFILES := $(shell find lambdas -name \*.go)
GOPROGS := $(GOFILES:%.go=%)
# We want to build all the progs
all: $(GOPROGS)
# How to build each prog from a .go file
$(GOPROGS): % : %.go
GOOS=linux go build -o $# $<
zip $(#D)/function.zip $#
Related
I am relatively new to make and don't know how to do one specific thing:
The overall process should look something like this:
the source files are java sources in a directory like src/org/path/to/packages/*.java
I want only to translate a specific java file, but the translation process will automatically translate all dependencies (I say 'translate' because I use j2objc to translate the java files to obj-c files - but that should be of no concern for this question)
The translated files will be put into the build/ directory with a folder structure reflecting the source folder structure (so build/org/path/to/packages/.m+.h)
These *.m and *.h files will then be compiled with j2objcc (a clang wrapper) into *.o files -> this step has to be done per file so every file is compiled with a command like j2objcc -c build/org/path/to/packages/file1.m -o build/org/path/to/package/file1.o
these shall be combined into a static library using ar
My problem is that I know which (one) java file I am starting with, but after step 2 I don't know which *.m and *.h files are generated/translated into the build directory. I'd like to read the contents of the build dir after step 2 with a command like find ./build -name '*.m' at make runtime but I don't know how to use this as a prerequisite in the make target.
I'm writing a Makefile to build a Latex document depending on plots whose data is generated from some other data by some python script.
It looks like this
% pdf plot needed by final document
build/tikz-standalone/%.pdf: build/tikz-standalone/%.tex xy_data
cd $$(dirname $#) && ../../latexrun $$(basename $<)
xy_data: $(PLOT_DATA) tools/plots/crunch.py | build
% crunch.py will create data for plots needed by build/tikz-standalone/%.tex
PYTHONPATH=. tools/plots/crunch.py
build:
mkdir -p build build/other_stuff ...
crunch.py generates several data files in build/data which are needed by build/tikz-standalone/%.tex. To create these files it uses other files stored in the variable PLOT_DATA. I could put a list of the intermediate data files in build/data into the Makefile at the position of xy_data. I don't like this as this would require me to update the list whenever a new file is added. What I want is that all data files are recreated whenever crunch.py or $(PLOT_DATA) has changed.
Is there a way to express this in Make?
If you do not want to provide and maintain the list of the generated files you can turn your (implicitly) phony xy_data target into an empty file used as a marker. Simply touch it at the end of the recipe:
BUILDDIRS := build build/other_stuff ...
build/tikz-standalone/%.pdf: build/tikz-standalone/%.tex xy_data
cd $(dir $#) && ../../latexrun $(notdir $<)
xy_data: $(PLOT_DATA) tools/plots/crunch.py | $(BUILDDIRS)
PYTHONPATH=. tools/plots/crunch.py
touch $#
$(BUILDDIRS):
mkdir -p $#
Note: I also improved a bit some other aspects:
Use of make functions dir and notdir instead of the shell equivalents.
Variable declaration for the build directories to avoid writing the same list several times, which is tedious and error prone.
Explicit list of all build directories as order-only prerequisites instead of just one, which could lead to unexpected results if this single one exists but not some others.
Generic rule for all build directories thanks to the $# automatic variable.
I have few txt files in a directory. I want to run a shell script only on the files which have been modified. How can I achieve this through Makefile?
Have written the following part but it builds all the txt files in the directory. Would be great to get some pointers on this.
FILENAME:= $(wildcard dir/txts/*/*.txt)
.PHONY: build-txt
build-txt: $(FILENAME)
sh build-txts.sh $^
I'm guessing you want something like this:
files := $(wildcard dir/txts/*/*.txt)
dummies := $(addprefix .mod_,$files)
all:$(dummies)
$(dummies): .mod_% : %
sh build-txts.sh $^
touch $#
For any new text file, it will run the script, and create a .mod counterpart. For any non-new text file, it will check if the timestamp is newer than the .mod files timestamp. If it is, it runs the script, and then touches the .mod (making the .mod newer than the text). For any text file that has not been modified since the last make, the .mod file will be newer and the script will not run. Notice that the .mod files are NOT PHONY targets. They are dummy files who exist solely to mark when the text file was last modified. You can stick them in a dummy directory for easy cleaning as well.
If you need something where you don't want to rebuild the text files by default on a fresh checkout, or your script criteria isn't based on timestamps, you would need something a bit more tricky:
files := $(wildcard dir/txts/*/*.txt)
md5s:= $(addprefix .md5_,$files)
all:$(md5s)
.PHONY:$(md5s)
$(md5s):
( [ -e $# ] && md5sum -c $# ) || \
( sh build-txts.sh $# && md5sum $(#:.md5_=) > $# )
Here, you run the rule for all text files regardless, and you use bash to determine if the file is out of date. If the text file does not exist, or the md5sum is not correct, it runs the script, then updates the md5sum. Because the rules are phony, they always run for all the .md5sum files regardless of whether they already exist.
Using this method, you could submit the .md5 files to your repository, and it would only run the script on those files whose md5 sum changed after checkout.
I am trying to use makefile to manage my building process in a small project, where the target number and target names are not known beforehand but depends on the input. Specifically, I want to generate a bunch of data files (say .csv files) according to a cities_list.txt file with a list of city names inside. For example, if the contents of the txt file are:
newyork
washington
toronto
then a script called write_data.py would generate three files called newyork.csv, washington.csv and toronto.csv. When the content of the cities_list.txt file changes, I want make to deal with this change cleverly, i.e. only update the new-added cities files.
I was trying to define variable names in target names to make this happen but didn't succeed. I'm now trying to create a bunch of intermediate .name files as below:
all: *.csv
%.name: cities_list.txt
/bin/bash gen_city_files.sh $<
%.csv: %.name write_data.py
python3 write_data.py $<
clean:
rm *.name *.csv
This seems to be very close to success, but it only gives me one .csv file. The reason is obvious, because make can't determine what files should be generated for the all target. How can I let make know that this *.csv should contain all the files where there exists a corresponding *.name file? Or is there any better way to achieve what I wanted to do here?
All right, this should do it. We'd like a variable assignment at the head of the file:
CITY_FILES := newyork.csv washington.csv toronto.csv
There are two ways to do this. This way:
-include cities.mak
# this rule can come later in the makefile, near the bottom
cities.mak: cities_list.txt
#sed 's/^/CITIES := /' $< > $#
and this way:
CITIES := $(shell cat cities_list.txt)
After we've done one of those two, we can construct the list of needed files:
CITY_FILES := $(addsuffix .csv, $(CITIES))
and build them:
# It is convenient to have this be the first rule in the makefile.
all: $(CITY_FILES)
%.csv: write_data.py
python3 $< $*.name
I want to generate my obj files in a subfolder, I have tried this:
lib/*.o: source/*.cpp
clang++ $(CC_FLAGS) -c -Iinclude source/*.cpp
But it still generates the obj files in the project root and not in the lib/
The project tree that I'm trying to have:
Project/
source/(cpp files)
include/(header files)
lib/(obj files)
You don't show your current makefile, but my suspicion is that it's wrong. However as we can't see it, we'll leave that alone.
The compiler does not support writing multiple object files to a different directory. If you pass multiple source files along with the -c flag then it will write out multiple object files but only to the current directory... as you've discovered the -o flag can't be specified on compile lines which generate multiple output files.
You can change your recipe to look like this:
cd lib && clang++ $(CC_FLAGS) -c -I../include ../source/*.cpp
this will cause all the object files to be written to the lib directory because it's now the current directory.
However, putting this into a makefile is not simple, because make itself is designed to have a single recipe create a single target. However you have this problem with your existing makefile which you don't show, as well.