This is my makefile
file1:
uglifyjs myfile1.js -c | gzip -c -9 > myfile1.min.js
file2:
uglifyjs myfile2.js -c | gzip -c -9 > myfile2.min.js
How can I change my makefile to remove duplicate code:
file1:
FILE=myfile1.js
#How to call build target?
file2:
FILE=myfile2.js
build:
uglifyjs $(FILE).js -c | gzip -c -9 > $(FILE).min.js
I know I can use make build but is there another way to do this without invoking make recursively?
Use automatic variables:
file1 file2:
uglifyjs my$#.js -c | gzip -c -9 > my$#1.min.js
I don't know why you're using targets like file1 when the file you're actually building is myfile1.min.js. That's not a good makefile.
But, that's not the question you asked.
Use a pattern rule to run the command, and then make your targets depend on the files you want:
file1: myfile1.min.js
file2: myfile2.min.js
%.min.js: %.js
uglifyjs $< -c | gzip -c -9 >$#
The pattern rule tells make how to build a .min.js file from a .js file, and the other rules tell it to build specific files.
Related
In a directory I have a config file with my db variables.
This file (db/database.ini) looks like this:
[PostgreSQL]
host=localhost
database=...
user=postgres
password=...
I have another file (db/create_stmts.sql) where I have all my raw create table statements, and i am trying to experiment the use of a Makefile to have a command like this:
make create-db from_file=db/create_stmts.sql
In order not to repeat myself, I thought of tailing the variables of db/database.ini to a file which I would then source, creating shell variables to pass to psql in the make file.
Here's my plan:
make-db:
# from_file: path to .sql file with all create statements to create the database where to insert
# how to run: make create-db from_file={insert path to sql file}
file_path=$(PWD)/file.sh
tail -n4 db/database.ini > file.sh && . $(file_path)
# -U: --user
# -d: --database
# -q: --quiet
# -f: --file
psql -U $(user) -d $(database) -q -f $(from_file) && rm file.sh
Which I run by: make create-db from_file=db/create_stmts.sql
Which gives me this message - from which i kindof understand that the sourcing just did not work.
#from_file: path to .sql file with all create statements to create the database where to insert
# how to run: make create-db from_file={insert path to sql file}
file_path=/home/gabriele/Desktop/TIUK/companies-house/file.sh
tail -n4 db/database.ini > file.sh && .
# -U: --user
# -d: --database
# -q: --quiet
# -f: --file
psql -U -d -q -f db/schema_tables.sql && rm file.sh
psql: FATAL: Peer authentication failed for user "-d"
Makefile:3: recipe for target 'create-db' failed
make: *** [create-db] Error 2
Any help?
Another solution, perhaps simpler to understand:
make-db:
file_path=$$PWD/file.sh; \
tail -n4 db/database.ini > file.sh && . $$file_path; \
psql -U $$user -d $$database -q -f $$from_file && rm file.sh
Note using ; and \ to convince make to run all commands in a single shell, and using $$ to escape the $ and use shell variable references.
The error is in the text, namely
psql -U -d -q -f db/schema_tables.sql && rm file.sh
This happens because the variables $(user) and $(database) aren't set. Every line within a target is executed in a sub shell. There is now way to use source like you would in a regular script.
You could create a file named database.mk in which you define these variables and use include database.mk at the top of your makefile to include them:
Makefile
CONFILE ?= database
include $(CONFILE).mk
test:
#echo $(user)
#echo $(database)
database.mk
user := user
database := data
If you want to parse the ini file you could do that as such
CONFILE := db/database.ini
make-db: _setup_con
echo $(user) $(database)
# your target
_setup_con:
$(eval user=$(shell grep "user=" $(CONFILE) | grep -Eo "[^=]*$$"))
$(eval database=$(shell grep "database=" $(CONFILE) | grep -Eo "[^=]*$$"))
# and so forward
I would make it more Make-way by using feature of automatic Makefile generation. Given that a configuration file is a simple properties file, its syntax is easily parseable by Make, it's sufficient to just get the lines with variables, i.e.:
include database.mk
database.mk: db/database.ini
grep -E '^\w+=\w+$$' $< > $#
.PHONY: create-db
create-db: $(from_file)
psql -U $(user) -d $(database) -q -f $<
Some additional notes:
create-db should be made .PHONY to avoid situation when nothing is done due to somebody creating (accidentally or not) a file named create-db,
by making create-db depending on from_file one can get a clean and readable error from make that a file does not exist instead of possibly cryptic error later.
I'm trying to download multiple files in parallel using xargs. Things worked so well if I only download the file without given name. echo ${links[#]} | xargs -P 8 -n 1 wget. Is there any way that allow me to download with filename like wget -O [filename] [URL] but in parallel?
Below is my work. Thank you.
links=(
"https://apod.nasa.gov/apod/image/1901/sombrero_spitzer_3000.jpg"
"https://apod.nasa.gov/apod/image/1901/orionred_WISEantonucci_1824.jpg"
"https://apod.nasa.gov/apod/image/1901/20190102UltimaThule-pr.png"
"https://apod.nasa.gov/apod/image/1901/UT-blink_3d_a.gif"
"https://apod.nasa.gov/apod/image/1901/Jan3yutu2CNSA.jpg"
)
names=(
"file1.jpg"
"file2.jpg"
"file3.jpg"
"file4.jpg"
"file5.jpg"
)
echo ${links[#]} ${names[#]} | xargs -P 8 -n 1 wget
With GNU Parallel you can do:
parallel wget -O {2} {1} ::: "${links[#]}" :::+ "${names[#]}"
If a download fails, GNU Parallel can also retry commands with --retry 3.
we have a huge project with very time consuming sub-tasks running on Unix. The whole make process runs multiple hours. So building in parallel is essential for us. The expensive jobs are preformed on a remote server via ssh. Everything works fine. But I'm afraid that accidentally a team member could forget to specify the number for the -j flag. The --load-average flag uses the load on the local server so no matter how busy the remote server is it would generate hundreds of sub-tasks on the remote server which would slow down the entire company. Please note that I don't have admin rights on either server. We could also live with limiting the number of jobs (hardcoded).
Thanks a lot in Advance
Karl
If you have GNU make 4.2 or better, you can check that the user didn't give a raw -j by looking at MAKEFLAGS:
ifeq (-j,$(filter -j,$(MAKEFLAGS)))
$(error You cannot use -j without specifying a number of jobs)
endif
If you have an earlier version of GNU make you can do this by checking if both -j was given AND the jobserver is active:
ifneq (,$(filter -j,$(MAKEFLAGS)))
ifneq (,$(filter --jobserver-%,$(MAKEFLAGS)))
$(error You cannot use -j without specifying a number of jobs)
endif
endif
(this version will work for newer versions as well).
It's possible this won't work with versions of GNU make <4.0; I didn't test it.
For older make versions you can consider the following. That is not pure makefile way, but works.
Solution
ifneq (0,$(shell ps --pid $$PPID -o args= | grep --perl-regexp -c '^make.*-j(?!\s*[0-9]+\b)'))
$(error You cannot use -j without specifying a number of jobs)
endif
# Or oneliner:
#$(if $(shell ps --pid $$PPID -o args= | grep --perl-regexp -c '^make.*-j(?!\s*[0-9]+\b)' | grep -v '^0$$'),$(error You cannot use -j without specifying a number of jobs))
# Below part just for testing
all a 1: ; #exit 0
Testing
$ make
$ make a
$ make 1
$ make -j1
$ make -j1 a
$ make -j1 1
$ make -j 1
$ make -j 1a
Makefile:6: *** You cannot use -j without specifying a number of jobs. Stop.
$ make -j a
Makefile:6: *** You cannot use -j without specifying a number of jobs. Stop.
$ make a -j
Makefile:6: *** You cannot use -j without specifying a number of jobs. Stop.
$ make 1 -j a
Makefile:2: *** You cannot use -j without specifying a number of jobs. Stop.
How it works
It is taking information from parent process (that is make process) using ps utility, and later filter out arguments using grep (regular expression is more or less: find make command with -j parameter that is not followed by a number).
Based on to the answer from Kuchara above I needed the following enhancement, since I call some subsystems with:
$(MAKE) $(MFLAGS) ...
and this will add the jobserver and remove the number from the -j flag in the subsystem:
yy:
#echo yy mflags: $(MFLAGS)
#echo yy ps: $(shell ps --pid $$PPID -o args=)
$(MAKE) $(MFLAGS) zz
zz:
#echo zz mflags: $(MFLAGS)
#echo zz ps: $(shell ps --pid $$PPID -o args=)
Testing:
make -j 3 yy
yy mflags: - --jobserver-fds=3,4 -j
yy ps: make -j 3 yy
zz mflags: -w --jobserver-fds=3,4 - --jobserver-fds=3,4 -j
zz ps: make - --jobserver-fds=3,4 -j zz
I needed to combine the answers from MadScientist and Kuchara:
ifneq (0,$(shell ps --pid $$PPID -o args= | grep --perl-regexp -c '^make.*-j(?!\s*[0-9]+\b)'))
ifeq (0,$(shell ps --pid $$PPID -o args= | grep --perl-regexp -c '^make.*--jobserver'))
$(error You cannot use -j without specifying a number of jobs)
endif
endif
Thanks a lot again at both for your great ideas.
How do I write a make target that will watch for any file changes in specific folders and execute some other make target to compile files? I am looking for a way that can do this with minimal dependency on tools in addition to make itself to keep things simple.
For the watching you can use fswatch. (There's also a go version of this program which may be easier to install: fswatch) For example:
fswatch -ext cpp,c,h make -f Makefile
Anytime you change a cpp, c or h file it will run make again.
Make can be a bit slow for this, so I tend to use ninja instead, but that really depends on the size of your project.
Another option is tup, which has watching built-in:
tup monitor
But, sadly, only for linux.
You can use entr and adjust your Makefile similar to this one
.DEFAULT_GOAL := run
SHELL := /bin/bash
run:
clear && \
cp one.txt two.txt && \
rm -f _* *.l2m *.o2m && \
Ganlib < testgan2.x2m
watch:
while sleep 1 ; do find . -name '*.x2m' -o -name '*.c2m' \
| entr -d make -f ./Makefile ; done
.PHONY: run watch
followed by
$ make watch
Need to test the performance of API's with variable parameters (x and y)
e.g. http://myapiurl?x=1&y=2&z=6
http://myapiurl?x=3&y=3&z=6
http://myapiurl?x=5&y=2&z=6
..........
now while bench marking I want to hit the urls randomly from a text file.
$ ab -c 100 -c 500 urls.txt
Patch - https://github.com/philipgloyne/apachebench-for-multi-url/blob/master/README.md
gcc -I /usr/include/apr-1.0 -I /usr/include/apache2 ab.c -o ab -lm -lapr-1 -laprutil-1
ab -c 100 -v 4 -n 2000 -L urls.txt > results.txt
This will load the target urls from text file.