How do I run nyc merge from Makefile? - makefile

I've inherited a JS code base with Jasmine unit tests. The testing framework uses karma and instanbul-combine to get code coverage. It seems istanbul-combine isn't working with present node modules, and besides is no longer maintained: the recommended replacement is nyc. I'm having trouble replacing istanbul-combine with nyc in the Makefile.
Here's are my attempts at merging the data (not even trying to get a report yet):
#1
#for dir in $(shell ls -d coverage/*/); do \
echo "Merging $${dir}"; \
npx nyc merge $${dir} coverage-final.json; \
done
#2
npx nyc merge coverage coverage-final.json
#3
npx nyc merge --include coverage/*/ coverage-final.json
The coverage data is in coverage/*/coverage-final.json, but none of these attempts succeeds in mergeing it into the result file coverage-final.json.
With #1, I'm pretty sure it's only actually merging a single set of results into the result file. With #2, there's an error; but if I put that command in the shell CLI, nothing is put into the result file.
With #3, at least there's no error, but only one of the coverage files is merged.
Here's the original Makefile line that I'm replacing:
PATH=$(PROJECT_HOME)/bin:$$PATH node_modules/istanbul-combine/cli.js \
-d coverage/summary -r html \
coverage/*/coverage-final.json

I wrote a little script in the Makefile to copy the coverage-final.json files from the child directories of the coverage directory to the coverage directory itself, and then merge them into a coverage-final.json file in the main JS directory.
#cd coverage; \
for dir in $(dir */coverage-final.json); do \
fn="$${dir}coverage-final.json"; \
newName="$${dir::-1}.json"; \
echo "cp $${fn} $${newName}"; \
cp $$fn $$newName; \
done;
npx nyc merge coverage coverage-final.json
The new filenames of the individual coverage files are taken from the name of the directories from which they come.

Related

How can I prioritize one rsync process over another?

I have two separate cronjobs running on a Red Hat instance. Both are copying log files from a remote server to the instance via rsync. The first cronjob runs rsync-new.sh, which copies any new log files (from today or yesterday) from various directories on the server. The second cronjob runs rsync-backfill.sh, which copies any log files older than yesterday. I separated the rsync processes so that the new files will always be copied quickly, and a large backfill job won't interfere with the copying of new files.
This generally works, except for the following case: if rsync-backfill.sh is already copying the old files from a folder, the rsync-new.sh won't copy its files until after rsync-backfill.sh has finished with the folder.
Is there any way to prioritize the rsync command from rsync-new.sh over the rsync command from rsync-backfill.sh? Or to at least let the rsync commands run in parallel so that the new files are always copied quickly?
Here's the general script structure:
rsync-new.sh
for SUBDIR in $(ls $SOURCEDIR)
do
rsyc -zt \
--exclude-from=$TRACKERFILE \
--out-format="%n" \
$SOURCEDIR/$SUBDIR/log-$TODAY*.log $DESTDIR/ | tee -a $TRACKERFILE
done
for SUBDIR in $(ls $SOURCEDIR)
do
rsyc -zt \
--exclude-from=$TRACKERFILE \
--out-format="%n" \
$SOURCEDIR/$SUBDIR/log-$YESTERDAY*.log $DESTDIR/ | tee -a $TRACKERFILE
done
rsync-backfill.sh
for SUBDIR in $(ls $SOURCEDIR)
do
rsyc -zt \
--exclude-from=$TRACKERFILE \
--exclude="log-$TODAY*.log" \
--exclude="log-$YESTERDAY*.log" \
--out-format="%n" \
$SOURCEDIR/$SUBDIR/log-*.log $DESTDIR/ | tee -a $TRACKERFILE
done
This is a non-issue, turns out it was just a coincidence that the new log files for one of the directories weren't syncing until after the backfill was complete (the new log files just happened to be much larger than the backfill files).
Cron runs jobs in isolated environments, so the rsync processes weren't interacting with each other.

How does "make dist" command work? I'm stuck on a script which I need to update to add new file and folder

I've to add a file and a folder to the zip that is created when I run make dist command. This is an open-source project.
After research, I understood I've to modify the Makefile.am but examples online don't work or match with my current Makefile.am
Makefile.am
SUBDIRS = bin data po src extensions docs
DISTCLEANFILES = \
intltool-extract \
intltool-merge \
intltool-update
EXTRA_DIST = \
$(bin_SCRIPTS) \
intltool-merge.in \
intltool-update.in \
intltool-extract.in
DISTCHECK_CONFIGURE_FLAGS = --disable-update-mimedb
check-po:
#for i in $(top_srcdir)/po/*.po ; do \
if ! grep -q ^`basename $$i | \
sed 's,.po,,'`$$ $(top_srcdir)/po/LINGUAS ; then \
echo '***' `basename $$i | \
sed 's,.po,,'` missing from po/LINGUAS '***' ; \
exit 1; \
fi; \
done;
lint:
flake8 --ignore E402 $(top_srcdir)/src $(top_srcdir)/extensions
test: lint check-po
PYTHONPATH=$(pkgdatadir)/extensions:$(PYTHONPATH) \
python -m sugar3.test.discover $(top_srcdir)/tests
configure.ac
AC_INIT([Sugar],[0.114],[],[sugar])
AC_PREREQ([2.59])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_SRCDIR([configure.ac])
SUCROSE_VERSION="0.114"
AC_SUBST(SUCROSE_VERSION)
AM_INIT_AUTOMAKE([1.9 foreign dist-xz no-dist-gzip])
AM_MAINTAINER_MODE
PYTHON=python2
AM_PATH_PYTHON
AC_PATH_PROG([EMPY], [empy])
if test -z "$EMPY"; then
AC_MSG_ERROR([python-empy is required])
fi
PKG_CHECK_MODULES(SHELL, gtk+-3.0)
IT_PROG_INTLTOOL([0.35.0])
GETTEXT_PACKAGE=sugar
AC_SUBST([GETTEXT_PACKAGE])
AM_GLIB_GNU_GETTEXT
AC_ARG_ENABLE(update-mimedb,
AC_HELP_STRING([--disable-update-mimedb],
[disable the update-mime-database after install [default=no]]),,
enable_update_mimedb=yes)
AM_CONDITIONAL(ENABLE_UPDATE_MIMEDB, test x$enable_update_mimedb = xyes)
GLIB_GSETTINGS
AC_CONFIG_FILES([
bin/Makefile
bin/sugar
data/icons/Makefile
data/Makefile
extensions/cpsection/aboutcomputer/Makefile
extensions/cpsection/aboutme/Makefile
extensions/cpsection/background/Makefile
extensions/cpsection/backup/Makefile
extensions/cpsection/backup/backends/Makefile
extensions/cpsection/datetime/Makefile
extensions/cpsection/frame/Makefile
extensions/cpsection/keyboard/Makefile
extensions/cpsection/language/Makefile
extensions/cpsection/modemconfiguration/Makefile
extensions/cpsection/Makefile
extensions/cpsection/network/Makefile
extensions/cpsection/power/Makefile
extensions/cpsection/updater/Makefile
extensions/cpsection/webaccount/services/Makefile
extensions/cpsection/webaccount/Makefile
extensions/deviceicon/Makefile
extensions/globalkey/Makefile
extensions/webservice/Makefile
extensions/Makefile
Makefile
po/Makefile.in
src/jarabe/config.py
src/jarabe/controlpanel/Makefile
src/jarabe/desktop/Makefile
src/jarabe/frame/Makefile
src/jarabe/intro/Makefile
src/jarabe/journal/Makefile
src/jarabe/Makefile
src/jarabe/model/Makefile
src/jarabe/model/update/Makefile
src/jarabe/util/Makefile
src/jarabe/util/telepathy/Makefile
src/jarabe/view/Makefile
src/jarabe/webservice/Makefile
src/Makefile
])
AC_OUTPUT
When I run the command make dist, the output zip doesn't include a file and folder which I now need to add. I'm not able to understand where in the code(Makefile.am or configure.ac) should I make changes.
I've to add a file and a folder to the zip that is created when I run make dist command.
I take it that these are not already included in the distribution. If you're not sure, then check -- Automake-based build systems such as yours identify a lot of files automatically for inclusion in distribution packages.
Supposing that these files are not already included, there are several ways to cause them to be. Easiest would be to add them to the EXTRA_DIST variable, yielding
EXTRA_DIST = \
$(bin_SCRIPTS) \
intltool-merge.in \
intltool-update.in \
intltool-extract.in \
a_directory \
some_file.ext
Don't forget the trailing backslashes if you continue with the multiline form (which I like, as I find it much easier to read). You can specify a path to the file, the directory, or both. Do note that in the case of the directory, it will be not just the directory itself but all its contents, recursively, that are included in the distribution. This is all documented in the manual.
If you need finer control, then there is also an extension point for managing the contents of the distribution in the form of the "dist hook". This comprises a make target named dist-hook. Like any other literal make rule in your Makefile.am, any rule you provide for building that target is copied to the final generated Makefile, and if such a rule is present then its recipe is run as part of building the distribution, after the distribution directory is otherwise populated but before the archive file is built from that. You can write more or less arbitrary shell code in that target's recipe to tweak the distribution. Follow the above link to the documentation for the gory details.
EXTRA_DIST variable sounds like the thing.

How to execute a make target on file change automatically?

How do I write a make target that will watch for any file changes in specific folders and execute some other make target to compile files? I am looking for a way that can do this with minimal dependency on tools in addition to make itself to keep things simple.
For the watching you can use fswatch. (There's also a go version of this program which may be easier to install: fswatch) For example:
fswatch -ext cpp,c,h make -f Makefile
Anytime you change a cpp, c or h file it will run make again.
Make can be a bit slow for this, so I tend to use ninja instead, but that really depends on the size of your project.
Another option is tup, which has watching built-in:
tup monitor
But, sadly, only for linux.
You can use entr and adjust your Makefile similar to this one
.DEFAULT_GOAL := run
SHELL := /bin/bash
run:
clear && \
cp one.txt two.txt && \
rm -f _* *.l2m *.o2m && \
Ganlib < testgan2.x2m
watch:
while sleep 1 ; do find . -name '*.x2m' -o -name '*.c2m' \
| entr -d make -f ./Makefile ; done
.PHONY: run watch
followed by
$ make watch

make test doesn't do anything for mocha tests

I'm following this example: http://brianstoner.com/blog/testing-in-nodejs-with-mocha/
I defined a Makefile in my root directory:
REPORTER = dot
test:
#NODE_ENV=test ./node_modules/.bin/mocha \
--reporter $(REPORTER) \
test-w:
#NODE_ENV=test ./node_modules/.bin/mocha \
--reporter $(REPORTER) \
--watch
.PHONY: test test-w
But when I run 'make test' it says "make: Nothing to be done for `test'."
Turns out Makefiles are tab-sensitive -- and those were clobbered when I cut-n-paste the file...
In vi, i turned on tabs and spacing to fix it:
vi Makefile
:set list
Now you can retab the file and ensure you are doing it correctly.

rsync exclude a directory but include a subdirectory

I am trying to copy a project to my server with rsync.
I have project specific install scripts in a subdirectory
project/specs/install/project1
What I am trying to do is exclude everything in the project/specs directory but the project specific install directory: project/specs/install/project1.
rsync -avz --delete --include=specs/install/project1 \
--exclude=specs/* /srv/http/projects/project/ \
user#server.com:~/projects/project
But like this the content of the specs directory gets excluded but the install/project1 directory does not get included.
I have tried everything but i just don't seem to get this to work
Sometime it's just a detail.
Just change your include pattern adding a trailing / at the end of include pattern and it'll work:
rsync -avz --delete --include=specs/install/project1/ \
--exclude=specs/* /srv/http/projects/project/ \
user#server.com:~/projects/project
Or, in alternative, prepare a filter file like this:
$ cat << EOF >pattern.txt
> + specs/install/project1/
> - specs/*
> EOF
Then use the --filter option:
rsync -avz --delete --filter=". pattern.txt" \
/srv/http/projects/project/ \
user#server.com:~/projects/project
For further info go to the FILTER RULES section in the rsync(1) manual page.
The other solution is not working here.
Reliable way
You have no choice but to manually descend for each level of your sub-directory. There is no risk to include unwanted files, as rsync doesn't include the files of included directories.
1) Create an include filter file, for instance "include_filter.txt":
+ /specs/
+ /specs/install/
+ /specs/install/project1/***
- /specs/**
2) Run it:
rsync -avz --delete --include-from=include_filter.txt \
/srv/http/projects/project/ \
user#server.com:~/projects/project
Don't forget the starting slash "/", otherwise you may match sub-directories named "**/specs/install/project1/".
By choosing an include type filter (--include-from=FILE), the starting plus "+" signs are actually optional, as this is the default action with no sign. (You can have the opposite "-" by default with --exclude-from=FILE.)
The double stars "**" means "any path"
The triple stars "***" means "any path, including this very directory"
Easy way
You can start your filters "*/", allowing rsync to descend all your sub-levels. This is convenient but:
All directories will be included, albeit empty. This can be fixed with the rysnc option -m, but then all empty dirs will be skipped.
1) Create an include filter file, for instance "include_filter.txt":
+ /**/
+ /specs/install/project1/***
- /specs/**
2) Run it:
rsync -avzm --delete --include-from=include_filter.txt \
/srv/http/projects/project/ \
user#server.com:~/projects/project
Note the added option -m.
Order of --include and --exclude affects what is being included or excluded.
When there are particular subdirectories to be included need to place them first.
Similar post available here.

Resources