I've got these two make targets:
NPM_OUT = node_modules/npm.sfv
NPM_BIN = $(shell command -v npm || command -v /usr/bin/npm || echo "npm")
HASH_CMD = $(shell command -v md5 || command -v md5sum)
$(NPM_OUT): npm-shrinkwrap.json
$(NPM_BIN) install --loglevel=error
#$(HASH_CMD) npm-shrinkwrap.json > $(NPM_OUT)
npm-shrinkwrap.json:
$(NPM_BIN) install --loglevel=error
$(NPM_BIN) prune
$(NPM_BIN) dedupe
$(NPM_BIN) shrinkwrap --dev
NPM_OUT is basically a bogus file I use just to determine if npm install has been ran yet. I don't know how to do this without a bogus file because that one command generates many output files, not just one object file like you'd see in c/c++.
The problem I'm having is that if npm-shrinkwrap.json doesn't exist then npm install --loglevel=error gets ran twice.
As you can see, it exists in both targets. If npm-shrinkwrap.json doesn't exist, then I need to run npm install before I can create the shrinkwrap file. But if I do that, then I don't need to run it again for $(NPM_OUT). The reason it's in $(NPM_OUT) is because I need to run it every time npm-shrinkwrap.json changes.
I thought maybe I could create a 3rd target for npm install which the other 2 targets could depend on, but unless I specify a dependency file for that too, then it will always run.
How can I handle this?
Say the npm install is up-to-date is the state of affairs you want to
proxy with the NPM_OUT.
You want the npm install is up-to-date to be dependent on npm-shrinkwrap.json,
if and only if npm-shrinkwrap.json exists. If you manually edit npm-shrinkwrap.json,
for instance, then that means the npm install is no longer up to up-to-date and
must be made so, using npm-shrinkwrap.json.
But if npm-shrinkwrap.json doesn't exist then in fact the dependency is the
other way round, because you need to make the npm install up-to-date in
order to make npm-shrinkwrap.json, ab initio.
So you have different dependencies depending on whether or not npm-shrinkwrap.json
exists:
NPM_BIRTH_CERT = node_modules/.npm_installed.timestamp
NPM_BIN = $(shell command -v npm || command -v /usr/bin/npm || echo "npm")
NPM_SHRINKWRAP := $(wildcard npm-shrinkwrap.json)
.PHONY: all clean really-clean
all: npm-shrinkwrap.json $(NPM_BIRTH_CERT)
$(NPM_BIRTH_CERT): $(NPM_SHRINKWRAP)
$(NPM_BIN) install --loglevel=error
touch $#
ifndef NPM_SHRINKWRAP
npm-shrinkwrap.json: $(NPM_BIRTH_CERT)
$(NPM_BIN) prune
$(NPM_BIN) dedupe
$(NPM_BIN) shrinkwrap --dev
touch $<
endif
clean:
rm -fr node_modules
really-clean: clean
rm -f npm-shrinkwrap.json
Here the rule:
$(NPM_BIRTH_CERT): $(NPM_SHRINKWRAP)
is:
$(NPM_BIRTH_CERT): npm-shrinkwrap.json
if npm-shrinkwrap.json exists and otherwise just:
$(NPM_BIRTH_CERT):
And whenever npm-shrinkwrap.json exists it's not a target at all.
I don't see the need make $(NPM_BIRTH_CERT) resolve to an MD5
hash of the npm-shrinkwrap.json, rather than just a file that
bears the timestamp of npm installs last completion or
npm-shrinkwrap.jsons last generation from an npm install, whichever is latest.
Using the ideas from Mike's answer, I think I can get this to behave the way I want:
NPM_SHRINKWRAP := $(wildcard npm-shrinkwrap.json)
ifdef NPM_SHRINKWRAP
$(NPM_OUT): npm-shrinkwrap.json
$(NPM_BIN) install --loglevel=error
touch $(NPM_OUT)
else
$(NPM_OUT): npm-shrinkwrap.json
endif
npm-shrinkwrap.json: package.json
$(NPM_BIN) install --loglevel=error --no-shrinkwrap
$(NPM_BIN) prune
$(NPM_BIN) dedupe
$(NPM_BIN) shrinkwrap --dev
touch $(NPM_OUT)
Basically, if the shrinkwrap file exists, then make $(NPM_OUT) will run npm install iff the shrinkwrap file has been updated.
If the shrinkwrap file doesn't exist, and since npm-shrinkwrap.json is still a dependency, the npm-shrinkwrap.json target will do the installation instead.
However, I introduced a new problem here: if npm-shrinkwrap.json does exist and package.json (wasn't in original question) has been updated, then the double-install will happen again. I don't know if that can be fixed, or if it's even worth fixing.
Related
my package doesn't requires virtualenv directly, some 3rd package does. However, when running test in tox, poetry install -E test -vvv alwasy fail due to:
poetry removes virtualenv first, which is created by tox
then it tries remove other parts and failed, due to virtualenv is removed, some packages cannot found.
the tox.ini:
[testenv]
skip_install = true
deps = poetry
commands =
poetry install -E test -vvv
the errors:
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file
Finding the necessary packages for the current system
Package operations: 73 installs, 1 update, 16 removals, 68 skipped
• Removing virtualenv (20.16.3): Pending...
• Removing virtualenv (20.16.3): Removing...
• Removing virtualenv (20.16.3)
• Removing webencodings (0.5.1): Pending...
• Removing webencodings (0.5.1): Removing...
• Removing webencodings (0.5.1): Failed
Command '['/apps/backtest/.tox/py38/bin/python', '/apps/backtest/.tox/py38/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'uninstall', 'webencodings', '-y']' returned non-zero exit status 2.
Command ['/apps/backtest/.tox/py38/bin/python', '/apps/backtest/.tox/py38/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'uninstall', 'zipp', '-y'] errored with the following return code 2, and output:
/apps/backtest/.tox/py38/bin/python: can't open file '/apps/backtest/.tox/py38/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip': [Errno 2] No such file or directory
of course pip doesn't exist since it belongs virtualenv and has been removed.
the question is:
how to find which 3rd packages requires virtualenv?
how to disallow poetry to remove virtualenv (it does this for install it later) if I can't remove dependency to virtualenv?
You are mixing two different installation concepts and the second overrides the first.
deps = poetry
This installs poetry (and it's dependencies including virtualenv) into into the virtualenvironment created by tox. The deps section is a tox concept that installs packages required for testing other than the package installation itself.
Then the commands run.
poetry install -E test -vvv
The poetry command will detect it is inside a virtualenv and then install the dependencies into that virtualenv, but also cleaning up unnecessary packages for your package. Thus, poetry is overriding it's own dependencies. Causing the errors you're encountering.
Solution is documented here. Usecase 1 does the trick for me.
You would need to include the pyproject.toml into your answer as that would be necessary for me to identify any erroneous setup there.
Tons of compile errors occur when make.
What I did are itemized:
My operation is shown itemized:
yum install git
yum install gcc-c++
yum install cmake
yum install python
yum install zeromq-devel
Then :
git clone https://github.com/google/googletest.git
cd /googletest/googletest
[root#VM_16_11_centos googletest]# cmake CMakeLists.txt
-- Configuring done
-- Generating done
-- Build files have been written to:
/usr/local/qiuxin/googletest/googletest
`[root#VM_16_11_centos googletest]# make
Huge Error here!!!
In file included from
/usr/local/qiuxin/googletest/googletest/src/gtest-all.cc:38:0:
/usr/local/qiuxin/googletest/googletest/include/gtest/gtest.h: In
member function 'virtual
testing::Test::Setup_should_be_spelled_SetUp*
testing::Test::Setup()':
solved. The mistake I made was:
1) I created a build folder in the googletest/googletest directory.
The right operation should be:
Create a build folder in googletest directory(NOT googletest/googletest directory.).
then:
cd build
cmake .. -DBUILD_GTEST=ON -DBUILD_SHARED_LIBS=ON
make
sudo make install
Everything goes well!
I was excited to use bash on windows but quickly ran into an issue. I am trying to install build-essential, but I get a dependency issue. Upon trying to resolve with sudo apt-get install -f, I run into another error that doesn't make much sense. I've tried update and upgrade, but that didn't work either. F
barzevp#UK-LT-8356:~$ sudo apt-get install build-essential
Reading package lists... Done
Building dependency tree
Reading state information... Done
build-essential is already the newest version.
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies.
libc6-dev : Depends: linux-libc-dev but it is not going to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
barzevp#UK-LT-8356:~$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
libfreetype6 os-prober
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
linux-libc-dev
The following NEW packages will be installed
linux-libc-dev
0 to upgrade, 1 to newly install, 0 to remove and 43 not to upgrade.
5 not fully installed or removed.
Need to get 0 B/767 kB of archives.
After this operation, 3,946 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 28660 files and directories currently installed.)
Preparing to unpack .../linux-libc-dev_3.13.0-123.172_amd64.deb ...
Unpacking linux-libc-dev:amd64 (3.13.0-123.172) ...
dpkg: error processing archive /var/cache/apt/archives/linux-libc-dev_3.13.0-123.172_amd64.deb (--unpack):
unable to install new version of /usr/include/linux/netfilter_ipv6/ip6t_hl.h': File exists
E: Sub-process /usr/bin/dpkg returned an error code (1)
Full log of what led to the error in Bash on Ubuntu on Windows terminal is here:
https://pastebin.com/dq2D2Gtz
I don't have a solution, but I see the root of the problem. It seems to have to do with case sensitivity on the filesystem. The package for linux-libc-dev puts two copies of a file in /usr/include/linux/netfilter_ipv6 with only a difference in case; ip6t_HL.h, and ip6t_hl.h. After placing ip6t_HL.h, it attempts to rename ip6t_hl.h.dpkg-new to ip6t_hl.h. The system call to rename the file fails, claiming that ip6t_hl.h already exists
On a "real" linux system, ip6t_HL.h and ip6t_hl.h would clearly be different files. Under WSL they probably had some strange incompatibilities to work out between NTFS's default case insensitive FS and unix's default case sensitivity.
You can duplicate the problem by hand e.g., with
echo hi > foo.H
echo hi > foo.h-new
mv foo.h-new foo.h
mv: cannot move 'foo.h-new' to 'foo.h': File exists
strace output:
rename("/usr/include/linux/netfilter_ipv6/ip6t_HL.h.dpkg-new", "/usr/include/linux/netfilter_ipv6/ip6t_HL.h") = 0
open("/usr/include/linux/netfilter_ipv6/ip6t_hl.h.dpkg-new", O_WRONLY) = 10
fsync(10) = 0
close(10) = 0
rename("/usr/include/linux/netfilter_ipv6/ip6t_hl.h.dpkg-new", "/usr/include/linux/netfilter_ipv6/ip6t_hl.h") = -1 EEXIST (File exists)
write(2, "dpkg: error processing archive /"..., 199dpkg: error processing archive /var/cache/apt/archives/linux-libc-dev_4.4.0-98.121_amd64.deb (--install):
unable to install new version of '/usr/include/linux/netfilter_ipv6/ip6t_hl.h': File exists
) = 199
I had the same problem. Like #dmattp I found that it this is because, unfortunately, the package contains some (header) files with names that are distinct only by letter case, and that the wonderful WSL file system has an inconsistency regarding case sensitiveness.
Here is a workaround, assuming that the package has name linux-libc-dev_3.13.0-123.172_amd64.deb:
cd any-temp-dir
apt-get download linux-libc-dev
ar x linux-libc-dev_3.13.0-123.172_amd64.deb
tar xJf data.tar.xz # ignore all erors like ./usr/include/linux/netfilter_ipv4/ipt_ttl.h: Cannot open: Input/output error
tar cJf data.tar.xz ./usr
ar rcs linux-libc-dev_3.13.0-123.172_amd64-patched.deb debian-binary control.tar.gz data.tar.xz
sudo dpkg -i linux-libc-dev_3.13.0-123.172_amd64-patched.deb
It says dependencies aren't being installed, so try:
sudo apt-get install linux-libc-dev
If that doesn't work try:
sudo apt-get install --reinstall build-essential
This will reinstall build-essential. Hope this helps, cheers!
I tried to install svnnotify(SVN-NOTIFY-2.8.4) and when I tried to execute the svnnotify which was saved in the bin of the download folder. I got the below response:
Can't locate SVN/Notify.pm in #INC (#INC contains: /Library/Perl/5.12/darwin-thread-multi-2level /Library/Perl/5.12 /Network/Library/Perl/5.12/darwin-thread-multi-2level /Network/Library/Perl/5.12 /Library/Perl/Updates/5.12.4 /System/Library/Perl/5.12/darwin-thread-multi-2level /System/Library/Perl/5.12 /System/Library/Perl/Extras/5.12/darwin-thread-multi-2level /System/Library/Perl/Extras/5.12 .) at /Users/bertils/Downloads/SVN-Notify-2.84/bin/svnnotify line 4.
BEGIN failed--compilation aborted at /Users/bertils/Downloads/SVN-Notify-2.84/bin/svnnotify line 4.
You skipped the install step.
Check the README.md file in the unzipped module, but basically you'll want to run:
perl Build.pl
if you're warned about missing dependencies, then (as suggested by the last command), install those. You probably want to do that as root:
sudo ./Build installdeps
next (again, from the README)
./Build
./Build test
if those steps went well, install (as root):
sudo ./Build install
svnnotify should now be in your path, ready to run.
I used node.js to install karma. My first try failed when running the following command on Terminal:
npm install -g karma
That failed so I decided to use:
sudo npm install -g karma
After entering my password it seemed to install correctly.
I am pasting part of the output of the install, maybe it will mean something to someone and it will be relevant to my question. After all the npm http messages this is what I see:
> ws#0.4.27 install /usr/local/share/npm/lib/node_modules/karma/node_modules/socket.io/node_modules/socket.io-client/node_modules/ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)
CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
SOLINK_MODULE(target) Release/bufferutil.node
SOLINK_MODULE(target) Release/bufferutil.node: Finished
CXX(target) Release/obj.target/validation/src/validation.o
SOLINK_MODULE(target) Release/validation.node
SOLINK_MODULE(target) Release/validation.node: Finished
/usr/local/share/npm/bin/karma -> /usr/local/share/npm/lib/node_modules/karma/bin/karma
karma#0.8.6 /usr/local/share/npm/lib/node_modules/karma
├── pause#0.0.1
├── dateformat#1.0.2-1.2.3
├── xmlbuilder#0.4.2
├── colors#0.6.0-1
├── chokidar#0.6.2
├── growly#1.1.1
├── mime#1.2.9
├── q#0.9.6
├── rimraf#2.1.4 (graceful-fs#1.2.3)
├── coffee-script#1.6.3
├── minimatch#0.2.12 (sigmund#1.0.0, lru-cache#2.3.0)
├── optimist#0.3.5 (wordwrap#0.0.2)
├── glob#3.1.21 (inherits#1.0.0, graceful-fs#1.2.3)
├── LiveScript#1.0.1 (prelude-ls#1.0.1)
├── log4js#0.6.6 (dequeue#1.0.3, semver#1.1.4, async#0.1.15, readable-stream#1.0.2)
├── lodash#1.1.1
├── http-proxy#0.10.3 (pkginfo#0.2.3, utile#0.1.7)
├── istanbul#0.1.22 (abbrev#1.0.4, which#1.0.5, fileset#0.1.5, nopt#2.0.0, wordwrap#0.0.2, async#0.1.22, mkdirp#0.3.5, esprima#0.9.9, escodegen#0.0.24, handlebars#1.0.12)
└── socket.io#0.9.16 (base64id#0.1.0, policyfile#0.0.4, redis#0.7.3, socket.io-client#0.9.16)
Then when I try to run the following command to create a karma config file with this command:
karma init karma.config.js
this is the message that gets returned:
-bash: karma: command not found
I have tried the same command with sudo before it but I get the same result.
Does anyone have any idea as to what is going on?
Any help is appreciated.
*Update!
I decided to check a file named: builderror.log
located in: /usr/local/share/npm/lib/node_modules/karma/node_modules/socket.io/node_modules/socket.io-client/node_modules/ws
This is what it shows:
gyp WARN EACCES user "root" does not have permission to access the dev dir "/Users/eperez/.node-gyp/0.10.5"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/share/npm/lib/node_modules/karma/node_modules/socket.io/node_modules/socket.io-client/node_modules/ws/.node-gyp"
gyp http GET http://nodejs.org/dist/v0.10.5/node-v0.10.5.tar.gz
gyp http 200 http://nodejs.org/dist/v0.10.5/node-v0.10.5.tar.gz
#mayankcpdixit gave the answer up there in a response to the OP's original question, but I'll put it here again in case anyone misses it.
You do not need to uninstall everything, and if I had to manually add a new path link for every npm package I try to install I'd probably shoot myself.
npm install -g karma-cli
Boom. Now you have karma command lines installed. Just like Grunt.
Edit: Please don't forget to upvote #mayankcpdixit as well, he commented directly on the original post, but didn't actually "answer" the question.
In your ~/.bash_profile (or similar) amend your PATH to include npm-installed binaries:
export PATH="$PATH:/usr/local/share/npm/bin"
I had this very same issue, and found this solution to be less time-consuming and impactful than completely re-installing node.
EDIT this has also worked for others in bash_profile
export PATH="$PATH:/usr/local/lib/node_modules/karma/bin"
It is recommended to install karma with its Command-Line-Interface (karma-cli) which will take care of fetching the appropriate karma. You can also install a different local version specific to each project you're working on and karma-cli will pick the appropriate one.
From the karma installation page:
Typing ./node_modules/karma/bin/karma start sucks so you might find it useful to install karma-cli globally:
npm install -g karma-cli
Now, check that karma was installed by typing:
karma start
You can also check that karma was installed by going to this directory:
cd /usr/local/lib/node_modules/karma
Good luck!
Don't need to completely uninstall node.js
Just
sudo rm -rf /usr/local/lib/node_modules/npm/
Then
install node.js
Then
reinstall karma
This worked for me.
I had to add export PATH="$PATH":/usr/local/lib/node_modules/npm/node_modules/karma/bin after installing karma with sudo npm install karma.
hope this helps.
Just go to test.sh:
Find: $BASE_DIR/../node_modules/karma/bin/karma start $BASE_DIR/../config/karma.conf.js $*
Replace with: /usr/local/bin/karma start $BASE_DIR/../config/karma.conf.js $*
Or: karma start $BASE_DIR/../config/karma.conf.js $*
I was also facing the same issue. It looks like karma for command line is a separate package which can be installed by
npm install -g karma-cli
When upgrading from Karma 0.10 to 0.12 the link to the karma executable is removed.
You can get it back with
cd node_modules/.bin
ln -s ../karma/bin/karma karma
Try re-installing node.js. There are lots of ways to install it, but I recommend downloading from nodejs.org
If that doesn't work, you may try to re-install karma.