When I install a package (say transformers) pip creates a directory named true
Original directory tree:
└─┬ .
├── .env/
└── .vscode/
after pip install transformers
└─┬ .
├── .env/
├── true/
└── .vscode/
Do not change global.cache-dir option. It takes paths only and does not take booleans. (so if you do not want to have a pip cache, do not set it to false using pip config set global.cache-dir false) as false or any other value is treated as a folder path and it defaults to being relative.
Related
I am trying to come up with an organization scheme for a package and an a series of contributed plugins that can also work standalone - i.e., they have their own setup.py. Those contributed packages are all hosted in a single repo, which can be used to install them all at once. The plugins are discovered as native namespace packages (as described here), and I manually point to the folders in setup.py as so:
package_dir={
"contrib_package_1": "my_package/contrib/contrib_package_1",
"my_package.contrib.contrib_package_1": "my_package/contrib/contrib_package_1",
}
I am having one major and one minor issue with this setup (and maybe a few more I haven't discovered yet).
Major issue: I can't discover the contrib packages when installing them in editable mode: pip install -e contrib-packages\contrib_package_1 works fine, the package is in pip freeze or conda list, and the correct path is specified in the egg-link file - but I simply cannot import it (ModuleNotFoundError). I guess this is somehow related to the nested structure, because when I pull out contrib_package_1 to the root level and install it in editable mode, it is discovered.
Minor issue: When installing all packages in regular mode they are discovered fine and harmonize as expected, but the path for my_package.contrib is empty until I do from my_package.contrib import contrib_package_1. Upon doing import my_package I would like users to see which plugins are installed, so, somehow the path for my_package.contrib would need to be updated upon loading the main package (I added from .config import * to the main packages __init__.py, but that didn't do the trick).
Similar issues have been described here: How to install multiple python namespace packages in editable mode
main-package # main repo
├-- setup.py
├── my_package
│ ├── __init__.py <- # contains `from .config import *`
│ ├── code.py
│ ├── main
│ │ ├── __init__.py
│ │ └── more_code.py
│ ├── contrib <- # empty
contrib-packages # contrib repo
├-- setup.py <- # install all contrib packages
├── contrib_package_1
│ ├── setup.py
│ └── my_package
│ └── contrib
│ └── contrib_package_1
│ ├── __init__.py
│ ├── code.py
│ └── _version.py
├── contrib_package_2
│ ├── setup.py
│ └── my_package
│ └── contrib
│ └── contrib_package_2
│ ├── __init__.py
│ ├── code.py
│ └── _version.py
...
I am trying to install some software (pulp) with Ansible. The actual installer comes in an Ansible collection, which in turn depends on a downloaded role (geerlingguy.postgresql).
The role is installed, and I can access it from one of my own playbooks. But when the tasks in the pulp.pulp_installer try to access the geerlingguy.postgresql role, they can't find it.
I followed the instructions on the Pulp Web site, which says to install the geerlingguy.postgresql role just like any other role with
ansible-galaxy install geerlingguy.postgresql
Specifically, when I try to run the installer, I get this error message:
ERROR! the role 'geerlingguy.postgresql' was not found in pulp.pulp_installer:ansible.builtin
The error appears to be in '/home/kkeane/.ansible/collections/ansible_collections/pulp/pulp_installer/roles/pulp_database/tasks/install_postgres.yml': line 130, column 15, but may
be elsewhere in the file depending on the exact syntax problem.
I installed the geerlingguy.postgresql role into the standard ~/.ansible location:
tree -L 3 ~/.ansible/
/home/kkeane/.ansible/
├── collections
│ └── ansible_collections
│ ├── ansible
│ ├── community
│ └── pulp
├── cp
├── galaxy_token
├── roles
│ └── geerlingguy.postgresql
│ ├── LICENSE
│ ├── README.md
│ ├── defaults
│ ├── handlers
│ ├── meta
│ ├── molecule
│ ├── tasks
│ ├── templates
│ └── vars
└── tmp
I can also access the role directly from one of my own playbooks.
The error message suggests that Ansible is only looking inside the pulp.pulp_installer and ansible.builtin collections, instead of using my full role path.
Do collections in fact use a different role path? If so, how do I get it to find the geerlingguy.postgresql role?
It appears that this was due to a bug in the specific version of Ansible that I used (2.9.6). Based on comparing the source code, version 2.9.19 seems to have this issue fixed. Unfortunately, on my version of Ubuntu (20.04), the latest version of Ansible is 2.9.6, and even the Ansible ppa does not have a newer version of the 2.9 series.
So I had to update to Ansible 2.10. That process is as follows. Note that you must use the package name ansible-base; if you install the package ansible, you will just reinstall the broken version.
Keep in mind that upgrading to 2.10 may have other side effects; there were substantial changes to Ansible between 2.9 and 2.10.
apt remove --purge --auto-remove ansible
add-apt-repository ppa:ansible/ansible
apt-get update
apt install ansible-base
Edit: there may be a possibility to install 2.9.19 on Ubuntu, after all, using pip instead of apt.
Copy files to pip's build directory
(Nasty) use-case: I have a 3rd party subproject that I have to endow with setup.py, so that it would run make producing certain artifacts that I want to list in scripts=. I've derived a class from setuptools.install.install that chdir()s into a folder adjacent to __file__ and makes.
However, when run with pip install, pip would only copy what seems to be source files to the build directory.
I've tried specifying data_files=, but that didn't help.
How can I hint pip to copy specific files to the build dir?
P.S. I know it may look like a wrong problem to solve, but I do specifically want to copy those files to build dir, to make and to add artifacts to the wheel.
UPD: in reply to the comment on setup.py MVE.
Example project structure:
├── libs
│ └── external
│ ├── Makefile
│ └── src
│ └── _...
├── setup.py
└── src
└── __init__.py
Example setup.py:
from setuptools import setup
from setuptools.command.install import install
from subprocess import Popen
import os
import glob
EXT_DIR = os.path.join(os.path.dirname(__file__), 'libs', 'external')
class RunMake(install):
"""Makefile on setuptools install."""
def run(self):
old_dir = os.getcwd()
try:
os.chdir(EXT_DIR)
self.spawn(['make'])
finally:
os.chdir(old_dir)
install.run(self)
setup(name='example',
version='0.0.1',
packages=['example'],
package_dir={'example': 'src'},
cmdclass={'install': RunMake},
scripts=['libs/external/build/bin/SomeBinary'],
data_files=[
('libs', glob.glob('libs/**'))
]
)
UPD: MANIFEST.in
Tried out just this, and Makefile still isn't there
include libs/external/Makefile
include libs/external/*
The resulting .egg-info:
setup.py
poisson_reconstruction.egg-info/PKG-INFO
poisson_reconstruction.egg-info/SOURCES.txt
poisson_reconstruction.egg-info/dependency_links.txt
poisson_reconstruction.egg-info/top_level.txt
src/__init__.py
I am trying some "cookbook-examples" on the site 'http://tech.pro/tutorial/1190/package-managers-an-introductory-guide-for-the-uninitiated-front-end-developer#front_end_developers'.
You should not have to look there now, but I thought it could be good to share the site.
So far so good, til it comes to the global installing.
When it comes to the point trying to install something globally I get stuck.
What I did so far for testing globally installing some package:
Created test-directory grunttest
Inside that directory:
npm install -g jshint
Output I can see:
npm http GET https://registry.npmjs.org/jshint
npm http 304 https://registry.npmjs.org/jshint
...
npm http 304 https://registry.npmjs.org/string_decoder
C:\Program Files\nodejs\node_modules\npm\jshint -> C:\Program Files\nodejs\node_modules\npm\node_modules\jshinnt
jshint#2.4.4 C:\Program Files\nodejs\node_modules\npm\node_modules\jshint
├── console-browserify#0.1.6
├── exit#0.1.2
├── underscore#1.4.4
├── shelljs#0.1.4
├── minimatch#0.2.14 (sigmund#1.0.0, lru-cache#2.5.0)
├── cli#0.4.5 (glob#3.2.9)
└── htmlparser2#3.3.0 (domelementtype#1.1.1, domutils#1.1.6, domhandler#2.1.0, readable-stream#1.0.26-2)
I just realize the 304, which should be ok, due to just says the resource was not modified since last installation (few minutes before).
Checking if the jshint exists with:
`npm -global list`
Output:
npm#1.4.3 C:\Program Files\nodejs\node_modules\npm
├── abbrev#1.0.4
├── ansi#0.2.1
├─...
├──
├── graceful-fs#2.0.2
├── inherits#2.0.1
├── ini#1.1.0
├─┬ init-package-json#0.0.14
│ └── promzard#0.2.1
├─┬ jshint#2.4.4 extraneous
│ ├─┬ cli#0.4.5
│ │ └─┬ glob#3.2.9
│ │ └── inherits#2.0.1
│ ├── console-browserify#0.1.6
│ ├── exit#0.1.2
│ ├─┬ htmlparser2#3.3.0
│ │ ├── domelementtype#1.1.1
│ │ ├── domhandler#2.1.0
│ │ ├── domutils#1.1.6
│ │ └─┬ readable-stream#1.0.26-2
│ │ └─... ├── text-table#0.2.0
├── uid-number#0.0.3
└── which#1.0.5
**npm ERR! extraneous: jshint#2.4.4 C:\Program Files\nodejs\node_modules\npm\node_modules\jshint npm**
Questions:
Why do I get npm ERR! extraneous ...?
What does it mean?
How can I resolve this issue?
Information:
I am on a windows-machine Windows 7, using cygwin as shell.
trying to just the jshint (jshint someTestfile.js) of course does not work.
npm ERR! extraneous means a package is installed but is not listed in your project's package.json.
Since you're listing packages that have been installed globally, it's going to give you a lot of extraneous errors that can be simply ignored because most things installed globally will not be in your project's package.json.
1 & 2: It means you don't have the jshint listed in your project's package.json file but that it is globally installed. So it is not a big problem.
3: To avoid this extraneous error, you can run or re-run the install with the option --save . This will update automatically you package.json file :
npm install -g jshint --save
Or need to update manually your package.json file with a "dependencies": {...}
I resolved this by doing an npm update in the parent package's folder which removed some of the extraneous packages from the list and then did npm uninstall <package> for the remaining few.
Seems to have worked, as I'm getting no errors after doing this.
I solved it by combining all the answers. At first I installed the package globally.
npm install -g packagename --save
Since npm installed this packaged as well globally but did not add it to my local package.json file, I had to do something about it.
I choose, the solution to remove the local one and then install it globally.
npm uninstall packagename
npm install -g packagename
This way I have no more warnings and do not mess up the package.json file.
I my case, I saw this 'npm ERR! extraneous' message in my cygwin terminal when i did an 'npm ls'. I thought this was some sort of a globally corrupted setup after having lots of tinkering. I learn the following observations here:
'npm ls' gives different outputs depending on what is your current folder location.
'npm ls' tries to detect the presence of a 'node_modules' folder in the current folder location, and list out those contents. NOT the global ones!
Furthermore, if the current folder containing 'node_modules' also has a package.json file containing fewer modules listed here, then the error shows.
I 'rm package.json' and 'npm ls' no longer shows error message. So I say, that always check the current location for the presence of 'node_modules' folder and the package.json file because these are prioritize first in the check and if these are missing, the check continues to to the parent folder and so on, and if you have tinkered a lot of code snippets a lot, then you may have scattered around lots and lots of node_modules folder and package.json file. Nothing is really corrupted here, unlike those experiences we have when doing J2EE Java development / eclipse IDE or during the days when we have to use regedit to change settings in Windows.
In my case it was because the package name in its package.json file was not the same as the depency name listed in the package.json of the dependent module. My error, since it's a new module I created, but hard to spot, since npm won't give any clue.
This happened when using the dependencies: { "my-module": "file:local-modules/mymodule" } syntax, with a typo in the name "my-module".
This is due to the fact that your package is not in your package.json. If you add it, the problem will be solved, please look at the image below:
I wanted to install some info document of libavl. I found some weird behavior of brew command with the following brew library that I made. What I did were:
First, I compile it with "make libavl.info."
Second, I created share/info under brew KEG PREFIX.
Third, I installed some info files to that share/info.
I do not know yet how I can symbolic-link those info files in system directory such as "/usr/local/share/info", but I want to have this brew formula. To make sure before and after installing info files that I have those info files in the corresponding KEG PREFIX, I printed directory structure in a file called "log," which is also installed to the KEG PREFIX. But, those info files disappeared after the installation. I do not know what happened. I do not know where I should ask in the homebrew website Brew.sh, so I ask this question here.
THIS IS THE FORMULA FILE called "avl.rb"
require 'formula'
class Avl < Formula
homepage 'http://savannah.gnu.org/projects/avl'
url 'ftp://ftp.gnu.org/pub/gnu/avl/avl-2.0.3.tar.gz'
sha1 'ec2e025838c846fbc05fb60cf49035f2e97f37f3'
def install
system "make", "libavl.info"
info.mkpath
system "/usr/local/bin/tree " "#{prefix} > #{prefix}/log"
info.install "libavl.info"
info.install "libavl.info-1"
info.install "libavl.info-2"
info.install "libavl.info-3"
info.install "libavl.info-4"
system "/usr/local/bin/tree " "#{prefix} >> #{prefix}/log"
end
end
The "log" file content before info.install command in the brew formula:
/usr/local/Cellar/avl/2.0.3
├── log
└── share
└── info
2 directories, 1 file
This is the content of "log" file after info.install command:
/usr/local/Cellar/avl/2.0.3
├── log
└── share
└── info
├── libavl.info
├── libavl.info-1
├── libavl.info-2
├── libavl.info-3
└── libavl.info-4
2 directories, 6 files
This is the final installed KEG PREFIX directory. There is the "log" file, but no "share/info" directory in it.
/usr/local/Cellar/avl/2.0.3
├── AUTHORS
├── COPYING
├── ChangeLog
├── INSTALL_RECEIPT.json
├── NEWS
├── README
├── TODO
└── log
0 directories, 8 files
I will appreciate any answers to this mystery.
Thank you,
Homebrew does not install info files by default. You can change this by setting HOMEBREW_KEEP_INFO=1.