My situation I have a specific library in codeartifact.
I have a project (not a library) managing dependencies with poetry consuming this library.
Now I need to install this library in the project but can't find instructions which
Don't change the actual pypi index url for all projects in my computer
Being able to install the specific library from codeartifact without it being proxy for pypi for ALL libraries
Any ideas or clues on how to achieve this?
In order to do that you need to add a source to your pyproject.toml and mark it as secondary:
[[tool.poetry.source]]
name = "my-repo"
url = "your-pypi-url"
secondary = true
And then point your library to this source.
With CLI:
poetry add my-library --source my-repo
Or directly in pyproject.toml
[tool.poetry.dependencies]
…
my-library = {version = "1.0.0", source = "my-repo"}
Related
After a fair bit of thrashing, I successfully installed the Python Camelot PDF table extraction tool (https://pypi.org/project/camelot-py/) and it works for the intended purpose. But in order to get it to work, aside from having to correct a deprecated dependency (by editing pyproject.toml and setting PyPDF2 =”2.12.1”) I used pip to install Camelot from within a Poetry (my preferred package manager) environment- because I haven’t yet figured out any other way.
Since I’m very new to Python and package management (but not to programming) I have some holes in my basic understanding that I need to patch up. I thought that using two package managers on the same project in principle defeats the purpose of using package managers, so I feel like I’m lucky that it works. Would love some input on what I’m missing.
The documentation for Camelot provides instructions for installing via pip and conda (https://camelot-py.readthedocs.io/en/master/user/install-deps.html), but not Poetry. As I understand (or misunderstand) it, packages are added to Poetry environments via the pyproject.toml file and then calling "poetry install."
I updated pyrpoject.toml as follows, having identified the current Camelot version as 0.10.1 (camelot --version):
[tool.poetry.dependencies]
python = "^3.8"
PyPDF2 = "2.12.1"
camelot = "^0.9.0"
This led to the error:
Because camelot3 depends on camelot (^0.9.0) which doesn't match any versions, version solving failed.
Same problem if I set (camelot = "0.10.1"). So I took the Camelot reference out of pyproject.toml, and ran the following command from within my Poetry virtual environment:
pip install “camelot-py[base]”
I was able to successfully proceed from here, but that doesn’t feel right. Is it wrong to try to force this project into Poetry, and should I instead consider using different package managers for different projects? Am I misunderstanding how Poetry works? What else am I missing here?
Whenever you see pip install 'Something[extra]' you can replace it with poetry add 'Something[extra]'.
Alternatively you can write it directly in the pyproject.toml and then run poetry install instead:
[tool.poetry.dependencies]
# ...
Something = { extras = ["extra"] }
Note that in your question you wrote camelot in the pyproject.toml but it is camelot-py that you should have written.
I'm trying to solve what I assume is a common problem with poetry but am unable to find the relevant documentation. My project includes multiple packages and uses pyproject.toml and poetry to manage dependencies with this structure
/pyproject.toml
/poetry.lock
/package1/pyproject.toml
/package1/poetry.lock
/package1/src/package1/...
/package1/pyproject.toml includes pypi dependencies in [tool.poetry.dependencies] and defines the buildable package as
packages = [
{ include = "package1", from = "./src" },
]
/pyproject.toml references package1 with
[tool.poetry.dependencies]
package1 = { path = "./package1", develop = true }
Finally, my Dockerfile installs the application with
WORKDIR /app/package1
RUN poetry install
WORKDIR /app
RUN poetry install
The problem is that Poetry installs each "project" (identified by the pyproject.toml file) in a separate virtual environment, and doesn't seem to support installing both in the same environment. When I execute the application it can find package1 but none of package1's dependencies.
How can I get everything installed in the same environment?
How am I supposed to handle this situation?
So I have a Yocto build and I need to install this 3rd party RPM. I've created a recipe for it using the link to the source RPM. Problem is when implementing the do_install() function.
I need to install it, and it's installed via rpm --install rpm_package, and then I need to enable the service.
For the service I know I have to inherit systemd in my recipe file, but for the installation I'm still confused.
The install command only creates directories and copies files over.
Any help is appreciated.
Indeed do_install only create install directories in your /tmp/work/cortex[...]/my_recipe_name/ directory
Normally if you bitbake your image which include your recipe you should have a warning like:
Files/directories were installed but not shipped in any package
So, after the installation you need to "package them" in order to be in your final linux image, to do so use FILES_$PN like below for instance:
FILES_${PN}_append = " /usr/sbin/"
where /usr/bin is the directory which contain what you want to "install" in your image.
Then the package will be shipped in your final image.
For the service, indeed inherit systemd is mandatory + you have to add in do_install
install -Dm0644 ${WORKDIR}/my_script.service ${D}${systemd_system_unitdir}/my_script.service
and at the end of your .bbfile
SYSTEMD_SERVICE_${PN} = "my_script.service"
You can install packages at runtime with rpm, but it is not recommended for production builds, as that is the whole idea behind Yocto which is creating a custom distribution with all your needs integrated.
In order to use rpm command in runtime you need to configure it to fetch the sources from the right location. And the right location is generally a Yocto build, because it is compatible with your board's architecture and system (Otherwise, you need to handle that manually).
You can link the Yocto build rpm files to the board's rpm command using smart, for more info check first the official Yocto documentation here.
Also, you can check more example here and here.
IMPORTANT
I do not recommend creating a systemd/sysvinit service to install an rpm/deb/ipk/tar package.
The idea is, if you install a package with rpm it means that it is already supported by Yocto and has a recipe. So, simply:
IMAGE_INSTALL_append = " package"
I am working on two projects, let's call them Project A and Project B.
Project B requires some modules in Project A, so I did a poetry build on Project A. I am able to access the module when I manually perform a pip install dist/blabla.whl on the build produced by poetry on Project A.
But when I do a poetry add project-a git+ssh://git#gitlab.blabla.co/nubela/project-a.git#develop, it says
Could not find a matching version of package project-a
Naturally, I understand because project-a is not classically packaged with setup.py and stuff. How can I then perform a poetry add <git-repo-uri> without involving self-hosted pypi instance?
I can push the .whl files to the project git repo, does that help?
The correct syntax would be
poetry add git+ssh://git#gitlab.blabla.co/nubela/project-a.git#develop
More examples are in the docs
I can push the .whl files to the project git repo, does that help?
While it would theoretically solve the issue, treating a repository as a file server sets you up for headaches - if your gitlab even allows uploading binaries in the first place.
Given the error message you're getting, it doesn't seem to be a poetry or setuptools issue, it is git that is complaining. Did you make sure that you have a tag or branch on your repo that is called develop? While it is not explicitly mentioned, the form of poetry add in the git-mode is
poetry add git+<protocol>://git#<repository>/<owner>/<project>.git#<git_ref>
The final #<git-ref> is optional, and if omitted poetry will install whatever is currently on the default branch (most likely "master"). What you probably want is just
poetry add git+ssh://git#gitlab.blabla.co/nubela/project-a.git
I'm having issues with pip failing to install editable packages from a local directory. I was able to install the packages manually using commands like pip install -e pkg1. I wanted to use a requirements.txt file to automate future installs, because my coworkers will be working on the same packages. My ideal development workflow is for each developer to checkout the source from version control and run pip install -r requirements.txt. The requirements file would designate all the packages as editable so we can import our code without the need for .pth files but we wouldn't have to keep updating our environments. And by using namespace packages, we can decouple the import semantics from the file structures.
But it's not working out.
I have a directory with packages like so:
index/
pkg1/
src/
pkg1/
__init__.py
pkg1.py
setup.py
pkg2/
src/
...etc.
Each setup.py file contains something like:
from setuptools import setup, find_packages
setup(
name="pkg1",
version="0.1",
packages=find_packages('src'),
package_dir={'':'src'},
)
I generated my requirements.txt file using pip freeze, which yielded something like this:
# Editable install with no version control (pkg1==0.1)
-e c:\source\pkg1
# Editable install with no version control (pkg2==0.1)
-e c:\source\pkg2
...etc...
I was surprised when pip choked on the requirements file that it created for itself:
(venv) C:\Source>pip install -r requirements.txt
c:sourcepkg1 should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
Also, some of our packages rely on other of our packages and pip has been absolutely useless at identifying these dependencies. I have resorted to manually installing packages in dependency order.
Maybe I'm pushing pip to its limits here. The documentation and help online has not been helpful, so far. Most sources discuss editable installation, installation from requirements files, package dependencies, or namespace packages, but never all these concepts at once. Usually when the online help is scarce, it means that I'm trying to use a tool for something it wasn't intended to do or I've discovered a bug.
Is this development process viable? Do I need to make a private package index or something?