I am trying to update a pre-existing project with another dependency by using the poetry add command but when I do, it erases all of the data in the [metadata.files] block.
I have looked through the documentation but can't find anyway of keeping/adding this information back into the poetry.lock file.
Any help would be appreciated!
You have to upgrade poetry to v1.1.14 at least.
It was breaking change on PyPI API.
Hashes could not be retrieved when locking.
Related
After a fair bit of thrashing, I successfully installed the Python Camelot PDF table extraction tool (https://pypi.org/project/camelot-py/) and it works for the intended purpose. But in order to get it to work, aside from having to correct a deprecated dependency (by editing pyproject.toml and setting PyPDF2 =”2.12.1”) I used pip to install Camelot from within a Poetry (my preferred package manager) environment- because I haven’t yet figured out any other way.
Since I’m very new to Python and package management (but not to programming) I have some holes in my basic understanding that I need to patch up. I thought that using two package managers on the same project in principle defeats the purpose of using package managers, so I feel like I’m lucky that it works. Would love some input on what I’m missing.
The documentation for Camelot provides instructions for installing via pip and conda (https://camelot-py.readthedocs.io/en/master/user/install-deps.html), but not Poetry. As I understand (or misunderstand) it, packages are added to Poetry environments via the pyproject.toml file and then calling "poetry install."
I updated pyrpoject.toml as follows, having identified the current Camelot version as 0.10.1 (camelot --version):
[tool.poetry.dependencies]
python = "^3.8"
PyPDF2 = "2.12.1"
camelot = "^0.9.0"
This led to the error:
Because camelot3 depends on camelot (^0.9.0) which doesn't match any versions, version solving failed.
Same problem if I set (camelot = "0.10.1"). So I took the Camelot reference out of pyproject.toml, and ran the following command from within my Poetry virtual environment:
pip install “camelot-py[base]”
I was able to successfully proceed from here, but that doesn’t feel right. Is it wrong to try to force this project into Poetry, and should I instead consider using different package managers for different projects? Am I misunderstanding how Poetry works? What else am I missing here?
Whenever you see pip install 'Something[extra]' you can replace it with poetry add 'Something[extra]'.
Alternatively you can write it directly in the pyproject.toml and then run poetry install instead:
[tool.poetry.dependencies]
# ...
Something = { extras = ["extra"] }
Note that in your question you wrote camelot in the pyproject.toml but it is camelot-py that you should have written.
I'm running a project that uses pip and a requirements.txt file to install and keep track of some dependencies. I want to write some sh scripts to run, build and test the application. For starters I would like a way to check if the current folder is in a pyenv and, if not, create one to enclose the application and not mess around other people's dependencies. Also, I would like an opinion of the best way to keep track of this kind of dependencies, if the requirements.txt is a good approach and if there's a way to keep track of installed versions just like happens with node packages.
Use Pipenv. It's a better way of tracking your depencies than requirements.txt and it uses Pyenv to automatically install your project's required Python version.
From the website:
The problems that Pipenv seeks to solve are multi-faceted:
You no longer need to use pip and virtualenv separately. They work together.
Managing a requirements.txt file can be problematic, so Pipenv uses Pipfile and Pipfile.lock to separate abstract dependency
declarations from the last tested combination.
Hashes are used everywhere, always. Security. Automatically expose security vulnerabilities.
Strongly encourage the use of the latest versions of dependencies to minimize security risks arising from outdated components.
Give you insight into your dependency graph (e.g. $ pipenv graph).
Streamline development workflow by loading .env files.
[...]
Pipenv Features
Enables truly deterministic builds, while easily specifying only what you want.
Generates and checks file hashes for locked dependencies.
Automatically install required Pythons, if pyenv is available.
Automatically finds your project home, recursively, by looking for a Pipfile.
Automatically generates a Pipfile, if one doesn’t exist.
Automatically creates a virtualenv in a standard location.
Automatically adds/removes packages to a Pipfile when they are installed or uninstalled.
Automatically loads .env files, if they exist.
Just tried to rebuild a container with DRF and drf-yasg. The exact same commit was passing all tests fine but is now falling over with the following exception:
ImportError: Could not import 'rest_framework.schemas.coreapi.AutoSchema' for API setting 'DEFAULT_SCHEMA_CLASS'. ModuleNotFoundError: No module named 'rest_framework.schemas.coreapi'.
Nothing else has changed, but it seems a newer package may have been included that broke the Swagger generator.
Anyone else experience similar?
So it seems pip was pulling DRF V3.10, which has some switch from CoreAPI to OpenAPI: https://www.django-rest-framework.org/community/3.10-announcement/ . Adding the line from the release documentation:
REST_FRAMEWORK = {
...
'DEFAULT_SCHEMA_CLASS': 'rest_framework.schemas.coreapi.AutoSchema'
}
did not seem to make any difference.
I would presume your dependencies in requirements.txt are not specific enough, and rebuilding the container has installed a later version of djangorestframework.
Check for a line in your pipfile like djangorestframework>=3.9, and this should be changed to either pin a specific version djangorestframework==3.9, or pin it to a specific minor release so you will still receive bug fixes and security updates djangorestframework>=3.9,<3.10.
These lines can also be used directly with pip, incase your container build uses pip directly, e.g. pip install "djangorestframework>=3.9,<3.10"
It seems that installing coreapi seperately may help: pip install coreapi
pip3 install packaging
solve it!
I know that we can always install a package via command:
composer require packageA
But I don't know if you guys ever have a situation like this:
You want to install a big size package "packageB" that your teammate added to composer.json and your wifi is slow so composer would take very very long to get the packageB. Then you have an idea:
"Maybe I try get the packageB zip from my teammate via flash drive and paste
it into my project."
And you did that, the package works as expected. Wonderful!
But then, you think again:
What if now I want to do the composer update other packages in my
project?
You try:
composer update
and then, what happen is composer will get the package again because you didn't use "composer install" or "composer update" to install packageB so composer doesn't know you have it.
(Sorry for the long explanation).
So my question is:
How do we let composer know that we have the package already so composer don't re-download the package again? Or this is the behavior of composer and I must always use "composer install/update", there is no other way?
And sorry, change to another wifi or find a faster internet connection is really not what I'm looking for. And I also know that we can install the package locally (see here: How to update a single composer package?).
Thanks in advance!
If we don't want to use repositories.
In my knowledge, the only option is to update you composer.json and composer.lock. Friend give you version 1.2 to vendor? Write in exactly version in composer.json and for composer.lock, you will need data from your friend too.
Run install then.
Should check, but not download any file. Still, problem is that all required libraries by this library, could be updated - you can only write down exactly version of them in file.
As default, I think, the didn't predict scenarios for that way.
This is the only solution for you, i know should work.
Composer does use caching heavily to reduce the amount of data to download. However this does not remove the need to download the package at least once.
Basically Composer has two modes to download: --prefer-dist will try to obtain a download URL for an archive file, and --prefer-source will try to obtain a copy of the version control system being used.
Both variants put the result into Composer's cache directory.
Over time you'll collect a couple of archive files locally, which allow for quick switches back and forth between existing version downloads, and newer versions will have to be downloaded once.
Also you can clone a git repository once, and Composer will try to reuse it when updating, by simply fetching new commits and checking out the appropriate tags. This still requires to clone the repository once.
You can work around cloning the repository by manually placing it at the correct spot, either by physically putting it there, or by symlinking the correct vendor directory. You can also make Composer aware of an official copy by adding the local copy as an entry to repositories. This will add this source to the existing collection of packages available from Packagist.
I'm trying to make composer update only newly added packages to composer.json i.e when I manually add a package dependency to the composer.json file, it should update the composer.lock file only for the new package; the rest of the packages should be at the same version as before. I tried running composer update --lock but I don't think it does what I'm trying to achieve and it took a lot of time to finish. I checked the commands on composer's documentation but can't find one to achieve my wish. Any advice or workaround will be appreciated.
Note: I'm using Laravel Forge, so there is a 2 minutes deployment limit.
In order to install only new packages with composer you should run
composer install
Because composer update will install your new packages but will update and all the other already installed packages.
You can specify the name of the package as an argument to the update command. This will perform a partial update: composer update the-package/you-want-to-update
I think your question is related to your (guessed) current workflow: To add a new package you edit the composer.json file and then run composer update - wishing to only add/update that new file.
If that is true, here is the solution:
composer require new/package will add the newest possible version (taking into account the currently installed packages) of the new package. Benefits: Only one command line, and no fiddling with JSON content.
If you already know which version you want, you could also run composer require new/package:^2.1.25#beta (or whatever version and stability level you want - this example is exaggerating a bit). If this version is incompatible with existing packages, nothing will get installed, everything will get rolled back, and you get an error message.