After a fair bit of thrashing, I successfully installed the Python Camelot PDF table extraction tool (https://pypi.org/project/camelot-py/) and it works for the intended purpose. But in order to get it to work, aside from having to correct a deprecated dependency (by editing pyproject.toml and setting PyPDF2 =”2.12.1”) I used pip to install Camelot from within a Poetry (my preferred package manager) environment- because I haven’t yet figured out any other way.
Since I’m very new to Python and package management (but not to programming) I have some holes in my basic understanding that I need to patch up. I thought that using two package managers on the same project in principle defeats the purpose of using package managers, so I feel like I’m lucky that it works. Would love some input on what I’m missing.
The documentation for Camelot provides instructions for installing via pip and conda (https://camelot-py.readthedocs.io/en/master/user/install-deps.html), but not Poetry. As I understand (or misunderstand) it, packages are added to Poetry environments via the pyproject.toml file and then calling "poetry install."
I updated pyrpoject.toml as follows, having identified the current Camelot version as 0.10.1 (camelot --version):
[tool.poetry.dependencies]
python = "^3.8"
PyPDF2 = "2.12.1"
camelot = "^0.9.0"
This led to the error:
Because camelot3 depends on camelot (^0.9.0) which doesn't match any versions, version solving failed.
Same problem if I set (camelot = "0.10.1"). So I took the Camelot reference out of pyproject.toml, and ran the following command from within my Poetry virtual environment:
pip install “camelot-py[base]”
I was able to successfully proceed from here, but that doesn’t feel right. Is it wrong to try to force this project into Poetry, and should I instead consider using different package managers for different projects? Am I misunderstanding how Poetry works? What else am I missing here?
Whenever you see pip install 'Something[extra]' you can replace it with poetry add 'Something[extra]'.
Alternatively you can write it directly in the pyproject.toml and then run poetry install instead:
[tool.poetry.dependencies]
# ...
Something = { extras = ["extra"] }
Note that in your question you wrote camelot in the pyproject.toml but it is camelot-py that you should have written.
Related
Just tried to rebuild a container with DRF and drf-yasg. The exact same commit was passing all tests fine but is now falling over with the following exception:
ImportError: Could not import 'rest_framework.schemas.coreapi.AutoSchema' for API setting 'DEFAULT_SCHEMA_CLASS'. ModuleNotFoundError: No module named 'rest_framework.schemas.coreapi'.
Nothing else has changed, but it seems a newer package may have been included that broke the Swagger generator.
Anyone else experience similar?
So it seems pip was pulling DRF V3.10, which has some switch from CoreAPI to OpenAPI: https://www.django-rest-framework.org/community/3.10-announcement/ . Adding the line from the release documentation:
REST_FRAMEWORK = {
...
'DEFAULT_SCHEMA_CLASS': 'rest_framework.schemas.coreapi.AutoSchema'
}
did not seem to make any difference.
I would presume your dependencies in requirements.txt are not specific enough, and rebuilding the container has installed a later version of djangorestframework.
Check for a line in your pipfile like djangorestframework>=3.9, and this should be changed to either pin a specific version djangorestframework==3.9, or pin it to a specific minor release so you will still receive bug fixes and security updates djangorestframework>=3.9,<3.10.
These lines can also be used directly with pip, incase your container build uses pip directly, e.g. pip install "djangorestframework>=3.9,<3.10"
It seems that installing coreapi seperately may help: pip install coreapi
pip3 install packaging
solve it!
I'm having issues with pip failing to install editable packages from a local directory. I was able to install the packages manually using commands like pip install -e pkg1. I wanted to use a requirements.txt file to automate future installs, because my coworkers will be working on the same packages. My ideal development workflow is for each developer to checkout the source from version control and run pip install -r requirements.txt. The requirements file would designate all the packages as editable so we can import our code without the need for .pth files but we wouldn't have to keep updating our environments. And by using namespace packages, we can decouple the import semantics from the file structures.
But it's not working out.
I have a directory with packages like so:
index/
pkg1/
src/
pkg1/
__init__.py
pkg1.py
setup.py
pkg2/
src/
...etc.
Each setup.py file contains something like:
from setuptools import setup, find_packages
setup(
name="pkg1",
version="0.1",
packages=find_packages('src'),
package_dir={'':'src'},
)
I generated my requirements.txt file using pip freeze, which yielded something like this:
# Editable install with no version control (pkg1==0.1)
-e c:\source\pkg1
# Editable install with no version control (pkg2==0.1)
-e c:\source\pkg2
...etc...
I was surprised when pip choked on the requirements file that it created for itself:
(venv) C:\Source>pip install -r requirements.txt
c:sourcepkg1 should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
Also, some of our packages rely on other of our packages and pip has been absolutely useless at identifying these dependencies. I have resorted to manually installing packages in dependency order.
Maybe I'm pushing pip to its limits here. The documentation and help online has not been helpful, so far. Most sources discuss editable installation, installation from requirements files, package dependencies, or namespace packages, but never all these concepts at once. Usually when the online help is scarce, it means that I'm trying to use a tool for something it wasn't intended to do or I've discovered a bug.
Is this development process viable? Do I need to make a private package index or something?
I have used a YAML file and have imported PyYAML into my project.
The code works fine in PyCharm, however on creation of an egg and running the egg gives an error as module not found on command prompt.
You have not provided quite enough information for an exact answer, but, for missing python modules, simply run
py -m pip install PyYaml
or, in some cases
python pip install PyYaml
You may have imported it in your project (on PyCharm) but you have to make sure it is installed and imported outside of the IDE, and on your system, where the python interpreter runs it
I have not made an .egg for some time (you really should be consider using wheels for distributing packages), but IIRC an .egg should have a requires.txt file with an entry that specifies dependency on pyyaml.
You normally get that when setup() in your setup.py has an argument install_requires:
setup(
...
install_requires=['pyyaml<4']
...
)
(PyYAML 4.1 was retracted because there were problems with that version, but it might be in your local cache of PyPI as it was in my case, hence the <4, which restricts installation to the latest 3.x release)
How would I install things on Pepper, since I don't know what package manager it uses. I usually use apt on my Ubuntu machine and want to install some packages on Pepper. I'm not sure what package manager Pepper has (if any) and want to install some packages, but also only know the name of the package using apt (not sure if the package name is the same on other package managers). And if possible, would I be able to install apt on Pepper. Thanks.
Note: From the research I've done, Pepper is using NaoQi which is based off Gentoo which uses portage.
You don't have root access on Pepper, which limits what you can install (and apt isn't on the robot anyway).
Some possibilities:
Include your content in Choregraphe projects - when you install a package, the whole directory structure is installed (more exactly, what's listed in the .pml); so you can put arbitrary files on your robot, and you can usually include whatever dependencies your code needs.
Install python packages with pip.
In NAOqi 2.5, a slightly older version of pip is installed that will not always work out of the box; I recommend upgrading it:
pip install --user --upgrade pip
... you can then use the upgraded pip to install other packages, using the upgraded pip, and always --user:
/home/nao/.local/bin/pip install --user whatever-package-you-need
Note however that if you do this and use your packages in your code running on Pepper, that code won't work on other robots until you do pip on them, which is why I usually only do this for tests; for production code I prefer packaging all dependencies in my app's package.
As a workaround if you need to install software (or just newer versions of software) using Gentoo Prefix is an option.
Gentoo Prefix builds a Gentoo OS on any location (no need of root, can be any folder). It includes it's own portage (package manager) to install new software.
I maintain a few projects to work with Pepper and use "any" software I want. Note that they are built for 64b (amd64) and 32b (x86) even though for Pepper only the 32b matter.
gentoo_prefix_ci and gentoo_prefix_ci_32b Which builds nightly the bootstrap of the Gentoo Prefix system. This is a process that takes a while to compile (3-6h depending on your machine) and that breaks from time to time (as upstream packages are updated and bugs are found, Gentoo is a rolling release distribution). Every night updated binary images ready to use can be found in the Releases section.
For ROS users that want to run it on the robot, based on the previous work, I maintain also ros_overlay_on_gentoo_prefix and ros_overlay_on_gentoo_prefix_32b. They provide nightly builds with binary releases of ROS Kinetic and ROS Melodic over Gentoo Prefix using ros-overlay. You can find ready-to-use 'ros_base' and 'desktop' releases.
For purposes related to the RoboCup#Home Social Standard Platform League where the Pepper robot is used I also maintain a specific build that contains a lot of additional software. This project is called pepper_os and it builds 270+ ROS packages, a lot of Python packages (250+ including Theano, dlib, Tensorflow, numpy...) and all the necessary dependencies for these to build (750+ packages). Note that the base image (it's built with Docker) is the actual Pepper 2.5.5.5 image, so it can be used for debugging as it if it was in the real robot (although without sensors and such).
Maybe this approach, or these projects, are useful.
The package manager on pepper is disabled. But you can copy the files to the robot and write your own service that imports any package you might need.
As a supplement on importing:
http://www.about-robots.com/how-to-import-python-files-in-your-pepper-apps.html
To get rid of error :
" SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
".
If you use python and requests package, just add verify=False at the end of your parameters.
r=requests.get(URL,params,header,verify=False)
Works with my Pepper
To get rid of
InsecurePlatformWarning: A true SSLContext object is not available.
install
/home/nao/.local/bin/pip install --user requests[security]
To get rid of:
CryptographyDeprecationWarning: Support for your Python version is deprecated.
install
/home/nao/.local/bin/pip install --user cryptography==2.2.2
If it based on Gentoo maybe we could try to install portage with pip.
pip install portage
Just a thought.
Jython Package installation issue, using pip
Hi, I have installed Jython2.7 configured with pydev in eclipse neon, also configured python 3.6 package
I am able to install packages for python using pip installer?
pip install "packagename"
Below are some of the packages in python/Lib/Site-packages directory
I was able to install all the packages
How do I use pip installer to install packages for jython?
I tried to install Jip package with
jython install setup.py
The binary File got installed in the Jython/Lib/Site-packages folder
However, I am not able to use it.
where and how do I get Jython package binaries like jip?
Also, Please let me know how to search jython packages?
Also, How to make pip install library packages in jython?
Any other configuration like jython home, etc that should be made?
This answer is going to be really generic but I just recently have slogged my way through the setup for jython/jip/pip and here's roughly what I had to do.
Firstly, I'm running Windows 7 64 Bit from behind a proxy (work machine.)
Had to install jython 2.7.0 instead of 2.7.1 because (I think anyway) 2.7.1 requires admin privileges which I don't have on my work PC.
Pip didn't install correctly during the Jython installation and I spent an obscene amount of time trying to get it installed and functioning as I knew it from my cpython days. NOTE: Just because you get pip installed, doesn't mean you can use any package on a python package repo. As of 2.7.0, Jython doesn't have end to end capability to interpret/compile some libraries that rely on certain python wrappers of native OS function calls. I believe 2.7.1 makes solid progress in the direction of supporting all needed native calls but don't quote me on that. For example, I tried to use wxPython to make a simple GUI to test my jython install. Trying to install it from pip kept causing really non-specific error info that took me a lot of time to figure out that the cause was jython simply couldn't compile the wxPython source so beware.
I had to set environment variables 'http_proxy' and 'https_proxy' in the form of http://proxyhosturl:port and https://proxyhosturl:port respectively to get out from behind the proxies without having to invoke pip with the proxy switch every time I called it.
To actually install pip, have a look here. These instructions are for Python and Linux/Unix but the principle is roughly the same. Just use jython -m instead of python -m and ignore the '$' at the start of each command line.
Also be sure to CD to your python_home/bin folder when invoking the ez_install exe.
If that doesn't work (didn't for me), try using get-pip.py script with these instructions https://pip.pypa.io/en/stable/installing/ (remember jython instead of python etc.). Download it, cd to the download location and follow the noted install steps. Worth noting is about half way down the install instructions where it details installing from local archives (source/binary zip or tar.gz archives of pip and setuptools as better described here: https://packaging.python.org/tutorials/installing-packages/#installing-from-local-archives).
The links to the bin archives of pip and setuptools are here:
https://pypi.python.org/pypi/setuptools
https://pypi.python.org/pypi/pip
It may also be worth making sure that your PATH environment variable has the jython/bin path in the variable value. The jython installer should do this but, again, mine did not.
If all goes well, you should be able to invoke pip with the --version switch and if it prints a line with the installed pip version info then you should be good to go
Another quirky issue I had was I could invoke a function of pip one time and any subsequent times I would get a stack trace ending with something along the lines of an object not having a certain property. I fixed that by finding my temp directory by opening a windows explorer instance and typing %TEMP% in the address bar and hitting enter, it should take you to a subdir of your AppData folder and there you may see a folder with the name of the package you were trying to install and the text "_pip" somewhere in the directory name. Delete the directory and try the pip install command again. I had to do this + invoke pip install pip -U to update my install to the latest version. Then pip began behaving correctly in my instance.
pip search numpy (or your library name) will generate a list of results with the same logic it uses to locate your desired package when you call pip install but, again, just because it returns a matching package doesn't mean it will compile when you install it (numpy doesn't work because of the missing java to C native function calls I described earlier.) The trade off is that you can import code artifacts from Java JAR files in your Jython script files and leverage their functionality with relative ease. Between the public Java APIs available and the python packages that work with the jython interpretor, you can (in my experience) come up with a way to accomplish your task. See the following info on JIP, Maven and IDEs.
IDE and jython integration (Eclipse)
- If you are stuck using Eclipse (like me) it actually has pretty decent support for python development. Install the PyDev plugin for Eclipse from Help -> Install Software. Put in this URL https://marketplace.eclipse.org/content/pydev-python-ide-eclipse, hit tab, and select the PyDev plugin and hit 'finish.'
- Setup the jython interpretor info from Windows -> Preferences -> PyDev. Provide the path to your jython.jar file.
- You should now be able to use File -> New PyDev project to create a basic python project and configure it to use your version of Jython and Java.
Brief Overview of Jip and Maven
- jip is a jython package that is invoked very similarly to pip but instead will download JAR files from the Maven Central Repository instead of python packages from pypi.com, for example. See the install instructions described here. Note the install procedure for a global jip install which differ from just pip install jip. https://pypi.python.org/pypi/jip/
- I never got jip to work exactly as I wished because there's not a ton of documentation on it outside of what I already linked. However, if you install a JAR using jip, you have to go to your project in Eclipse and actually add the JARs themselves to your PYTHONPATH in order for import statements and editing to have intellisense and so that you don't get a classnotfound exception at runtime. See following screen shot.
- There is a JIP config file that you can use similar to the pip config ini file but I have yet to find any exhaustive documentation on it's setup.
Note in the above screen shot the first entry in the External libraries entries. By default, pip places installed packages in that directory so to enable eclipse to find them, you need to also ensure that location is entered.
In Conclusion
- I have more to add to this answer and I will do so as soon as possible. In the meantime, see this example project I've loaded into github.
https://github.com/jheidlage1222/jython_java_integration_example
It shows basic config and how to interface with JARs from python code. I used the apache httpcomponents library as an example. Good luck amigo.