I am trying to install some python requirements from a local package directory containing wheel archives. I am installing the requirements inside a Docker container.
The steps I'm following are:
$ pip install wheel
# wheel runs, outputs .whl files to wheelhouse directory
$ pip wheel --wheel-dir wheelhouse -r requirements.txt
Then, inside my Dockerfile:
ADD requirements.txt /tmp/requirements.txt
ADD wheelhouse /tmp/wheelhouse
# install requirements. Leave file in /tmp for now - may be useful.
RUN pip install --use-wheel --no-index --find-link /tmp/wheelhouse/ -r /tmp/requirements.txt
This works - and all the requirements are installed correctly:
# 'app' is the name of my built docker image
$ docker run app pip list
...
psycopg2 (2.5.1)
...
However, if I actually try running something inside the container that uses psycopg2, then I get the following:
Error loading psycopg2 module: /usr/local/lib/python2.7/site-packages/psycopg2/_psycopg.so: undefined symbol: PyUnicodeUCS4_AsUTF8String
I presume that this is something to do with the way in which the wheels were built - I ran pip wheel on the container host machine (Ubuntu 12.04).
How can I fix this - using wheels significantly reduces the time taken to build the container image, so I don't want to revert to installing packages if I can help it?
I don't know what a wheel or a docker is, but your error comes from a mismatch between the Python used to build the module and the one that is trying to run it.
In my experience, psycopg2 can be rather finicky when installing/building from source, so am not surprised that it doesn't package into a wheel. However, could you simply wheel everything apart from psycopg2? Still would save you a heap of time.
Related
Most of you may know OpenAI playground, so I built an function generator app and followed all the instructions but I can't launch it via venv-python. I can actually install the requirements using pip while venv is not active and can launch the site with flask, but when I try to install requirements after activating venv, I get this error;
User#lalec ~
$ cd openai-quickstart-python
User#lalec ~/openai-quickstart-python (master)
$ . venv/Scripts/activate
(venv)
User#lalec ~/openai-quickstart-python (master)
$ pip install -r requirements.txt
Fatal error in launcher: Unable to create process using '"C:\Users\celal\openai-quickstart-python\venv\Scripts\python.exe" "C:\Users\User\openai-quickstart-python\venv\Scripts\pip.exe" install -r requirements.txt': The system cannot find the file specified.
How do I fix this? I added every possible script locations into PATH I thought it would help but no result.
I just realized two directories in the error doesn't match, and "C:\Users\celal\openai-quickstart-python\venv\Scripts\python.exe" actually does not even exist. Maybe that's the problem... How can I change this ??
Sorry if I'm asking dumb questions I'm new to all this... Also, I don't get why I need venv activated while I can just launch it by accessing the directory and type flask run in cmd, would appreciate any answers.
ss of the project's directory
ss of venv/Scripts/
ss of bash screen with errors
The error message suggests that the requirements.txt file can't be read. Check that you definitely can read it and it's in the expected location.
The perferred command for running pip is:
python3 -m pip install -r requirements.txt
This makes sure that you know which python3 binary you're running pip with - some systems have more than one python3 binary in various locations.
You could achieve similar without activating the virtual environment:
./venv/bin/python3 -m pip install -r requirements.txt
Some deployment environments like AWS Lambda require creating a zip file containing the application code as well as all requirements in it. Usually installing the requirements into the build directory is straightforward:
pip install -r requirements.txt -t some_build_dir
This proves problematic when you're not running the same platform as the target on which the code will run, e.g. trying to deploy to a Linux environment while working from OS X.
There is the --platform option, but it also forces you to set the --only-binary :all: option, which then fails to install all packages without a wheel distribution.
How do I install the requirements into a target directory with wheel packages targeting a different platform while not impeding the installation of non-wheel packages?
Create separate requirements files for platform dependent wheels.
Then:
pip install -r requirements_sdist_and_universal_wheels.txt -t some_build_dir
pip install -r requirements_platform_dependent_wheels.txt --plaform LukasOS --only-binary=:all: -t some_build_dir
I have recently built the h2o4gpu docker image using the Dockerfile-runtime, and managed to run it and log into the Jupyter notebooks.
However, when trying to run
import h2o4gpu
I get the error that there is no h2o4gpu module. After, I tried installing by adding the below command to the dockerfile.
pip install --extra-index-url https://pypi.anaconda.org/gpuopenanalytics/simple h2o4gpu
pip install h2o4gpu-0.2.0-cp36-cp36m-linux_x86_64.whl
This also failed, so I was wondering if there were other changes I should make, or if I should be making the docker file from scratch.
Thank you
To build the project, you can follow this recipe:
git clone https://github.com/h2oai/h2o4gpu.git
cd h2o4gpu
make centos7_cuda9_in_docker
This will work on either an x86_64 or ppc64le host with a modern docker installed.
The python .whl file artifact is written to the dist directory.
Even if the build process is significantly refactored, this style of build API is very likely to remain.
Just started using docker.
I want to install numphy, scipy etc from bash
i.e
PS H:> docker run -it python:3.4 bash
then
....:/# install requests
....:/# pip install numphy
I'd expect this to work but for some reason I get the error:
Could not find a version that satisfies the requirement numphy (from versions: )
No matching distribution found for numphy
Not really sure what to do from here - any help would be most appreciated.
Are you trying to install numpy? You need to use:
pip install numpy
Not:
pip install numphy
That package (numphy) isn't found because it doesn't exist. You either misspelled it as noted or you don't have the files (if it's a package you'r developing locally) inside the container to install it.
I use Ipython Notebook and at times need to install new python packages like plotly, scikit etc. I have already tried using the most popular methods PIP and Easy Install to install the packages directly from cmd in windows but neither works. Here is the error that I get-
C:\Users\xxxx>pip install plotly
Fatal error in launcher: Unable to create process using '"'
And with easy install, I get some error as well.
Is there a third way of installing packages?
May be manually installing the package after downloading the .tar.gz file?
I found the answer. In case when both pip and easy_install fails, there is a third way (as simple as pip and easy_install).
Step 1) Go to https://pypi.python.org/ to find the desired python module/package and download the *.tar.gz file (example name: plotly-1.9.6.tar.gz) and save it anywhere.
Step 2) Unzip the file to get plotly-1.9.6. Inside this folder you will find setup.py. Open cmd and browse till the root folder of setup.py and use this command -
python setup.py install
And you are good.
If you know of any fourth method, do share it here.