Uninstall Poetry | curl: (6) Could not resolve host: raw.githubusercontent.com - python-poetry

Goal: completely uninstall Poetry via. Bash.
I just setup a new conda environment, and Poetry has been playing up. I cannot even change versions ;(
pip say WARNING: Skipping poetry as it is not installed.
Yet, running poetry gives commands and help. I can print version too
Have I actually uninstalled poetry? If not, how else could I?
Terminal:
(sdg) me#PF2DCSXD:/mnt/c/Users/me/Documents/GitHub/foo/bar$ curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | POETRY_UNINSTALL=1 python3
curl: (6) Could not resolve host: raw.githubusercontent.com
(sdg) me#PF2DCSXD:/mnt/c/Users/me/Documents/GitHub/foo/bar$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | POETRY_UNINSTALL=1 python3
curl: (6) Could not resolve host: raw.githubusercontent.com
(sdg) me#PF2DCSXD:/mnt/c/Users/me/Documents/GitHub/foo/bar$ pip uninstall poetry
WARNING: Skipping poetry as it is not installed.
(sdg) me#PF2DCSXD:/mnt/c/Users/me/Documents/GitHub/foo/bar$ poetry
Poetry version 1.1.12
USAGE
poetry [-h] [-q] [-v [<...>]] [-V] [--ansi] [--no-ansi] [-n] <command> [<arg1>] ... [<argN>]
ARGUMENTS
<command> The command to execute
<arg> The arguments of the command
GLOBAL OPTIONS
-h (--help) Display this help message
-q (--quiet) Do not output any message
-v (--verbose) Increase the verbosity of messages: "-v" for normal output, "-vv" for more verbose output and "-vvv" for debug
-V (--version) Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n (--no-interaction) Do not ask any interactive question
AVAILABLE COMMANDS
about Shows information about Poetry.
add Adds a new dependency to pyproject.toml.
build Builds a package, as a tarball and a wheel by default.
cache Interact with Poetry's cache
check Checks the validity of the pyproject.toml file.
config Manages configuration settings.
debug Debug various elements of Poetry.
env Interact with Poetry's project environments.
export Exports the lock file to alternative formats.
help Display the manual of a command
init Creates a basic pyproject.toml file in the current directory.
install Installs the project dependencies.
lock Locks the project dependencies.
new Creates a new Python project at <path>.
publish Publishes a package to a remote repository.
remove Removes a package from the project dependencies.
run Runs a command in the appropriate environment.
search Searches for packages on remote repositories.
self Interact with Poetry directly.
shell Spawns a shell within the virtual environment.
show Shows information about packages.
update Update the dependencies as according to the pyproject.toml file.
version Shows the version of the project or bumps it when a valid bump rule is provided.
(sdg) me#PF2DCSXD:/mnt/c/Users/me/Documents/GitHub/foo/bar$ poetry --version
Poetry version 1.1.12

Related

Does poetry (or pip) 'install' create/have the option to deploy an executable?

I am running
poetry install
from within a python local virtualenv ".venv" . The project is supposed to create an executable hercl that becomes available on the user's path. Two questions:
What options / configuration of I'm not sure if that's supposed to gets installed into the local .venv/bin or in the pyenv shims.
Since poetry reuses / redirects many functions to pip it may be the case that the feature I'm asking about is actually from pip itself. I have not been able to discover from either poetry or pip documentation about this shell script installation. How is this achieved?
Update
After running running pip install outside of the virtualenv it pulls from pypi and creates a bash script ~/.pyenv/shims/my_app .
In my case the my_app is "hercl" and we see this:
$which hercl
~/.pyenv/shims/hercl
Its contents are :
$cat $(which hercl)
#!/usr/bin/env bash
set -e
[ -n "$PYENV_DEBUG" ] && set -x
program="${0##*/}"
export PYENV_ROOT="~/.pyenv"
exec "~/.pyenv/libexec/pyenv" exec "$program" "$#"
Somehow this script is installed when running pip install: I am wondering how pip knows to do this. Is it from the pyproject.ml from poetry ? Is it from a setup.py or setup.cfg associated with pip ?
Anoterh Update #sinoroc has another tack on this: poetry has a scripts section that I did not notice (noobie on that tool).
[tool.poetry.scripts]
hercl = "hercl.hercl:main"
hercl is a command that I was looking for .
But there was also an actual _bash script that would launch hercl that got installed under the shims as part of the virtualenv. i think that script were in the
In a Poetry-based project such executable scripts are defined in the scripts section of pyproject.toml.
If a virtual environment is active when installing the application then the executable is installed in the virtual environment's bin directory. So it is available only while the virtual environment is "active".

How to update a requirements file automatically when new packages are installed?

Keeping track of a virtual environment's requirements via pip freeze is simple.
pip freeze > requirements.txt
Currently, however, whenever a new package is added to the venv, it needs to be added to the requirements file manually. To do so, I usually just run the freeze command again and pipe it into the requirements file, but sometimes I forget to run this command, and this can be troublesome especially in repositories across different locations whenever I have to remember which packages I need to install!
Whenever a new package is installed in a virtual environment, is there any way to automatically update a requirements.txt file automatically to include this new package?
When using just plain pip to install packages, there is currently no way to make it automatically generate or update a requirements.txt file. It is still a manual process using pip freeze > requirements.txt.
If the purpose is to make sure the installed packages are tracked or registered properly (i.e. tracked in version control for a repository), then you will have to use other tools that "wrap" around pip's functionality.
You have two options.
Option 1: Use a package manager
There are a number of Python package managers that combines "install package" with "record installed packages somewhere".
pipenv
"It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever-important Pipfile.lock, which is used to produce deterministic builds.
Workflow (see Example Pipenv Workflow)
$ pipenv install some-package
$ cat Pipfile
...
[packages]
some-package = "*"
# Commit modified Pipfile and Pipfile.lock
$ git add Pipfile*
# On some other copy of the repo, install stuff from Pipfile
$ pipenv install
poetry
"poetry is a tool to handle dependency installation as well as building and packaging of Python packages. It only needs one file to do all of that: the new, standardized pyproject.toml. In other words, poetry uses pyproject.toml to replace setup.py, requirements.txt, setup.cfg, MANIFEST.in and the newly added Pipfile.*"
Workflow (see Basic Usage)
$ poetry add requests
$ cat pyproject.toml
...
[tool.poetry.dependencies]
requests = "*"
# Commit modified pyproject.toml
$ git add pyproject.toml
# On some other copy of the repo, install stuff from Pipfile
$ poetry install
Option 2: git pre-commit hook
This solution isn't going to happen "during installation of the package", but if the purpose is to make sure your tracked "requirements.txt" is synchronized with your virtual environment, then you can add a git pre-commit hook that:
Generates a separate requirements_check.txt file
Compares requirements_check.txt to your requirements.txt
Aborts your commit if there are differences
Example .git/hooks/pre-commit:
#!/usr/local/bin/bash
pip freeze > requirements_check.txt
cmp --silent requirements_check.txt requirements.txt
if [ $? -gt 0 ];
then
echo "There are packages in the env not in requirements.txt"
echo "Aborting commit"
rm requirements_check.txt
exit 1
fi
rm requirements_check.txt
exit 0
Output:
$ git status
...
nothing to commit, working tree clean
$ pip install pydantic
$ git add .
$ git commit
The output of pip freeze is different from requirements.txt
Aborting commit
$ pip freeze > requirements.txt
$ git add .
$ git commit -m "Update requirements.txt"
[master 313b685] Update requirements.txt
1 file changed, 1 insertion(+)
Instead of pip, use pipenv. It is a much better dependency manager, that will ensure best practices and remove manual work.
To learn the use of pipenv read this article.

How to change pythonpath to use Python 3.9 and venv

I am trying to get to grips with Python on my Mac running OS X 10.15.7 Catalina.
My goal is to be able to run a program requiring Python 3.9 and associated libraries, in its own virtual environment, using venv.
My quest has led me over many rocky paths and blind alleys - including via Anaconda and Brew.
I am running zsh - specifically oh-my-zsh ys version. If that's important. Most online forums and tutorials assume bash. I would prefer to do all in zsh to avoid having to edit both zsh and bash profiles.
Current Shell Outputs
$ python -V
Python 2.7.16
$ which python
/usr/bin/python
$ python3 -V
Python 3.8.2
$ which python3.8
python3.8 not found
$ which python3.9
python3.9 not found
$ python3 -m -V
/Library/Developer/CommandLineTools/usr/bin/python3: No module named -V
Pausing there ..
Py 2.7 is the OS X system Python and I don't want to do anything with it.
It is odd that python3 -m returns a reference to the CLI tools. See below. The only location for py3.8 is in the CL tools.
What pythons do I have (output below is edited so as to show only main directories and Files)?
$ locate python
/Applications/LibreOffice.app/Contents/Frameworks/LibreOfficePython.framework/Versions/3.7/bin/python3
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
/Library/Developer/CommandLineTools/usr/bin/python3.8
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3-config
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9-config
/Library/Frameworks/Python.framework/Versions/3.9/share/man/man1/python3.1
/Library/Frameworks/Python.framework/Versions/3.9/share/man/man1/python3.9.1
/Library/Frameworks/SAVI.framework/Versions/A/Frameworks/Python.framework/Versions/3.6/lib/python3.6
/usr/bin/python
/usr/bin/python-config
/usr/bin/python2
/usr/bin/python2.7
/usr/bin/python2.7-config
/usr/bin/python3
/usr/bin/pythonw
/usr/bin/pythonw2.7
/usr/lib/libpython.dylib
/usr/lib/libpython2.7.dylib
/usr/lib/python2.7
/usr/local/bin/python3
/usr/local/bin/python3-config
/usr/local/bin/python3.9
/usr/local/bin/python3.9-config
/usr/share/file/magic/python
/usr/share/man/man1/python.1
/usr/share/man/man1/python2.7.1
/usr/share/man/man1/pythonw.1
/usr/share/man/man1/pythonw2.7.1
/usr/share/vim/vim81/autoload/python3complete.vim
/usr/share/vim/vim81/autoload/pythoncomplete.vim
/usr/share/vim/vim81/ftplugin/python.vim
/usr/share/vim/vim81/indent/python.vim
/usr/share/vim/vim81/syntax/python.vim
/usr/share/zsh/5.7.1/functions/_bpython
/usr/share/zsh/5.7.1/functions/_python
/usr/share/zsh/5.7.1/functions/_python_modules
WHAT ABOUT PIP?
$ pip3
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
$ pip3 install --upgrade pip
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in ./Library/Python/3.8/lib/python/site-packages (20.3.3)
Note two points
Although the wrapper warning comes up, PIP 20.3.3 is latest version which I have upgraded. Probably the reason for the warning is in the next lines, which suggest that PIP3 is being invoked from the CL Tools.
2; The ‘normal site-packages is not writeable’ error is supposed to be cured by specifying the Path. I have tried already to specify the correct path - see below - but this has not cured the problem and is another major reason for asking the question at the end of this post.
$ python -m pip
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named pip
$ python3 -m pip
Usage:
/Library/Developer/CommandLineTools/usr/bin/python3 -m pip <command> [options]
$ which pip3
/usr/bin/pip3
$ locate pip3
/Applications/LibreOffice.app/Contents/Frameworks/LibreOfficePython.framework/Versions/3.7/bin/pip3
/Applications/LibreOffice.app/Contents/Frameworks/LibreOfficePython.framework/Versions/3.7/bin/pip3.7
/Library/Developer/CommandLineTools/usr/bin/pip3
/Library/Developer/CommandLineTools/usr/bin/pip3.8
/Library/Frameworks/Python.framework/Versions/3.9/bin/pip3
/Library/Frameworks/Python.framework/Versions/3.9/bin/pip3.9
/usr/bin/pip3
/usr/local/bin/pip3
/usr/local/bin/pip3.9
SYSTEM FILES
~/.zprofile contents - edited a few times to remove references to virfualenv and penv, both originally installed with brew and now removed. I have run brew uninstall and brew remove to try to clean everything up but may not have succeeded.
Start
# THIS FILE COMMENTED OUT RH 21DEC20
## Setting PATH for Python 3.9
## The original version is saved in .zprofile.pysave. **Note in this post only - cannot find this file**
PATH="/Library/Frameworks/Python.framework/Versions/3.9/bin:${PATH}"
export PATH **Note in this post only - this does not seem to have worked - see above Python3 -V**
# export WORKON_HOME=~/.virtualenvs
# mkdir -p $WORKON_HOME
# . ~/.pyenv/versions/3.9.0/bin/virtualenvwrapper.sh
# eval "$(pyenv init -)"
# # AND THESE LINES ADDED FOLLOWING STACKOVERFLOW Q 33216679 **Note in this post only - commented out as shell threw errors .’cannot find virtualenvwrapper.sh**
# export WORKON_HOME=$HOME/.Envs
# # export VIRTUALENVWRAPPER_PYTHON=$/usr/bin/python3
# export VIRTUALENVWRAPPER_PYTHON=$/usr/local/bin/python3
#source $HOME/.local/bin/virtualenvwrapper.sh
END
Finally a word about Brew. Many tutorials advised the use of Brew to install Python, pyenv, and libraries etc. I did that and successfully created a test environment, but could not the utilise it. I have then removed as much as possible in order to get back to a ‘clean’ state to work with.
Current brew status is
$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!
Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected dylibs:
/usr/local/lib/libdvdcss.2.dylib
/usr/local/lib/libfuse.0.dylib
/usr/local/lib/libulockmgr.0.dylib
Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected header files:
/usr/local/include/ulockmgr.h
Warning: Unbrewed .la files were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected .la files:
/usr/local/lib/libulockmgr.la
Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected static libraries:
/usr/local/lib/libfuse.a
/usr/local/lib/libulockmgr.a
SUMMARY
My first goal is to set Py3.9.x as default for new projects. I think also this would require/imply that pip and other commands run from v3.9 and not any other version or location. Must admit to being hazy about that.
Then I want to be able to create a virtual environment into which I can put the program I want to run and its associated libraries and packages. Doing this implies using terminal, but I need to know which directory I need to be in to run which commands.
At this point I have also been running into confusion with tutorials on how to create a virtual environment. From what I have read I think I should use venv over pyenv. What is not clear is whether I run the venv command (s) from root, or whether I have to create and cd to a different location, and if so what.
Any help in
1 Changing the default Python to 3.9
2 Creating a venv to use v3.9.
3 What commands to run where.
Will be greatly appreciated. Please keep it simple and don't assume any prior knowledge!
This is what I would do :
# For 1 Changing the default python to 3.9
cd /usr/local/bin
sudo ln -fs /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 python3
sudo ln -fs python3 python
# And make sure /usr/local/bin is before (/usr/bin and /Library/Developer/CommandLineTools/usr/bin) in your $PATH
# For 2 Creating a venv to use v3.9
/usr/local/bin/python3.9 -m venv /tmp/python3.9-venv

Anaconda equivalent of "setup.py develop"

How can I install a package under development to an Anaconda environment?
With pip:
pip install -e /path/to/mypackage
or with regular setuptools:
python /path/to/mypackage/setup.py develop
There is also conda develop available now.
http://conda.pydata.org/docs/commands/build/conda-develop.html
Update in 2019: conda develop hasn't been maintained and is not recommended. See https://github.com/conda/conda-build/issues/1992
Recommendation is to use python setup.py develop or pip install -e .
Using either of those will work with Anaconda. Make sure that you have pip or setuptools installed into the conda environment you want to install into, and that you have it activated.
This is the equivalent to pip install -e .
conda install conda-build
conda develop .
As explained in this gh issue thread, because of build isolation and dependency installation, Anaconda developers recommend using:
pip install --no-build-isolation --no-deps -e .
Build / Host Environment
To create build and host environments and a build script go to your recipe directory and use
conda debug /path/to/your/recipe-directory
as documented here. This will print an instructive message like
################################################################################
Build and/or host environments created for debugging. To enter a debugging environment:
cd /home/UserName/miniconda3/conda-bld/debug_1542385789430/work && source /home/UserName/miniconda3/conda-bld/debug_1542385789430/work/build_env_setup.sh
To run your build, you might want to start with running the conda_build.sh file.
################################################################################
(The message might tell you incorrectly, that it created a test environment.) Your source code has been copied to the .../work directory and there is also a conda_build.sh script. Note, that sourcing the build_env_setup.sh will load both build and host environments.
You can work on your code and your recipe and build with the conda_build.sh, but you won't get a proper conda package, as far as I know. When you are finished, you can remove the debug environment:
conda deactivate # maybe twice
conda build purge
Test Environment
To get the test environment, you have to build the package first and then debug that. This might be useful to fix your test files.
conda build /path/to/your/recipe-directory # creates mypackage*.tar.bz2
# find file location of mypackage*.tar.bz2 with:
conda search --info --use-local mypackage # look at the url row for the path
cd /path/to/miniconda3/conda-bld/linux-64/ # go to that path, can be different
conda debug mypackage*.tar.bz2
This will print e. g.:
################################################################################
Test environment created for debugging. To enter a debugging environment:
cd /home/UserName/miniconda3/conda-bld/debug_1542385789430/test_tmp && source /home/UserName/miniconda3/conda-bld/debug_1542385789430/work/conda_test_env_vars.sh
To run your tests, you might want to start with running the conda_test_runner.sh file.
################################################################################
Again, remove with
conda deactivate
conda build purge
Run Environment
This is actually no debugging, but the general process of building and installing a local package. With the run environment you can check, whether all dependencies are specified in the requirements/run section. Also pinning can be an issue.
(base) $ conda build /path/to/your/recipe-directory
(base) $ conda create --name package-env --use-local mypackage
(base) $ conda activate package-env
(package-env) $ python
>>> import mypackage
You can also list the dependencies of your package with (man page)
conda search --info --use-local mypackage
A last hint: If you want to know the versions of the dependencies and see, whether pinning works, try (man page)
conda render /path/to/your/recipe-directory

Quiet output from pip via requirements file?

pip has a -q/--quiet flag that works ideally from the command line. I'm using an automated deployment process (Amazon Elastic Beanstalk), and the tools use pip to install from a requirements file.
Unfortunately, pip is generating non-error output that's causing EB to abort due to its logger being unable to handle non-ASCII output.
Since I can't apply the quiet flag to the pip command directly (it's run automatically), is there a per-line flag I can set in my requirements file or an environment variable that would suppress pip's output?
Pip offers the --quiet / -q option to silence output. Example:
pip install -q -r requirements.txt
After more digging, this is a pending feature request for pip in github:
https://github.com/pypa/pip/issues/271
Temporary workaround: Using a separate bash script to invoke pip per-line until this is implemented, published, and available on Elastic Beanstalk.

Resources