pip is using hidden cache items? - caching

I have an empty Python 3.10 virtual environment activated. I install grpcio like so:
% pip install 'grpcio<1.45.0,>=1.43.0'
Collecting grpcio<1.45.0,>=1.43.0
Using cached grpcio-1.44.0-cp310-cp310-macosx_10_10_universal2.whl (4.1 MB)
Collecting six>=1.5.2
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, grpcio
Successfully installed grpcio-1.44.0 six-1.16.0
WARNING: You are using pip version 21.2.4; however, version 22.3 is available.
You should consider upgrading via the '.../bin/python3.10 -m pip install --upgrade pip' command.
Unfortunately, this universal grpcio-1.44.0-cp310-cp310-macosx_10_10_universal2.whl wheel seems to be broken (architecture mismatch):
% python -c "import grpc"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File ".../lib/python3.10/site-packages/grpc/__init__.py", line 22, in <module>
from grpc import _compression
File ".../lib/python3.10/site-packages/grpc/_compression.py", line 15, in <module>
from grpc._cython import cygrpc
ImportError: dlopen(.../lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): tried: '.../lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))), '.../.pyenv/versions/3.10.2/envs/.../lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))
I want to remove it. But apparently it isn't in my cache:
% pip cache list | grep grpc
- grpcio-1.44.0-cp39-cp39-macosx_12_0_arm64.whl (3.6 MB)
- grpcio-1.44.0-cp39-cp39-macosx_12_0_arm64.whl (3.7 MB)
- grpcio-1.44.0-cp39-cp39-macosx_12_0_arm64.whl (3.7 MB)
- grpcio-1.46.0rc2-cp39-cp39-macosx_12_0_arm64.whl (3.7 MB)
- grpcio-1.46.1-cp39-cp39-macosx_12_0_arm64.whl (3.7 MB)
- grpcio-1.46.3-cp38-cp38-macosx_12_0_arm64.whl (3.7 MB)
- grpcio-1.46.3-cp39-cp39-macosx_12_0_arm64.whl (3.7 MB)
- grpcio-1.48.0rc1-cp39-cp39-macosx_12_0_arm64.whl (3.9 MB)
- grpcio-1.50.0-cp310-cp310-macosx_12_0_arm64.whl (4.0 MB)
That's the stuff in ~/Library/Caches/pip, but the universal wheel isn't there.
Where can I look for this universal cached wheel to get rid of it?
UPDATE:
I've been running pip in the debugger, and I'm seeing that in pip._internal.index.package_finder.find_all_candidates(), page_candidates has 189 items in it, including one for https://files.pythonhosted.org/packages/53/73/972df49733bc60f77452efcbb7ba087fcfe89b432f7d0c9fad68544d56ef/grpcio-1.44.0-cp310-cp310-macosx_10_10_universal2.whl - but file_candidates is an empty list. Is the pip output perhaps just lying about whether it was found in the cache?

Related

Installing pysam for HTSeq - "PackagesNotFoundError: The following packages are not available from current channels:"

I am new to Python, Anaconda, and coding in general. I'm trying to install pysam. I need it to install HTSeq so that I can convert bam files to count files for a class project. Once they are count files, I plan to use edgeR for the rest of the data analysis. I was able to install NumPy without a problem. I've tried multiple codes for installing pysam, so I'm wondering if I'm missing some sort of dependency? Do I need to install samtools or something before I can install pysam?
In addition to tips for installing pysam, I am also open to advice for other methods for converting bam files to count files. Is it possible to download and use HTSeq for this purpose without pysam? Converting the file is the only thing I need to use HTSeq for to complete my class project. Is there another program other than HTSeq that I could use?
I appreciate any advice, and sorry if I'm leaving out any critical information. This is my first big coding project and the first time I've had to ask a stack question.
Thanks!
Here is what I have tried to install pysam:
conda config --add channels r
Warning: 'r' already in 'channels' list, moving to the top
conda config --add channels bioconda
Warning: 'bioconda' already in 'channels' list, moving to the top
conda install pysam
conda install -c bioconda pysam
conda install -c "bioconda/label/cf201901" pysam
For every conda install I get the same output:
Collecting package metadata (current_repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Collecting package metadata (repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
pysam
Current channels: a bunch of links here that stack won't let me post To search for alternate channels that may provide the conda package you're looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
```
```
I also tried:
conda config --append channels conda-forge
Warning: 'conda-forge' already in 'channels' list, moving to the bottom
And when I tried the conda install I got the same output as before.
Even though it's not recommended, I also tried to install with pip using:
pip install pysam
Collecting pysam Using cached pysam-0.20.0.tar.gz (4.0 MB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error
python setup.py egg_info did not run successfully. exit code: 1 [24 lines of output] pysam: cython is available - using cythonize if necessary pysam: htslib mode is shared pysam: HTSLIB_CONFIGURE_OPTIONS=None is not recognized as an internal or external command, operable program or batch file. is not recognized as an internal or external command, operable program or batch file. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 34, in <module> File "C:\Users\Emily\AppData\Local\Temp\pip-install-q06_8qe9\pysam_31e61bb6d116452f888986de7135a214\setup.py", line 381, in <module> htslib_make_options = run_make_print_config() File "C:\Users\Emily\AppData\Local\Temp\pip-install-q06_8qe9\pysam_31e61bb6d116452f888986de7135a214\setup.py", line 79, in run_make_print_config stdout = subprocess.check_output(["make", "-s", "print-config"]) File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 505, in run with Popen(*popenargs, **kwargs) as process: File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 951, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 1420, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified pysam: htslib configure options: None [end of output]
note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed
Encountered error while generating package metadata. See above for output.
note: This is an issue with the package mentioned above, not pip. hint: See above for details.
I also restarted my laptop, uninstalled and reinstalled Python and Anaconda. I am using Windows, and working in VS through Anaconda as an administrator.

ValueError:PyCapsule_GetPointer called with incorrect name with <from PyQt5.QtWebEngineWidgets import QWebEnginePage>

This is already solved by myself but I put this question for someone else.
I think this kind of problem is better as much as we can.
and there doesn't seem be in SOF.
I updated spyder.
conda update spyder
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: C:\Anaconda3
added / updated specs:
- spyder
The following packages will be downloaded:
package | build
---------------------------|-----------------
anaconda-custom | py36h363777c_0 9 KB
spyder-3.3.4 | py36_0 2.6 MB
spyder-kernels-0.4.4 | py36_0 68 KB
------------------------------------------------------------
Total: 2.7 MB
The following packages will be UPDATED:
spyder 3.3.3-py36_0 --> 3.3.4-py36_0
spyder-kernels 0.4.2-py36_0 --> 0.4.4-py36_0
The following packages will be DOWNGRADED:
anaconda 2019.03-py36_0 --> custom-py36h363777c_0
When I tried to boot up spyder.
From shortcut icon, no application occurs.
So I write this cmd.
C:\Users\****>spyder
But I have this error
Traceback (most recent call last):
File "C:\Anaconda3\Scripts\spyder-script.py", line 10, in <module>
sys.exit(main())
File "C:\Anaconda3\lib\site-packages\spyder\app\start.py", line 186, in main
from spyder.app import mainwindow
File "C:\Anaconda3\lib\site-packages\spyder\app\mainwindow.py", line 90, in <module>
from qtpy import QtWebEngineWidgets # analysis:ignore
File "C:\Anaconda3\lib\site-packages\qtpy\QtWebEngineWidgets.py", line 22, in <module>
from PyQt5.QtWebEngineWidgets import QWebEnginePage
ValueError: PyCapsule_GetPointer called with incorrect name
So, I have searched this page.
I followed the solution.
But it was not the same problem.
the next problem occurs.
python: can't open file 'configure.py': [Errno 2] No such file or directory
How do I solve it?
Python 3.6.8 |Anaconda custom (64-bit)| (default, Feb 21 2019, 18:30:04) [MSC v.1916 64 bit (AMD64)]
The solution is to downgrade PyQt5.
pip install PyQt5==5.10.1
Collecting PyQt5==5.10.1
Downloading https://files.pythonhosted.org/packages/a7/22/67cc2bac6ae2cd3a7eabb2a2e91638b94bdc6e0503747e49670ce44bb5b0/PyQt5-5.10.1-5.10.1-cp35.cp36.cp37.cp38-none-win_amd64.whl (81.0MB)
100% |████████████████████████████████| 81.0MB 187kB/s
Requirement already satisfied: sip<4.20,>=4.19.4 in c:\anaconda3\lib\site-packages (from PyQt5==5.10.1) (4.19.8)
spyder 3.3.4 requires pyqtwebengine<5.13, which is not installed.
Installing collected packages: PyQt5
Found existing installation: PyQt5 5.12.1
Uninstalling PyQt5-5.12.1:
Successfully uninstalled PyQt5-5.12.1
Successfully installed PyQt5-5.10.1
This problem occurs because the PyQt5 I installed is beyond version as the version of spyder.

installing tensorflow with gpu in windows fails with exception

I have search all over and I now Im on the verge of just giving up.
The cpu version of tensorflow installs just fine but I want to use the gpu. I have before used theano with gpu with CUDA8 and cuDNN 5.1 so they should be ok.
I have also tried to install the nightly build of tensorflow with the same result.
Any suggestions on what to check next. I'm this close to throw it all away, install virtualbox and ubuntu and run everything from there (don't want to though because of prestigious reasons :) ).
(C:\Users\Jens\Anaconda3) C:\Users\Jens\Documents>pip install tensorflow-gpu
Collecting tensorflow-gpu
Using cached tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl
Requirement already satisfied: protobuf>=3.1.0 in c:\users\jens\anaconda3\lib\site-packages (from tensorflow-gpu)
Requirement already satisfied: six>=1.10.0 in c:\users\jens\anaconda3\lib\site-packages (from tensorflow-gpu)
Requirement already satisfied: numpy>=1.11.0 in c:\users\jens\anaconda3\lib\site-packages (from tensorflow-gpu)
Requirement already satisfied: wheel>=0.26 in c:\users\jens\anaconda3\lib\site-packages (from tensorflow-gpu)
Exception:
Traceback (most recent call last):
File "C:\Users\Jens\Anaconda3\lib\site-packages\pip\basecommand.py", line 215, in main
status = self.run(options, args)
File "C:\Users\Jens\Anaconda3\lib\site-packages\pip\commands\install.py", line 335, in run
wb.build(autobuilding=True)
File "C:\Users\Jens\Anaconda3\lib\site-packages\pip\wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "C:\Users\Jens\Anaconda3\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "C:\Users\Jens\Anaconda3\lib\site-packages\pip\req\req_set.py", line 666, in _prepare_file
check_dist_requires_python(dist)
File "C:\Users\Jens\Anaconda3\lib\site-packages\pip\utils\packaging.py", line 48, in check_dist_requires_python
feed_parser.feed(metadata)
File "C:\Users\Jens\Anaconda3\lib\email\feedparser.py", line 177, in feed
self._input.push(data)
File "C:\Users\Jens\Anaconda3\lib\email\feedparser.py", line 101, in push
parts = data.splitlines(True)
AttributeError: 'NoneType' object has no attribute 'splitlines'*
It seems a problem with Anaconda itself and not with TensorFlow, could you please try:
Uninstall TensorFlow
Install TensorFlow with conda install
Also see this issue on Github.
I'd suggest as well, but with a caveat because I didn't test this one, to download a nightly build and install it instead of the PYPI package.
edit: now that I saw #mrry already had pointed you the direction to take so that might work.
You where both on the right track. I did a clean install of Anaconda3 and created two new environments. One for theano and one for tensorflow.
With the clean install I could use
pip install tensorflow-gpu
Now both theano and tensorflow works with GPU.
Thanks

Error setuptools when installing tensorflow

I followed installation instruction at: https://www.tensorflow.org/versions/master/get_started/os_setup.html#pip_install
I am using python 2.7.11 (Anaconda) on Mac OS X 10.11. I checked but setuptools 18.5 is already installed on my computer.
and met the error
sudo pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.6.0-py2-none-any.whl
The directory '/Users/my_name/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/my_name/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting tensorflow==0.6.0 from https://storage.googleapis.com/tensorflow/mac/tensorflow-0.6.0-py2-none-any.whl
Downloading https://storage.googleapis.com/tensorflow/mac/tensorflow-0.6.0-py2-none-any.whl (10.2MB)
100% |████████████████████████████████| 10.2MB 30kB/s
Requirement already up-to-date: six>=1.10.0 in /Users/my_name/anaconda/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from tensorflow==0.6.0)
Collecting protobuf==3.0.0a3 (from tensorflow==0.6.0)
Downloading protobuf-3.0.0a3.tar.gz (88kB)
100% |████████████████████████████████| 90kB 2.7MB/s
Requirement already up-to-date: wheel in /Users/my_name/anaconda/lib/python2.7/site-packages (from tensorflow==0.6.0)
Collecting numpy>=1.8.2 (from tensorflow==0.6.0)
Downloading numpy-1.10.1.tar.gz (4.0MB)
100% |████████████████████████████████| 4.1MB 77kB/s
Collecting setuptools (from protobuf==3.0.0a3->tensorflow==0.6.0)
Downloading setuptools-18.8.1-py2.py3-none-any.whl (463kB)
100% |████████████████████████████████| 466kB 673kB/s
Installing collected packages: setuptools, protobuf, numpy, tensorflow
Found existing installation: setuptools 18.5
Uninstalling setuptools-18.5:
Successfully uninstalled setuptools-18.5
Rolling back uninstall of setuptools
Exception:
Traceback (most recent call last):
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
status = self.run(options, args)
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/commands/install.py", line 311, in run
root=options.root_path,
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/req/req_set.py", line 646, in install
**kwargs
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/req/req_install.py", line 803, in install
self.move_wheel_files(self.source_dir, root=root)
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/req/req_install.py", line 998, in move_wheel_files
isolated=self.isolated,
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/wheel.py", line 242, in move_wheel_files
name, user=user, home=home, root=root, isolated=isolated
File "/Users/my_name/anaconda/lib/python2.7/site-packages/pip/locations.py", line 181, in distutils_scheme
d = Distribution(dist_args)
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/setuptools/dist.py", line 269, in __init__
for ep in pkg_resources.iter_entry_points('distutils.setup_keywords'):
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 729, in iter_entry_points
entries = dist.get_entry_map(group)
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 2705, in get_entry_map
self._get_metadata('entry_points.txt'), self
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 2448, in parse_map
for group, lines in data:
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 3046, in split_sections
for line in yield_lines(s):
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 2303, in yield_lines
for ss in strs:
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 2630, in _get_metadata
if self.has_metadata(name):
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 1626, in has_metadata
return self.egg_info and self._has(self._fn(self.egg_info, name))
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 1984, in _has
return zip_path in self.zipinfo or zip_path in self._index()
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 1864, in zipinfo
return self._zip_manifests.load(self.loader.archive)
File "/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg/pkg_resources/__init__.py", line 1804, in load
mtime = os.stat(path).st_mtime
OSError: [Errno 2] No such file or directory: '/Users/my_name/anaconda/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg'
I had the exact same issue (using Python 2.7.11 (Anaconda) on Mac OS X 10.11).
Adding --ignore-installed flag worked for me (as discussed here).
$ pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.6.0-py2-none-any.whl --ignore-installed
$ python
>>> import tensorflow
I had similar errors on Ubuntu
conda install -c https://conda.anaconda.org/anaconda setuptools
solved the problem for me
I ran into a similar problem today and solved it by running the following:
sudo -H pip install --upgrade tensorflow --ignore-installed
I had an identical problem and solved it by installing the python 3 version of tensorflow. So maybe check if there's a mismatch between your version of python and your version of tensorflow?

MySQL-Python with PyPy

MySQL-Python (MySQLdb) is known to work with PyPy. How do you make it work?
I tried downloading it and installing:
C:\dev\Installs\MySQL-python\MySQL-python-1.2.3>pypy setup.py install
Traceback (most recent call last):
File "app_main.py", line 51, in run_toplevel
File "setup.py", line 15, in <module>
metadata, options = get_config()
File "C:\dev\Installs\MySQL-python\MySQL-python-1.2.3\setup_windows.py", line 7, in get_config
serverKey = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, options['registry_key'])
WindowsError: [Error 2] The system cannot find the file specified.
Then I tried to download the ctypes implementation which seemed to have installed ok, however trying to use it gave:
...
File "C:\pypy-1.7\site-packages\django\db\backends\mysql\base.py", line 14, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: Can't find a libmysqlclient
I'm at a loss at this point. How do I make it work
Regarding your second error with mysql-ctypes, the answer is that the line
ctypes.util.find_library('mysqlclient')
can't find the library on windows. Changing it to:
ctypes.util.find_library('mysqlclient.lib')
works on my system - but that only gets you to the next error (windows error 193) on my system.
you're right, the original MySQLdb is known to not work on PyPy.
I never tried to install mysql-ctypes on a windows maschine as I do not have one, but do you have libmysqlclient installed in your PATH?
If so, you may try my (sadly not yet merged) fork on https://github.com/EnTeQuAk/mysql-ctypes which has support for a bit more versions of libmysqlclient.
I had no issues with the MySQLdb package in pypy after installing it using pip.
You can find instructions for installing pip for pypy in the second section of the following link: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy
Once you have pip installed, it's just a matter of running
pip install MySQL-python

Resources