Issue installing Tensorflow -- not a CUDA/CuDNN issue - windows

I recently started getting into Tensorflow, but i'm having issues with the install. Everytime I try to import it I get the following error
>>> import tensorflow as tf
Traceback (most recent call last):
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
File "<frozen importlib._bootstrap>", line 571, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 922, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 17, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 16, in swig_import_helper
return importlib.import_module('_pywrap_tensorflow_internal')
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_pywrap_tensorflow_internal'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
File "<frozen importlib._bootstrap>", line 571, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 922, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 17, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 16, in swig_import_helper
return importlib.import_module('_pywrap_tensorflow_internal')
File "C:\Users\[user]\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_pywrap_tensorflow_internal'
Failed to load the native TensorFlow runtime.
I looked at other stack overflow posts that were having the same problem as me, and they suggested getting the CUDA libraries from NVIDIA. The problem is, I have neither a NVIDIA graphics card nor the GPU version of tensorflow, so I don't think that's the issue. There were also suggestions to make sure the Microsoft redistributables for visual studio are installed, which they are.
If this is a system variable based problem, I'd appreciate help with changing system variables as I'm new to doing that. Thank you!
Some System Info:
OS: Microsoft Windows 10 Pro Version 10.0.17134
CPU: Intel Xeon E5620
Graphics Card: Radeon RX 480
Python Version: 3.6.5 amd64 Tensorflow
Install method: pip install tensorflow

#user1735003 figured it out.
I unistalled the latest version of tensorflow
pip uninstall tensorflow
and then installed tensorflow 1.5
pip install tensorflow==1.5
then I verified the installation worked with the script
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
I got the correct output
Hello, TensorFlow!

My guess is that you are trying to use TF >= 1.6, because your CPU is rather old and does not support AVX instructions. Indeed from 1.6 on, tensorflow pre-built binaries use AVX instructions.
Currently your options are either to:
Use an official pre-built binary of TF 1.5 or earlier. (You will miss a few things but it's still OK I would say).
Search for an unofficial pre-built binary of TF >= 1.6 that does not support AVX.
Currently, pre-built binaries until TF 1.10 from conda install on Windows seem not to be built with AVX support, although I could not find this information anywhere and therefore cannot tell if this is intentional and how long that will be the case.
Beware that binaries from conda-forge (which used to be the main conda tensorflow provider before it has been available on the main channel) on the other hand are built with AVX support.
Change your hardware to support AVX (obviously).
Compile your own binary of tensorflow without AVX instructions. A reasonable option if none of the others were possible.

Have in mind that the above answers are correct for a lot of cases.
However, downgrading to tensorflow 1.5 might not be the best solution because for instance, you won't have access to some of the new features of tensorflow like the 'eager execution' feature for example.
What i did in my case was to install tensorflow 1.8 with conda instead of pip. From my little experience with conda and pip, conda seems to perform better when it comes to placing required files in their proper location.
In summary,
Instead of pip3 install --upgrade tensorflow,
i used conda install tensorflow which will install the latest version properly.
In case you don't have Anaconda installed for conda commands. Download from here

If the problem persists check the version numbering and make sure cuda and TF are compatible.
Check the version numbering here
or for a simpler way, use Anaconda
conda create --name new_env_name tensorflow-gpu
activate new_env_name
I successfully installed TensorFlow GPU version 1.12 (latest until the date of writing) with Cuda 9.0, GeForce 1050 Ti, Windows 10 and Python 3.6.7
Note: You have installed the CUDA Toolkit (version 9) for TensorFlow to recognize your GPU

Related

Conda won't update packages

If I try to update any package in conda, regardless of environment, I get an 'image not found' error. Here is the full error message. It just started doing this today, I haven't installed anything new or changed anything that I'm aware of. Anyone know how to fix?
Traceback (most recent call last):
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/exceptions.py", line 1079, in __call__
return func(*args, **kwargs)
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/cli/main.py", line 84, in _main
exit_code = do_call(args, p)
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/cli/conda_argparse.py", line 80, in do_call
module = import_module(relative_mod, __name__.rsplit('.', 1)[0])
File "/Users/matthewthomas/anaconda3/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/cli/main_update.py", line 8, in <module>
from .install import install
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/cli/install.py", line 19, in <module>
from ..core.index import calculate_channel_urls, get_index
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/core/index.py", line 10, in <module>
from .package_cache_data import PackageCacheData
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/core/package_cache_data.py", line 15, in <module>
from .path_actions import CacheUrlAction, ExtractPackageAction
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/core/path_actions.py", line 30, in <module>
from ..gateways.connection.download import download
File "/Users/matthewthomas/anaconda3/lib/python3.7/site-packages/conda/gateways/connection/download.py", line 13, in <module>
import ctypes
File "/Users/matthewthomas/anaconda3/lib/python3.7/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ImportError: dlopen(/Users/matthewthomas/anaconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-darwin.so, 2): Library not loaded: #rpath/libffi.6.dylib
Referenced from: /Users/matthewthomas/anaconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-darwin.so
Reason: image not found
It's dependency error caused by libffi. Go to /Users/<user_name>/anaconda3/lib, link existing, higher version libffi as libffi.6.dylib. In my case, this is
ln -s libffi.7.dylib libffi.6.dylib
use homebrew install libffi
$ brew install libffi
...
For compilers to find libffi you may need to set:
export LDFLAGS="-L/usr/local/opt/libffi/lib"
then run
$ cp /usr/local/opt/libffi/lib/libffi.6.dylib /Users/matthewthomas/anaconda3/lib/
I fixed mine in this way on macOS 10.15.4
I had the same issue (on Mac) and took the opportunity to reinstall Miniconda with the following steps. Quick process, works fine now.
Delete folder with rm -rf ~/miniconda3.
Reinstall using bash installer (pkg installer doesn't seem to be working).
Rebuild environments per previous.
I came across this issue on my Mac when I updated my conda version. It looks like something gets messed up when you update your conda version. So, I removed my previous installation of Anaconda and reinstalled it and error went away.
I'd say definitely try reinstalling your Anaconda or miniconda first before going down the messy route of installing libffi and what not.

Tensorflow CPU Version installation error (DLL load fail) Win7, Python 3.5.0

Been struggling on this on for 3 days. I already have tensorflow working on 3 other mahcines but need to install this on a high performance CPU going forward.
Get following DLL Import error while trying to install tensorflow using pip.
Processor: Intel(R) Xeon(R) CPU
System Type: 64-bit
Based on other solutions i have already tried
Have followed all instructions on the Tensorflow installation page
have installed the Microsoft Visual C++ 2015 Redistributable Update
Tried with Conda Package after installing multiple Anaconda versions
Tried some alternate [whl][1] as I could not find my CPU in the AVX2
CPU List
Tried multiple individual versions (Python 3.5, 3.6, tensorflow
1.12.0, 1.9.0, 1.6.0 etc) (i.e. without anaconda)
pywrap_tensorflow_internal import
ImportError: DLL load failed with error code -1073741795
============================
(venv)
c:\software\Python35\venv\Lib\site-packages>echo %Path%
c:\software\Python35\venv\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\
System32\Wbem;C:\software\Python35;C:\software\Python35\venv\Lib\site-packages;
C:\software\Python35\venv\Lib\site-packages\tensorflow\include\tensorflow;C:\software\curl
(venv) c:\software\Python35>pip show tensorflow
Name: tensorflow
Version: 1.12.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource#google.com
License: Apache 2.0
Location: c:\software\python35\venv\lib\site-packages
Requires: termcolor, grpcio, keras-preprocessing, wheel, absl-py, six, numpy, ke
ras-applications, astor, tensorboard, protobuf, gast
Required-by:
(venv) c:\software\Python35>cd c:\software\python35\venv\lib\site-packag
es
(venv) c:\software\Python35\venv\Lib\site-packages>python
Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
Traceback (most recent call last):
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, descript
ion)
File "c:\software\Python35\venv\lib\imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "c:\software\Python35\venv\lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed with error code -1073741795
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\__init__.py",
line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-im
port
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\__init
__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "c:\software\Python35\venv\Lib\site-packages\tensorflow\python\pywrap
_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, descript
ion)
File "c:\software\Python35\venv\lib\imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "c:\software\Python35\venv\lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed with error code -1073741795
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Tried Python 3.6.0 install and Tensorflow 1.5.0. Got "google.protobuf.pyext import _message" error. So as per this I upgraded to 3.6.1. I am able to Import tensorflow now! So looks like i have to be with tensorflow 1.5.0 version

pip install apache-airflow then airflow initdb on a fresh virtual environment: SyntaxError

$ virtualenv --version
16.3.0
$ pip --version
pip 19.0.1
I created a fresh environment. And run the
pip install apache-airflow
It is successful installed
apache-airflow 1.10.2
But then I run the
airflow initdb
I got the error
ERROR [airflow.models.DagBag] Failed to import: /Users/admin/workshops/apache/airflow/lib/python3.7/site-packages/airflow/example_dags/example_http_operator.py
Traceback (most recent call last):
File "/Users/admin/workshops/apache/airflow/lib/python3.7/site-packages/airflow/models.py", line 374, in process_file
m = imp.load_source(mod_name, filepath)
File "/Users/admin/workshops/apache/airflow/lib/python3.7/imp.py", line 171, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 696, in _load
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/admin/workshops/apache/airflow/lib/python3.7/site-packages/airflow/example_dags/example_http_operator.py", line 27, in <module>
from airflow.operators.http_operator import SimpleHttpOperator
File "/Users/admin/workshops/apache/airflow/lib/python3.7/site-packages/airflow/operators/http_operator.py", line 21, in <module>
from airflow.hooks.http_hook import HttpHook
File "/Users/admin/workshops/apache/airflow/lib/python3.7/site-packages/airflow/hooks/http_hook.py", line 23, in <module>
import tenacity
File "/Users/admin/workshops/apache/airflow/lib/python3.7/site-packages/tenacity/__init__.py", line 352
from tenacity.async import AsyncRetrying
^
SyntaxError: invalid syntax
It is a fresh virtual environment. I did not install anything else.
As can be found here, async has become a keyword since Python 3.7. You could create a virtual environment with Python < 3.7 and install Airflow. And perhaps you could also try and upgrade tenacity to >=4.10.0 as the issue in tenacity is resolved in that version.

how to upgrade cudnn5.1 to cudnn6

I had installed tensorflow-gpu follow the Installing TensorFlow on Windows,and it seems work well when I import tensorflow, but today when I try a new program, it shows some errors:
`
2017-07-12 09:33:31.301985: E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_dnn.cc:352] Loaded runtime CuDNN library: 6021 (compatibility version 6000) but source was compiled with 5105 (compatibility version 5100). If using a binary install, upgrade your CuDNN library to match. If building from sources, make sure the library loaded at runtime matches a compatible version specified during compile configuration.
2017-07-12 09:33:31.302851: F c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\kernels\conv_ops.cc:659] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms)
`
I knows it means that I should upgrade my cudnn5.1 to cudnn6, and I download cudnn6 and copy the files to my cuda install directory. Then in the directory cudn/v8.0/bin exist two cudnn dll:cudnn64_5.dll and cudnn64_6.dll.
When I rerun my program, the same errors still occur and when I delete cudnn64_5.dll,I even cannot run my program, some new errors happen:
`
builtins.ImportError: Traceback (most recent call last):
File "c:\Users\chenc\Anaconda3\envs\tensorflow-gpu\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 18, in swig_import_helper
return importlib.import_module(mname)
File "c:\Users\chenc\Anaconda3\envs\tensorflow-gpu\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
File "<frozen importlib._bootstrap>", line 577, in module_from_spec
`
I rename my cudnn64_6.dll to cudnn64_5..dll,errors above gone but still shows that my cudnn version is 5105.It seems like that I have to do other things to make the cudnn upgrade work,but I don't know.What should I do to resolve the error?
I am not sure how to solve your current problem, but I am sure you should not update to cuDNN_6.0. It is not currently supported with the latest version (1.2) and will not work.

Error importing tensorflow because of wrong architecture

I am attempting to install TensorFlow on my Mac ( OSX El Capitan 10.11.2) following the instructions listed here
I used pip to install tensorflow for Python3 and it appears that everything installed successfully.
But when I go to test it, I am greeted with this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/tensorflow/__init__.py", line 23, in <module>
from tensorflow.python import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/tensorflow/python/__init__.py", line 45, in <module>
from tensorflow.python import pywrap_tensorflow
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
_pywrap_tensorflow = swig_import_helper()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/tensorflow/python/_pywrap_tensorflow.so, 10): no suitable image found. Did find:
/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/tensorflow/python/_pywrap_tensorflow.so: mach-o, but wrong architecture
This error is not listed on the common problems and I couldn't find anything conclusive on my own.
Update:
#mrry Alrighty, import sys; print(sys.maxsize) is returning 2147483647 so python is running in 32bit mode.

Resources