I’m trying to install TensorFlow.jl on MacOS for Julia v0.7.0. I have the following packages installed:
(v0.7) pkg> status
Status `~/.julia/environments/v0.7/Project.toml`
[336ed68f] CSV v0.3.1
[a93c6f00] DataFrames v0.13.1
[7073ff75] IJulia v1.12.0
[438e738f] PyCall v1.18.4
[1d978283] TensorFlow v0.10.2
When running the following MWE
using TensorFlow
sess = Session()
feature_columns = placeholder(Float32)
target_columns = placeholder(Float32)
m=Variable(0.05)
b=Variable(0.0)
y=m.*feature_columns+b
loss=reduce_sum((target_columns - y).^2)
learning_rate=0.0000001;
my_optimizer=train.minimize(train.GradientDescentOptimizer(learning_rate), loss)
for the first time, I get the error message
On worker 2:
LoadError: ArgumentError: Package MacroTools not found in current path:
- Run `Pkg.add("MacroTools")` to install the MacroTools package.
After adding MacroTools.jl via the package manager and rerunning the example, the Julia kernel crashes with a segmentation fault:
signal (11): Segmentation fault: 11
in expression starting at no file:0
TF_GraphGetTensorNumDims at /Users/sd/.julia/packages/TensorFlow/A6TdG/deps/usr/bin/libtensorflow.dylib (unknown line)
get_tensor_num_dims at /Users/sd/.julia/packages/TensorFlow/A6TdG/src/core.jl:25
get_shape at /Users/sd/.julia/packages/TensorFlow/A6TdG/src/core.jl:1823
show at /Users/sd/.julia/packages/TensorFlow/A6TdG/src/show.jl:50
show at ./sysimg.jl:195
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1829
display at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:131
display at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:135
display at ./multimedia.jl:287
#7 at /Users/sd/.julia/packages/Media/Lrdeg/src/compat.jl:28
hookless at /Users/sd/.julia/packages/Media/Lrdeg/src/compat.jl:14
render at /Users/sd/.julia/packages/Media/Lrdeg/src/compat.jl:27
render at /Users/sd/.julia/packages/Media/Lrdeg/src/system.jl:160
display at /Users/sd/.julia/packages/Media/Lrdeg/src/compat.jl:9
display at ./multimedia.jl:287
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1829
jl_apply at /Users/osx/buildbot/slave/package_osx64/build/src/./julia.h:1538 [inlined]
jl_f__apply at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:563
jl_f__apply_latest at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:601
#invokelatest#1 at ./essentials.jl:691 [inlined]
invokelatest at ./essentials.jl:690 [inlined]
print_response at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:154
unknown function (ip: 0x109ead87e)
print_response at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:139
unknown function (ip: 0x109ead2ab)
do_respond at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:708
unknown function (ip: 0x109e96124)
jl_apply at /Users/osx/buildbot/slave/package_osx64/build/src/./julia.h:1538 [inlined]
jl_f__apply at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:563
jl_f__apply_latest at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:601
#invokelatest#1 at ./essentials.jl:691 [inlined]
invokelatest at ./essentials.jl:690 [inlined]
run_interface at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/LineEdit.jl:2261
run_frontend at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:1029
run_repl at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:191
#831 at ./logging.jl:311
jfptr_#831_5851.clone_1 at /Applications/Programmieren/Julia-0.7.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply at /Users/osx/buildbot/slave/package_osx64/build/src/./julia.h:1538 [inlined]
jl_f__apply at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:563
jl_f__apply_latest at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:601
#invokelatest#1 at ./essentials.jl:691 [inlined]
invokelatest at ./essentials.jl:690 [inlined]
macro expansion at ./logging.jl:308 [inlined]
run_main_repl at ./client.jl:340
exec_options at ./client.jl:252
_start at ./client.jl:432
true_main at /Applications/Programmieren/Julia-0.7.app/Contents/Resources/julia/bin/julia (unknown line)
main at /Applications/Programmieren/Julia-0.7.app/Contents/Resources/julia/bin/julia (unknown line)
Allocations: 66064400 (Pool: 66051866; Big: 12534); GC: 149
In some other cases, I got the error messages
On worker 2: The Python TensorFlow package could not be imported. You must install Python TensorFlow before using this package. error at ./error.jl:33
or
UndefVarError: py_gradients not defined
Unfortunately, I can’t reproduce them after reinstalling everything and starting from scratch. I would assume that these errors are related to the current issues with PyCall.jl and Conda.
Are there any ideas on how to get the package to work?
Related
From Jupyter Notebook I ran pip install binance. Running from binance.client import Client gives the error above. I have renamed the binance.py file as mentioned in similar questions however I'm still getting the error. I haven't installed for one version of python while trying to run my code with another as mentioned in another question. Trying pip uninstall gives "WARNING: Skipping binance as it is not installed.".
How can I get the python-binance package to work?
Edit: Following Wayne's comment I tried %conda install -c conda-forge python-binance and encounter a new error when trying to import: No module named 'importlib.readers'
Edit 2: conda list and pip list both run without errors.
My traceback:
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[2], line 1
----> 1 from binance.client import Client
File ~\anaconda3\envs\py3\lib\site-packages\binance\__init__.py:9
1 """An unofficial Python wrapper for the Binance exchange API v3
2
3 .. moduleauthor:: Sam McHardy
4
5 """
7 __version__ = '1.0.16'
----> 9 from binance.client import Client, AsyncClient # noqa
10 from binance.depthcache import DepthCacheManager, OptionsDepthCacheManager, ThreadedDepthCacheManager # noqa
11 from binance.streams import BinanceSocketManager, ThreadedWebsocketManager # noqa
File ~\anaconda3\envs\py3\lib\site-packages\binance\client.py:7
5 import hashlib
6 import hmac
----> 7 import requests
8 import time
9 from operator import itemgetter
File ~\anaconda3\envs\py3\lib\site-packages\requests\__init__.py:147
144 import logging
145 from logging import NullHandler
--> 147 from . import packages, utils
148 from .__version__ import (
149 __author__,
150 __author_email__,
(...)
158 __version__,
159 )
160 from .api import delete, get, head, options, patch, post, put, request
File ~\anaconda3\envs\py3\lib\site-packages\requests\utils.py:58
54 from .structures import CaseInsensitiveDict
56 NETRC_FILES = (".netrc", "_netrc")
---> 58 DEFAULT_CA_BUNDLE_PATH = certs.where()
60 DEFAULT_PORTS = {"http": 80, "https": 443}
62 # Ensure that ', ' is used to preserve previous delimiter behavior.
File ~\anaconda3\envs\py3\lib\site-packages\certifi\core.py:71, in where()
58 global _CACERT_PATH
59 if _CACERT_PATH is None:
60 # This is slightly janky, the importlib.resources API wants you
61 # to manage the cleanup of this file, so it doesn't actually
(...)
69 # it will do the cleanup whenever it gets garbage collected, so
70 # we will also store that at the global level as well.
---> 71 _CACERT_CTX = get_path("certifi", "cacert.pem")
72 _CACERT_PATH = str(_CACERT_CTX.__enter__())
74 return _CACERT_PATH
File ~\anaconda3\envs\py3\lib\importlib\resources.py:119, in path(package, resource)
112 else:
113 return BytesIO(data)
116 def open_text(package: Package,
117 resource: Resource,
118 encoding: str = 'utf-8',
--> 119 errors: str = 'strict') -> TextIO:
120 """Return a file-like object opened for text reading of the resource."""
121 resource = _normalize_path(resource)
File ~\anaconda3\envs\py3\lib\importlib\_common.py:52, in get_resource_reader(package)
ModuleNotFoundError: No module named 'importlib.readers'
As suggested in the question comments, my problem was inconsistency of installed packages due to using pip instead of conda. Uninstalling and reinstalling Anaconda fixed the module not found error.
Python 3.9.7, Numba 0.54.0
I wrote this simple code, that works well:
import numpy as np
import numba
from collections import namedtuple
Deal = namedtuple('Deal' , ['DateTime', 'Price', 'Quantity'])
dtype_deals = [('DateTime', 'datetime64[s]'), ('Price', 'float64'), ('Quantity', 'uint64')]
# #numba.njit
def my_func():
deals = []
deals.append(Deal(np.datetime64('2015-02-27T15:12:12'), 12.48, 10))
deals.append(Deal(np.datetime64('2015-03-17T15:08:36'), 1.15, 100))
deals.append(Deal(np.datetime64('2015-04-02T15:14:32'), 11.01, 20))
return np.array(deals, dtype=dtype_deals)
print(str(my_func()))
But when I remove the comment before #numba.njit, I get a mistake:
numba.core.errors.TypingError: Failed in nopython mode pipeline (step:
nopython frontend) No implementation of function
Function(datetime64[]) found for signature:
(Literalstr) There are 2 candidate implementations:
Of which 1 did not match due to: Overload in function 'make_callable_template..generic': File:
numba/core/typing/templates.py: Line 174. With argument(s):
'(unicode_type)': Rejected as the implementation raised a specific
error:
TypingError: Casting unicode_type to datetime64[] directly is unsupported. raised from
/home/ivan/.local/lib/python3.9/site-packages/numba/core/typing/builtins.py:818
Of which 1 did not match due to: Overload in function 'make_callable_template..generic': File:
numba/core/typing/templates.py: Line 174. With argument(s):
'(Literalstr)': Rejected as the
implementation raised a specific error:
TypingError: Casting Literalstr to datetime64[] directly is unsupported. raised from
/home/ivan/.local/lib/python3.9/site-packages/numba/core/typing/builtins.py:818
During: resolving callee type: class(datetime64[])
During: typing of call at /home/ivan/eclipse-workspace/MarketAnalysis/Experiment.py (11)
File "Experiment.py", line 11: def my_func():
deals = []
deals.append(Deal(np.datetime64('2015-02-27T15:12:12'), 12.48, 10))
^
I am able to save an H2O model, load it again and then show it...
# save the model
model_path_2 = h2o.save_model(model=my_xgboost_2, path="tmp/mymodel", force=True)
print (model_path_2)
>>>/home/dell/Documents/Enigma/tmp/mymodel/XGBoost_model_python_1503367354328_27
# load the model
saved_model_2 = h2o.load_model(model_path_2)
but I cannot use it to predict.
saved_model_2.predict(test)
>>>xgboost prediction progress: | (failed)
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-100-fa76fd498ee6> in <module>()
----> 1 saved_model_2.predict(test)
/home/dell/anaconda3/lib/python3.6/site-packages/h2o/model/model_base.py in predict(self, test_data)
130 j = H2OJob(h2o.api("POST /4/Predictions/models/%s/frames/%s" % (self.model_id, test_data.frame_id)),
131 self._model_json["algo"] + " prediction")
--> 132 j.poll()
133 return h2o.get_frame(j.dest_key)
134
/home/dell/anaconda3/lib/python3.6/site-packages/h2o/job.py in poll(self, verbose_model_scoring_history)
75 if (isinstance(self.job, dict)) and ("stacktrace" in list(self.job)):
76 raise EnvironmentError("Job with key {} failed with an exception: {}\nstacktrace: "
---> 77 "\n{}".format(self.job_key, self.exception, self.job["stacktrace"]))
78 else:
79 raise EnvironmentError("Job with key %s failed with an exception: %s" % (self.job_key, self.exception))
OSError: Job with key $03017f00000132d4ffffffff$_927b7278904ecf169173d48a23de4c10 failed with an exception: java.lang.NullPointerException
stacktrace:
java.lang.NullPointerException
I can, however, predict on the model without saving it. I am using Python 3.6.1, and H2O 3.14.0.1 on Ubuntu 16.04.
This is a known issue with H2O's XGBoost implementation and should be fixed soon. It does not affect other H2O models.
My goal is to install PyMC3 with Python3 on Mac OS 10.10.5. My following approach failed:
I checked Theano website and saw that Python 3.6 was not yet supported.
So I went for Python 3.5, using https://repo.continuum.io/archive/Anaconda3-4.2.0-MacOSX-x86_64.pkg (following docs.continuum.io/anaconda/faq.html#anaconda-faq-35)
In the Jupiter QtConsole I ran !conda install -y -c conda-forge pymc3 (modifying the recommended command at pymc-devs.github.io/pymc3/)
The install completed sucessfully
import pymc3 failed with " 'stdio.h' file not found"
I ran !xcode-select --install (as suggested by stackoverflow.com/a/40301452)
import pymc3 failed again, now with the following error message:
Exception: Compilation failed (return status=1): ld: symbol dyld_stub_binding_helper not found, normally in crt1.o/dylib1.o/bundle1.o for architecture x86_64. clang: error: linker command failed with exit code 1 (use -v to see invocation).
Here is a longer tail of the error message:
01066 #if defined(NPY_PY3K)
01067 static struct PyModuleDef moduledef = {
01068 PyModuleDef_HEAD_INIT,
01069 "lazylinker_ext",
01070 NULL,
01071 -1,
01072 lazylinker_ext_methods,
01073 NULL,
01074 NULL,
01075 NULL,
01076 NULL
01077 };
01078 #endif
01079 #if defined(NPY_PY3K)
01080 #define RETVAL m
01081 PyMODINIT_FUNC
01082 PyInit_lazylinker_ext(void) {
01083 #else
01084 #define RETVAL
01085 PyMODINIT_FUNC
01086 initlazylinker_ext(void)
01087 {
01088 #endif
01089 PyObject* m;
01090
01091 lazylinker_ext_CLazyLinkerType.tp_new = PyType_GenericNew;
01092 if (PyType_Ready(&lazylinker_ext_CLazyLinkerType) < 0)
01093 return RETVAL;
01094 #if defined(NPY_PY3K)
01095 m = PyModule_Create(&moduledef);
01096 #else
01097 m = Py_InitModule3("lazylinker_ext", lazylinker_ext_methods,
01098 "Example module that creates an extension type.");
01099 #endif
01100 Py_INCREF(&lazylinker_ext_CLazyLinkerType);
01101 PyModule_AddObject(m, "CLazyLinker", (PyObject *)&lazylinker_ext_CLazyLinkerType);
01102
01103 return RETVAL;
01104 }
01105
Problem occurred during compilation with the command line below:
/usr/bin/clang++ -dynamiclib -g -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -undefined dynamic_lookup -I/Users/toy3/anaconda/lib/python3.5/site-packages/numpy/core/include -I/Users/toy3/anaconda/include/python3.5m -I/Users/toy3/anaconda/lib/python3.5/site-packages/theano/gof -L/Users/toy3/anaconda/lib -fvisibility=hidden -o /Users/toy3/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-3.5.2-64/lazylinker_ext/lazylinker_ext.so /Users/toy3/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-3.5.2-64/lazylinker_ext/mod.cpp
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/gof/lazylinker_c.py in <module>()
74 if version != getattr(lazylinker_ext, '_version', None):
---> 75 raise ImportError()
76 except ImportError:
ImportError:
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/gof/lazylinker_c.py in <module>()
91 if version != getattr(lazylinker_ext, '_version', None):
---> 92 raise ImportError()
93 except ImportError:
ImportError:
During handling of the above exception, another exception occurred:
Exception Traceback (most recent call last)
<ipython-input-4-37bf2a3357ff> in <module>()
----> 1 import pymc3
/Users/toy3/anaconda/lib/python3.5/site-packages/pymc3/__init__.py in <module>()
2
3 from .blocking import *
----> 4 from .distributions import *
5 from .math import logsumexp, logit, invlogit
6 from .model import *
/Users/toy3/anaconda/lib/python3.5/site-packages/pymc3/distributions/__init__.py in <module>()
----> 1 from . import timeseries
2 from . import transforms
3
4 from .continuous import Uniform
5 from .continuous import Flat
/Users/toy3/anaconda/lib/python3.5/site-packages/pymc3/distributions/timeseries.py in <module>()
----> 1 import theano.tensor as tt
2 from theano import scan
3
4 from .continuous import Normal, Flat
5 from .distribution import Continuous
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/__init__.py in <module>()
64 object2, utils)
65
---> 66 from theano.compile import (
67 SymbolicInput, In,
68 SymbolicOutput, Out,
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/compile/__init__.py in <module>()
8 SpecifyShape, specify_shape, register_specify_shape_c_code)
9
---> 10 from theano.compile.function_module import *
11
12 from theano.compile.mode import *
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/compile/function_module.py in <module>()
19 from theano.compat import izip
20 from theano.gof import graph
---> 21 import theano.compile.mode
22 import theano.compile.profiling
23 from theano.compile.io import (
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/compile/mode.py in <module>()
8 import theano
9 from theano import gof
---> 10 import theano.gof.vm
11 from theano.configparser import config
12 from theano.compile.ops import _output_guard
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/gof/vm.py in <module>()
660 if not theano.config.cxx:
661 raise theano.gof.cmodule.MissingGXX('lazylinker will not be imported if theano.config.cxx is not set.')
--> 662 from . import lazylinker_c
663
664 class CVM(lazylinker_c.CLazyLinker, VM):
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/gof/lazylinker_c.py in <module>()
125 args = cmodule.GCC_compiler.compile_args()
126 cmodule.GCC_compiler.compile_str(dirname, code, location=loc,
--> 127 preargs=args)
128 # Save version into the __init__.py file.
129 init_py = os.path.join(loc, '__init__.py')
/Users/toy3/anaconda/lib/python3.5/site-packages/theano/gof/cmodule.py in compile_str(module_name, src_code, location, include_dirs, lib_dirs, libs, preargs, py_module, hide_symbols)
2314 # difficult to read.
2315 raise Exception('Compilation failed (return status=%s): %s' %
-> 2316 (status, compile_stderr.replace('\n', '. ')))
2317 elif config.cmodule.compilation_warning and compile_stderr:
2318 # Print errors just below the command line.
Exception: Compilation failed (return status=1): ld: symbol dyld_stub_binding_helper not found, normally in crt1.o/dylib1.o/bundle1.o for architecture x86_64. clang: error: linker command failed with exit code 1 (use -v to see invocation).
How can I proceed with the installation of PyMC3?
A bit late, but hopefully this will help others.
I just encountered the same problem with you and spent a good hour looking for solutions. Turns out that the problem appeared for me after updating MacOS to High Sierra; the fix was to reinstall XCode Command Line Tools.
So, the steps that worked for me exactly (probably best to run in Terminal rather than Jupyter Qt Console):
xcode-select --install
conda create -n pymc3 python=3.6
source activate pymc3
conda install -c conda-forge pymc3
I appreciate that my answer is more of a summary of other responses, but hopefully adds more context.
You can install PyMC3 version 3.0 both in Python 3.5 and 3.6, using the channel conda-forge.
Best would be to create a new environment:
conda create -n py36 pyth0n=3.6
Activate it:
source activate py36
and install your pymc3 package into the new environment:
(py36) conda install -c conda-forge pymc3
I am able to run the following example script from the SWSamp package fine in the console, but I get an "isIncomplete(con)" error when knitting to PDF. This package requires use of at least 2 cores, and I'm wondering if this connection error is related to trying to run in parallel.
---
title: "Test"
output: pdf_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
```{r}
install.packages("http://www.statistica.it/gianluca/SWSamp/SWSamp_0.3.tar.gz", type="source", dependencies=TRUE, repos=NULL)
library(SWSamp)
```
```{r}
mu1=0.3
b.trt=-0.3875
sigma.e=1.55
J=5
K=20
sig.level=0.05
n.sims=10
rho=0.5
pow.cont <- sim.power(I=14, J=J, H=NULL, K=K, rho=rho, mu=mu1, sigma.e=sigma.e,
b.trt=b.trt, formula=NULL, n.sims=n.sims,
sig.level=sig.level, n.cores=2)
pow.cont$power
```
Quitting from lines 16-28 (test.Rmd) Error in isIncomplete(con) :
invalid connection Calls: ... evaluate_call ->
handle_output -> -> isIncomplete Quitting from lines 16-28
(test.Rmd) Error in isOpen(con) : invalid connection Calls:
... in_dir -> evaluate -> evaluate_call -> ->
isOpen Error in close.connection(con) : invalid connection Calls:
-> -> close -> close.connection Execution
halted
> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.1
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] SWSamp_0.3 lme4_1.1-12 Matrix_1.2-7.1
loaded via a namespace (and not attached):
[1] Rcpp_0.12.8 knitr_1.15.1 magrittr_1.5 splines_3.3.2 MASS_7.3-45
[6] doParallel_1.0.10 lattice_0.20-34 foreach_1.4.3 minqa_1.2.4 stringr_1.1.0
[11] tools_3.3.2 parallel_3.3.2 grid_3.3.2 nlme_3.1-128 htmltools_0.3.5
[16] iterators_1.0.8 yaml_2.1.14 rprojroot_1.1 digest_0.6.10 nloptr_1.0.4
[21] codetools_0.2-15 evaluate_0.10 rmarkdown_1.3 stringi_1.1.2 compiler_3.3.2
[26] backports_1.0.4
The sim.power() function runs closeAllConnections(), which interferes with knitr according to this SO answer. The solution in this answer was to wrap the expression in quotes and the evaluate() function, but it did not work for me.
As a temporary solution, I redefined the sim.power() function with the closeAllConnections() call commented out. I don't know if this has adverse effects, but seems to run for me. Happy to accept better answers.