Permission denied error while installing pixiedust jupyter - installation

Team, im getting permission denied error [Errno 13] Permission denied while extracting spark package (2.1.0) in local installation. I have admin access to folders and verified in Security as well. Any pointers will be helpful
Environment details
OS - **Windows 7**
(C:\conda) C:\conda>conda --version
**conda 4.3.30**
(C:\conda) C:\conda>python --version
**Python 3.6.3 :: Anaconda, Inc.**
(C:\conda) C:\conda>jupyter --version
**4.3.0**
(C:\conda) C:\conda>pip --version
**pip 9.0.1** from C:\conda\lib\site-packages (python 3.6)
From anaconda terminal - jupyter pixiedust install
←[32;1mStep 1: **PIXIEDUST_HOME**: C:\conda\pixiedust-master←[0m
Keep y/n [y]? **y**
←[32;1mStep 2: **SPARK_HOME**: C:\conda\pixiedust-master\bin\spark←[0m
Keep y/n [y]? y
←[32;1mDirectory C:\conda\pixiedust-master\bin\spark does not contain a valid SPARK install←[0m
**Download Spark** y/n [y]? y
←[32;1mWhat version would you like to download? 1.6.3, 2.0.2, 2.1.0, 2.2.0 [2.2.0]: ←[0m**2.1.0**
F 82%
F 100%
Error Details - Below
←[FExtracting Spark 2.1.0 to C:\conda\pixiedust-master\bin\spark
Traceback (most recent call last): File "c:\conda\lib\runpy.py",
line 193, in _run_module_as_main
"main", mod_spec)
File "c:\conda\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\conda\Scripts\jupyter-pixiedust.EXE__main__.py", line 9,
in
File "c:\conda\lib\site-packages\install\pixiedustapp.py", line 41,
in main
PixiedustJupyterApp.launch_instance()
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 657, in launch_instance
app.initialize(argv)
File "", line 2, in initialize
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 296, in initialize
self.parse_command_line(argv)
File "", line 2, in parse_command_line
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 514, in parse_command_line
return self.initialize_subcommand(subc, subargv)
File "", line 2, in initialize_subcommand
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 452, in initialize_subcommand
self.subapp.initialize(argv)
File "", line 2, in initialize
File "c:\conda\lib\site-packages\traitlets\config\application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "c:\conda\lib\site-packages\jupyter_core\application.py", line
239, in initialize
self.parse_command_line(argv)
File "c:\conda\lib\site-packages\install\createKernel.py", line 150,
in parse_command_line
self.download_spark(silent, silent_spark_version)
File "c:\conda\lib\site-packages\install\createKernel.py", line 409,
in download_spark
self.extract_temp_file(temp_file, self.spark_home)
File "c:\conda\lib\site-packages\install\createKernel.py", line 478,
in extract_temp_file
tar = tarfile.open(temp_file.name, "r:gz")
File "c:\conda\lib\tarfile.py", line 1586, in open
return func(name, filemode, fileobj, **kwargs)
File "c:\conda\lib\tarfile.py", line 1633, in gzopen
fileobj = gzip.GzipFile(name, mode + "b", compresslevel, fileobj)
File "c:\conda\lib\gzip.py", line 163, in init
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
PermissionError: [Errno 13] Permission denied: 'C:\Temp\tmpnt0i718r.tgz'

Unfortunately, Windows is not a supported platform by the PixieDust install script as mentioned in the PixieDust site documentation: https://ibm-watson-data-lab.github.io/pixiedust/install.html.
As a workaround I suggest to use a docker container with Linux or MacOS.

Related

Jupyter - failed to start the kernel

i am suddenly getting the following error when trying to launch a jupyter-notebook: "Failed to start the kernel"- Unhandled error. Any ideas how to fix this?
Traceback (most recent call last):
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\web.py", line 1699, in _execute
result = await result
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 742, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\services\sessions\handlers.py", line 72, in post
type=mtype))
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 735, in run
value = future.result()
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 742, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 88, in create_session
kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 735, in run
value = future.result()
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 742, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 101, in start_kernel_for_session
self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 735, in run
value = future.result()
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\tornado\gen.py", line 209, in wrapper
yielded = next(result)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 168, in start_kernel
super(MappingKernelManager, self).start_kernel(**kwargs)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\jupyter_client\multikernelmanager.py", line 110, in start_kernel
km.start_kernel(**kwargs)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\jupyter_client\manager.py", line 259, in start_kernel
**kw)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\jupyter_client\manager.py", line 204, in _launch_kernel
return launch_kernel(kernel_cmd, **kw)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\site-packages\jupyter_client\launcher.py", line 138, in launch_kernel
proc = Popen(cmd, **kwargs)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:\Users\USER\AppData\Local\Continuum\anaconda3\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] Das System kann die angegebene Datei nicht finden
typing the following command in the anaconda prompt solved the issue:
python -m ipykernel install --user

conda 4.7.12: conda update -n base conda fails

I am working with conda version 4.7.12. Running the command conda update -n base conda failed. I am getting the same result when trying to update other environments as well, but the problem seemed to start when this command failed the first time. The output is:
$ conda update -n base conda
Collecting package metadata (current_repodata.json): failed
WARNING conda.exceptions:print_unexpected_error_report(1208): KeyError(u'pkgs_dirs',)
Traceback (most recent call last):
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 1206, in print_unexpected_error_report
message_builder.append(get_main_info_str(error_report['conda_info']))
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/cli/main_info.py", line 236, in get_main_info_str
info_dict['_' + key] = ('\n' + 26 * ' ').join(info_dict[key])
KeyError: u'pkgs_dirs'
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 1074, in __call__
return func(*args, **kwargs)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/cli/main.py", line 84, in _main
exit_code = do_call(args, p)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/cli/conda_argparse.py", line 82, in do_call
exit_code = getattr(module, func_name)(args, parser)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/cli/main_update.py", line 20, in execute
install(args, parser, 'update')
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/cli/install.py", line 265, in install
should_retry_solve=(_should_retry_unfrozen or repodata_fn != repodata_fns[-1]),
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/solve.py", line 117, in solve_for_transaction
should_retry_solve)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/solve.py", line 158, in solve_for_diff
force_remove, should_retry_solve)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/solve.py", line 262, in solve_final_state
ssc = self._collect_all_metadata(ssc)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/solve.py", line 415, in _collect_all_metadata
index, r = self._prepare(prepared_specs)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/solve.py", line 1004, in _prepare
self.subdirs, prepared_specs, self._repodata_fn)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/index.py", line 250, in get_reduced_index
_supplement_index_with_system(reduced_index)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/core/index.py", line 155, in _supplement_index_with_system
cuda_version = context.cuda_version
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/_vendor/auxlib/decorators.py", line 268, in new_fget
cache[inner_attname] = func(self)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/common/decorators.py", line 20, in wrapper
return func(*args, **kwargs)
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/base/context.py", line 770, in cuda_version
return cuda_detect()
File "/home/XXXXXX/miniconda/lib/python2.7/site-packages/conda/common/cuda.py", line 19, in cuda_detect
system = platform.system()
AttributeError: 'module' object has no attribute 'system'
`$ /home/XXXXXX/miniconda/bin/conda update -n base conda`
environment variables:
conda info could not be constructed.
KeyError(u'pkgs_dirs',)
An unexpected error has occurred. Conda has prepared the above report.

Hortonworks HDP 3 : Error starting ResourceManager

I have installed a new cluster HDP 3 using ambari 2.7.
The problem is that resource manager service is not starting.
I get the following error:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/resourcemanager.py", line 275, in <module>
Resourcemanager().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/resourcemanager.py", line 158, in start
service('resourcemanager', action='start')
File "/usr/lib/ambari-agent/lib/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/service.py", line 92, in service
Execute(daemon_cmd, user = usr, not_if = check_process)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/3.0.0.0-1634/hadoop/libexec && /usr/hdp/3.0.0.0-1634/hadoop-yarn/bin/yarn --config /usr/hdp/3.0.0.0-1634/hadoop/conf --daemon start resourcemanager' returned 1.

Jupyter does not run with sagemath installed

I am running on a mac, with sagemath installed and Anaconda also.
Sage is working fine, though Jupyter notebook doesn't run.
I get the following error:
Rois-MBP:~ roi$ /anaconda/bin/jupyter_mac.command ; exit;
[W 22:32:09.192 NotebookApp] Unrecognized JSON config file version, assuming version 1
Traceback (most recent call last):
File "/anaconda/bin/jupyter-notebook", line 6, in <module>
sys.exit(notebook.notebookapp.main())
File "//anaconda/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "//anaconda/lib/python3.5/site-packages/traitlets/config/application.py", line 595, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "//anaconda/lib/python3.5/site-packages/traitlets/config/application.py", line 74, in catch_config_error
return method(app, *args, **kwargs)
File "//anaconda/lib/python3.5/site-packages/notebook/notebookapp.py", line 1069, in initialize
self.init_configurables()
File "//anaconda/lib/python3.5/site-packages/notebook/notebookapp.py", line 837, in init_configurables
parent=self,
File "//anaconda/lib/python3.5/site-packages/nb_conda_kernels/manager.py", line 19, in __init__
specs = self.find_kernel_specs() or {}
File "//anaconda/lib/python3.5/site-packages/nb_conda_kernels/manager.py", line 129, in find_kernel_specs
self.conda_info = self._conda_info()
File "//anaconda/lib/python3.5/site-packages/nb_conda_kernels/manager.py", line 29, in _conda_info
p = subprocess.check_output(["conda", "info", "--json"]
File "//anaconda/lib/python3.5/subprocess.py", line 626, in check_output
**kwargs).stdout
File "//anaconda/lib/python3.5/subprocess.py", line 693, in run
with Popen(*popenargs, **kwargs) as process:
File "//anaconda/lib/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "//anaconda/lib/python3.5/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'conda'
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Other apps like spyder do run successfully.
Can I solve it somehow?
Found a way to run jupyter notebook through sage:
Just run from the terminal: ./sage -n jupyter

IPython installation (Mac OS X 10.9.4) error requires pyzmq

I have successfully installed IPython and its dependencies, it works in command line perfectly, but it doesn't launch as "ipython notebook" in browser. The error it gives:
Traceback (most recent call last):
File "/usr/local/bin/ipython", line 5, in <module>
start_ipython()
File "/Library/Python/2.7/site-packages/IPython/__init__.py", line 120, in start_ipython
return launch_new_instance(argv=argv, **kwargs)
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 563, in launch_instance
app.initialize(argv)
File "<string>", line 2, in initialize
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 92, in catch_config_error
return method(app, *args, **kwargs)
File "/Library/Python/2.7/site-packages/IPython/terminal/ipapp.py", line 321, in initialize
super(TerminalIPythonApp, self).initialize(argv)
File "<string>", line 2, in initialize
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 92, in catch_config_error
return method(app, *args, **kwargs)
File "/Library/Python/2.7/site-packages/IPython/core/application.py", line 381, in initialize
self.parse_command_line(argv)
File "/Library/Python/2.7/site-packages/IPython/terminal/ipapp.py", line 316, in parse_command_line
return super(TerminalIPythonApp, self).parse_command_line(argv)
File "<string>", line 2, in parse_command_line
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 92, in catch_config_error
return method(app, *args, **kwargs)
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 475, in parse_command_line
return self.initialize_subcommand(subc, subargv)
File "<string>", line 2, in initialize_subcommand
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 92, in catch_config_error
return method(app, *args, **kwargs)
File "/Library/Python/2.7/site-packages/IPython/config/application.py", line 406, in initialize_subcommand
subapp = import_item(subapp)
File "/Library/Python/2.7/site-packages/IPython/utils/importstring.py", line 42, in import_item
module = __import__(package, fromlist=[obj])
File "/Library/Python/2.7/site-packages/IPython/html/notebookapp.py", line 39, in <module>
check_for_zmq('2.1.11', 'IPython.html')
File "/Library/Python/2.7/site-packages/IPython/utils/zmqrelated.py", line 37, in check_for_zmq
raise ImportError("%s requires pyzmq >= %s"%(required_by, minimum_version))
ImportError: IPython.html requires pyzmq >= 2.1.11
I have tried upgrading the packages. I also read that the problem is probably that IPython is installed in a different environment from its dependencies or a path is broken. Since I'm totally new to Mac, I have no idea how to correct that. Any help or any alternative solution on how to make the IPython Notebook work would be much appreciated!
try
easy_install --upgrade IPython
easy_install --upgrade pyzmq
After hours of searching and troubleshooting, I installed ipython notebook on Mac OS X Yosemite (10.10).
To avoid any permission related errors, download and install virtualenv with your python first.
Then continue as follows
$ virtualenv venv
$ source venv/bin/activate
(venv)[user#host ]$ pip install ipython[all]
Successfully installed ...
(venv)[user#host ]$ ipython notebook
would bring up the notebook with all the modules installed. When you're done with your environment, you can easily $deactivate it.

Resources