I am looking to allow users on a cluster to set up their own conda environments.
However, across all users, I want to force them to use a password to log in to Jupyter, regardless of whether or not they are launching from their own conda environment or from the root environment. I know I can globally set conda parameters by modifying the .condarc file in the root Anaconda directory. Is there an equivalent file for globally requiring jupyter password use/setting jupyter configs?
You do not want users to launch their own notebook, if you have a case with multiple users using notebooks you should use JupyterHub. JupyterHub is your solution; JupyterHub has been developed for that, and not using JupyterHub will lead to the same pains a Parsing HTML with RegEx.
JupyterHub works with Containers, if you use container, but works also if you only have conda envs, or just system Python. If you have centralized auth (OAuth/LDAP) JupyterHub hook into that, if you prefer PAM, PAM work. If you have SLURM/MPI, JupyterHub hook into that. If you need to use SSH, JupyterHub can undersand it. There is a full org on GitHub conveniently also named JupyterHub of plugins for JupyterHub to bend it to your will and needs... and if you have a specific case note covered, open a bug report or write a plugin that cover yours, and maybe later it can be added to the org !
So I believe JupyterHub is what you are looking for.
And also, you shouldn't install the notebook in each environment. The notebook server should only be installed once system wide (or once per users sometime). You should only install ipykernel in each env.
Related
Before my hdd got destroyed I was working on this virtual environment.
Where I have installed all the necessary packages like python, jupyter notebook. I used to run jupyter notebook using terminal. Then I replaced my hdd and copied the whole virtual environment. But now its saying permission denied.
There was an article explaining you need to use chmod u+x appname but it is not working
Previously I used to write in terminal like.
source virtualenvironment/bin/activate
jupyter-notebook
After entering jupyter-notebook it should run the notebook
Simply copying and pasting will not work as there may be paths that are not being set. So you simply need to create virtual environment again with same name. Then inside the virtual environment you need to install the same packages. Note that all the requirements must be fulfilled but this time the paths are also set. So running packages now will work fine.
I am trying to install gcloud SDK while using a coporate network which works behind a proxy (and a VPN sometimes). I get proxy settings automatically using a pac.fcgi file (Automatic Proxy Configuration).
Now when I try to run ./install.sh in the terminal, I am constantly getting the following error
➜ google-cloud-sdk ./install.sh
Welcome to the Google Cloud SDK!
To help improve the quality of this product, we collect anonymized usage data
and anonymized stacktraces when crashes are encountered; additional information
is available at <https://cloud.google.com/sdk/usage-statistics>. You may choose
to opt out of this collection now (by choosing 'N' at the below prompt), or at
any time in the future by running the following command:
gcloud config set disable_usage_reporting true
Do you want to help improve the Google Cloud SDK (Y/n)?
ERROR: (gcloud.components.list) Failed to fetch component listing from server. Check your network settings and try again.
I found this SO question which was having the same issue, but their problem was related to ipv6. I already have ipv6 disabled (I can't even enable it) and I am using ethernet to connect to the network. Thus the answer is not useful to me.
I searched for proxy related info about installation and found this page. It asks for using a non-interactive installer instead and using gcloud command to set the proxy after installation. My problem is that I can't even install gcloud. I have tried with both interactive and non-interactive installers.
Any way I can install gcloud while being behind a corporate proxy.
PS: I am on macOS High Sierra and using zsh shell (already tried bash), in case it matters.
So, the problem was I was unable to set proxy before installation of gcloud. The installation of gcloud basically set the CLI in the path and installs some required components (core, bq, gsutil etc).
So what I did is I added the following lines in my .zshrc
source <PATH to gcloud sdk>/google-cloud-sdk/path.zsh.inc
source <PATH to gcloud sdk>/google-cloud-sdk/completion.zsh.inc
or if you're using bash, add these lines in your .bashrc or .bash_profile
source <PATH to gcloud sdk>/google-cloud-sdk/path.bash.inc
source <PATH to gcloud sdk>/google-cloud-sdk/completion.bash.inc
After that I restarted the terminal (or just run source ~/.zshrc or source ~/.bashrc or source ~/.bash_profile on terminal)
Now I was able to use gcloud commands. I ran gcloud init, set the proxy and then installed the required components using the following command -
gcloud components install core bq gsutil
And I installed gcloud without using the install.sh script.
PS: Run scutil --proxy to know the proxy settings set in your system.
For Windows 10 users who experience this issue. I installed the google SDK from the archived versions page. then used git bash to install using ./google-cloud-sdk/install.sh. then added in environment variable manually to PATH to use commands.
I am in the process of installing jupyterhub. I successfully install jupyterhub using:
python3 -m pip install jupyterhub
npm install -g configurable-http-proxy
However, when I run jupyterhub -h in the Windows command prompt it gives:
"jupyterhub" is not recognized as an internal or external command, operable
program or batch file.
I added C:\Users\User\AppData\Local\Continuum\Anaconda3\Lib\site-packages\jupyterhub\ to my user environment variable, however still receive the message. What path should I be using?
Please note that according to this, Jupyterhub is not officially supported for Windows yet.
That aside, you could dockerize it to make your life easier. For this error, please check if you can see the executable in C:\Program Files\Continuum Analytics\Anaconda3\scripts. The lib directory you're specifying contains python source files and not the executables.
I had this same issue, and I saw this occurred because jupyterhub is a python script rather than an executable. So to run this on Windows I needed to execute it like python C:\Program Files\Continuum Analytics\Anaconda3\scripts\jupyterhub.
However, I still was unable to run jupyterhub on Windows because it depends on the pwd module, which is a Unix/Linux only module.
As others have said, Windows is not a supported platform. JupyterHub is best used on Linux-like platforms where you have Docker or something similar to conatainerize each user's session.
A good alternative is to install Oracle VirtualBox and run a local VM. I run a 64-bit Ubuntu and it's quite good performance. It makes things much easier to run JupyterHub on. Asides depending on pwd, there are also assumptions around user-creation and other activities that Windows isn't going to handle well.
In short, if you want to run on native Windows, you're going to become the first JupyterHub Windows contributor. I looked at doing it but it looked like too much effort.
The upside of running a VM is that behaviour in the VM is going to more closely resemble what you have running on the server anyway. If you don't plan running on a server, then just "jupyter notebook", as this is all JupyterHub ends up running...
Can the anaconda tools that manipulate environments use a search path when looking for environments?
I would like to have a single anaconda installation shared among our group, with a shared set of read-only environments and each user also having their own set of writable personal environments.
Commands like conda env list and conda info -e would have to iterate over an environment search path. So would activate.
Does conda support such an architecture?
I think the answer yes, the envs_dirs configuration parameter can be set in your $HOME/.configrc file (looks like yaml) to change the list of directories searched for environments. You can also set the conda environment search path via the CONDA_ENVS_PATH shell environment variable
https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#specify-environment-directories-envs-dirs
However the specific usage scenario outlined above is already handled without modifying any configuration parameters.
If the anaconda install directory is not writable, then conda searches both the shared environments and the environments under $HOME/.conda/envs.
If the anaconda installation directory is writable, the per-user environments in $HOME/envs are silently ignored.
I just want to spool up some micro-instances and install a few gems.
I recommend you just start an instance of a clean base AMI running the Linux distro of your preference from a trusted source. Then run the standard commands (e.g., apt-get, yum) to install the standard Ruby provided by the distro and go from there.
You don't need to find an AMI that has exactly the set of packages you want to run. Modern Linux makes it easy to install these cleanly.
With AMIs like the official Ubuntu or Amazon Linux you can even pass in a script as user-data and it will run it on first boot. You can use this to make sure that your desired packages and gems are installed when the instance runs.