Run Jupyter Notebook in the Background on Docker - terminal

I am trying to run a jupyter notebook in the background without printing anything to the console. I found this solution in a question for bash:
jupyter notebook &> /dev/null &
But I am running jupyter in a docker container and want it to start in the background via CMD. How can I do the same in sh?

I got it to work using the setup from: https://github.com/jupyter/docker-stacks/tree/master/minimal-notebook
the trick was to install tini and put the following code into a start-notebook.sh script:
#!/bin/bash
exec jupyter notebook &> /dev/null &
this is than added to the path with:
COPY start-notebook.sh /usr/local/bin/ and
RUN chmod +x /usr/local/bin/start-notebook.sh
Then I could set CMD ["start-notebook.sh"] to start up the container with jupyter running in the background on start.

You can do that, executing the below command
jupyter notebook --allow-root &> /dev/null &
You might see the warning that jupyter command needs --allow-root option if you execute jupyter notebook command as a root in a docker container.

Related

bash script to run jupyter notebook in virtualenv

To speed up launching projects I created a small bash script which does the following:
takes an argument (project name)
moves to the directory of that project
starts a virtual environment
starts a jupyter notebook
#!/bin/bash
if [ "$1" == "k3" ]; then
project_path="tau-code/k3-analysis/"
fi
codepath="/media/peter/somedrive/code"
full_path="$codepath/$project_path"
# Go to directory of project
cd $full_path
# Start environment & notebook if available
pipenv shell
jupyter notebook --ip=0.0.0.0
It activates the environment, but does not run the jupyter command. When I exit the environment I see the error:
line 16: jupyter: command not found
I can manually type jupyter notebook --ip=0.0.0.0 in my newly created environment and that does work.
What may be the problem?
pipenv shell starts a new shell which must be deactivated by using exit. In your script any commands following the call to pipenv shell are not executed in that new shell. Instead they are executed in the same bash shell after the virtual environment shell is closed. You should use pipenv run jupyter notebook --ip=0.0.0.0
See pipenv documentation:
shell will spawn a shell with the virtualenv activated. This shell can be deactivated by using exit.
run will run a given command from the virtualenv, with any arguments forwarded (e.g. $ pipenv run python or $ pipenv run pip freeze).
Hi you need to add this
pipenv run jupyter notebook

Opening files in VS Code in docker bash ubuntu

I am trying to open a file inside docker bash (I am using ubuntu terminal btw) using the ff commands:
sudo docker container exec -it rubybox bash
mkdir rubyfiles
cd rubyfiles
touch Dockerfile
And now when I run:
code .
It's suppose to open the file in Visual Studio code but instead I got this error:
bash: code: command not found
Any idea what's the correct command or how to configure this in order to open a file inside docker or terminal after running code .?
You cannot open a GUI app with Docker. You need to share your screen with it.
See: https://medium.com/#SaravSun/running-gui-applications-inside-docker-containers-83d65c0db110
What you could do is to share the local PWD with Docker and work from here:
docker run -d -v $PWD:/t -w /t rubybox

Run command from .bashrc in detached docker container

I'm running a docker container in detached mode, and I'm having difficulty running a command that's in the container's ~/.bashrc.
Inside the docker container's ~/.bashrc:
labserve () {
jupyter lab --ip=0.0.0.0 --allow-root --no-browser --NotebookApp.token=''
}
If I start the container with
$ docker run -d image -p 8888:8888 --name jupyterbox
I'd like to be able to tell the image to tell the detached container to open up jupyterlab with the following command:
$ docker exec jupyterbox bash -c labserve
however, when I do that, I get the error
bash: labinit: command not found
On the other hand, if I do
$ docker run -it jupyterbox bash
root#4e344655fc31:/home# labserve
The command runs correctly, and opens up a jupyterlab within my docker container running on port 8888.
How do I exec this command in a detached container?

docker error on windows : the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty' [duplicate]

This question already has answers here:
Error "The input device is not a TTY"
(16 answers)
Closed 2 years ago.
After I run this
docker run --rm -v "/c/users/vipul rao/documents/github/wappalyzer:/opt/wappalyzer" -it wappalyzer/dev
I am getting the following error
the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty'
What should I use here? I am running Docker on Windows 8 in MINGW64.
As suggested by the error message you obtain, you should try to use winpty (which is installed by default with Git-Bash) and thus run:
winpty docker run --rm -v "/c/users/vipul rao/documents/github/wappalyzer:/opt/wappalyzer" -it wappalyzer/dev
If this works, you may want to set a Bash alias to avoid manually prepending winpty all the time:
echo "alias docker='winpty docker'" >> ~/.bashrc
or
echo "alias docker='winpty docker'" >> ~/.bash_profile
If you are using Git Bash you can try like this
winpty docker run -it ubuntu
This problem occurs when running with -it option using bash terminal on windows. You can use Powershell to resolve this issue.
This works for me.
I am using git bash on windows
winpty docker-compose exec app ls -l
Remove -it from the command. If you want to keep it interactive then keep -i
Don't use alias docker="winpty docker". It solves your problem but break pipes.
$ winpty docker run -ti ubuntu
root#e85cff7d1670:/# exit
$ wintpy docker run ubuntu bash HELLO
HELLO
$ wintpy docker run ubuntu bash HELLO | cat
stdout is not a tty
Copy this to your .bashrc. This script uses winpty docker only if -ti is used.
function docker(){
for param; do if [[ "$param" == "-ti" ]] || [[ "$param" == "-it" ]]; then
winpty docker "$#"; return
fi; done;
command docker "$#"
}
docker run -ti ubuntu becomes winpty docker run -ti ubuntu avoids error: the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty'"
docker run ubuntu echo "what's up" | cat becomes command docker run echo "what'up" | cat avoids error: stdout is not a tty
The script only looks if there is a '-it' parameter without checking if it is inside a 'docker run' sentence... but it does the trick for my uses.
Did you start "Docker Quickstart Terminal"? I was trying to run
$ docker run -i -t redcricket/react-tutorial:latest /bin/bash
on windows from a Cygwin bash shell and got the same error:
the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty'
Then I remembered that when I installed docker on my windows 10 system something called "Docker Quickstart Terminal" got installed. You need to start that first from that dumb windows 'Type here to search' thing on the task bar:
That launches this …
… you can run your docker commands there without getting that error or running winpty.
The problem is with gitbash however with the powershell it is working fine ..
Happened to me. From Git Bash, on Windows 8 running Docker Toolbox. There are two things happening. From git bash, we do not seem to have complete escalated privilege to the docker daemon (even though i'm running git bash with administrative privileges).
Thus:
Try running the command from your docker terminal. (gives you privilege).
To compensate for errors from Window's folder naming formats, don't forget to quote the path.. (to escape spaces and/or capitalization errors) say
From:
docker run -v $(pwd):/data image_ref
To:
docker run -v "$(pwd):/data" image_ref
(notice the enclosing quotes in the latter around $(pwd):/data).
Just add 'winpty' in start of your cmd ,Try below:
$ winpty docker.exe run --rm -v "/c/users/vipul rao/documents/github/wappalyzer:/opt/wappalyzer" -it wappalyzer/dev
Why this happens? More details here:
http://willi.am/blog/2016/08/08/docker-for-windows-interactive-sessions-in-mintty-git-bash/
Got this error for running docker-compose exec workspace bash
So just prefix with winpty winpty docker-compose exec workspace bash
It may be that you're not running your commands within the Docker terminal. If you don't, you may not be properly connected to the Docker daemon and won't be able to interact correctly.
Make sure you're running commands in the actual Docker Terminal.
For those using WSL and running Docker for windows inside of cmder or conemu I would recommend to not to use Docker which is installed on windows in 'Program Files' but instead install Docker inside WSL on ubuntu/linux. Do remember though that you can't run Docker itself from within WSL, you must connect to Docker running on windows from the linux Docker client installed in WSL.
To install Docker on WSL
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
Your options for running actual Docker commands are to either:
Connect to Docker using the switch -H
docker -H localhost:2375 run -it -v /mnt/c/code:/var/app -w "/var/app" centos:7
Or set the environment variable docker_host
export DOCKER_HOST=tcp://localhost:2375
Either way you are now be able to interactively connect to a running Docker container
you can try with Cmder tool it will work. Its not working with Gitbash
In addition to above mentioned solutions.
In case you are getting this error for docker attach
example: docker attach alpine1
error: the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty'
Solution: Adding winpty before docker command i.e. winpty docker attach should work.
example: winpty docker attach alpine1
Note: I was getting this error while using base on windows and this solution worked for me.
I had the same error when trying to run the docker-compose exec command.
In the help documentation docker-compose exec --help it shows how you can disable the pseudo-tty allocation by adding -T to your command options in the following way:
docker-compose exec -T
From the help documentation:
-T Disable pseudo-tty allocation. By default docker-compose exec allocates a TTY.
If you are using gitbash the problem is when setting the terminal emulator for for using with Git bash.setting the emurator
Instead you can change the emulator to the first options or use the
winpty command before your docker run command

Commands to execute background process in Docker CMD

I am creating a docker image using a Dockerfile. I would like to execute some scripts while starting the docker container. Currently I have a shell script to execute all the necessary processes
CMD ["sh","start.sh"]
I would like to execute a shell command with a process running in background example
CMD ["sh", "-c", "mongod --dbpath /test &"]
Besides the comments on your question that already pointed out a few things about Docker best practices you could anyway start a background process from within your start.sh script and keep that start.sh script itself in foreground using the nohup command and the ampersand (&). I did not try it with mongod but something like the following in your start.sh script could work:
#!/bin/sh
...
nohup sh -c mongod --dbpath /test &
...
Of course there is also the official Docker documentation of how to start multiple services, again using a script file not the CMD. The docker documentation also states how to use supervisord as a process manager:
FROM ubuntu:latest
RUN apt-get update && apt-get install -y supervisor
RUN mkdir -p /var/log/supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY my_first_process my_first_process
COPY my_second_process my_second_process
CMD ["/usr/bin/supervisord"]
If it is an option you could use a phusion base images which allows running multiple processes in one container. Thus you can run system services such as cron or other processes using a service supervisor like runit.
More information about whether or not a phusion base image is a good choice in your use case can be found here
A ruby focused description of how to avoid running more processes in your container except for your app you can find here. The elaborations are too detailed to repeat on SO.

Resources