I created an singularity image with my script. I need to run my app using a local file with parameters for the script. for this I want o use --bind parameter. It works perfectly fine if I try to do:
singularity run --bind /my/source/file:/destination/file my_app_image.img <input.txt >output.txt
But I would like to run it directly like this:
./my_app_image.img --bind /my/source/file:/destination/file <input.txt >output.txt
but this does not work - file is not mounted in the destination. How can I mount a file during runtime?
Although I worked for some time with singularity containers, I don't know if this is possible.
If you have to call an executable in your use case, you could write a short wrapper script that calls singularity run ... and binds the file into the container.
You can also ask your question on the Singularity user list.
P.S. I wanted to comment, but do not have enough reputation.
you can export your bind directory/file before a call the container like an executable.
export SINGULARITY_BINDPATH="/my/source/file:/destination/file"
./my_app_image.img <input.txt >output.txt
Related
Problem:
I'm using Terminator but I think this question is relevant to any terminal.
To be more precise let me please explain my problem on concrete example.
I'm running Android cts tests, so I need to use cts-tradef script. This is how it looks like:
The script just runs jar (which I don't want to modify), however I would like to change color of cts-tf, so it looks like on a picture below:
My unsuccessful attempts to solve the problem:
For now I've tried doing something like this:
echo -e "\033[01;32m" && ./cts-tradefed
However it will color everything (as on below picture) while I want to color only cts-tf string (as above):
I also tried using sed, however although it works and replace some strings it also finishes cts-tradefed, so it's useless to me. Same thing (cts-tradefed finishes) happens when piping it through grep:
./cts-tradefed | grep --color "cts-tf\|$"
Another try was with grc tool mentioned by Anthony Geoghegan. Running it without config file doesn't do anything but cts-tradefed doesn't finish but when I run it with config file cts-tradefed finishes the same as with grep or sed. My config file was ok, since it works good with cat or similar commands.
I haven’t used it myself but Radovan Garabík’s Generic Colouriser looks like it should do what you want. It’s written in Python “for beautifying your logfiles or output of commands”.
Packages are available for Debian-based operating systems but I imagine it should not be too hard to install if you are familiar with Python.
GitHub repository
I'm learning about Docker at the moment, and going through the Dockerfile reference, specifically the RUN instruction. There are two forms of RUN - the shell form, which runs the command in a shell, and the exec form, which "does not invoke a command shell" (quoted from the Note section).
If I understood the documentation correctly, my question is - If, and how can, Docker run a command without a shell?
Note that the answers to Can a command be executed without a shell?'s don't actually answer the question.
If I understand your question correctly, you're asking how something can be run (specifically in the context of docker) without invoking a command shell.
The way things are run in the linux kernel are usually using the exec family of system calls.
You pass it the path to the executable you want to run and the arguments that need to be passed to it via an execl call for example.
This is actually what your shell (sh, bash, ksh, zsh) does under the hood anyway. You can observe this yourself if you run something like strace -f bash -c "cat /tmp/foo"
In the output of that command you'll see something like this:
execve("/bin/cat", ["cat", "/tmp/foo"], [/* 66 vars */]) = 0
What's really going on is that bash looks up cat in $PATH, it then finds that cat is actually an executable binary available at /bin/cat. It then simply invokes it via execve. and the correct arguments as you can see above.
You can trivially write a C program that does the same thing as well.
This is what such a program would look like:
#include<unistd.h>
int main() {
execl("/bin/cat", "/bin/cat", "/tmp/foo", (char *)NULL);
return 0;
}
Every language provides its own way of interfacing with these system calls. C does, Python does and Go, which is what's used to write Docker for the most part, does as well. A RUN instruction in the docker likely translates to one of these exec calls when you hit docker build. You can run an strace -f docker build and then grep for exec calls in the log to see how the magic happens.
The only difference between running something via a shell and directly is that you lose out on all the fancy stuff your shell will do for you, such as variable expansion, executable searching etc.
A program can execute another program without a shell; you just create a new process and load an executable onto it, so you don't need the shell for that. The shell is needed for a user to start a program because it is the user interface to the system. Also, a program is not able to run a built-in command like cd or rm without a shell because there's no executable to be found (there are alternative ways, thought, but not as simple).
In very general - docker run will start container with its default process, when docker exec allow you to run any process you want inside the container.
For example, running microsoft/iis container by docker run microsoft/iiswill run default process, which is powershell.
But you can run cmd by running docker exec -i my_container cmd
See this answer for more details.
..in the following shell script?
$USER1$=/usr/lib/nagios/plugins
As far as i know variable defining is done as-
export USER1=/usr/lib/nagios/plugins
Source:
Ok, the command works. Now I have to implement it into Nagios. Because
all my "local" command not installed by the package-manager shall be
in /usr/lib/nagios/plugins_local I define a $USER2$ variable for this
path:
# vim resource.cfg
...
# Sets $USER1$ to be the path to the plugins
$USER1$=/usr/lib/nagios/plugins
# my own check-commands live here:
$USER2$=/usr/lib/nagios/plugins_local
$USERn$ (more specifically, $USER1 to $USER255$) is the way to declare a user-defined Macro in Nagios.
See also "Understanding Macros and how they work."
More specifically and more interestingly this is a good way to hide usernames/passwords needed withing database/http checks for instance.
This means that you can attempt soemthing like the following directly in your configuration files and thus you do not fear that you are committing or backing up usernames/passwords.
./nrpe -c check_http -H $IP -a $USER1$:$USER2$ -u $LINK
An aside: Unfortunetly Nagios only supports up to 32 of the USER variables.
I am using SpiderMonkey 1.8.5 on Debian.
I am starting a script through the command-line using
js -f <myScript>
I'd like to pass some arguments to my script, but I don't know how to do that. It's supposedto be possible since the documentation tells you about a special object gathring all parameters provided to a script.
I tried the following:
js -f <myScript> <1stArg>
But SpiderMonkey consider both parameters as different scripts to execute and thus sends en error saying the '<1stArg>' file doesn't exist.
What is the correct way to do what I wish?
Like this:
In script.js
#!/usr/bin/js
print(arguments);
Then from system shell:
$ chmod +x script.js
$ ./script.js
While playing with Heroku, I found their approach of using environment variables for server-local configuration brilliant. Now, while setting up an application server of my own, I find myself wondering how hard that would be to replicate.
I'm deploying a sinatra application, riding Unicorn and Nginx. I know nginx doesn't like to play with the environment, so that one's out. I can probably put the vars somewhere in the unicorn config file, but since that's under version control with the rest of the app, it sort of defeats the purpose of having the configuration sit in the server environment. There is no reason not to keep my app-specific configuration files together with the rest of the app, as far as I'm concerned.
The third, and last (to my knowledge) option, is setting them in the spawning shell. That's where I got lost. I know that login and non-login shells use different rc files, and I'm not sure whether calling something with sudo -u http stuff is or not spawning a login shell. I did some homework, and asked google and man, but I'm still not entirely sure on how to approach it. Maybe I'm just being dumb... either way, I'd really appreciate it if someone could shed some light on the whole shell environment deal.
I think your third possibility is on the right track. What you're missing is the idea of a wrapper script, whose only function is to set the environment and then call the main program with whatever options are required.
To make a wrapper script that can function as a control script (if prodEnv use DB=ProdDB, etc), there is one more piece that simplifies this problem. Bash/ksh both support a feature called sourcing files. This an operation that the shell provides, to open a file and execute what is in the file, just as if it was in-lined in the main script. Like #include in C and other languages.
ksh and bash will automatically source /etc/profile, /var/etc/profile.local (sometimes), $HOME/.profile. There are other filenames that will also get picked up, but in this case, you'll need to make your own env file and the explicitly load it.
As we're talking about wrapper-scripts, and you want to manage how your environment gets set up, you'll want to do the sourcing inside the wrapper script.
How do you source an environment file?
envFile=/path/to/my/envFile
. $envFile
where envFile will be filled with statements like
dbServer=DevDBServer
webServer=QAWebServer
....
you may discover that you need to export these variable for them to be visble
export dbServer webServer
An alternate assignment/export is supported
export dbServer=DevDBServer
export webServer=QAWebServer
Depending on how non-identical your different environments are, you can have your wrapper script figure out which environment file to load.
case $( /bin/hostame ) in
prodServerName )
envFile=/path/2/prod/envFile ;;
QASeverName )
envFile=/path/2/qa/envFile ;;
devSeverName )
envFile=/path/2/dev/envFile ;;
esac
. ${envFile}
#NOW call your program
myProgram -v -f inFile -o outFile ......
As you develop more and more scripts in your data processing environment, you can alway source your envFile at the top. When you eventually change the physical location of a server (or it's name), then you have only one place that you need to make the change.
IHTH
Also a couple of gems dealing with this. figaro that works both with or without heroku. Figaro uses a yaml file (in config and git ignored) to keep track of variables. Another option is dotenv that reads variables from an .env file. And also another article with all them options.
To spawn an interactive shell (a.k.a. login shell) you need to invoke sudo like this:
sudo -i -u <user> <command>
Also you may use -E to preserve the environment. This will allow some variables to be pased for your current environment to the command invoked with sudo.
I solved a similar problem by explicitly telling Unicorn to read a variables file as part of startup in its init.d script. First I created a file in a directory above the application root called variables. In this script I call export on all my environment variables, e.g. export VAR=value. Then I defined a variable GET_VARS=source /path/to/variables in the /etc/init.d/unicorn file. Finally, I modified the start option to read su - $USER -c "$GET_VARS && $CMD" where $CMD is the startup command and $USER is the app user. Thus, the variables defined in the file are exported into the shell of Unicorn's app user on startup. Note that I used an init.d script almost identical to the one from this article.