Essentially, I have a standard format for file naming conventions. It breaks down to this:
target_dateUTC_timeUTC_tool
So, for instance, if I run tcpdump on a target of 'foo', then the file would be foo_dateUTC_timeUTC_tcpdump. Simple enough, but a pain for everyone to constantly (and consistently) enter... so I've tried to create a bash script which sets system variables like so:
FILENAME=$TARGET\_$UTCTIME\_$TOOL
Then, I can just call the variable at runtime, like so:
tcpdump -w $FILENAME.lpc
All of this works like a champ. I've got a menu-driven .sh which gives the user the options of viewing the current variables as well as setting them... file generation is a breeze. Unfortunately, by setting the date/time variable, it is locked to the value at the time of creation (naturally). I set the variable like so:
UTCTIME=$(/bin/date --utc +"%Y%m%d_%H%M%Z")
What I really need is either a way to create a variable which updates at runtime, or (more likely) another way to skin this cat.
While scouring for solutions, I came across a similar issues... like this.
But, to be honest, I'm stumped on how to marry the two approaches and create a simple, distributable solution.
.sh file is posted via pastebin, here.
Use a function:
generate_filename() { echo "${1}_$(/bin/date --utc +"%Y%m%d_%H%M%Z")_$2"; }
And use it like this:
tcpdump -w "$(generate_filename foo tcpdump).lpc"
It's hard to get the function to automatically determine the command name. You can use bash history to get it and save a couple of characters typing:
tcpdump -w "$(generate_filename foo !#:0).lpc"
Related
Problem:
I'm using Terminator but I think this question is relevant to any terminal.
To be more precise let me please explain my problem on concrete example.
I'm running Android cts tests, so I need to use cts-tradef script. This is how it looks like:
The script just runs jar (which I don't want to modify), however I would like to change color of cts-tf, so it looks like on a picture below:
My unsuccessful attempts to solve the problem:
For now I've tried doing something like this:
echo -e "\033[01;32m" && ./cts-tradefed
However it will color everything (as on below picture) while I want to color only cts-tf string (as above):
I also tried using sed, however although it works and replace some strings it also finishes cts-tradefed, so it's useless to me. Same thing (cts-tradefed finishes) happens when piping it through grep:
./cts-tradefed | grep --color "cts-tf\|$"
Another try was with grc tool mentioned by Anthony Geoghegan. Running it without config file doesn't do anything but cts-tradefed doesn't finish but when I run it with config file cts-tradefed finishes the same as with grep or sed. My config file was ok, since it works good with cat or similar commands.
I haven’t used it myself but Radovan Garabík’s Generic Colouriser looks like it should do what you want. It’s written in Python “for beautifying your logfiles or output of commands”.
Packages are available for Debian-based operating systems but I imagine it should not be too hard to install if you are familiar with Python.
GitHub repository
This question has been posted here many times, but it never seems to answer my question.
I have two scripts. The first one contains one or multiple variables, the second script needs those variables. The second script also needs to be able to change the variables in the first script.
I'm not interested in sourcing (where the first script containing the variables runs the second script) or exporting (using environment variables). I just simply want to make sure that the second script can read and change (get and set) the variables available in the first script.
(PS. If I misunderstood how sourcing or exporting works, and it applies to my scenario, please let me know. I'm not completely closed to those methods, after what I've read, I just don't think those things will do what I want)
Environment variables are per process. One process can not modify the variables in another. What you're asking for is not possible.
The usual workaround for scripts is sourcing, which works by running both scripts in the same shell process, but you say you don't want to do that.
I've also given this some thought. I would use files as variables. For example in script 1 you use for writing variable values to files:
echo $varnum1 > /home/username/scriptdir/vars/varnum1
echo $varnum2 > /home/username/scriptdir/vars/varnum2
And in script 2 you use for reading values from files back into variables:
$varnum1=$(cat /home/username/scriptdir/vars/varnum1)
$varnum2=$(cat /home/username/scriptdir/vars/varnum2)
Both scripts can read or write to the variables at any given time. Theoretically two scripts can try to access the same file at the same time, I'm not sure what exactly would happen but since each file only contains one value, the time to read or write should be extremely short.
In order to even reduce those times you can use a ramdisk.
I think this is much better than scripts editing each other (yuk!). Live editing of scripts can mess up scripts and only works when you initiate the script again after the edit was made.
Good luck!
So after a long search on the web and a lot of trying, I finally found some kind of a solution. Actually, it's quite simple.
There are some prerequisites though.
The variable you want to set already has to exist in the file you're trying to set it in (I'm guessing the variable can be created as well when it doesn't exist yet, but that's not what I'm going for here).
The file you're trying to set the variable in has to exist (obviously. I'm guessing again this can be done as well, but again, not what I'm going for).
Write
sudo sed -i 's/^\(VARNAME=\).*/\1VALUE/' FILENAME
So i.e. setting the variable called Var1 to the value 5, in the file
test.ini:
sudo sed -i 's/^\(Var1=\).*/\15/' test.ini
Read
sudo grep -Po '(?<=VARNAME=).*' FILENAME
So i.e. reading the variable called Var1 from the file test.ini
sudo grep -Po '(?<=Var1=).*' test.ini
Just to be sure
I've noticed some issues when running the script that sets variables from a different folder than the one where your script is located.
To make sure this always go right, you can do one of two things:
sudo sed -i 's/^\(VARNAME=\).*/\1VALUE/' `dirname $0`/FILENAME
So basically, just put `dirname $0`/ (including the backticks) in front of the filename.
The other option is to make `dirname $0`/ a variable (again including the backticks), which would look like this.
my_dir=`dirname $0`
sudo sed -i 's/^\(VARNAME=\).*/\1VALUE/' $my_dir/FILENAME
So basically, if you've got a file named test.ini, which contains this line: Var1= (In my tests, the variable can start empty, and you will still be able to set it. Mileage may vary.), you will be able to set and get the value for Var1
I can confirm that this works (for me), but since you all, with way more experience in scripting then me, didn't come up with this, I'm guessing this is not a great way to do it.
Also, I couldn't tell you the first thing about what's happening in those commands above, I only know they work.
So if I'm doing something stupid, or if you can explain to me what's happening in the commands above, please let me know. I'm very curious to find out what you guys think if this solution.
Okay I know there is a bash debugger. But what I'm seeking is if I had an environment variable in one of my startup scripts and I don't know how it was set or where it might be, is there a way to find it other than exhaustively searching the scripts?
I mean is there a mechanism/tool that provides such a thing? Does bash keep track of variable setting locations?
Even though this might not seem very important but it crossed my mind the other day when I was helping a friend install OpenCL and the package supposedly set the variable $ATISTREAMSDKROOT automatically. Anyway the package was supposed to add a file to /etc/profile.d to allow for setting the variable, but it didn't. And luckily the variable came out blank.
But I was wondering if it hadn't come out blank, and the package added it to some random file, I would have probably had no way of telling where it is other than looking for it.
Of course I know one could write a sed command or two and search through the scripts but I'd consider that exhaustive search :D
One option would be to start an instance of bash with:
bash -x
... and look for where the variable is set in that output. To redirect that output to a file, you could do:
bash -x -ls -c "exit" 2> shell-startup-output
You should see in the output where each file is sourced.
While playing with Heroku, I found their approach of using environment variables for server-local configuration brilliant. Now, while setting up an application server of my own, I find myself wondering how hard that would be to replicate.
I'm deploying a sinatra application, riding Unicorn and Nginx. I know nginx doesn't like to play with the environment, so that one's out. I can probably put the vars somewhere in the unicorn config file, but since that's under version control with the rest of the app, it sort of defeats the purpose of having the configuration sit in the server environment. There is no reason not to keep my app-specific configuration files together with the rest of the app, as far as I'm concerned.
The third, and last (to my knowledge) option, is setting them in the spawning shell. That's where I got lost. I know that login and non-login shells use different rc files, and I'm not sure whether calling something with sudo -u http stuff is or not spawning a login shell. I did some homework, and asked google and man, but I'm still not entirely sure on how to approach it. Maybe I'm just being dumb... either way, I'd really appreciate it if someone could shed some light on the whole shell environment deal.
I think your third possibility is on the right track. What you're missing is the idea of a wrapper script, whose only function is to set the environment and then call the main program with whatever options are required.
To make a wrapper script that can function as a control script (if prodEnv use DB=ProdDB, etc), there is one more piece that simplifies this problem. Bash/ksh both support a feature called sourcing files. This an operation that the shell provides, to open a file and execute what is in the file, just as if it was in-lined in the main script. Like #include in C and other languages.
ksh and bash will automatically source /etc/profile, /var/etc/profile.local (sometimes), $HOME/.profile. There are other filenames that will also get picked up, but in this case, you'll need to make your own env file and the explicitly load it.
As we're talking about wrapper-scripts, and you want to manage how your environment gets set up, you'll want to do the sourcing inside the wrapper script.
How do you source an environment file?
envFile=/path/to/my/envFile
. $envFile
where envFile will be filled with statements like
dbServer=DevDBServer
webServer=QAWebServer
....
you may discover that you need to export these variable for them to be visble
export dbServer webServer
An alternate assignment/export is supported
export dbServer=DevDBServer
export webServer=QAWebServer
Depending on how non-identical your different environments are, you can have your wrapper script figure out which environment file to load.
case $( /bin/hostame ) in
prodServerName )
envFile=/path/2/prod/envFile ;;
QASeverName )
envFile=/path/2/qa/envFile ;;
devSeverName )
envFile=/path/2/dev/envFile ;;
esac
. ${envFile}
#NOW call your program
myProgram -v -f inFile -o outFile ......
As you develop more and more scripts in your data processing environment, you can alway source your envFile at the top. When you eventually change the physical location of a server (or it's name), then you have only one place that you need to make the change.
IHTH
Also a couple of gems dealing with this. figaro that works both with or without heroku. Figaro uses a yaml file (in config and git ignored) to keep track of variables. Another option is dotenv that reads variables from an .env file. And also another article with all them options.
To spawn an interactive shell (a.k.a. login shell) you need to invoke sudo like this:
sudo -i -u <user> <command>
Also you may use -E to preserve the environment. This will allow some variables to be pased for your current environment to the command invoked with sudo.
I solved a similar problem by explicitly telling Unicorn to read a variables file as part of startup in its init.d script. First I created a file in a directory above the application root called variables. In this script I call export on all my environment variables, e.g. export VAR=value. Then I defined a variable GET_VARS=source /path/to/variables in the /etc/init.d/unicorn file. Finally, I modified the start option to read su - $USER -c "$GET_VARS && $CMD" where $CMD is the startup command and $USER is the app user. Thus, the variables defined in the file are exported into the shell of Unicorn's app user on startup. Note that I used an init.d script almost identical to the one from this article.
Writing a bash script, and I want to get user input. Awesome,
read -p "What directory should we save in? " -e FOLDER
Except that what I'd like to do, ideally, is have the user see something like:
What directory should we save in? /home/user/default/
with the cursor at the end of the line, and the ability to delete backwards or append or whatever. Essentially, pre-filling the user's input, but giving them the ability to edit it.
Readline obviously has the capability, but it appears to be not exposed in the read command. Any alternatives? I'd prefer to not have to use perl or such.
The constraint I'm working under is that I'm writing a single shell script that would be nice to disseminate widely, so should rely on as little pre-existing infrastructure as possible. rlwrap and read -i both work if their dependencies (rlwrap and bash version >> whatever I have, respectively) are available. Both good answers, choose whichever works for you.
$ read -p "What directory should we save in? " -i "/home/user/default/" -e FOLDER
What directory should we save in? /home/user/default/
that should work, right?
You can wrap the command in rlwrap, which provides instant readline capabilities: https://github.com/hanslub42/rlwrap
(rlwrap -P does what you want)
As far as a pure bash solution is concerned for the 3.2 line (which i am presuming you are using), I dont think its possible