I need to access the folder that is created dynamically during each bot integration. On one of the run it is something like this -
/Library/Developer/XcodeServer/Integrations/Caches/a3c682dd0c4d569a3bc84e58eab88a48/DerivedData/Build/Products/Debug-iphonesimulator/my.app
I would like to get to this folder in an post trigger, how do I go about it? Based on the wwdc talk it seems like some environment variables like 'XCS_INTEGRATION_RESULT' and XCS_ERROR_COUNT etc.. are being used. Also I can see in logs something like PROJECT_DIR.
But I can't access any of these variables from my command line(is it because I am a different user than the bot?)
Also where can I find the list of variables created by this CI system?
I have been echoing set to the bot log, the first line of my bot script is simply
set
When you view the log after the integration is complete it will be in your trigger output.
XCS_ANALYZER_WARNING_CHANGE=0
XCS_ANALYZER_WARNING_COUNT=0
XCS_ARCHIVE=/Library/Developer/XcodeServer/Integrations/Integration-76eb5292bd7eff1bfe4160670c2d4576/Archive.xcarchive
XCS_BOT_ID=4f7c7e65532389e2a741d29758466c18
XCS_BOT_NAME='Reader'
XCS_BOT_TINY_ID=00B0A7D
XCS_ERROR_CHANGE=0
XCS_ERROR_COUNT=0
XCS_INTEGRATION_ID=76eb5292bd7eff1bfe4160670c2d4576
XCS_INTEGRATION_NUMBER=15
XCS_INTEGRATION_RESULT=warnings
XCS_INTEGRATION_TINY_ID=FF39BC2
XCS_OUTPUT_DIR=/Library/Developer/XcodeServer/Integrations/Integration-76eb5292bd7eff1bfe4160670c2d4576
XCS_PRODUCT='Reader.ipa'
XCS_SOURCE_DIR=/Library/Developer/XcodeServer/Integrations/Caches/4f7c7e65532389e2a741d29758466c18/Source
XCS_TESTS_CHANGE=0
XCS_TESTS_COUNT=0
XCS_TEST_FAILURE_CHANGE=0
XCS_TEST_FAILURE_COUNT=0
XCS_WARNING_CHANGE=36
XCS_WARNING_COUNT=36
#Viktor is correct, these variables only exist during their respective sessions. #Pappy gave a great list of those variables.
They can be used in a script like so:
IPA_PATH="${XCS_OUTPUT_DIR}/${XCS_BOT_NAME}.ipa"
echo $IPA_PATH
I'm not familiar with Xcode Server but generally Unix/CI systems when export environment variables they only export it to the current session.
If you want to set an environment variable persistently you have to set it in an initializer file like ~/.bash_profile or ~/.bashrc so it always gets set/loaded when a shell session starts (ex: when you log in with Terminal - the exact file depends on what kind of shell you start).
It wouldn't make much sense to export these persistently either, because in that case if you run different integrations these would simply overwrite each others exported environment variables (they would set the same environment variables).
That's why the systems which communicate through environment variables usually don't write the variables into persistent initialiser file rather just export the variables. With export the variable is accessible from the process which exports it and from the child processes the process starts.
For example in a bash script if you export a variable you can access it from the bash script after the export and from any command/program you start from the bash script, but when the bash script finishes the environment won't be accessible anymore.
edit
Just to clarify it a bit: You should be able to access these environment variables from a post trigger script, run by Xcode Server but you most likely won't be able to access these from your Terminal/command line.
Also where can I find the list of variables created by this CI system?
You can print all the available environment variables with the env command. In a bash script simply type env in a new line like this:
#!/bin/bash
env
This will print all the available environment variables (not just the ones defined by Xcode Server!) - you can simply pipe it to a file for inspection if you want to, like this:
#!/bin/bash
env > $HOME/envinspect.txt
After this script runs you can simply open the envinspect.txt file in the user's home folder.
Related
I am trying to use fish as my shell. When I login with LightDM, I want to start certain Xsession apps, but only when the shell in invoked at the outset by LightDM.
I have tried this in ~/.config/fish/config.fish:
###################################################################
# Start xsession applications, but only once.
if test -z "$XSESSION_STARTED"
set -xg XSESSION_STARTED 'f'
end
if test "$XSESSION_STARTED" = 'f'
xsession-apps
end
The function xsession-apps then starts all the apps in the background and sets the environment variable at the end like this:
set -xg XSESSION_STARTED "t"
But XSESSION_STARTED does not appear to get set to 't' and causes the xsession-apps function to get called every time, even when I start a new terminal within gnome-term.
What am I missing. Is there a better way to approach this?
even when I start a new terminal within gnome-term.
That is to be expected. Global variables are set within that particular fish. If you start another fish, it won't have it (unless you start it inside that fish, because the variable is exported).
There's a few ways to approach this:
Don't do it in config.fish at all - use the DE's autostart mechanism or at least ~/.xinitrc. This is the best and cleanest approach, and independent of your shell.
Use universal variables - these are stored persistently and shared for all fish sessions on the machine. The issue here is invalidating it - you need to unset the variable again once you logout/reboot, but if your machine crashed that wouldn't happen
Use a flag file on a tmpfs (i.e. in RAM) - this will be automatically invalidated if your machine stops, whatever the cause. You need to setup a tmpfs for it, though.
Here is the code I used in ~/.xsessionrc:
# Apps launched directly by X window managers don't have their environment set
if [ $SHELL = "/usr/bin/fish" ]
then
/usr/bin/fish -c xsession-apps
else
source ~/src/dotfiles/keychain.sh
source ~/src/dotfiles/shell/aliases
source ~/src/dotfiles/shell/env
source ~/src/dotfiles/xsession-apps
eval "$(~/.rbenv/bin/rbenv init -)"
fi
It initializes fish or bash/zsh, depending on what I'm in the mood for at the time. Lately, I'm liking fish. I then defined a fish function called xsession-apps to launch the things I wanted started up in my X session, such as dropbox, hplip, xmobar, etc. I have a similar setup as a bash script, also called xsession-apps that gets sourced if I'm not using fish.
By the way, I use xmonad as my windowing environment.
I was curious about how the famous Bourne shell is written.So I started digging into the source code.I understand that,One of the initial things which bash does is it loads the ~/.bash_profile file for the user env.
My question is how it loads the environment variables in the current shell.
Does it fork and execute it?
Does it read the bash_profile file and does putenv(),one for each
variable?
In the source code bash-2.0/shell.c, I found a function call like,
maybe_execute_file ("~/.bash_profile", 1);
Even after digging further I was unable to get the exact logic as of how the values are pushed in environment.
Neither - it executes it without forking (just like the . or source built in command). If it forked first then environment variable changes in the subshell wouldn't be visible in the original parent.
I have fairly often seen that PATH variable is exported at the beginning of the script. For example in /etc/init.d/rc script in Debian Wheezy:
PATH=/sbin:/usr/sbin:/bin:/usr/bin
export PATH
While I understand that this ensures that executables used in the script are started from correct directories, I don't fully understand which shells are affected by this export statement. For example here I start the script named rc(PID 6582; command is "/bin/sh /etc/init.d/rc") in bash(PID 3987):
init(1)-+-acpid(1926)
|-sshd(2139)-+-sshd(2375)---bash(2448)---screen(3393)---screen(3394)-+-bash(3395)---vim(3974)
| | |-bash(3397)---pstree(6584)
| | `-bash(3987)---rc(6582)---sleep(6583)
Am I correct that this PATH export statement in rc script affects only the /bin/sh with PID 6582 because parent shells(bash with PID 3987 in my example) do not inherit variables from children? In addition, am I correct that all the commands executed in script rc are started under the /bin/sh with PID 6582 and thus use this PATH=/sbin:/usr/sbin:/bin:/usr/bin variable? If yes, then hasn't the simple PATH=/sbin:/usr/sbin:/bin:/usr/bin been enough?
The environment variables are inherited by all the processes run from the script. PATH in particular affects the behaviour of the C functions execlp() and execvp(), so all the processes launched by the init.d script that started sshd and their descendants are impacted, but only until the point where one of these descendants changes and exports it.
In particular, bash(2448) most probably changes it, as it is a login shell, to match the system's and the user's config, so all it descendans are impacted by this change.
Then when you run manually the /etc/init.d/rc script by hand, the change is inherited by the sleep command (but that one never tries to run anithing).
If yes, then hasn't the simple PATH=/sbin:/usr/sbin:/bin:/usr/bin been enough?
If you mean just setting the variable instead of also exporting it, it depends on what the rc script runs. If it launches anything that tries to run commands with any of those functions, then no, only after exporting PATH it affects the children.
PATH should already be exported by the parent shell when the script run, so indeed, there is no need.
I can imagine corner cases where the shell which runs your script might not be properly initialized, such as for a startup script running very early in the boot process, but for regular userspace scripts, things should be set up the way you want them already.
i have a tcl file and defining my environment variable inside that file. something like below
set env(ET_PINASSIGN_SCRIPT) $ET_PINASSIGN_SCRIPT
where $ET_PINASSIGN_SCRIPT variable will have a user incoming input value. Now I need to read this env variable in a shell file (#!/bin/ksh
). This is what i am trying and not working
$env ET_PINASSIGN_SCRIPT .
Any suggestions?
Thanks
I understand this is not possible. A program which is running, eg. your script, receives a duplicate of the environment, and can modify it. But when the program stops running, it disappears, together with its environment and changes.
There are few direct methods to communicate 'from the dead' process. You could create a temporary file, or return simple integers from the exit code.
Another way, would be to run the other program concurrently in a way that they share the same environment,
More info on the environment:
http://en.wikipedia.org/wiki/Environment_variable
Edit: I feel I wasn't clear enough: I just wanted to point out that programming that modifies the environment for other programs is 'extreme' (i.e. dangerous) programming. All kind of tricks can be done which can have lasting influences on the use of the computer.
The simplest method would be to make the Tcl code write a shell script to set the variable to standard out (with puts) and to then do:
eval `tclsh yourscript.tcl`
in your shell.
The Tcl code would probably be something like:
puts "ET_PINASSIGN_SCRIPT='$ET_PINASSIGN_SCRIPT'"
except you'll have to worry about any embedded ' characters, so might do…
puts "ET_PINASSIGN_SCRIPT='[string map {' '\\''} $ET_PINASSIGN_SCRIPT]'"
While playing with Heroku, I found their approach of using environment variables for server-local configuration brilliant. Now, while setting up an application server of my own, I find myself wondering how hard that would be to replicate.
I'm deploying a sinatra application, riding Unicorn and Nginx. I know nginx doesn't like to play with the environment, so that one's out. I can probably put the vars somewhere in the unicorn config file, but since that's under version control with the rest of the app, it sort of defeats the purpose of having the configuration sit in the server environment. There is no reason not to keep my app-specific configuration files together with the rest of the app, as far as I'm concerned.
The third, and last (to my knowledge) option, is setting them in the spawning shell. That's where I got lost. I know that login and non-login shells use different rc files, and I'm not sure whether calling something with sudo -u http stuff is or not spawning a login shell. I did some homework, and asked google and man, but I'm still not entirely sure on how to approach it. Maybe I'm just being dumb... either way, I'd really appreciate it if someone could shed some light on the whole shell environment deal.
I think your third possibility is on the right track. What you're missing is the idea of a wrapper script, whose only function is to set the environment and then call the main program with whatever options are required.
To make a wrapper script that can function as a control script (if prodEnv use DB=ProdDB, etc), there is one more piece that simplifies this problem. Bash/ksh both support a feature called sourcing files. This an operation that the shell provides, to open a file and execute what is in the file, just as if it was in-lined in the main script. Like #include in C and other languages.
ksh and bash will automatically source /etc/profile, /var/etc/profile.local (sometimes), $HOME/.profile. There are other filenames that will also get picked up, but in this case, you'll need to make your own env file and the explicitly load it.
As we're talking about wrapper-scripts, and you want to manage how your environment gets set up, you'll want to do the sourcing inside the wrapper script.
How do you source an environment file?
envFile=/path/to/my/envFile
. $envFile
where envFile will be filled with statements like
dbServer=DevDBServer
webServer=QAWebServer
....
you may discover that you need to export these variable for them to be visble
export dbServer webServer
An alternate assignment/export is supported
export dbServer=DevDBServer
export webServer=QAWebServer
Depending on how non-identical your different environments are, you can have your wrapper script figure out which environment file to load.
case $( /bin/hostame ) in
prodServerName )
envFile=/path/2/prod/envFile ;;
QASeverName )
envFile=/path/2/qa/envFile ;;
devSeverName )
envFile=/path/2/dev/envFile ;;
esac
. ${envFile}
#NOW call your program
myProgram -v -f inFile -o outFile ......
As you develop more and more scripts in your data processing environment, you can alway source your envFile at the top. When you eventually change the physical location of a server (or it's name), then you have only one place that you need to make the change.
IHTH
Also a couple of gems dealing with this. figaro that works both with or without heroku. Figaro uses a yaml file (in config and git ignored) to keep track of variables. Another option is dotenv that reads variables from an .env file. And also another article with all them options.
To spawn an interactive shell (a.k.a. login shell) you need to invoke sudo like this:
sudo -i -u <user> <command>
Also you may use -E to preserve the environment. This will allow some variables to be pased for your current environment to the command invoked with sudo.
I solved a similar problem by explicitly telling Unicorn to read a variables file as part of startup in its init.d script. First I created a file in a directory above the application root called variables. In this script I call export on all my environment variables, e.g. export VAR=value. Then I defined a variable GET_VARS=source /path/to/variables in the /etc/init.d/unicorn file. Finally, I modified the start option to read su - $USER -c "$GET_VARS && $CMD" where $CMD is the startup command and $USER is the app user. Thus, the variables defined in the file are exported into the shell of Unicorn's app user on startup. Note that I used an init.d script almost identical to the one from this article.