I've created a build pipeline for my chatbot that uses mocha tests as part of the process. All tests pass on my local with VS Code, but they are failing during the pipeline build. The issue is missing environmental variables. However, I have added for each run via variables and it is still not working only in some cases. Some of the variables are coming through fine, but other continue to be null (and are causing failure of the build). I have checked and all the variable names are identical to my local .env file. Any ideas why some of the variables are coming across as null during pipeline build where others are fine?
This turned out to be an issue between my local test environment being Windows command line and the build pipeline being ubuntu. On Windows environmental variables are not case sensitive, but they are on Linux platforms such as ubuntu. I changed by yml file to use windows-latest instead of ubuntu-latest for the build and that fixed the problem. The key takeaway is:
Use Linux env variable naming regardless of your platform (all caps and underscores for spaces)
In case you already have many env variables in your project with mixed case, you can switch the OS in your build pipeline to Windows, but that is more of a workaround than a solution to the problem. That said, dealing with something relatively simple like botframework it should be fine to avoid having to recode and redefine all of your env variables.
Related
I'm getting this error when trying to run this jmeter sentence from a maven project in java.
jmeter -g DIRECTORY -o DIRECTORY
When I run it in cmd it works perfectly but when I try to run it using this:
processBuilder.command("cmd.exe","/c","jmeter -g DIRECTORY -o DIRECTORY");
processBuilder.directory(new File("C:\\apache-jmeter-5.3\\bin"));
try {
Process process = processBuilder.start();
It says The JMETER_HOME environment variable is not defined correctly
When I run the command echo %JMETER_HOME% it returns C:\apache-jmeter-5.3
What I found weird is that in other PC it all works fine and that one doesn't have the JMETER_HOME variable defined, so I tried deleting the variable from this PC but it won't delete, it's not appearing in the system variables window but when I echo it's there.
What is going on?
Thanks in advance
So you want us to guess what is the difference between one your PC and the other one without providing any information apart from incomplete code?
I can only state that I cannot reproduce your issue and yes, I don't have JMETER_HOME variable set anywhere.
If you're using Maven maybe it worth considering looking at JMeter Maven Plugin which can automatically download JMeter, run the test and generate the HTML reporting dashboard.
Also be informed that according to JMeter Best Practices you're supposed to use the latest version of JMeter so consider upgrading to JMeter 5.4.3 (or whatever is the latest version of JMeter available at JMeter Downloads page) on next available opportunity.
I faced same issue of JMETER_HOME not been set, even though I tried setting that Env variable.
I was tying to launch the Jmeter on the Virtual system (VM) using command prompt, and didn't succeed.
I able to solve it by running the command prompt as admin and launching Jemter, it worked fine.
Note: make sure you have the set the Env. PATH set to the bin folder of Jmeter
I have this script on an Ubuntu VM that uses an environment variable that is set in a script in /etc/profile.d/appsetup.sh.
The variable is used in my script, server-up.sh, to start my Java app:
export path=$KEYSTORE_PATH
java -jar -Dsecurity.keystore.path=$path [jarfile]
If I echo the variable, it works:
$ echo $KEYSTORE_PATH
/etc/ssl/certs
And if I run the script on my own (sudo sh server-up.sh) it runs and uses the environment variable just fine.
However, when the script is executed from Jenkins' "Execute Shell" step (on the same VM), it apparently can't access the environment variable, even though supposedly it's available system-wide.
I've tried setting the owner of server-up.sh to both root and jenkins and Jenkins runs it either way, but in neither case does it get the environment variables. In Jenkins, I also tried using the command sudo -E /[path]/server-up.sh but then the job fails and an error says sudo: sorry, you are not allowed to preserve the environment.
I've googled a dozen times for various things, but almost everything that comes up is people asking how to set environment variables in Jenkins, and I don't need to do that; I just want a script that Jenkins can execute have access to system environment variables.
What do I need to do to get this working?
Make a small change to allow the /etc/profile.d/appsetup.sh script to output the variable to a file, where the Jenkins job can access this variable to create an environment variable available for your job to run successfully.
I don't think the context and needs are explained sufficiently well to properly answer the question with a here's how you do it.
On a server, jenkins.war launches from a shell (or a root shell which invokes a shell script with commands which launches jenkins), which has an environment and to which you can set and pass parameters. Those exist in the context of the jenkins.war process. If you run from a daemon (initd / systemd) you get a non-in=teractive shell, which is set differently to your normal shell.
Your Jenkins will typically have Nodes launching agents on remote servers. Those are typically launched via a non-interactive shell (so no user .profile settings).
Then the jobs themselves run on one of the agents where the executor launches a shell for the job execution. Sub-shells may be launched for specific steps.
The two context you mention sudo sh server-up.sh and Jenkins' "Execute Shell" step (on the same VM), even on the same VM do not inherit the same environment as the Node is launched on it's own process using a non-interactive shell and is not aware of anything in your server-up.sh script; it (generally) just gets the /etc/profile.
You have options. You can set Global variables within Jenkins: ${JENKINS_URL}/configure
Global Properties
[ X ] Environment variables
[ X ] Prepare jobs environment (requires Env Inject plugin)
The same options also exist at the Node level
You can install the slaves-setup plugin, which allows you some customization when launching agents (aka slaves).
You can install the Environment Injector plugin, which adds the previously mentioned Prepare jobs environment feature.
It also adds jobs specific configuration options for:
[ X ] Prepare an environment for the run
Plus under the Build Environment section,
[ X ] Inject environment variables to the build process
and
[ X ] Inject passwords to the build as environment variables
Those are encrypted, and I believe are masked
Finally, you can add a build step to Inject environment variables, useful if you need to have different values for different steps.
BUT it's certs in a keystore!
Given that you also mention what you are trying to make available is $KEYSTORE_PATH=/etc/ssl/certs, I wonder if you've explored using the Credentials plugin? Is supports a wide variety of credential types, including:
Password
Username and password
SSH private key
Public Certificate and private key
Binary blob data
That OAuth thingy
The obvious benefit to using this approach vs cooking your own is it's been designed to work securely with Jenkins so your secrets don't get inadvertently exposed. Aside from the extensive documentation on the plugin site, there's more in the book on Using credentials, using them in a pipeline which also mentions the Snippet generator will stub it for you, and on the Cloudbees site - Injecting secrets into builds. You can probably find plenty of help here in S/O and DevOps.
You may also wish to explore the newly introduced Git credentials binding for sh, bat, and powershell, though not sure that's applicable in your case.
Ok, this seems easy enough for Linux containers but I am trying to get this done using Windows Containers and its annoying that its so difficult.
I have a Windows Dockerfile which builds a project and part of the build process is to reversion the C# AssemblyInfo.cs files so that the built assemblies have a build version from the CI environment (Devops)
I am using a Powershell script https://github.com/microsoft/psi/blob/master/Build/ApplyVersionToAssemblies.ps1, it expects 2 Environment variables, one which I can hardcode so is not a problem, but the BUILD_BUILDNUMBER environment variable needs to be injected from Devops build system.
I have tried the following, none of which work
ARG BUILD_BUILDNUMBER
ENV BUILD_BUILDNUMBER=$BUILD_BUILDNUMBER
RUN ApplyVersionToAssemblies.ps1
and running
docker build -f Dockerfile --build-arg BUILD_BUILDNUMBER=1.2.3.4 .
also
RUN SETX BUILD_BUILDNUMBER $BUILD_BUILDNUMBER
RUN SETX BUILD_BUILDNUMBER %BUILD_BUILDNUMBER%
and a few other combinations that I dont recall, what I ended up doing which works but seems like a hack is to pass the BUILDNUMBER as a file via a COPY and then modifying the the Powershell script to read that into its local variable
So for the moment it works but I would really like to know how this is supposed to work via ARG and ENV for Windows Container builds
Windows Containers definitely feel like Linux containers poor cousin :)
Example for CMD in Docker Windows Containers:
ARG FEED_ACCESSTOKEN
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{\"endpointCredentials\": [{\"endpoint\":\"https://URL.com/_packaging/Name/nuget/v3/index.json\", \"username\":\"PATForPackages\", \"password\":\"${FEED_ACCESSTOKEN}\"}]}"
SHELL ["cmd", "/S", "/C"]
RUN echo %VSS_NUGET_EXTERNAL_FEED_ENDPOINTS%
I am building a visual studio solution containing number of project. I wanted to disable multiprocess build, so , i tried setting an enviroment variable CL to /MP1. But it didn't worked in Jenkins while working in running the batch script for building solution using command line.
Good morning,
Log to your Jenkins server, and stop the Jenkins from the command line. While doing this, open your web-browser and refresh the Jenkins webpage to make sure it stopped(it will take around 5 seconds to stop the service). Then start again from the command line, it will update the variable. I did yesterday, to run my unit tests. It should work.
To set environment variables for individual projects, use the checkbox 'Prepare an environment for the run' and set what environment variables you want in the format 'ENV=value' in the Properties content box.
Otherwise, all I can suggest is that you haven;t restarted the Jenkins service after setting your variable in Windows.
You can also used the EnvInject plugin, it works well.
https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin
I'm using Jenkins to do continuous integration builds. I have quite a few jobs that have much of the same configuration code. I'm in the midst of pulling this all out into a common script file that I'd like to run pre and post build.
I've been unable to figure out how to set some environment variables within that script, so that both the Xcode build command, and the Jenkins build can see them.
Does anyone know if this is possible?
It is not possible to do exactly what you ask. A process cannot change the environment variables of another process. The pre and post and actual build steps run in different processes.
But you can create a script that sets the common environment variables and share that script between all your builds.
The would first call your shell to execute the commands in the script and then call xcodebuild:
# Note the dot in the beginning of the next line. It is not a typo.
. set_environment.sh
xcodebuild myawesomeapp.xcodeproj
The script could look like this:
export VARIABLE1=value1
export VARIABLE2=value2
How exactly your jobs will share the script depends on your environment and use case. You can
place the script in some well-known location on the Jenkins host or
place the script in the version controlled source tree if all your jobs share the same repository or
place the script in a repository of its own and make a Jenkins build which archives the script as a build artifact. All the other jobs would then use Copy Artifact plugin to get a copy of the script from the artifacts of script job.
From Apple's Technical Q&A QA1067 it appears that if you create the file /Users/YOU/.MacOSX/environment.plist and populate it with your desired environment variables that all processes (launched by the user with the environment.plist file in their home dir) will pick up these environment variables. You may need to restart your computer (or just log out and back in) before a newly launched process will pick up the variables.
This article also claims that Xcode will also pass these variables to a build phase script. I have not tested it yet but next time I restart my MacBook I will let you know if it worked.
From http://developer.apple.com/library/mac/#/legacy/mac/library/qa/qa1067/_index.html
Q: How do I set environment for all processes launched by a specific
user?
A: It is actually a fairly simple process to set environment variables
for processes launched by a specific user.
There is a special environment file which loginwindow searches for
each time a user logs in. The environment file is:
~/.MacOSX/environment.plist (be careful it's case sensitive). Where
'~' is the home directory of the user we are interested in. You will
have to create the .MacOSX directory yourself using terminal (by
typing mkdir .MacOSX). You will also have to create the environment
file yourself. The environment file is actually in XML/plist format
(make sure to add the .plist extension to the end of the filename or
this won't work).