I have used already built pipeline using iris data and created a wheel and egg file using "kedro package". After this I created a virtual environment using python and installed both wheel and egg files there. I tried to run the pipeline file from the scripts folder in environment and (exe file too) but its throwing "could not find kedro.yml error".
Can you please help if this is the right way to transfer the project and run the pipeline and if not, can you help me with the right way to deploy pipelines and run them in the new environment?
This is covered by this issue kedro/issues/370. I was able to fully package a kedro project by moving /conf and .kedro.yml into the package, and adding them to the Manifest.in.
# MANIFEST.in
include **/*.kedro.yml
recursive-include <project-name>/conf/ *.yml *.yaml
Related
How does the config file provider plugin work?
I have a Jenkinsfile for a declarative pipeline (multi-branch build) that contains:
configFileProvider([configFile(fileId: 'maven-settings', variable: 'MAVEN_SETTINGS')]) {
sh 'mvn -B -s $MAVEN_SETTINGS -DWHERE="$WHERE" deploy'
}
I have tried running this on two different Jenkins installations one which is installed directly and runs as daemon and another running as a container (jenkinsci/blueocean).
When run on the direct the Jenkins Config File Provider plugin is able to provide the required settings.
provisioning config files...
copy managed file [Maven settings] to file:/var/lib/jenkins/workspace/redacted#tmp/config8989354118161621860tmp
When run under the jenkinsci/blueocean container it fails with:
provisioning config files...
not able to provide the file [ManagedFile: id=maven-settings, targetLocation=null, variable=MAVEN_SETTINGS], can't be resolved by any provider - maybe it got deleted by an administrator
I have created a managed maven-settings.xml file with id maven-settings for blue/ocean but it is not being picked up.
I've also tried copying it to ~/.m2/settings.xml
By contrast the working installation does not have any managed files (settings.xml or otherwise) and I am unable to locate any maven settings file in the workspace. I'm not sure what the #tmp directory is. It is deleted by the time a build finishes.
So my question is:
Where should I put the settings to make the configFileProvider pass them on for the jenkinsci/blueocean build job?
How does the config file provider plugin work?
I have no idea what its doing so its hard to debug. The source is here but Java, Maven & Jenkins are not my main area.
What differences are there when Jenkins itself run as a container?
This answer suggests the the config file provider is unnecessary.
There is a similar question which is unanswered but it relates to a maven plugin.
I have part of the answer. My maven-settings file wasn't being picked up because I was using the file name rather than the file Id which is different.
The remaining part of the question is how is the original Jenkins instance able to generate this file without it being listed as a managed file.
It seems that when Gradle 3.1 downloads the dependencies for your project it stores them in:
C:\Users\username\.gradle\caches
I tried copying the contents of this folder to a different machine that has a fresh Gradle 3.1 installation, but when I try to build my project it still tries to download all of the dependencies from scratch, which then fails because the new machine has no internet connection.
How do I get around this?
Compress main folder C:\Users\username.gradle to gradleHome.zip
Copy and uncompress this gradleHome.zip file to another folder for example D:\gradle_home
set gradle_home environment variable to this new folder. See how to change environment variables in windows here.enter link description here
I need to execute groovy script with gradle, but the thing is that these scripts are located in external directory (let's say it's Desktop). I've heard, that in previous versions of Gradle (currently working on 3.2.1) it was not possible, since it is not part of the gradle project. I wonder if it is possible now, or do I have to copy all these scripts into some folder located in gradle project, to execute it.
User story:
I found script in external directory (based on some properties passed to console) - I have absolute path to the script
Now I want to execute this script (build.gradle) without copying it into my gradle project
I'm pretty green with gradle and I hope I made my point clear.
Thanks for help and every attempt :)
Is that what you're looking for? To run the script clone the repository, navigate to 42556631/project folder and run the command:
gradle -b ../script/build.gradle clean build
yes you need to move build.gradle file into project/Build Script folder and then run it.
I am doing the following steps:
dotnet restore
dotnet publish
octopusDeploy: Push packages
The second step creates a 'published-app' folder and the third step is meant to take that and create a .zip file and send it to the Octopus server.
The third step is connecting to the Octopus server but gives the error:
Running command: octo.exe push --server http://server.com/ --apikey SECRET
Pushing packages to Octopus server
Please specify a package to push
I am following this https://stackoverflow.com/a/38927027 so my third step has:
%teamcity.build.workingDir%/published-app/**/* => App.zip
Any ideas why the zip file is not being created?
Not sure if you ever got this working for yourself, however just in case it helps anyone we recently came across the same issue deploying an AspNetCore 2.0 web application running on net471 being built by TeamCity 2017.1.4 (build 47070).
After some tinkering I noticed that the "OctopusDeploy: Create and Push Packages" build step ran at our git checkout root directory, so I ending up having to use the following values for the "Package path patterns"
%ProjectDirectory%/published-app/**/* => %ProjectName%.%GitVersion.NuGetVersion%.zip
NB: %ProjectDirectory%, %ProjectName% and %GitVersion.NuGetVersion% are build parameters we have manually defined elsewhere in the build process that TeamCity can replace. %ProjectDirectory% is simply the application's source directory relative to the root of the git checkout i.e. WebApplication1 so the full path would be <full checkout path>/WebApplication1
Another gotcha that we experienced was that at the time of writing the combination of TeamCity and octo.exe (from Octopus.TeamCity v4.15.10) didn't like creating nupkg files, so make sure you try to produce a ".zip" file. In the error instances we would receive the following error:
Error from Octo.exe: Cannot run program "C:\BuildAgent\temp\buildTmp\octo-temp\3.0\octo.exe" (in directory "C:\BuildAgent\work\4e62985fa616fa1f"): CreateProcess error=206, The filename or extension is too long
What I have done :
I have Jenkins set up on my Ubuntu in :
/var/lib/jenkins/
I have a job, that runs every 45 minutes that does a hg pull and hg update --clean default
from my bitbucket repository.So this is running fine.I have a folder
/var/lib/jenkins/jobs/Code Deployement
which contains the latest updated code from my repository.
Problem :
However, I want to access my updated code from
/var/www/html/[project-name]
Query :
Is there anyway i can make jenkins job update this folder instead of the /var/lib/jenkins/Code Deployment folder ? I certainly dont want to make /var/www/html/[project-name] as my jenkins home folder.
How can I achieve what I described above.Will I have to copy the folder from the jobs folder to my desired location after every time the job runs ? Please help me out with the solution,I'am a beginner with automated deployment using Jenkins.
Thank you.
Under "Advanced" you can explicitly choose a working directory for the projects without changing the Jenkins home directory. Check the "Use custom workspace" box and set the directory that Jenkins will pull the code to and build in.
In our setup we wait for the build process to complete in the working directory and add a build step for Jenkins to copy (most) of the files out to the directory that serves up the website. We had issues with file locking preventing the build process if someone (e.g. the testers) were using the site.
One simple way to do it is to create a symlink under /var/www/html/ that points to your code directory, e.g.:
sudo ln -s /var/lib/jenkins/jobs/"Code Deployement" /var/www/html/[project-name]