Download files from Artifactory to Teamcity without retaining their full path - teamcity

I am using
TeamCity Enterprise 2017.1.2 (build 46812)
Artifactory Professional 5.3.1 rev 50046
Teamcity has the Artifactory plug-in installed (ver 2.3.0)
The task is simple - download files from Artifactory to Teamcity build:
From Artifactory MyRepo/RootFolder/ProjectFolder/1.2.3/<files>
To TC %checkoutdir%/artifacts/<files>
The <files> part of the path contains both folders and files and I want to retain their structure.
The download spec json is:
{
"files": [
{
"pattern": "MyRepo/RootFolder/ProjectFolder/1.2.3/",
"target": "artifacts/"
}
]
}
However, the files get downloaded into a different location than I would expect:
Actual: artifacts/RootFolder/ProjectFolder/1.2.3/<files>
Expected: artifacts/<files>
The whole path from Artifactory gets appended after the target directory. How do I tell the plugin to only use the relative path of files after the specified root? I have tried fiddling about with wildcards, slashes etc, but nothing helped.
I had to create an extra build step where I manually move files to the structure I expect, but I would prefer not to have to do that.
WORKING ANSWER:
{
"files": [
{
"pattern": "MyRepo/RootFolder/ProjectFolder/1.2.3/(*)",
"target": "artifacts/{1}",
"flat": "true"
}
]
}

You can customize your target structure by using Placeholders in your File Specs as described here.
Placeholders allow you to capture a specific section of your File Spec "pattern" property value, and use it inside the "target" property value.
In your case, the download File Spec should look like this:
{
"files": [
{
"pattern": "MyRepo/RootFolder/ProjectFolder/1.2.3/(*)",
"target": "artifacts/{1}"
}
]
}

Related

Not able to Upload Artifacts along with their folder structure to Jfrog using Pipeline

I am using Pipeline Job which should upload all the jars to the Jfrog, it's working but it is uploading all the jars without its folder structure to Jfrog.
eg:
libs-release-local/one.jar
libs-release-local/two.jar
But I want to upload all the jars along with their folder structure like below.
eg:
libs-release-local/abc/efg/abc/one.jar
libs-release-local/ABC/EFG/ABC/two.jar
Note: here the folder structure may change based on the jar.
So how do I make changes in a script which will catch the folder structure and upload it for every jar?
Here is the current script am using
stage('Uploading to artifactory'){
steps{
rtUpload (
serverId:"<server id>" ,
spec: '''{
"files": [
{
"pattern": "**/*.jar",
"target": "libs-bt-test-local/"
}
]
}''',
)
}
}
Let me know if there a possible way to include a loop which will dynamically change the directory structure for every jar.
The target value can be edited with a placeholders in order to dynamically determine the uploaded path.
For example: libs-bt-test-local/{1}
For further information and examples, you may refer to the Artifactory REST API documentation page:
https://www.jfrog.com/confluence/display/JFROG/Using+File+Specs#UsingFileSpecs-UsingPlaceholders

The target deployment path '...' does not match the POM's expected path prefix

I am trying to upload a pom.xml file to a Maven repository hosted on an Artifactory server. The <project> section of the pom.xml looks like this:
<groupId>com.x.y.z</groupId>
<artifactId>common</artifactId>
<version>2.3.0-RELEASE</version>
<packaging>jar</packaging>
I am using the Artifactory plugin for Jenkins in a Pipeline script and here is the uploadSpec
{
"files": [
{
"pattern": "target/common-2.3.0-RELEASE.jar",
"target": "REPOSITORY/com/x/y/z/common/2.3.0-RELEASE/common-2.3.0-RELEASE.jar"
},
{
"pattern": "pom.xml",
"target": "REPOSITORY/com/x/y/z/common/2.3.0-RELEASE/common-2.3.0-RELEASE.pom"
}
]
}
When I now try to upload the artifact, I'm getting the following error message:
java.io.IOException: Failed to deploy file.
Status code: 409
Response message: Artifactory returned the following errors:
The target deployment path 'com/x/y/z/common/2.3.0-RELEASE/common-2.3.0-RELEASE.pom'
does not match the POM's expected path prefix 'com/x/y/z/common/2.2.7'.
Please verify your POM content for correctness and make sure the source path is a valid Maven repository root path. Status code: 409
Before I upload the RELEASE, I upload a SNAPSHOT which (in this case) had the version 2.2.7-SNAPSHOT. After that I bump the version to 2.3.0, re-build the project with mvn clean install and then start another upload to Artifactory. Somehow Artifactory still seems to expect the "old" version, when I try to upload the new version.
Edit
When I upload the file with curl, everything works as expected:
curl -user:password-T pom.xml \
"http://DOMAIN/artifactory/REPOSITORY/com/x/y/z/common/2.3.0-RELEASE/common-2.3.0-RELEASE.pom"
So it seems like this is related to the Jenkins Artifactory Plugin.
You upload your pom file to an incorrect location. You use
REPOSITORY/com/x/y/z/common-2.3.0-RELEASE.pom
as a path, when the path should be
REPOSITORY/com/x/y/z/common/2.3.0-RELEASE/common-2.3.0-RELEASE.pom
Note version-named directory that is missing.
The good news are you don't even need to bother with it. When you use our Artifactory.newMavenBuild for Maven builds, we'll take care of the correct deployment. See the example.
Can you try the below code in pipeline script?
{
"pattern": "pom.xml",
"target": "REPOSITORY/com/x/y/z/common/2.3.0-RELEASE/common-2.3.0-RELEASE.pom"
}
or if it doesn't work you can utilize maven deploy in pipeline script using
def mvnHome = tool mvnName
sh "$mvnHome/bin/mvn deploy -deploy-file -Durl=file:///C:/m2-repo \
-DrepositoryId=some.id \
-Dfile=path-to-your-artifact-jar \
-DpomFile=path-to-your-pom.xml

Exclude .js, include .min.js when publishing to Azure with VS 2015

I found this (seemingly) related SO post, but following the suggestions from both answers didn't help, all my js files are getting pushed to Azure (not just the *.min.js files from my js folder.)
What am I doing wrong? Is this possible? I could update my gulp script I suppose to read an environment variable ("Development", or "Production") and then delete the source js files conditionally. It just seems to be better to make the build task function as I wish (especially since it looks doable.)
Js files for my project are in [solution folder][project folder]\wwwroot\js.
According to the comment you added in the related SO post you mentioned, I assumed that your application is Based on ASP.NET Core. As far as I know, we could determine that which file/folder could be included or excluded when publishing your web application by configuring the publishOptions section in your project.json file as follows:
"publishOptions": {
"include": [
"wwwroot",
"wwwroot/js/**/*.min.js",
"Views",
"Areas/**/Views",
"appsettings.json",
"web.config"
],
"exclude": [
"wwwroot/js/**/*.js"
]
}
But, as this tutorial mentioned that the exclude patterns have higher priority than the include patterns, hence a file found in both will be excluded. In this situation, you need to configure all the included/excluded files in the includeFiles/excludeFiles node of the publishOptions.
According to your requirement, Using Gulp would be an ideal approach to achieve it.
Additionally, if your project is an ASP.NET MVC application, you could add the following to your .pubxml file.
<ItemGroup>
<ExcludeFromPackageFiles Include="wwwroot\js\**\*.js" Exclude="wwwroot\js\**\*.min.js">
<FromTarget>Project</FromTarget>
</ExcludeFromPackageFiles>
</ItemGroup>
I ended up going the gulp route thus far (still interested in the other alternatives if viable.)
Created a new task in my gulpfile...
var del = require("del");
gulp.task("remove-non-minjs", function () {
return del([
paths.scripts.dest + "**/*.js",
"!" + paths.scripts.dest + "**/*.min.js"
]);
});
And then added this to my project.json's prepublish script...
"scripts": {
"prebuild": [ "gulp default" ],
"prepublish": [ "npm install", "gulp default", "gulp remove-non-minjs" ],
"postpublish": [ "dotnet publish-iis --publish-folder %publish:OutputPath% --framework %publish:FullTargetFramework%" ]
}
I verified it works, but just seems a bit hacky.
This seemed to do the trick for me:
https://learn.microsoft.com/en-us/aspnet/web-forms/overview/deployment/advanced-enterprise-web-deployment/excluding-files-and-folders-from-deployment
Via the Solution Explorer, you can edit the Properties of each file in your project and set the Build Action to NONE. It looks like you might be able to select multiple files at one time while doing this.

Global.json and running web application

I am trying to create a new web application using OS X and VS Code from scratch without using any scaffolding tool. My starting point is Scott Allen's tutorial on pluralsight:
https://app.pluralsight.com/library/courses/aspdotnet-core-1-0-fundamentals/table-of-contents
My project structure is:
The global.json file contains
{
"projects": [ "src" ],
"sdk": {
"version": "1.0.0-rc1-update2"
}
}
And the project.json currently contains
{
"version": "1.0.0-*",
"compilationOptions": {
"emitEntryPoint": false
},
"dependencies": {
"Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final",
"Microsoft.AspNet.Hosting": "1.0.0-rc1-final"
},
"frameworks": {
"dnx451": {},
"dnxcore50": {}
},
"commands": {
"web": "Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.Kestrel --server.urls http://localhost:5000"
},
"exclude": [
"wwwroot",
"node_modules"
]
}
I have run dnu restore to get the packages and now I would like to run the web. I need to go to the web app folder and run dnx web in order to do so and the app starts
Is it possible to run the application directly from the root folder, not from the web app folder? Is the global.json file needed in such setup? And how do I change the hosting environment? I have gone through the documentation, but the hosting environment is only clear when using VS 2015.
You cannot simply run from the root because there could be multiple projects that are "executable". But you can pass the project to dnx using the --project/-p argument.
The environment is set using the ASPNET_ENVIRONMENT environment variable.
The global.json file is useful for two things:
The sdk section is only by VS.
The projects section is used all the time and it's useful if you have the projects in multiple folders (for example src and test). If everything is in a single folder, you don't need it.
So, the bare minimum in order to run an web application is:
A folder for your project
A project.json file
A startup file

Sublime Text project folders : display path as name

I have a Sublime Text project with several folders. My problem is that these folders have the same name (the name of my app). So the folder name in the sidebar is not obvious.
For example, given a "helloworld" app :
d:\repos\helloworld is my Git repository for the app.
d:\temp\helloworld is a temporary folder where I extract jar files to search about a class or work on images.
d:\help\helloworld is where I put some useful files like Unix commands or SQL scripts that I use to debug my app. (I share other ones in the Git repo, but these ones are private.)
So my ST sidebar is :
Folders
├─ helloworld
├─ helloworld
└─ helloworld
I know I can change my ST project file to add a name. This is what I do now:
"folders": [ {
"path": "D:\\temp\\helloworld",
"name": "D:\\temp\\helloworld"
} ]
What I want to do is something like:
"folders": [ {
"path": "D:\\temp\\helloworld",
"name": ${path}
} ]
Or with a user setting like:
"use_path_as_name_in_sidebar" : true
As of today (Sublime Text 3 Build 3059), Sublime Text's project does not support such feature.
http://www.sublimetext.com/docs/3/projects.html
Sublime Text's forum may be a better place for such feature request.
This is possible now, and quite easy. As per Sublime Documentation: just edit your .project file thus:
{
"folders":
[
{
"path": "./path/to/your/docs",
"name": "Documentation"
}
]
}

Resources