I have a question regarding the AQL (Artifactory Query Language) used by JFrog Artifactory to find "things" in the artifactory. The AQL seems to be very powerful, but i'm wondering how to build (correct) search queries, using the correct terms.
The AQL documentation (https://www.jfrog.com/confluence/display/RTF/Artifactory+Query+Language) offers several object types. But what's the exact difference between an item, artifact, build and entry?
If i'm searching for a specific "file" (JAR) - is it an artifact, an item, or both?
Responding your last question, this is indeed an artifact. Every file in Artifactory is an artifact. With that being said, every artifact is an Item. Therefore use the item.artifact closer. :)
Build is something a bit different, as Artifactory is a binary repository manager, it can connect and serve different build agents, such as Maven, Gradle, Nuget, PyPi etc... When using those clients in a CI server, that JFrog has plugin for (For example, Jenkins, Bamboo, TeamCityetc..) , it will produce a build info JSON that includes all of the information on your build. Also, the artifacts produced during this build will be associated to that build using properties. Long story short, you can use the 'build' closer in AQL to search for details in a specific build or to search for builds that contains specific data.
Hope this was helpful :)
Related
my project is organized as a Gradle multi-project build with five Java modules/sub-projects. When building them, it results in five different JAR artifacts.
Four of those artifacts contain helper classes or small, isolated portions of code doing very specific things (for example efficient graph search that is optimised towards my specific use case domain). Only one project is the "main" artifiact that makes sense to use in a standalone way, but all five artifacts are required for it to run.
I would like to make this core artifact available to users, and I have been successful in uploading all five artifacts to a Bintray account. When mirroring to JCenter, I have two concerns:
Do I have to actively link all 5 projects to JCenter, or is there a way to only expose the "core" artifact to the general public?
What does the "Is Pom Project" checkbox do? As I understand it, Gradle creates POM files for every Maven publication artifact, so this box should always be checked for Maven-style builds. Is this correct?
(potential duplicate that does not contain a solution apart from "I work at Bintray and I fixed it for you in our system!": Linking Bintray Package to JCenter)
Thanks!
- Gregor
I hope I can answer all.
Do I have to actively link all 5 projects to JCenter, or is there a way to only expose the "core" artifact to the general public?
As the other answer you have attached says, it is linked on path level, this means that if you include org/worldcubeassociation/tnoodle/lib-scrambles/ as the path, then only those modules will be linked to Jcenter.
What does the "Is Pom Project" checkbox do? As I understand it, Gradle creates POM files for every Maven publication artifact, so this box should always be checked for Maven-style builds. Is this correct?
Yes, you are correct. The POM file is created and uploaded. You can see the POM file in your path.
For more information you can always check the central repositories guide.
I need a tool that will help to find all artifacts that reference another artifact.
When I rebuild an artifact, I need to update/rebuild all artifacts that were using the old version. But I work in a big organisation, and nobody knows really where the artifact is spread in the organisation, so nobody is ever completely sure that everybody use the latest versions.
What I need would be a tool - maybe an artifactory plugin or feature, or a maven plugin doing a lookup in the repository - that indexes all the known poms, and is able to make a listing of all artifacts that have the updated artifact in their dependencies, either directly and transitively. Thus a list of artifacts I would need to rebuild. Quite the opposite of dependency:tree.
Filtering that list by repository, groupId, packaging, etc. is a nice to have. But I can live without.
Any idea?
You can use the Artifactory Query Language with the REST API to do that. For example, if you want to find all builds that use "MySuperAwesomeDependency-1.0.2" your AQL statement would be something like:
//Find builds that use a dependency that is a snapshot
builds.find({"module.dependency.item.name":{"$match":"MySuperAwesomeDependency-1.0.2*"}})
The key in the above statement would be the module.dependency.item.name, which allows you to search for dependencies by name, assuming you store the dependencies in Artifactory.
I am working on a research project and I need to find some case studies to test my tool.
A great way to find such case studies would be to look for all the projects that use a given maven dependency.
Is there any website where I can query for example for "All the projects depending on guava"?
mvnrepository.com will also list the artifacts that directly depend upon a certain artifact.
Wherever you read about continuous delivery or continuous integration it's recommended to use an artifact repository to store the artifacts even though Jenkins already stores them for each build.
So why is it recommended to use an artifact repository? Is there a smooth solution to work with the artifacts of the Jenkins builds, ex. to use these artifacts for deployment?
An artifact repository and continuous integration tools serve two different purposes and one cannot be substituted with the other. Check this video from Artifactory, one of the providers of artifact repositories, about why one should use an artifact repository.
Jenkins stores the artifacts as plain files without versioning while artifacts in an artifact repository can be version controlled. So you have a lot more flexibility in retrieving artifacts and governing them. Read this very good article on why we need them. Surely not all of those things are supported by continuous integration tools like Jenkins.
Moreover, you can also look at the Artifactory plugin for Jenkins which integrates the two.
An artifact repository is needed but the artifact repo is a conceptual piece an not always a distinct tool. With Jenkins you should have MD5 signatures and (I think) a way of downloading the files you want (web service call, right?) from your remote server. Certainly, if you're doing something simple like using the Jenkins build pipeline plugin, it should be able to access the right versions of the files smoothly.
Alternatively, if you are using a separate deployment tool, the better ones bundle an artifact repository.
Regardless, you want what the ITIL folks call a Definitive Media/Software Library. Definitive in that the bits are secure, trusted, and official. And a library in that they can be easily looked up and accessed. When working with an artifact repository, you need to make sure its adequately secure. It is backed up. It is accessible for your deployments (including to production). If you look at Jenkins and it meets your criteria in those categories, consider yourself done. If it's lacking, and I wouldn't be surprised if it was, then you need either a dedicated tool like the Maven repos, or something bundled with the deploy tooling.
For more of my rambling on the subject, there's a recorded webcast. The slides for that are up on Slideshare.
I haven't kept up to date with Jenkins, we still use a version of the CI when it was orginally called Hudson.
In your projects your poms you should normally point to your own artifact repository were you can fetch and deploy your own (company) projects.
Using an artifact repository with your CI server, it can then deploy successfully built snapshot and releases which can be available to other developers.
I am using Artifactory to support an enterprise multi-module project. Often, we change the names of modules and the associated dependencies in POM files are not updated to use the new module name. Because SNAPSHOT dependencies are not automatically cleaned up on a regular interval, these old module references can stay there for months. I discovered a few when I migrated Artifactory to another server and the old module dependencies resulted in build errors. I am building these SNAPSHOT artifacts nightly using Jenkins so I would like some way to automate cleaning up the SNAPSHOT artifacts.
Does Artifactory (or another artifact server such as Nexus) support a concept where if a SNAPSHOT artifact is older than X days, the artifact is deleted? Is there another way to automate artifact server cleanup to accomplish what I want to do? The only thing I can think of is to create a cron job to clear out libs-snapshot-local on a regular interval before the nightly build starts. Has someone already built this capability?
As far as I know, Artifactory doesn't have an automated way to delete modules that are older than a certain value. At my shop we've written a Groovy client that uses Artifactory's REST API to do exactly this.
Note that, if your artifacts are shared libraries, you need to be careful that nothing depends on them before you delete them. Our script takes this into account, too.
If you're interested in following up, post a comment and I'll see if it's OK to share our script with you.
Another solution might be a user plugin. You can write a simple Groovy script that will run in Artifactory itself (as opposite to remote invocation by REST Gareth proposed) on a scheduled basis, searching for artifacts not downloaded for a long time and deleting them.
I've made a Ruby script to delete artifacts which aren't download for X days. The way it works just like what JBaruch mentioned in his answer.
It isn't a plugin. It works with Artifactory Open Source. Plugin is only supported by Artifactory Pro.
The source code: https://gist.github.com/aleung/5203736