Building open source dependencies using gradle - maven

I really don't have much experience in developing let alone using build tools.
I was assigned a task to build dependencies locally and get the jar files.
say I have a list of deps (GAV) like this:-
1. org.jetbrains.kotlin:kotlin-stdlib:1.6.0-RC
2. com.auth0:java-jwt:3.18.2, etc
3. openapi4j:openapi-operation-validator:1.0.7, etc
So i was able to download the source code url from maven repository and source code from github programmatically, for example :-
org.jetbrains.kotlin:kotlin-stdlib:1.6.0-RC - https://github.com/JetBrains/kotlin
com.auth0:java-jwt:3.18.2 - https://github.com/auth0/java-jwt
openapi4j-openapi-operation-validator-1.0.7 https://github.com/openapi4j/openapi4j
but there are many build.gradle files in different directories, how do I know which directory should I move into before running the gradle build command.
Things I have already tried and failed:-
For deps like this openapi4j:openapi-operation-validator:1.0.7, i can directly go into the openapi-operation-validator folder in the Github repo (https://github.com/openapi4j/openapi4j ) and run the gradle build command, but not all projects are structured like that I guess?
For deps like this com.auth0:java-jwt:3.18.2, the artifactId (java-jwt
) is already present in the github path (https://github.com/auth0/java-jwt), so i can run the gradle build command on the root github repo.
From the spring guides , among all the Gradle.build files available I can check which file has:-
jar {
archiveBaseName = <artifactId>
archiveVersion = <version>
}
, then I can move to that dir and run Gradle build, but not all build.gradle files have this.
None of the above approaches are concrete, is there any other firm approach that I can use to tackle the problem?

Your approach is generally correct.
You need to find the source code in github/gitlab/wherever, read the readme file and try to build it with whatever build tool was used there.
This may or may not work.

Related

Where m2 file is stored when installing apache maven on a unix box

I downloaded maven gz file, unzipped same, but i dont know where the m2 is stored. I imagine im missing a step but i cant see what one?
Is there aninstall script etc?
[root#atddpvm5 apache-maven-3.5.4]# cd /var/tmp/apache-maven-3.5.4/
[root#atddpvm5 apache-maven-3.5.4]# ls
apache-maven DEPENDENCIES doap_Maven.rdf LICENSE maven-builder-
support maven-core maven-model maven-plugin-api
maven-resolver-provider maven-settings-builder NOTICE README.md
CONTRIBUTING.md deploySite.sh Jenkinsfile maven-artifact maven-compat
maven-embedder maven-model-builder maven-repository-metadata maven-
settings maven-slf4j-provider pom.xml src
By default the .m2 folder is stored in the home folder of the user. In this case since you are using root, the path is most likely /root/.m2. You also have to use the -a switch with ls to see that folder, since it's a hidden folder (it starts with a .). Note that the folder will only be created on the first usage of Maven, i.e. when you call a maven command on a maven project, like mvn clean install.
Additionally it looks like you have downloaded the source distribution of Maven, which only makes sense if you want to work on Maven itself. You might want to download the binary distribution, if you just want to use it.

Using Gradle for packing multiple artifacts from Artifactory

I have multiple non-java artifacts stored in Artifactory that I would like to pack into single zip/tar file.
I tried using Gradle for this and https://www.jfrog.com/confluence/display/RTF/Gradle+Artifactory+Plugin with "plugins" notation. I have created separate configuration and started fighting on how to get those dependencies into one archive. This is where I started doubting whether Gradle is a good tool for the job. If it isn't can you recommend something? If it is good tool, where can I find some example of how to accomplish it?
I was thinking of something more advanced than Bash script so that it leaves good room for future extensions.
If you have all of those artifacts in a single location (same folder / path) in Artifactory, you can use the "Retrieve Folder or Repository Archive" REST API.
If you would like to stick to Gradle I found the following in the Gradle documentation the following that might assist you:
task zip(dependsOn: jar, type: Zip) {
from { configurations.runtime.allArtifacts.files }
into(project.name + '-' + project.version)
}

How to download dependencies using Bazel?

I am new to Bazel. I need to download external dependency jar files using Bazel. Where to configure to download external jar files. I tried to add jar files in BUILD and WORKSPACE files , but not working.
Workspace file
maven_server(name="myserver",url="https://mvnrepository.com/artifact/org.apache.flink/flink-java")
maven_jar(name="flink-java",artifact="org.apache.flink:filnk-java:jar:1.2.0",server="myserver")
bind(name="flink-java",actual="#flink-java//jar")
Build file
java_binary(
name="read_files",
srcs = glob(["ReadFiles.java"]),
main_class="com.ibm.cdo.gts.contracts.pipeline.preprocess.ReadFiles",
deps=["//external:flink-java"],
)
The logical mistake you have there is that the workspace name (flink-java) is illegal see here and here. What is needed is to change the - to _ in maven_jar and of course the reference in the bind.
maven_jar(name="flink_java",artifact="org.apache.flink:flink-java:jar:1.2.0")
bind(name="flink-java",actual="#flink_java//jar")
Two more mistakes you had in the above were:
In the artifact coordinates you wrote filnk-java when you needed flink-java
The mvnrepository URL you used doesn't seem to be legal. I tried a few variants but couldn't get it to work. Additionally from the site they seem to link downloads to Maven Central itself so I'm not sure they server the jars. Having said that since Bazel fallbacks to Maven Central I just omitted the maven_server and the build passes. If you need help with the maven_server part and you need to use a private Maven repository please double check the URL first.
PS: It would have really helped if you added to your question the Bazel outputs since I needed to create my own workspace to know what were the problems you encountered.

Downloaded path for Gradle comple statement [duplicate]

How does Gradle store downloaded jar files on the local file system? Maven stores them in the .m2 directory under USER_HOME, but where does Gradle store them? I checked the .gradle folder there, but saw only compiled scripts.
On Mac, Linux and Windows i.e. on all 3 of the major platforms, Gradle stores dependencies at:
~/.gradle/caches/modules-2/files-2.1
Gradle caches artifacts in USER_HOME/.gradle folder. The compiled scripts are usually in the .gradle folder in your project folder.
If you can't find the cache, maybe it's because you have not cached any artifacts yet. You can always see where Gradle has cached artifacts with a simple script:
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
compile 'com.google.guava:guava:12.0'
}
task showMeCache doLast() {
configurations.compileClasspath.each { println it }
}
Now if you run gradle showMeCache it should download the dependencies into the cache and print the full path.
In Windows 10 PC, it is saved at:
C:\Users\%USERNAME%\.gradle\caches\modules-2\files-2.1\
Gradle's local repository folder is:
$USER_HOME/.gradle/caches/modules-2/files-2.1
Defined dependencies will be loaded from remote repositories into gradle's local repository folder. For each loaded file, gradle will be create a new folder named with md5 value of the original file (pom,jar,..). Full path for the dependency file is made up from :
groupid + artifactid + version + FILE_MD5_VALUE + FILE_NAME
If our defined dependency is:
compile 'org.springframework:spring-jdbc:4.3.4.RELEASE'
Then the library will be loaded into :
/$USER_HOME/.gradle/caches/modules-2/files-2.1/org.springframework/spring-jdbc/4.3.4.RELEASE/42175d194cf6aa7c716c0887f30255e5c0a5262c/spring-jdbc-4.3.4.RELEASE.jar
In fact the cache location depends on the GRADLE_USER_HOME environment variable value.
By default, it is $USER_HOME/.gradle on Unix-OS based and %userprofile%.\gradle on Windows.
But if you set this variable, the cache directory would be located from this path.
And whatever the case, you should dig into caches\modules-2\files-2.1 to find the dependencies.
If you want your dependency files to be in some specific folder you can simply use a copy task for it. For Eg.
task copyDepJars(type: Copy) {
from configurations.compile
into 'C:\\Users\\athakur\\Desktop\\lib'
}
I am on windows,
You should be able find the dependencies inside
$USER_HOME.gradle\caches\artifacts-24\filestore
Many answers are correct!
I want to add that you can easily find your download location with
gradle --info build
like described in https://stackoverflow.com/a/54000767/4471199.
New downloaded artifacts will be shown in stdout:
Downloading https://plugins.gradle.org/m2/org/springframework/boot/spring-boot-parent/2.1.7.RELEASE/spring-boot-parent-2.1.7.RELEASE.pom to /tmp/gradle_download551283009937119777bin
In this case, I used the docker image gradle:5.6.2-jdk12.
As you can see, the docker container uses /tmp as download location.
You can use the gradle argument --project-cache-dir "/Users/whatever/.gradle/" to force the gradle cache directory.
In this way you can be darn sure you know what directory is being used (as well as create different caches for different projects)
I just stumbled onto this while searching for this answer. If you are using intellij, you can navigate to the file location, but opening the external lib folder in the project explorer, right clicking on the jar, and select Open Library Settings.
It took me a while to realize this, hence the additional answer. Hopefully it can save folks time. Note that if you are running sudo gradle the dependencies may not be in your home directory, even if sudo echo $HOME returns /Users/<my-non-root-user>/. On my Mac, Gradle was caching the dependencies in /private/var/root/.gradle/caches/.
In case it is an Android gradle project - you can find the android libraries below your $ANDROID_HOME/extras/android/m2repository folder
In android studio do the following steps to check the gradle downloaded jar file.
Set project structure view to "Project"
At bottom External library section available, expand it.
Here you can see downloaded jar files.
On my windows machine with "Buildship 2.0.2" plugin installed in eclipse, dependencies are stored :
$USER_HOME.gradle\caches\modules-2\files-2.1
For my case, I was using an Ivy repository, and my Gradle dependencies were stored in ~/.ivy2/.

How to use leiningen to develop using local jars?

I realize that this question is pretty much the exact question found here. However, seeing as that question is 1.5 years old (or so), I would like to revisit it. How does one add local dependencies using leiningen? Surely this capability must exist by now?
Create a private Maven Repository, and then, add the following to your project.clj
:repositories {"local" ~(str (.toURI (java.io.File. "your_local_repository")))}
If the jars are based on your own projects, you can use lein install to put them into your local .m2, or use the checkout-dependencies feature.
You can also use the extra-classpaths feature, etc.
I found that the easiest (albeit somewhat hacky) solution is to do the following:
For an existing project that you're using as a dependency:
In your local project that has the dependency you want to modify, ensure you run lein deps
Clone the repo of this dependency so you can modify it locally (obv. make sure you're using the same tag as the version you specify in your project.clj file)
Run lein uberjar in this dependency dir (where the relevant project.clj file lives)
Copy the generated standalone jar in target/ to the exact path/file of your local maven files... (something like: ~/.m2/repository/project/.../file.jar); Ensure that you backup the original jar file so you can restore it later on if that is desirable
For development of your own project:
Within the project or plugin you're developing, simply run lein install
Find out where your local maven repo is (see above for an example path)
Enter dependency information in your test project like you would for any other leiningen project
Again, this is a quick hack and perhaps not the way you'd go about doing serious local development, but I found it easy enough for what I wanted. Check out lein help tutorial for much more info

Resources