Dockerize a multi maven project (not multi-module) - maven

In my maven application i have multiple projects:
Core
Application 1
Application 2
Application 1 and Application 2 are two projects that uses the core (for example, those application are built for two different customers)
In order to Dockerize all of them, the simplest way would be to create a multi-module project, but the downside is that i have all inside a single project (core + Application 1 + Application 2).
I would like to have the core separated from them.
The main problem with this configuration is that the core project need to built before the other two, and App 1 and App 2 use this as maven dependency:
App 1
<dependency>
<groupId>it.myorg</groupId>
<artifactId>core-project</artifactId>
<version>1.12.0-SNAPSHOT</version>
</dependency>
If i try to dockerize the App 1 its fail when i package it, because inside the docker container core-project 1.12.0-SNAPSHOT do not exists.
I was thinking to setup a "local maven repo", pushing the core there and App 1 will "pull" the jar from the repo and not from .m2 folder, but i dont like this soulution.
I can provide more information, sorry if i dont provide examples, but my paper is white right now :(
Folder structure
+- Core
--- pom.xml
--- src
+- Application1
--- pom.xml
--- src
The solution i'm trying to do now is create a Dockerfile for core project (FROM maven:latest), building the image with a tag and in Dockerfile of App1 use this image (so, creating a multi stage build but in two separate moments).
The best would be
FROM maven:latest as core-builder
## build the core
FROM maven:latest
## Copy jar from builder
Because the project are in separate folder, i cant build the core in this way. I need to build del core BEFORE (running docker build -t) and later copy from them.
UPDATE
After the correct answer from #mihai i'm asking if its possible a structure like this:
-- myapp-docker
- Dockerfile
- docker-compose.yml
-- core-app
-- application_1
Having Dockerfile at the same level of core-app and application_1 its totally fine and 100% working. The only "problem" is that i should put all the projects in the same repo.

This is the proposed solution with multi-stage builds.
To replicate your setup I created this structure:
.
├── Dockerfile-app1
├── application1
│ ├── pom.xml
│ └── src
│ └── main
│ ├── resources
│ └── webapp
│ ├── WEB-INF
│ │ └── web.xml
│ └── index.jsp
├── core
│ ├── pom.xml
│ └── src
│ ├── main
│ │ └── java
│ │ └── com
│ │ └── test
│ │ └── App.java
│ └── test
│ └── java
│ └── com
│ └── test
│ └── AppTest.java
In the pom.xml file from Application 1 I added the dependency to core:
<dependency>
<groupId>com.test</groupId>
<artifactId>core</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
I named the Dockerfile Dockerfile-app1, this way you can have more than 1 of them.
This is the Dockerfile-app1:
FROM maven:3.6.0-jdk-8 as build
WORKDIR /apps
COPY ./core .
RUN mvn clean install
FROM maven:3.6.0-jdk-8
# If you comment this out then the build fails because it cannot find the dependency to 'core'
COPY --from=build /root/.m2 /root/.m2
COPY ./application1 ./
RUN mvn clean install
You should probably add an entrypoint at the end to run your project or even better add another 3rd stage that only copies the generated artefacts and runs your project (this way the final image will not have your sourced in).
The first stage only builds the core submodule.
The second stage used the results of the first stage, copies only the source for application1 and builds it.
You can easily replicate this for application2 by creating a similar file Dockerfile-app2.

Since you're using maven, try dockerfile-maven to build the image. You don't want any of your build information inside of your image (like what the dependencies are), you should just add the jar at the end. I usually use it together with spring-boot-maven-plugin and repackage, to get a fully self-contained jar.

Related

How to tell Gradle and Intellij that the project's folder structure is different?

I'm using Gradle with the wrapper, and the folder structure by default is like so:
.
├── settings.gradle
├── build.gradle
├── gradle.properties
├── gradle
│ └── wrapper
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
├── gradlew
└── gradlew.bat
However, I would like to change it to so:
.
├── gradle
| ├── build.gradle
│ ├── settings.gradle
│ ├── gradle.properties
│ └── wrapper
│ ├── gradlew
│ ├── gradlew.bat
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
└── src
├── main
└── test
Other than the fact that I don't know how to tell IntelliJ about the folder structure, I don't know how to change it for Gradle since the Environment Options related with changing the folder structure are deprecated:
-b, --build-file (deprecated)
Specifies the build file. For example: gradle --build-file=foo.gradle. The default is build.gradle, then build.gradle.kts.
-c, --settings-file (deprecated)
Specifies the settings file. For example: gradle --settings-file=somewhere/else/settings.gradle
You can't tell Gradle and Intellij IDEA that you use a non-standard Gradle build layout. And in all honesty, you shouldn't even consider that unless you have strong reasons to do so. There are mainly two reasons for that:
Developers familiar with one Gradle project feel immediately at home when starting with your Gradle project.
A non-standard build file and directory layout requires additional logic in IDE's (which is not present) and requires to provide extra parameters when building on the command line.
To put things into context, please have look at Gradle issue #16402.
Deprecate command-line options that describe the build layout
The -b and -c command-line options are effectively used to describe a non-standard build layout to Gradle. This is problematic because it means that a specific combination of options must be used whenever Gradle is used on that build, for example whenever invoked from the IDE, CI, command-line or some other tool. These command-line options also have some potentially surprising behaviours, such as running a settings script present in the target directory.
We don't think there are any use cases that are strong enough to justify keeping these options, and we should remove them (via deprecation). If we discover there are some use cases, we might consider replacing the options with more self-describing contracts, for example conventions for build script names.

Go workspace dependencies

I'm trying to get to grips with working with workspaces in go 1.18, and how to make it work well in a monorepo.
As a minimum example, I have the following project structure:
.
├── docker-compose.yml
├── go.work
├── makefile
└── project
├── go.mod
├── module1
│ ├── Dockerfile
│ ├── go.mod
│ └── main.go
├── module2
│ ├── Dockerfile
│ ├── go.mod
│ └── main.go
└── shared-module
├── go.mod
└── shared.go
module1 and module2 can both be built into binaries with their respective main.go files. However, module1 uses shared-module, but module2 does not.
When building the binaries without Docker, I can cd into module1 or module2 and run go build and everything works fine. However problems occur when I want to use Docker for the builds.
Here is the Dockerfile for module1:
# FIRST STAGE
FROM golang:1.18 AS builder
WORKDIR /app/
# copy workfiles
COPY go.* ./
WORKDIR /app/project/
# dependency management files
COPY project/go.* ./
COPY project/module1/go.* ./module1/
COPY project/shared-module/go.* ./shared-module/
WORKDIR /app/project/module1/
RUN go mod download
WORKDIR /app/project/
# copy shared module
COPY project/shared-module/ ./shared-module/
# copy module to compile
COPY project/module1/ ./module1/
WORKDIR /app/project/module1/
RUN CGO_ENABLED=0 GOOS=linux go build -o bin/module1
# SECOND STAGE
FROM alpine:3.14.2
WORKDIR /app
COPY --from=builder /app/project/module1/bin/module1 /app
ENTRYPOINT [ "./module1" ]
With this build i'm trying to maximise caching by specifying files which change infrequently first. I'm also excluding any files which I don't need for the module (like module2).
Running docker-compose build module1 to build the image using that Dockerfile, the build fails with error:
go: open /app/project/module2/go.mod: no such file or directory
This initially surprised me, as module2 is not a dependency of either module1, or shared-module, but after a bit of consideration I realised it was because of the go.work file which specifies ./project/module2. Removing the line in go.work that specifies that module allows the image to be built successfully.
My question is therefore, if I want to have streamlined image builds, do I have to create multiple go.work files for each of the modules I want to build in Docker? For example, I would need another go.work file for module2 which omits module1 and shared-module.

Make gradle point to subdirectory and treat it as a rootProject

I encountered a problem with gradle project structure. I have a task that needs to be realized and some tests are meant to be executed to check whether my project structure is correct and the tasks in gradle execute correctly. However I think I misunderstood instruction a bit and I'm wondering whether I can do something with my current folders structure or If I will have to rewrite the whole project. My current project structure looks like this:
main-repo-folder/
├── docker-related-file
├── rootProject
│ ├── sub-project-1
│ ├── build(output from tasks is created here)
│ ├── build.gradle
│ ├── sub-project-2
│ ├── gradle
│ ├── gradlew
│ ├── gradlew.bat
│ ├── settings.gradle
│ └── src
As you can see, the root project is a directory inside a repo. In order for my tests to execute I think the repo itself must be a root folder (or act as one) because the tests seem to be trying executing there. And here is my question, is it possible to add f.e settings.gradle file in main-repo-folder (at the same level as rootProject folder) to "point" gradle to build from rootProject and treat that folder as the root?(I mean f.e if I call gradle clean build task_name in main-repo-folder I want to make gradle execute it as I would be in rootProject folder)
I've tried to find some information but I'm at the path of learning gradle and I don't know if it is even possible :/ .
Rename main-repo-folder/rootProject to main-repo-folder.

How to debug maven dependencies logic?

Maven download package XYZ on compile time on step "maven-compiler-plugin:3.1:compile (default-compile)"
but mvn -U -X -e dependency:tree -Dverbose | grep XYZ doesn't show anything.
How can i find out why is maven downloading dependency XYZ?
edit:
don't want to turn this into a collective POM debug exercise, but here is what i am trying to debug:
after mvn package (with an empty ~/.m2/repository) i end up with
.m2/repository/log4j/log4j
├── 1.2.12
│   ├── log4j-1.2.12.jar
│   ├── log4j-1.2.12.jar.sha1
│   ├── log4j-1.2.12.pom
│   ├── log4j-1.2.12.pom.sha1
│   └── _remote.repositories
└── 1.2.17
├── log4j-1.2.17.jar
├── log4j-1.2.17.jar.sha1
├── log4j-1.2.17.pom
├── log4j-1.2.17.pom.sha1
└── _remote.repositories
i do not have any direct or indirect use of log4j. I'm pretty sure as soon as I post my pom someone will answer "of course you have ABC which everyone knows brings in XYZ"... but i would very much to understand how to force maven to show more information that helps me to get to that conclusion in different cases.
So some plugin you are using in your POM brings a log4j dependency.
First of all, this not really anything to worry about because these dependencies are not just for your JAR/WAR/EAR, but only during the build.
Secondly, excluding them at the plugin level will likely cause the build to fail.
Lastly, your problem might just be solved by updating your plugins to the latest version.

Vendoring package which resides in another project's vendor folder

I'm writing a library package which depends on certain imports but I'm not sure how to handle it correctly.
Let me start with the directory structure:
go/src/github.com/
├── developer A/
│ ├── project 1
│ └── project 2
│
└── developer B/
└── project 3
└── vendor
└── project 4
Project 1 is a library. It is used in project 2 and gets pulled into 2s vendor folder. Therefore, project 1 should contain all its dependencies such that clients (e.g. project 2) don't need to pull them as well. However, one dependency of project 1 is project 4 which is contained in project 3s vendor folder. It is essential that this dependency is always exactly the version vendored by project 3. Go doesn't allow imports to point to packages inside vendor folders, so I can't import it directly from there. How do I solve this with govendor?
Go won't let you reach into another project's vendor directory. It sounds like your intention is to ensure versions. This is what go modules are tasked to do. Take a look at the wiki for more information.

Resources