How to consume a published OLP schema dependency? - maven

Question

This question is a minimum reproducible example based on the tutorial Build a Batch Pipeline with Maven Archetypes (Scala), though we are unable to consume a schema artifact in a pipeline we are building for a customer.
We have a separate repository we want to publish (mvn deploy) the schema from. We ran setup equivalent to (replace djv with your own unique string):
$ mvn archetype:generate -DarchetypeGroupId=org.codehaus.mojo.archetypes \
-DarchetypeArtifactId=pom-root \
-DarchetypeVersion=1.1 \
-DgroupId=com.example.djv \
-DartifactId=nodecardinality \
-Dversion=1.0.0 \
-Dpackage=com.example.djv.nodecardinality
...
$ cd nodecardinality
$ mvn archetype:generate -DarchetypeGroupId=com.here.platform.schema \
-DarchetypeArtifactId=project_archetype \
-DarchetypeVersion=1.0.13 \
-DgroupId=com.example.djv.nodecardinality \
-DartifactId=schema \
-Dversion=1.0.0 \
-Dpackage=com.example.djv.nodecardinality.schema \
-DmajorVersion=1
...
$ cat << EOF > schema/proto/src/main/proto/com/example/djv/nodecardinality/schema/v1/schema.proto
syntax = "proto3";
>
> package com.example.djv.nodecardinality.schema.v1;
>
> message NodeCardinalityPartition {
> repeated NodeCardinality node_cardinality = 1;
> }
>
> message NodeCardinality {
> string id = 1;
> uint32 cardinality = 2;
> }
> EOF
$ # <edit schema/ds/pom.xml per the tutorial>
$ cd schema
$ mvn deploy
Primary symptom
In the OLP Portal, I can see my new schema in the list of Schemas. It explains that I can add a dependency on it like so:
<dependency>
<groupId>com.example.djv.nodecardinality</groupId>
<artifactId>schema_v1_scala_2.11</artifactId>
<version>1.0.0</version>
<type>jar</type>
</dependency>
However, I cannot download the artifact in the processor project:
$ cd processor
$ mvn install
...
[INFO] Building processor Direct1toN Batch Processor in Scala 1.0.0
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from HERE_PLATFORM_REPO: https://repo.platform.here.com/artifactory/open-location-platform/com/example/djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
Downloading from OLP Public repo: https://artifactory.in.here.com/artifactory/here-olp-sit/com/example/djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/example/djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
...
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project processor: Could not resolve dependencies for project com.example.nodecardinality:processor:jar:1.0.0: Failed to collect dependencies at com.example.djv.nodecardinality:schema_v1_scala_2.11:jar:1.0.0: Failed to read artifact descriptor for com.example.djv.nodecardinality:schema_v1_scala_2.11:jar:1.0.0: Could not transfer artifact com.example.djv:nodecardinality:pom:1.0.0 from/to HERE_PLATFORM_ARTIFACT (here+artifact-service://artifact-service): Cannot access here+artifact-service://artifact-service with type here using the available connector factories: BasicRepositoryConnectorFactory: Cannot access here+artifact-service://artifact-service with type here using the available layout factories: Maven2RepositoryLayoutFactory: Unsupported repository layout here -> [Help 1]
...
We see this failure only because we are running the processor project build on the same machine that we ran mvn deploy for the schema project so some files exists in the ~/.m2 cache. Without these files the download just fails:
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from HERE_PLATFORM_REPO: https://repo.platform.here.com/artifactory/open-location-platform/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom
Downloading from OLP Public repo: https://artifactory.in.here.com/artifactory/here-olp-sit/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom
[WARNING] The POM for com.example.djv.nodecardinality:schema_v1_scala_2.11:jar:1.0.0 is missing, no dependency information available
Downloading from HERE_PLATFORM_REPO: https://repo.platform.here.com/artifactory/open-location-platform/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.jar
Downloading from OLP Public repo: https://artifactory.in.here.com/artifactory/here-olp-sit/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.jar
Downloading from central: https://repo.maven.apache.org/maven2/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
...
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project processor: Could not resolve dependencies for project com.example.nodecardinality:processor:jar:1.0.0: Could not find artifact com.example.djv.nodecardinality:schema_v1_scala_2.11:jar:1.0.0 in HERE_PLATFORM_REPO (https://repo.platform.here.com/artifactory/open-location-platform/) -> [Help 1]
Secondary symptom
As we saw above, running mvn deploy from a local schema directory doesn't put everything required to build in the local ~/.m2 repository (that is, to build the processor sub-project). To put the required files in the local ~/.m2 repository we need to run mvn install from the parent directory (nodecardinality) of the schema repository.
This allows us to at least develop locally (temporarily).
Partial resolution
Follow the instructions in the section Artifact Service on the page Dependency Management · OLP SDK. Although the build gets farther, it still isn't able to pull the dependency:
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from HERE_PLATFORM_REPO: https://repo.platform.here.com/artifactory/open-location-platform/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom
Downloading from OLP Public repo: https://artifactory.in.here.com/artifactory/here-olp-sit/com/example/djv/nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom
Downloading from HERE_PLATFORM_ARTIFACT: here+artifact-service://artifact-service/com.example.djv.nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom
Downloaded from HERE_PLATFORM_ARTIFACT: here+artifact-service://artifact-service/com.example.djv.nodecardinality/schema_v1_scala_2.11/1.0.0/schema_v1_scala_2.11-1.0.0.pom (3.4 kB at 1.6 kB/s)
Downloading from HERE_PLATFORM_REPO: https://repo.platform.here.com/artifactory/open-location-platform/com/example/djv/nodecardinality/schema_v1/1.0.0/schema_v1-1.0.0.pom
Downloading from OLP Public repo: https://artifactory.in.here.com/artifactory/here-olp-sit/com/example/djv/nodecardinality/schema_v1/1.0.0/schema_v1-1.0.0.pom
Downloading from HERE_PLATFORM_ARTIFACT: here+artifact-service://artifact-service/com.example.djv.nodecardinality/schema_v1/1.0.0/schema_v1-1.0.0.pom
Downloaded from HERE_PLATFORM_ARTIFACT: here+artifact-service://artifact-service/com.example.djv.nodecardinality/schema_v1/1.0.0/schema_v1-1.0.0.pom (8.0 kB at 6.3 kB/s)
Downloading from HERE_PLATFORM_REPO: https://repo.platform.here.com/artifactory/open-location-platform/com/example/djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
Downloading from OLP Public repo: https://artifactory.in.here.com/artifactory/here-olp-sit/com/example/djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
Downloading from HERE_PLATFORM_ARTIFACT: here+artifact-service://artifact-service/com.example.djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/example/djv/nodecardinality/1.0.0/nodecardinality-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
...
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project processor: Could not resolve dependencies for project com.example.nodecardinality:processor:jar:1.0.0: Failed to collect dependencies at com.example.djv.nodecardinality:schema_v1_scala_2.11:jar:1.0.0: Failed to read artifact descriptor for com.example.djv.nodecardinality:schema_v1_scala_2.11:jar:1.0.0: Could not find artifact com.example.djv:nodecardinality:pom:1.0.0 in HERE_PLATFORM_REPO (https://repo.platform.here.com/artifactory/open-location-platform/) -> [Help 1]

Answer 1

OLP Schemas are deployed to HERE Artifact Service. The service requires special authorization. In order to download a schema or java/scala binding you should include Maven Wagon in your project. Please refer to "Artifact Service" section in https://developer.here.com/olp/documentation/sdk-developer-guide/dev_guide/topics/dependency-management.html
In short:
Add version of the wagon as a property:
<artifact.wagon.version>1.6.1</artifact.wagon.version>
Add a placeholder repository to the list of repositories:
<repositories>
<repository>
<id>HERE_PLATFORM_ARTIFACT</id>
<layout>default</layout>
<url>here+artifact-service://artifact-service</url>
</repository>
</repositories>
Put the plugin reference in build section:
<extensions>
<extension>
<groupId>com.here.platform.artifact</groupId>
<artifactId>artifact-wagon</artifactId>
<version>${artifact.wagon.version}</version>
</extension>
</extensions>
Note that your need valid 'credentials.properties' file saved in ~/.here directory. How to get it please read https://developer.here.com/olp/documentation/sdk-developer-guide/dev_guide/topics/get-credentials.html
Best regards,
Dima

Answer 2

The issue is deployment with a parent Maven project. See the last sentence of the last error in the question:
Could not find artifact com.example.djv:nodecardinality:pom:1.0.0 in HERE_PLATFORM_REPO
The com.example.djv:nodecardinality:pom:1.0.0 POM was never uploaded when we ran mvn deploy from the schema directory. You cannot run mvn deploy from the top-level nodecardinality directory.
The simple solution is to remove the parent project pointer from the pom.xml in the schema directory before you run mvn deploy:
<!-- <parent>
<artifactId>nodecardinality</artifactId>
<groupId>com.example.djv</groupId>
<version>1.0.0</version>
</parent> -->