I am trying to generate class files for gRPC. I have a proto file containing:
syntax = "proto3";
option java_multiple_files = true;
package grpc.greeting;
message HelloRequest {
string firstName = 1;
string secondName = 2;
}
message HelloResponse {
string greeting = 1;
}
service HelloService {
rpc hello(HelloRequest) returns (HelloResponse);
}
I have put proto file in a package grpc/protos.
I have added dependencies and plugin in pom.xml.
'mvn clean install' is successful, but I don't see any files generated.
Seems location of proto is wrong. So where do I need to place it? I also tried by placing it at project root.
Related
I have the following types:
src/main/proto/mypackage/TypeA.proto:
syntax = "proto3";
package mypackage;
option java_package = "tld.domain.protos.mypackage";
option java_outer_classname = "TypeA";
import "mypackage/TypeB.proto";
message TypeA {
string field1 = 1;
TypeB field2 = 2;
}
src/main/proto/mypackage/TypeB.proto:
syntax = "proto3";
package mypackage;
option java_package = "tld.domain.protos.mypackage";
option java_outer_classname = "TypeB";
message TypeB {
string field1 = 1;
}
I am using Gradle to compile the types:
plugins {
...
id "com.google.protobuf" version "0.9.1"
}
...
sourceSets {
main {
java {
srcDirs 'build/generated/sources/proto/main/java'
}
}
}
...
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.21.9"
}
}
...
dependencies {
implementation 'com.google.protobuf:protobuf-java:3.21.9'
implementation 'com.google.protobuf:protobuf-java-util:3.21.9'
}
When I run ./gradlew clean build, I get the following error:
Execution failed for task ':generateProto'.
> protoc: stdout: . stderr: mypackage/TypeB.proto: File not found.
bop/WorkInstruction.proto:6:1: Import "mypackage/TypeB.proto" was not found or had errors.
bop/WorkInstruction.proto:65:3: "mypackage.TypeB" seems to be defined in "mypackage/TypeB.proto", which is not imported by "mypackage/TypeA.proto". To use it here, please add the necessary import.
I tried the following imports, but they all fail with the same error:
import "TypeB.proto";
import "/TypeB.proto";
import "mypackage/TypeB.proto";
import "mypackage.TypeB.proto";
import "proto/TypeB.proto";
I also tried leaving out the import (since files are in same package and directory):
> protoc: stdout: . stderr: mypackage/TypeA.proto:63:3: "mypackage.TypeB" seems to be defined in "mypackage/TypeB.proto", which is not imported by "mypackage/TypeA.proto". To use it here, please add the necessary import.
What is the correct way to import to use a type from another proto file?
I am trying to publish some jar artefacts on gitlab package registry but I get this error from the server :
Received status code 415 from server: Unsupported Media Type
Here is the publishing section of my build.gradle.kts :
publishing {
publications {
create<MavenPublication>("maven"){
artifact(tasks["bootJar"])
}
}
repositories {
maven {
url = uri("https://gitlab.com/api/v4/groups/my-group/-/packages/maven")
name = "Gitlab"
credentials(HttpHeaderCredentials::class) {
name = "Token"
value = System.getenv("CI_JOB_TOKEN")
}
authentication {
create<HttpHeaderAuthentication>("header")
}
}
}
}
In my gitlab-ci, I added a task for publish the artefacts :
deploy:
stage: deploy
script: gradle publish
only:
- master
Any help would be appreciated
Quick answer
Replace your publishing url pointing to the group-scope with the one pointing to the specific-package-repository, e.g. on gitlab.com:
https://gitlab.com/api/v4/projects/<your-project-id>/packages/maven
You need to replace <your-project-id> with your specific project-id of course.
Related to this a quote from docs.gitlab:
Note: In all cases, you need a project specific URL for uploading a package in the distributionManagement section.
Or in other words: Only the general repositories section can use your groups-url for searching other already published artifacts! (I also had to understand that). So:
you cannot publish to the group-package-store on gitlab, you can just search there.
Publication goes always to the project-specific package-store, which will then be visible at group-scope too.
Example gradle config (kotlin-dsl)
repositories {
mavenCenter()
jcenter()
// Here you USE the group api/v4 url for SEARCHING packages
maven {
name = "GitLab"
url = uri("https://gitlab.com/api/v4/groups/my-group/-/packages/maven")
credentials(HttpHeaderCredentials::class) {
name = "Job-Token"
value = System.getenv("CI_JOB_TOKEN")
}
authentication {
create<HttpHeaderAuthentication>("header")
}
}
}
publishing {
publications {
create<MavenPublication>("maven"){
artifact(tasks["bootJar"])
}
}
repositories {
maven {
// here your PROVIDE the PROJECT-URI for publishing your package
// in the project-specific package-space which is also visible at
// the group scope above
url = uri("https://gitlab.com/api/v4/projects/<your-project-id>/packages/maven")
name = "Gitlab"
credentials(HttpHeaderCredentials::class) {
name = "Job-Token"
value = System.getenv("CI_JOB_TOKEN")
}
authentication {
create<HttpHeaderAuthentication>("header")
}
}
}
}
More Info
There are multiple scenarios on how you may interact with the maven-repository-space on GitLab. The three switches are:
The place where you want to look for existing published packages
project-scope (https://.../api/v4/projects/<project-id>/packages/maven)
group-scope (https://.../api/v4/groups/<group-id>/-/packages/maven)
instance-scope (https://.../api/v4/packages/maven)
The authorization-method you want to use
PERSONAL_ACCESS_TOKEN
DEPLOY_TOKEN
CI_JOB_TOKEN
The place where your want to publish your package
this must always be a specific project-url (https://.../api/v4/projects/<project-id>/packages/maven)
I think the most important thing is to make sure you've enabled archives in your project:
Go to Project Settings
Expand Permissions
Switch on "Packages"
Apparently, there are also other reasons for the response status 415 Unsupported Media Type:
I ran into the same error message while trying to publish to the project repository. The error that I made was to use the URL-encoded path of the project instead of the project ID in the repository URL.
From the Gitlab documentation (emphasis added):
For retrieving artifacts, use either the URL-encoded path of the project
(like group%2Fproject) or the project's ID (like 42). However, only the
project's ID can be used for publishing.
This is the build.gradle.kts configuration that worked for me on my self-hosted Gitlab instance for publishing a Spring Boot fat JAR:
plugins {
/* ... other stuff ... */
`java-library`
`maven-publish`
}
publishing {
publications {
create<MavenPublication>("bootJava") {
artifact(tasks.getByName("bootJar"))
}
}
repositories {
maven {
val projectId = System.getenv("CI_PROJECT_ID")
name = "Project Name"
url = uri("https://gitlab.example.com/api/v4/projects/${projectId}/packages/maven")
credentials(HttpHeaderCredentials::class) {
name = "Job-Token"
value = System.getenv("CI_JOB_TOKEN")
}
authentication {
create<HttpHeaderAuthentication>("header")
}
}
}
}
I have a situation where I have two Gradle subprojects that are basically dumb protobuf containers. One sub project needs to import the definitions from the other, but I can't seem to figure out how to get the protobuf Gradle plugin to work correctly (I'm fairly new to Gradle).
Here's an example.
Directory layout:
build.gradle
settings.gradle
gradle.properties
dependency/
|
- build.gradle
- src/main/proto/dependency.proto
main/
|
- build.gradle
- src/main/proto/main.proto
build.gradle (top level)
plugins {
id "com.google.protobuf" version "0.8.12"
id "java"
}
configure (allprojects) {
apply plugin: 'java'
apply plugin: 'com.google.protobuf'
}
settings.gradle
rootProject.name = 'sample'
include 'main'
include 'dependency'
gradle.properties
group=sample
version=0.1.0
dependency/build.gradle
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.11.0"
}
}
dependencies {
compile 'com.google.protobuf:protobuf-java:3.11.4'
compile 'com.google.protobuf:protobuf-java-util:3.11.4'
}
dependency/src/main/proto/dependency.proto
syntax = "proto3";
package dependency;
option java_package = "dependency";
message DependencyRequest {
string foo = 1;
}
message DependencyResponse {
string bar = 1;
}
main/build.gradle
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.11.0"
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.27.2'
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
dependencies {
compile 'com.google.protobuf:protobuf-java:3.11.4'
compile 'com.google.protobuf:protobuf-java-util:3.11.4'
protobuf project(':dependency')
}
main/src/main/proto/main.proto
syntax = "proto3";
package main;
option java_package = "main";
import "dependency/dependency.proto";
service MainService {
rpc CallDependency(DependencyRequest) returns (DependencyResponse) {}
}
When I try to build, I get this error:
$ ./gradlew :main:compileJava
> Task :main:generateProto FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':main:generateProto'.
> protoc: stdout: . stderr: dependency/dependency.proto: File not found.
main.proto:7:1: Import "dependency/dependency.proto" was not found or had errors.
main.proto:11:24: "DependencyRequest" is not defined.
main.proto:11:52: "DependencyResponse" is not defined.
I read documentation for the plugin and this answer, but I haven't been able to get it to work. Any idea why the protobuf task isn't picking up dependency.proto?
Apparently the protobuf files were getting set up correctly, but there was a mismatch in the package names that cause the main.proto to not be able to find dependency.proto. I was able to get this working by removing the package names in the protobuf definitions.
If anyone runs into this problem in the future, check your package names and imports to make sure they're correct.
I'm trying to write tests for a build process, so my unit test wants to replace the actual repository locations with local locations to avoid poisoning the real server. (Plus, I suppose, the person running the test might not have access to publish anyway.)
In the build itself:
publishing {
repositories {
maven {
name = 'snapshot'
url = "${artifactory_contextUrl}/libs-snapshot-local"
credentials {
username artifactory_user
password artifactory_password
}
}
maven {
name = 'release'
url = "${artifactory_contextUrl}/libs-release-local"
credentials {
username artifactory_user
password artifactory_password
}
}
}
}
In my test build, I'm trying to override it with this:
publishing {
repositories {
getByName('snapshot') {
url = uri('/tmp/local-repo/snapshots')
}
getByName('release') {
url = uri('/tmp/local-repo/release')
}
}
}
When I try to run the build, I get:
Execution failed for task ':publishMavenJavaPublicationToReleaseRepository'.
> Failed to publish publication 'mavenJava' to repository 'release'
> Authentication scheme 'all'(Authentication) is not supported by protocol 'file'
There are a lot of posts out on the web about this specific error, but it always seems to be people who accidentally put a file path in when they should have put a URI. I'm putting in a URI deliberately, though, so is there a way to get this to work?
I have also tried this:
publishing {
repositories {
clear()
maven {
name = 'snapshot'
url = uri('/tmp/local-repo/snapshots')
}
maven {
name = 'release'
url = uri('/tmp/local-repo/release')
}
}
}
That fails with:
A problem occurred configuring root project 'test-common-plugin1913987501683151177'.
> Exception thrown while executing model rule: PublishingPluginRules#publishing(ExtensionContainer)
> Cannot add task 'publishMavenJavaPublicationToSnapshotRepository' as a task with that name already exists.
I was surprised that deleting all the repositories doesn't also delete all the tasks they own. When I try to programmatically delete the task it's complaining about, Gradle claims that it doesn't exist.
Alright, I ended up having to read the source of Gradle (again), but I found a way to do it. Essentially you can directly set credentials back to null, like this:
publishing {
repositories {
getByName('snapshot') {
url = uri('/tmp/local-repo/snapshots')
configuredCredentials = null
}
getByName('release') {
url = uri('/tmp/local-repo/release')
configuredCredentials = null
}
}
}
I am writing a little gradle plugin that needs to expose configuration for two source locations. I am presently doing this via a project extension and the consumer gradle file would have syntax like:
generator {
idl {
baseDir = "src"
include = "*.idl"
exclude = "**/.svn"
}
template {
baseDir = "src"
include = "*.template"
exclude = "**/.svn"
}
}
My first attempt at this was accomplished by having two properties on the extension class like so:
class GeneratorExtension {
public static final NAME = "generator"
private Project project
GeneratorExtension(Project project) {
this.project = project
}
#Input ConfigurableFileTree idl
#Input ConfigurableFileTree template
void idl(String dir, Closure closure) {
idl = project.fileTree(dir, closure)
}
void template(String dir, Closure closure) {
template = project.fileTree(dir, closure)
}
}
Which changes the gradle file syntax to:
generator {
idl (dir = "src") {
include = "*.idl"
exclude = "*"
}
template (dir = "src") {
include = "*.template"
exclude = ""
}
}
Now this works, but it doesn't give me the clean usability I would prefer. I'm fairly new to both gradle and groovy, it looks like I might be able to accomplish what I want through the use of a DefaultNamedDomainObjectSet, but that references internal classes and so that's why I'm here.
What would be the correct way to add an extension uses FileTree, but keeps the clean consumer syntax?
Thanks in advance.
Thanks to this and Peter's comment above (which means doubly thanks to Peter) I have come up with a solution. The introduction of the NamedDomainObjectContainer into my thinking allowed an even better syntax for the consumer.
Here's what I ended up with:
generator {
idl {
dir = 'src'
include = "*.idl"
exclude = "*"
}
templates {
java {
dir = 'src'
include = '*.template'
}
javascript {
dir = 'src'
include = '*.template'
}
}
}
And the plugin code:
class GeneratorPlugin implements Plugin<Project> {
private GeneratorExtension extension
void apply(Project project) {
def templates = project.container(FileTreeContainer)
extension = project.extensions.create(GeneratorExtension.NAME, GeneratorExtension, project, templates)
applyTasks(project)
}
void applyTasks(final Project project) {
project.task('generateJava', type: GenerateJavaTask, group: 'Preprocess', description: 'Generate Java files based on provided IDL files and templates') { }
project.task('generateJavascript', type: GenerateJavascriptTask, group: 'Preprocess', description: 'Generate Javascript files based on provided IDL files and templates') { }
}
}
class GeneratorExtension {
public static final NAME = "generator"
private Project project
#Input FileTreeContainer idl
#Input NamedDomainObjectContainer<FileTreeContainer> templates
GeneratorExtension(Project project, NamedDomainObjectContainer<FileTreeContainer> templates) {
this.project = project
idl = new FileTreeContainer("idl")
this.templates = templates
}
void idl(Closure closure) {
project.configure(idl, closure)
idl.fileTree = project.fileTree(idl.dir) {
include idl.include
exclude idl.exclude
}
}
void templates(Closure closure) {
templates.configure(closure)
String foo = "hello"
}
}