My directory structure is src/ps/proto. My build.gradle file is located in the src directory. I've set sourceSets to
sourceSets {
ps {
proto {
srcDir 'ps/proto'
}
}
}
Yet, a gradlew generatePsProto gives me a slew of error, one of which is that my source directory is src/src/ps/proto.
Execution failed for task ':generatePsProto'.
> protoc: stdout: . stderr: /home/build/tree/src/src/ps/proto: warning: directory does not exist.
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: cldb.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
ps/proto/security.proto: File not found.
ps/proto/common.proto: File not found.
ps/proto/cli.proto: File not found.
ps/proto/volumemirrorcommon.proto: File not found.
ps/proto/metrics.proto: File not found.
cldb.proto: Import "ps/proto/security.proto" was not found or had errors.
cldb.proto: Import "ps/proto/common.proto" was not found or had errors.
mldb.proto: Import "ps/proto/cli.proto" was not found or had errors.
mldb.proto:214:12: "CredentialsMsg" is not defined.
mldb.proto:218:12: "CredentialsMsg" is not defined.
[...]
mldb.proto:3614:12: "Key" is not defined.
mldb.proto:3618:12: "CredentialsMsg" is not defined.
mldb.proto:3619:12: "ServerKeyType" is not defined.
I don't want the Gradle default of src/ps/proto. I want ps/proto. Can this be done? My goal is to remove the hard coded exec calls to protoc and use the protobuf pluging to compile the *.proto files into their respective *.h, *.cc, and eventually into *.o files.
Gradle 4.7 Build time: 2018-04-18 09:09:12 UTC Revision:
b9a962bf70638332300e7f810689cb2febbd4a6c
Groovy: 2.4.12 Ant: Apache Ant(TM) version 1.9.9
compiled on February 2 2017 JVM: 1.8.0_144 (Oracle
Corporation 25.144-b01) OS: Linux 3.10.0-514.el7.x86_64
amd64
build.gradle
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.5'
}
}
apply plugin: 'c'
apply plugin: 'cpp'
apply plugin: 'java'
apply plugin: 'com.google.protobuf'
repositories {
mavenCentral()
}
dependencies {
compile "com.google.protobuf:protobuf-java:2.4.1"
}
sourceSets {
ps {
proto {
srcDir 'ps/proto'
}
}
}
protobuf {
protoc {
artifact = 'com.google.protobuf:protoc:2.4.1'
}
generateProtoTasks {
all().each { task ->
task.plugins {
cpp {}
}
}
}
}
Gradle has a widely applicable convention that collection properties can be configured with methods of the same name. The key difference between setting a property using assignment (=) and using a method is that the latter appends its values.
So, to override the default source directory for the source set you need to use assignment:
sourceSets {
ps {
proto {
srcDirs = ['ps/proto']
}
}
}
I don't really understand why you're getting an error message about src/src/ps/proto, so there may be some other issue. That error seems to be coming from protoc rather than Gradle.
EDIT
To solve your import problem, you should use
import security.proto;
i.e. without the source directory path. This is because the Protobuf Gradle Plugin appears to add each source directory as an "include directory". protoc only looks in such include directories for imports.
This is at least what I have garnered from the documentation and source code.
You seemed to have typo either for 'fs' or 'ps'.
The problem is the Import.
because you have for example
cldb.proto: Import "ps/proto/security.proto"
mldb.proto: Import "ps/proto/cli.proto"
Edited:
you have to put every .proto file in src/yourCustomSourceSetName/proto/ps/proto/ and add
sourceSets {
yourCustomSourceSetName {}
}
dependencies {
compile ... // dependencies for compileJava task
yourCustomSourceSetNameCompile ... // dependencies for custom sourceSet
compile sourceSets.yourCustomSourceSetName.output // compileJava uses output of the custom sourceSet
}
Then the import will look for other .proto files in the right place.
Related
I have a situation where I have two Gradle subprojects that are basically dumb protobuf containers. One sub project needs to import the definitions from the other, but I can't seem to figure out how to get the protobuf Gradle plugin to work correctly (I'm fairly new to Gradle).
Here's an example.
Directory layout:
build.gradle
settings.gradle
gradle.properties
dependency/
|
- build.gradle
- src/main/proto/dependency.proto
main/
|
- build.gradle
- src/main/proto/main.proto
build.gradle (top level)
plugins {
id "com.google.protobuf" version "0.8.12"
id "java"
}
configure (allprojects) {
apply plugin: 'java'
apply plugin: 'com.google.protobuf'
}
settings.gradle
rootProject.name = 'sample'
include 'main'
include 'dependency'
gradle.properties
group=sample
version=0.1.0
dependency/build.gradle
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.11.0"
}
}
dependencies {
compile 'com.google.protobuf:protobuf-java:3.11.4'
compile 'com.google.protobuf:protobuf-java-util:3.11.4'
}
dependency/src/main/proto/dependency.proto
syntax = "proto3";
package dependency;
option java_package = "dependency";
message DependencyRequest {
string foo = 1;
}
message DependencyResponse {
string bar = 1;
}
main/build.gradle
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.11.0"
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.27.2'
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
dependencies {
compile 'com.google.protobuf:protobuf-java:3.11.4'
compile 'com.google.protobuf:protobuf-java-util:3.11.4'
protobuf project(':dependency')
}
main/src/main/proto/main.proto
syntax = "proto3";
package main;
option java_package = "main";
import "dependency/dependency.proto";
service MainService {
rpc CallDependency(DependencyRequest) returns (DependencyResponse) {}
}
When I try to build, I get this error:
$ ./gradlew :main:compileJava
> Task :main:generateProto FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':main:generateProto'.
> protoc: stdout: . stderr: dependency/dependency.proto: File not found.
main.proto:7:1: Import "dependency/dependency.proto" was not found or had errors.
main.proto:11:24: "DependencyRequest" is not defined.
main.proto:11:52: "DependencyResponse" is not defined.
I read documentation for the plugin and this answer, but I haven't been able to get it to work. Any idea why the protobuf task isn't picking up dependency.proto?
Apparently the protobuf files were getting set up correctly, but there was a mismatch in the package names that cause the main.proto to not be able to find dependency.proto. I was able to get this working by removing the package names in the protobuf definitions.
If anyone runs into this problem in the future, check your package names and imports to make sure they're correct.
I'm fairly new to Groovy and I'm trying to wrap my head around Gradle. If I import the org.jvnet.hudson.plugins through Grapes it works perfectly and the dependency is resolved. But if I try to retrieve the dependency using Gradle the dependency is not resolved.
The package org.eclipse.hudson:hudson-core:3.2.1 works with both Gradle and Grape.
A dependency that is not resolved using Gradle
compile 'org.jvnet.hudson.plugins:checkstyle:3.42'
A dependency which is resolved using Grape
#Grab('org.jvnet.hudson.plugins:checkstyle:3.42')
A dependency which is resolved using Gradle
compile 'org.eclipse.hudson:hudson-core:3.2.1'
Error during Gradle build
line 3, column 1.
import hudson.plugins.checkstyle.CheckStyleResultAction;
^
The build.gradle
apply plugin: 'groovy'
repositories {
mavenCentral()
maven {
url "http://repo.jenkins-ci.org/releases/"
}
}
configurations {
ivy
}
sourceSets {
main {
groovy {
srcDirs = ['src/']
}
}
test {
groovy {
srcDirs = ['test/']
}
}
}
dependencies {
compile 'org.codehaus.groovy:groovy-all:2.4.11'
compile "org.apache.ivy:ivy:2.4.0"
ivy "org.apache.ivy:ivy:2.3.0"
// Works
compile 'org.eclipse.hudson:hudson-core:3.2.1'
// Does not work
compile 'org.jvnet.hudson.plugins:checkstyle:3.42'
}
tasks.withType(GroovyCompile) {
groovyClasspath += configurations.ivy
}
You're probably not actually downloading the jar you think you are. Looks like the default artifact that comes back from the org.jvnet.hudson.plugins:checkstyle:3.42 dependency is actually a file named checkstyle-3.42.hpi.
To get the jar which contains the classes instead, use:
compile group: 'org.jvnet.hudson.plugins', name: 'checkstyle', version:'3.42', ext: 'jar'
Then that class will be found on your classpath (and you'll be on to locating the next missing dependency).
Gradle project deprecated 'classesDir' so the previously working method:
sourceSets {
main {
output.classesDir = "myDir"
}
}
should be replaced with something else. Documentation talks about 'output.classesDirs' but this is read-only property.
What is the method to specify custom compilation output directory in Gradle 4.x scripts?
If you are working with java you can do this
apply plugin: 'java'
sourceSets {
main {
// Compiled Java classes should use this directory
java.outputDir = file('myDir')
}
}
See more: https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/SourceSetOutput.html
As per Gradle 6.5.1 docs the java.outputDir property is has been replaced by classesDirectory:
Gradle 6.5.1 docs for SourceDirectorySet
However I think that the destinationDirectory property should be used to read or modify the compiler output dir. So the docs should say that it is replaced by the destinationDirectory property rather than the classesDirectory property.
The compiler output directory can be changed using either of the following ways:
sourceSets {
main {
java {
destinationDirectory.set(file("${project.buildDir}/classes/${sourceSets.main.name}/java"))
}
}
}
OR
sourceSets {
main {
java {
destinationDirectory.value(project.getLayout().getBuildDirectory().dir("classes/${sourceSets.main.name}/java"));
}
}
}
In my opinion, the second option is better.
To read the output dir for a particular sourceSet use:
project.sourceSets.main.java.destinationDirectory.get()
I'm trying to compile some protobuf definitions as a gradle task, but get the following error with no source generation:
* What went wrong:
Execution failed for task ':generateProto'.
> protoc: stdout: . stderr: /Users/ash/IdeaProjects/kotlin/grpc/build/extracted-protos/main: warning: directory does not exist.
service-definitions.proto:10:17: "Empty" is not defined.
service-definitions.proto:10:33: "Empty" is not defined.
service-definitions.proto:15:17: "SimpleRequest" is not defined.
service-definitions.proto:15:41: "SimpleResponse" is not defined.
service-definitions.proto:21:27: "StreamingOutputCallRequest" is not defined.
service-definitions.proto:21:71: "StreamingOutputCallResponse" is not defined.
service-definitions.proto:27:33: "StreamingInputCallRequest" is not defined.
service-definitions.proto:27:69: "StreamingInputCallResponse" is not defined.
service-definitions.proto:34:29: "StreamingOutputCallRequest" is not defined.
service-definitions.proto:34:73: "StreamingOutputCallResponse" is not defined.
service-definitions.proto: warning: Import empty.proto but not used.
service-definitions.proto: warning: Import messages.proto but not used.
This only occurs when I try to compile a definition that imports other definitions. If I remove the service-definitions.proto definition, the others compile and sources are generated.
The protobuf definitions are available from the vertx-grpc examples and I have them in $projectDir/src/main/proto. I've not changed their default build location, so sources are being generated to $projectDir/build/generated/source/proto. Instead of an extracted-protos directory, however, I have extracted-include-protos (probably explains the directory not found error).
Here's my gradle.build file, should it be of use. Note that the gRPC plugin uses the Vert.x artifact rather than the version from io.grpc.. (it's actually just a wrapper over the top of it). I have tried swapping the Vert.x version out for the io.grpc version and get the same error.
group 'grpc'
version '1.0-SNAPSHOT'
buildscript {
ext.kotlin_version = '1.1.3'
repositories {
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "com.google.protobuf:protobuf-gradle-plugin:0.8.1"
}
}
apply plugin: 'kotlin'
apply plugin: "java"
apply plugin: "com.google.protobuf"
apply plugin: "idea"
protobuf {
//generatedFilesBaseDir = "$projectDir/build/generated/source/proto"
protoc {
artifact = "com.google.protobuf:protoc:3.3.0"
}
plugins {
grpc {
artifact = "io.vertx:protoc-gen-grpc-java:1.3.0"
}
}
generateProtoTasks {
all().each { task ->
task.plugins {
grpc {}
}
}
}
}
clean {
delete protobuf.generatedFilesBaseDir
}
idea {
module {
sourceDirs += file("${protobuf.generatedFilesBaseDir}/main/java")
sourceDirs += file("${protobuf.generatedFilesBaseDir}/main/grpc")
}
}
sourceSets {
main {
java {
srcDirs += "${protobuf.generatedFilesBaseDir}/main/java"
srcDirs += "${protobuf.generatedFilesBaseDir}/main/grpc"
}
}
}
repositories {
jcenter()
}
dependencies {
compile "org.jetbrains.kotlin:kotlin-stdlib-jre8:$kotlin_version"
compile "io.vertx:vertx-core:3.4.2"
compile "io.vertx:vertx-web:3.4.2"
compile "io.vertx:vertx-grpc:3.4.2"
}
compileKotlin {
kotlinOptions {
jvmTarget = "1.8"
apiVersion = "1.1"
languageVersion = "1.1"
}
}
compileTestKotlin {
kotlinOptions {
jvmTarget = "1.8"
apiVersion = "1.1"
languageVersion = "1.1"
}
}
I've not changed any settings, why is it (a) looking for extracted files in the wrong location, and (b) looking for extracted files at all when importing, given that the necessary definitions are all in the src/main/proto directory?
Okay, after much head scratching, the clue was in the last two lines of the error message:
service-definitions.proto: warning: Import empty.proto but not used.
service-definitions.proto: warning: Import messages.proto but not used.
In my proto definitions, I had specified messages and empty to be in the messages package but hadn't qualified the types when used in the importing definition, which is in the services package. Alas, a simple change and it works:
package services;
service EmptyPingPongService {
// One empty request followed by one empty response.
rpc EmptyCall(Empty) returns (Empty);
}
Becomes:
package services;
service EmptyPingPongService {
// One empty request followed by one empty response.
rpc EmptyCall(messages.Empty) returns (messages.Empty);
}
I am using the below mentioned protobuf gradle plugin in one project where its working fine but when I referenced the same plugin in a different project, 'gradle clean' is consistently giving me the error copied below:
relevant parts of build.grade (v3.4)
apply plugin: 'com.google.protobuf'
buildscript {
repositories {
mavenCentral()
mavenLocal()
jcenter()
}
dependencies {
// classpath "net.ltgt.gradle:gradle-errorprone-plugin:0.0.9"
// classpath 'com.github.jengelman.gradle.plugins:shadow:1.2.4'
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.0'
}
}
def grpcVersion = '1.1.2'
dependencies {
compile "io.grpc:grpc-netty:${grpcVersion}"
compile "io.grpc:grpc-protobuf:${grpcVersion}"
compile "io.grpc:grpc-stub:${grpcVersion}"
}
protobuf {
protoc {
Artifact = 'com.google.protobuf:protoc:3.2.0'
}
plugins {
grpc {
Artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}"
}
}
generateProtoTasks {
all()*.plugins {
grpc {
// To generate deprecated interfaces and static bindService method,
// turn the enable_deprecated option to true below:
option 'enable_deprecated=false'
}
}
}
}
error when I run gradle clean
* What went wrong:
Could not compile build file '/xyz/xyz/build.gradle'.
> startup failed:
build file '/xyz/xyz/build.gradle': 102: you tried to assign a value to the class 'org.gradle.api.component.Artifact'
# line 102, column 9.
Artifact = 'com.google.protobuf:protoc:3.2.0'
^
build file '/xyz/xyz/build.gradle': 106: you tried to assign a value to the class 'org.gradle.api.component.Artifact'
# line 106, column 13.
Artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}"
I have tried protobuf plugins 0.8.0. and 0.8.1 but both give the same error. v0.8.0 works as is in a different project. Any thoughts on how to troubleshoot this further would be appreciated.
It should be artifact, not Artifact. The latter is a class that you try to assign to which will not work, the former is a property you assign to.