gRPC-server (which is spring boot app) to upload a file to GCS (google cloud storage) bucket - spring-boot

Question

My team created two spring boot projects named grpc-client and grpc-server.
grpc-client will call grpc-server and then grpc-server will save the data to the Mongodb.
I need to write the code for uploading the file to a GCS (google cloud storage) bucket.
I'm getting below error when I run as SpringBoot App for grpc-server.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'app': Invocation of init method failed; nested exception is com.google.cloud.storage.StorageException: storage.googleapis.com
I wrote the code in grpc-server as below
App class:
import java.io.IOException;
import javax.annotation.PostConstruct;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import com.hsbc.grpc.cdc.server.FileWriter;
#SpringBootApplication
public class App
{
public static void main(String[] args) throws IOException {
SpringApplication.run(App.class, args);
App app = new App();
app.myMethod();
}
#Autowired
private FileWriter myservice;
#Value("${file.storage}")
public String filePath;
#PostConstruct
private void myMethod() throws IOException{
System.out.println("storing the data into the bcuket");
myservice.uploadObject("hsbc-9802305-cde-dev","risk-hsbc-9802305-cde-dev-europe-west2-journaling","data",filePath);
}
/**
* #return the myservice
*/
public FileWriter getMyservice() {
return myservice;
}
/**
* #param myservice the myservice to set
*/
public void setMyservice(FileWriter myservice) {
this.myservice = myservice;
}
}
FileWriter Class:
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
#Component
public class FileWriter {
// The ID of your GCP project
String projectId = "hsbc-9802305-cde-dev";
// The ID of your GCS bucket
String bucketName = "cde-env-sidecar-test";
// The ID of your GCS object
String objectName = "data";
// The path to your file to upload
#Value("${file.storage}")
String filePath;
public static void uploadObject(
String projectId, String bucketName, String objectName, String filePath) throws IOException {
Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
BlobId blobId = BlobId.of(bucketName, objectName);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();
storage.create(blobInfo, Files.readAllBytes(Paths.get(filePath)));
System.out.println(
"File " + filePath + " uploaded to bucket " + bucketName + " as " + objectName);
}
}
Application.properties:
mongodb.serverNames=gbl20075862.hc.cloud.uk.hsbc:5255,gbl20075872.hc.cloud.uk.hsbc:5255,gbl20076400.hc.cloud.uk.hsbc:5255,gbl20078778.hc.cloud.uk.hsbc:5255,gbl20076399.hc.cloud.uk.hsbc:5255
mongodb.username=CREUAT
mongodb.password=cre#1234
mongodb.database=DAIcebergTest
mongodb.connectionsPerHost=3
mongodb.connectionTimeOutMillis=10000
mongodb.authenticationDatabase=DAIcebergTest
mongodb.maxWaitTime=120000
mongodb.sslEnabled=true
file.storage=C:/SHARVANI/CodeBase/GRPC/json.json
spring.cloud.gcp.credentials.location=file:C:/SHARVANI/CodeBase/GRPC/CDE-Grpc-Server/src/main/resources/hsbc-json.json
I also set the environmental variable GOOGLE_APPLICATION-CREDENTAILS to PATH in my local user account.
I also added spring.cloud.gcp.credentials.location in application.properties
Please help me solve the issue as earliest.
The whole stacktrace:
The Class-Path manifest attribute in C:\sandbox\.m2\my-repo\io\grpc\grpc-netty-shaded\1.27.1\grpc-netty-shaded-1.27.1.jar referenced one or more files that do not exist: file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/grpc-core-1.27.1.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/grpc-api-1.27.1.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/gson-2.8.6.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/annotations-4.1.1.4.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/error_prone_annotations-2.3.4.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/perfmark-api-0.19.0.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/grpc-context-1.27.1.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/jsr305-3.0.2.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/animal-sniffer-annotations-1.18.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/guava-28.1-android.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/failureaccess-1.0.1.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/checker-compat-qual-2.5.5.jar,file:/C:/sandbox/.m2/my-repo/io/grpc/grpc-netty-shaded/1.27.1/j2objc-annotations-1.3.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.6.RELEASE)
2020-07-04 19:23:23.575 INFO 38364 --- [ restartedMain] c.h.g.App : Starting App on A35600L58EZ7D6T with PID 38364 (C:\SHARVANI\CodeBase\GRPC\CDE-Grpc-Server\target\classes started by 45063911 in C:\SHARVANI\CodeBase\GRPC\CDE-Grpc-Server)
2020-07-04 19:23:23.587 INFO 38364 --- [ restartedMain] c.h.g.App : No active profile set, falling back to default profiles: default
2020-07-04 19:23:23.706 INFO 38364 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2020-07-04 19:23:23.706 INFO 38364 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : For additional web related logging consider setting the 'logging.level.web' property to 'DEBUG'
2020-07-04 19:23:25.692 INFO 38364 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data repositories in DEFAULT mode.
2020-07-04 19:23:25.818 INFO 38364 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 118ms. Found 0 repository interfaces.
2020-07-04 19:23:27.398 INFO 38364 --- [ restartedMain] o.s.b.w.e.t.TomcatWebServer : Tomcat initialized with port(s): 0 (http)
2020-07-04 19:23:27.457 INFO 38364 --- [ restartedMain] o.a.c.c.StandardService : Starting service [Tomcat]
2020-07-04 19:23:27.458 INFO 38364 --- [ restartedMain] o.a.c.c.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.21]
2020-07-04 19:23:27.827 INFO 38364 --- [ restartedMain] o.a.c.c.C.[.[.[/] : Initializing Spring embedded WebApplicationContext
2020-07-04 19:23:27.827 INFO 38364 --- [ restartedMain] o.s.w.c.ContextLoader : Root WebApplicationContext: initialization completed in 4121 ms
storing the data into the bcuket
2020-07-04 19:23:28.621 INFO 38364 --- [ restartedMain] c.g.a.o.ComputeEngineCredentials : Failed to detect whether we are running on Google Compute Engine.
2020-07-04 19:23:29.214 WARN 38364 --- [ restartedMain] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'app': Invocation of init method failed; nested exception is com.google.cloud.storage.StorageException: storage.googleapis.com
2020-07-04 19:23:29.219 INFO 38364 --- [ restartedMain] o.a.c.c.StandardService : Stopping service [Tomcat]
2020-07-04 19:23:29.240 INFO 38364 --- [ restartedMain] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-07-04 19:23:29.243 ERROR 38364 --- [ restartedMain] o.s.b.SpringApplication : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'app': Invocation of init method failed; nested exception is com.google.cloud.storage.StorageException: storage.googleapis.com
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:139) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at com.hsbc.grpc.App.main(App.java:21) [classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_66]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_66]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_66]
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-2.1.6.RELEASE.jar:2.1.6.RELEASE]
Caused by: com.google.cloud.storage.StorageException: storage.googleapis.com
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:227) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:308) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.StorageImpl$3.call(StorageImpl.java:203) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.StorageImpl$3.call(StorageImpl.java:200) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) ~[gax-1.57.0.jar:1.57.0]
at com.google.cloud.RetryHelper.run(RetryHelper.java:76) ~[google-cloud-core-1.93.6.jar:1.93.6]
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) ~[google-cloud-core-1.93.6.jar:1.93.6]
at com.google.cloud.storage.StorageImpl.internalCreate(StorageImpl.java:199) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:161) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.hsbc.grpc.cdc.server.FileWriter.uploadObject(FileWriter.java:39) ~[classes/:?]
at com.hsbc.grpc.App.myMethod(App.java:35) ~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_66]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_66]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_66]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
... 23 more
Caused by: java.net.UnknownHostException: storage.googleapis.com
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[?:1.8.0_66]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[?:1.8.0_66]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_66]
at java.net.Socket.connect(Socket.java:589) ~[?:1.8.0_66]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668) ~[?:1.8.0_66]
at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[?:1.8.0_66]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) ~[?:1.8.0_66]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) ~[?:1.8.0_66]
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264) ~[?:1.8.0_66]
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367) ~[?:1.8.0_66]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) ~[?:1.8.0_66]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105) ~[?:1.8.0_66]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999) ~[?:1.8.0_66]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) ~[?:1.8.0_66]
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283) ~[?:1.8.0_66]
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258) ~[?:1.8.0_66]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250) ~[?:1.8.0_66]
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:113) ~[google-http-client-1.35.0.jar:1.35.0]
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84) ~[google-http-client-1.35.0.jar:1.35.0]
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) ~[google-http-client-1.35.0.jar:1.35.0]
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithoutGZip(MediaHttpUploader.java:551) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequest(MediaHttpUploader.java:568) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.api.client.googleapis.media.MediaHttpUploader.directUpload(MediaHttpUploader.java:360) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:334) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:551) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592) ~[google-api-client-1.30.9.jar:1.30.9]
at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:305) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.StorageImpl$3.call(StorageImpl.java:203) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.StorageImpl$3.call(StorageImpl.java:200) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) ~[gax-1.57.0.jar:1.57.0]
at com.google.cloud.RetryHelper.run(RetryHelper.java:76) ~[google-cloud-core-1.93.6.jar:1.93.6]
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) ~[google-cloud-core-1.93.6.jar:1.93.6]
at com.google.cloud.storage.StorageImpl.internalCreate(StorageImpl.java:199) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:161) ~[google-cloud-storage-1.109.1.jar:1.109.1]
at com.hsbc.grpc.cdc.server.FileWriter.uploadObject(FileWriter.java:39) ~[classes/:?]
at com.hsbc.grpc.App.myMethod(App.java:35) ~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_66]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_66]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_66]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
... 23 more
Picked up JAVA_TOOL_OPTIONS: -Duser.home=C:\Users\45063911

Answer 1

Caused by: java.net.UnknownHostException: storage.googleapis.com
Looks like you have a network problem.
Check that you can reach the server (Google storage) with ping :
ping storage.googleapis.com
You may also test the url from your browser
If you're behind a proxy or on an enterprise network you may have to contact your network administrators and ask then to allow acces to google API.