I develop a web service whit springboot and I need it save some info in my impala database.
I searching about save in impala and I only find how to use the JDBC.
Someone know if I can use spring boot and spring jpa to save data in impala?
My .yml
server:
port: 8080
spring:
config:
name: **
cloud:
loadbalancer:
ribbon:
enable: false
datasource:
url: jdbc:impala://...
username: ...
password: ...
driver-class-name: io.opentracing.contrib.jdbc.TracingDriver (I think this is not the correct but i find another)
My pom.xml
<dependencies>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<dependency>
<groupId>com.cloudera.impala.jdbc</groupId>
<artifactId>ImpalaJDBC41</artifactId>
<version>2.6.4.1005</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>es.caser.archit.spring</groupId>
<artifactId>db-caf-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>es.caser.archit.spring</groupId>
<artifactId>rest-caf-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>es.caser.archit.spring</groupId>
<artifactId>test-caf-boot-starter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>es.caser.archit.spring</groupId>
<artifactId>security-caf-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>com.github.ulisesbocchio</groupId>
<artifactId>jasypt-spring-boot-starter</artifactId>
<version>2.1.0</version>
</dependency>
</dependencies>
Thanks :D
The driver class name should be
driverClassName: org.apache.hive.jdbc.HiveDriver
And use this dependency
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>3.1.2</version>
</dependency>
Related
StreamBridge cannot be injected (using #Autowired) into a service and causes the following error:
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.cloud.stream.function.StreamBridge' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {#org.springframework.beans.factory.annotation.Autowired(required=true)}
pom.xml (partial)
<repositories>
<repository>
<id>confluent</id>
<url>https://packages.confluent.io/maven/</url>
</repository>
</repositories>
<properties>
<!-- Build properties -->
<maven.version>3.2.5</maven.version>
<java.version>11</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.build.timestamp.format>yyyyMMddHHmmss</maven.build.timestamp.format>
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<start-class>com.smarthome.actuatorms.ActuatormsApp</start-class>
<argLine>-Djava.security.egd=file:/dev/./urandom -Xmx1G</argLine>
<m2e.apt.activation>jdt_apt</m2e.apt.activation>
<run.addResources>false</run.addResources>
<jhipster-dependencies.version>7.9.3</jhipster-dependencies.version>
<spring-boot.version>3.0.2</spring-boot.version>
<archunit-junit5.version>0.22.0</archunit-junit5.version>
<mapstruct.version>1.5.2.Final</mapstruct.version>
<!-- Plugin versions -->
<maven-clean-plugin.version>3.2.0</maven-clean-plugin.version>
<maven-site-plugin.version>3.12.1</maven-site-plugin.version>
<maven-compiler-plugin.version>3.10.1</maven-compiler-plugin.version>
<maven-javadoc-plugin.version>3.4.1</maven-javadoc-plugin.version>
<maven-eclipse-plugin.version>2.10</maven-eclipse-plugin.version>
<maven-enforcer-plugin.version>3.1.0</maven-enforcer-plugin.version>
<maven-failsafe-plugin.version>3.0.0-M7</maven-failsafe-plugin.version>
<maven-jar-plugin.version>3.2.2</maven-jar-plugin.version>
<maven-idea-plugin.version>2.2.1</maven-idea-plugin.version>
<maven-resources-plugin.version>3.3.0</maven-resources-plugin.version>
<maven-surefire-plugin.version>3.0.0-M7</maven-surefire-plugin.version>
<maven-war-plugin.version>3.3.2</maven-war-plugin.version>
<maven-checkstyle-plugin.version>3.1.2</maven-checkstyle-plugin.version>
<checkstyle.version>10.3.2</checkstyle.version>
<nohttp-checkstyle.version>0.0.10</nohttp-checkstyle.version>
<git-commit-id-plugin.version>5.0.0</git-commit-id-plugin.version>
<modernizer-maven-plugin.version>2.4.0</modernizer-maven-plugin.version>
<jacoco-maven-plugin.version>0.8.8</jacoco-maven-plugin.version>
<properties-maven-plugin.version>1.1.0</properties-maven-plugin.version>
<sonar-maven-plugin.version>3.9.1.2184</sonar-maven-plugin.version>
<avro.version>1.11.1</avro.version>
<avro-compiler.version>1.11.1</avro-compiler.version>
<avro-maven-plugin.version>1.11.1</avro-maven-plugin.version>
<kafka-avro-serializer.version>7.3.1</kafka-avro-serializer.version>
<spring-cloud-stream-schema.version>2.2.1.RELEASE</spring-cloud-stream-schema.version>
<spring-cloud-stream.version>4.0.1</spring-cloud-stream.version>
<spring-boot-starter-validation.version>3.0.2</spring-boot-starter-validation.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>tech.jhipster</groupId>
<artifactId>jhipster-dependencies</artifactId>
<version>${jhipster-dependencies.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
<version>${spring-boot-starter-validation.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
<version>${spring-cloud-stream.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-schema</artifactId>
<version>${spring-cloud-stream-schema.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>${avro.version}</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-compiler</artifactId>
<version>${avro-compiler.version}</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>${avro-maven-plugin.version}</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-avro-serializer</artifactId>
<version>${kafka-avro-serializer.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Spring Cloud -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
</dependencies>
Please note that these dependencies are not exhaustive and I've included the ones that are most relevant to Spring Cloud Stream.
In the application.yml properties I have included the properties for Spring Cloud stream.
spring:
cloud:
stream:
kafka:
binder:
brokers:
- localhost:9092
producer-properties:
key.serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
value.serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
schema.registry.url: http://localhost:8081
consumer-properties:
key.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
value.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
schema.registry.url: http://localhost:8081
specific.avro.reader: true
bindings:
switchArrayStateProducer-out-0:
destination: switch_array_state
content-type: application/*+avro
group: switch-array-state-group-0
I used Avro for this project and I have created a topic in Kafka with the name switch_array_state.
There is a service where I am injecting StreamBridge (I tried both constructor injection and autowiring). But this is where the dependency issue is found. It seems there is no bean of this type and I have no idea why this is not instantiated. I have used this approach before on a different application with a different Spring Boot version and it works.
I found out the cause of this. It is a mismatch in versions between Spring Cloud and Spring Boot.
Check this link for more information
Config client cannot resolve configuration property and cannot connect to config server.
Here's my service application.properties file:
spring.application.name = idmanager-service
spring.cloud.config.uri = http://localhost:8888
server.port=8081
#enable actuator endpoints
management.endpoints.web.exposure.include=*
management.endpoint.health.show-details=always
info.app.name=Id management Application
info.app.description=managing citizens identities
info.app.version=0.1.0
The dependencies in pom.xml:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<!-- Lmobok Dependency-->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.2</version>
<scope>provided</scope>
</dependency>
<!-- / Lmobok Dependency-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<!--<version>RELEASE</version>-->
<scope>compile</scope>
</dependency>
<!-- Eureka discovery client-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-commons</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
<!-- Config client-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<!-- Actuator -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
The config server and the discovery server started successfully and other services are successfully registered.
Here's the config server application.properties file:
server.port=8888
spring.cloud.config.server.git.uri=file:./src/main/resources/myConfig
The application.properties file in myConfig folder:
global=xxxxx
Other services are successfully registered and fetching config server:
c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
I resolved this issue by fixing Spring cloud version like below:
<spring-cloud.version>Hoxton.SR3</spring-cloud.version>
Then I specified the Spring cloud dependency version using dependency management:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
I created a simple ( and almost empty ) spring boot project, I packaged into a jar and I deployed it on appengine.
But it doesn't start!
When I run https://.appspot.com I Have a 500 error page
I saw the database was created and the tables too, but there is something wrong into application
These are the last few lines of the log
Started VibesCoreApplication in 9.729 seconds (JVM running for 11.147)
Shutting down ExecutorService 'taskScheduler'
Closing JPA EntityManagerFactory for persistence unit 'default'
HikariPool-1 - Shutdown initiated...
HikariPool-1 - Shutdown completed.
this is a part of the pom file
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!-- for GAE -->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- for GAE -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<type>jar</type>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-sqladmin</artifactId>
<version>v1beta4-rev76-1.25.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.23.0</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter</artifactId>
<version>LATEST</version>
</dependency>
<!-- JPA -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<version>2.1.4.RELEASE</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-jdbc -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
<!-- MySQL -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.17</version>
<scope>runtime</scope>
</dependency>
</dependencies>
this is the app.yaml file
runtime: java11
instance_class: F2
env: standard
network:
session_affinity: true
entrypoint: java -Xmx64m -jar target/vibes-core-0.4.0.jar
handlers:
- url: /(.*\.(gif|png|jpg|js|css|env))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg)$
here there is the full logs
I was able to find a similar issue with yours here. The fact is that ( even this should not happen ) the configuration loading order may be broken.
In the post that I have sent you, the solution was removing the #EnableAutoConfiguration annotation.
How to configure my Spring Boot service to have max 2 open connections to the Postgres database? Application is used on the PRODUCTION only by a few people and I don't want to
my pom:
(...)
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.8.RELEASE</version>
<relativePath/>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-hibernate5</artifactId>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.6</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1200-jdbc41</version>
<scope>provided</scope>
<exclusions>
<exclusion>
<artifactId>slf4j-simple</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>com.caucho</groupId>
<artifactId>hessian</artifactId>
<version>4.0.51</version>
</dependency>
<dependency>
<groupId>com.vladmihalcea</groupId>
<artifactId>hibernate-types-52</artifactId>
<version>2.4.3</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
(...)
</build>
</project>
and my application.properties file:
spring.datasource.url=jdbc:postgresql://xxx
spring.datasource.username=xxx
spring.datasource.password=xxx
spring.datasource.maximum-pool-size=2 //I think it is not sufficient
info.app.name: xxx
I think spring.datasource.maximum-pool-size=2 is not sufficient. What should I add to the properties file? Should I maybe exclude HikariCP from my pom file?
Since Spring Boot 2.0, Hikari is the default DataSource implementation. Therefore, if you want to configure the maximumPoolSize (a parameter for Hikeri Configuration) in Spring Boot, you are supposed to set it as follows:
For application.properties
spring.datasource.hikari.maximum-pool-size=2
For application.yml
spring:
datasource:
hikari:
maximum-pool-size: 2
If you haven't changed the connection implementation pool that Spring is using, it should be HikariCP, and then the setting you're looking for is:
spring.datasource.hikari.maximum-pool-size
its listed here: https://docs.spring.io/spring-boot/docs/current/reference/html/appendix-application-properties.html
I am having problem persisting the zipkin data. I did not get any error message. So I am sharing configuration to get a help.
I can see my logs in zipkin UI, but not able to persist in elastic search.
My zipkin-service pom file shared below.
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
<dependency>
<groupId>io.zipkin.java</groupId>
<artifactId>zipkin-autoconfigure-ui</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-sleuth-zipkin-stream</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-rabbit</artifactId>
</dependency>
<dependency>
<groupId>io.zipkin.java</groupId>
<artifactId>zipkin-autoconfigure-storage-elasticsearch-http</artifactId>
<version>2.0.0</version>
</dependency>
<!-- add the dependency for elasticsearch via http -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
And my properties looks like this
spring:
profiles: local
rabbitmq:
host: ${RABBIT_HOST:localhost}
username: test
password: test
zipkin:
storage:
type: elasticsearch
elasticsearch:
jest:
uris: http://localhost:9200