I'm trying to scrape some data and display it on my spring boot app however the workings of Selenium seems to be different than the usual Java program. I'm getting all sorts of errors while trying to implement Selenium. Also, should I put chromedriver.exe under resources?
Web scraping class
#Service
public class WebScrape {
final static private String url = "https://www.worldometers.info/coronavirus/";
WebDriver driver = new ChromeDriver();
public List<Data>getAllData(){
return allData;
}
}
pom.xml
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-chrome-driver</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
Edit: Even with Selenium in Pom and default spring boot dependency, the import statements do not work
If you run your Selenium code as part of application logic itself you should not provide test scope to your dependencies. Use compile.
When you use test scope, your libraries are only included when you run test phase.
Related
Current project is an API that uses google cloud functions. I'd like to improve the coverage using unit tests and therefore would like to use PowerMock to mock static GCP functions. However, I receive the following error:
NoClassDefFoundError: Could not initialize class javax.xml.transform.FactoryFinder
#RunWith(PowerMockRunner.class)
#PowerMockRunnerDelegate(SpringRunner.class)
#SpringBootTest(webEnvironment = WebEnvironment.MOCK)
#AutoConfigureMockMvc
#PrepareForTest(MockedClass.class)
public class TestControllerTest {
... code here ...
}
Other details: Springfox 3.0.0, Java 11, uses Lombok.
Relevant POM:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.3</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>4.5.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-core</artifactId>
<version>$(powermock.version)</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>$(powermock.version)</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<version>${powermock.version)</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4-rule</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-classloading-xstream</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<!-- API, java.xml.bind module -->
<dependency>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>9
<version>2.3.3</version>
</dependency>
<!---Runtime, com.sun.xml.bind-module-->
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>"
<version>2.3.6</version>
</dependency>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.3.1</version>
</dependency>
Power mock version is 2.0.9.
I've tried adding the classes that it says is missing.
I've tried changing up the versions of powermock.
I've tried messing with jacoco.
I've tried to ignore the errors using PowerMockIgnore but that started causing other issues.
Referenced several of the popular existing answers on stackoverflow.
Tests works without powermock (but obviously cover less).
I'm trying to learn Spring Security by following this tutorial https://spring.io/guides/gs/securing-web/
But I'm stuck on the setup phase. The tutorial essentially tells me to add these items to my pom:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
But my IDE complains that spring security doesn't exist:
I've tried other tutorials and have run into the same issue. I don't understand.
I've tried to do a mvn clean install (the tutorial doesn't say to do that) but it didn't work.
Also tried File -> Invalidate Caches ... -> Invalidate and Restart in my IDE, that didn't help either.
Anybody know what the issue is?
Erros after running mvn clean install:
I am trying to connect Spring WebFlux (spring boot version 2.0.6) with datastax Cassandra DSE3.x. But when I am trying to query Cassandra Table using ReactiveCassandraRepository with custom query I am getting following error. However I am able to get count of the records from that table.
Unsupported feature with the native protocol V1 (which is currently in use): Binary values are not supported; nested exception is com.datastax.driver.core.exceptions.UnsupportedFeatureException: Unsupported feature with the native protocol V1 (which is currently in use): Binary values are not supported"
My Repository Code is
public interface SystemLookupReactiveRepository extends ReactiveCrudRepository<SystemLookup, String> {
#AllowFiltering
#Query("{SELECT * FROM \"SystemLookup\" WHERE LookupId = ?0 ALLOW FILTERING}")
Mono<SystemLookup> findByLookupId(String lookupId);}
My POM file is :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra-reactive</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
<exclusions>
<exclusion>
<artifactId>spring-boot-starter-reactor-netty</artifactId>
<groupId>org.springframework.boot</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
I probably have an issue with my POM in my SpringBoot App.
Currently I am trying to access my Keycloak Server with the
"admin-client-keycloak"
But on the call:
Response response = getInstance().realm(REALM).users().create(user);
I get Error:
java.lang.NoSuchMethodError: javax.ws.rs.core.UriBuilder.resolveTemplates(Ljava/util/Map;)Ljavax/ws/rs/core/UriBuilder;
at org.jboss.resteasy.client.jaxrs.internal.ClientWebTarget.resolveTemplates(ClientWebTarget.java:178) ~[resteasy-client-3.5.1.Final.jar:3.5.1.Final]
at org.jboss.resteasy.client.jaxrs.internal.proxy.SubResourceInvoker.invoke(SubResourceInvoker.java:65) ~[resteasy-client-3.5.1.Final.jar:3.5.1.Final]
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:76) ~[resteasy-client-3.5.1.Final.jar:3.5.1.Final]
at com.sun.proxy.$Proxy198.realm(Unknown Source) ~[na:na]
at org.keycloak.admin.client.Keycloak.realm(Keycloak.java:118) ~[keycloak-admin-client-3.4.3.Final.jar:3.4.3.Final]
To be honest: I am not a Maven expert and stitched it together from several web soultions and I expect that some parts are still outdated or something.
the POM dependecys:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>1.5.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<version>1.5.3.RELEASE</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>6.0.6</version>
<scope>runtime</scope>
</dependency>
<!-- Keycloak -->
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-admin-client</artifactId>
<version>3.4.3.Final</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-client</artifactId>
<version>3.5.1.Final</version>
</dependency>
<!-- JUnit -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${junit.jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jersey</artifactId>
<version>2.0.2.RELEASE</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.keycloak.bom</groupId>
<artifactId>keycloak-adapter-bom</artifactId>
<version>3.4.3.Final</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
The Error occurs from the 'keycloak-admin-client' module.
Now the main question is:
Whats the problem? Or is it realy a bug inside the version of 'keycloak-admin-client'?
I guess not which leads my to a second question:
How can I found out what I need to change in a situation like this?
Thanks in advance for all suggestions. I primary want to learn how to solve such issues by my own in future.
Kind regards
Gregor
EDIT:
Well I am a step further but the issue is still there.
The POM I show you here is a POM of library which is used in other projects.
These Project have a 'spring-Boot-starter' which are using the 'jersey' version:1.54.
So the problem seems clear.
I added the 'jersey 2.0.2' dependency in the final project POM
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jersey</artifactId>
<version>2.0.2.RELEASE</version>
</dependency>
But the issue still exists.
How can I force maven use the 2.0.2 version and override the 1.5.?
Only adding the dependecy doesnt seems to help.
Some of the other dependencies are using a different version of the lib jsr311-api. In my case it was the eureka client. I just added an exclude with this dependency in the pom and it worked
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
<exclusions>
<exclusion>
<groupId>javax.ws.rs</groupId>
<artifactId>jsr311-api</artifactId>
</exclusion>
</exclusions>
</dependency>
Please make sure UriBuilder class exist on your project. And make sure only one UriBuilder class (version) exist in your project. (It can be multiple on your project with different versions.) Different versions can be overlapped.
keycloak-admin-client version should same with your keycloak server version.
add additional dependency to pom (versions must be added, for keycloak 3.0.0.Final => resteasy dependecies 3.5.0.Final works for me. Dependencies must be complied.)
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-admin-client</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-client</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jackson2-provider</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-multipart-provider</artifactId>
</dependency>
Note: This is my suggestion, if you are develop multi-layer app. You can divide your project to layers. You can divide your spring boot starter app and keycloak access layer. And use this new project as dependecy.
edit,
see Failed adding user by keycloak-admin-client to Keycloak due to "unknown resource"
i hope these can helps.
I've a Spring Boot application where one of the dependencies is using spring and a embedded jetty to start an ad-hoc web server. This causes my spring boot app to start in a jetty instead of a tomcat.
My spring-boot-starter-web:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-websocket</artifactId>
</exclusion>
</exclusions>
</dependency>
The dependencies pom:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-http</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
</dependency>
Is there a possibility to configure the server to use by spring boot explicitly instead of being inferred by the dependency tree?
EDIT
I investigated the issue a little bit further and created a repo to reproduce the issue: github.com/svettwer/spring-server-test
org.eclipse.jetty.websocket:javax-websocket-server-impl causes spring to start with jetty without any other config required.
EDIT 2
The issue is not present anymore in Spring Boot 2.x
EDIT 3
I'll deleted the repo mentioned earlier, but here is the dependency setup that caused the issue:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>1.5.7.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<version>1.5.7.RELEASE</version>
<scope>test</scope>
</dependency>
<!-- Comment that in to start spring with jetty-->
<dependency>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>javax-websocket-server-impl</artifactId>
<version>9.4.8.v20171121</version>
</dependency>
</dependencies>
Normally, if you have the spring-boot-starter-tomcat on your classpath (through spring-boot-starter-web), it should always select Tomcat since it has priority over other servlet containers. Even if you have the following dependencies, Spring boot will start with Tomcat:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-webapp</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
</dependency>
You can programmatically override the chosen servlet container by registering your own ServletWebServerFactory, for example:
#Bean
public ServletWebServerFactory factory() {
return new TomcatServletWebServerFactory();
}
You can choose the predefined TomcatServletWebServerFactory, JettyServletWebServerFactory or the UndertowServletWebServerFactory.
This guide may help you: https://docs.spring.io/spring-boot/docs/current/reference/html/howto-embedded-web-servers.html
If you run an mvn dependency:tree and search for jetty you might find that you need to exclude it, e.g:
spring-boot-starter-jetty
excluded like in the example:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!-- Exclude the Tomcat dependency -->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Use Jetty instead -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
which is favouring jetty over tomcat - but you get the idea hopefully...
Hope this helps.