I created an example Livy (Spark) application using the com.cloudera.livy.Job class for calculating an approximate value for Pi (Source: https://github.com/cloudera/livy#using-the-programmatic-api), exported as jar file to e.g. C:/path/to/the/pijob.jar.
Actually I'm running this job from another Main class like this (also copied from the link above and adapted):
import java.io.File;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Properties;
import java.util.concurrent.ExecutionException;
import com.cloudera.livy.LivyClient;
import com.cloudera.livy.LivyClientBuilder;
public class Main {
public static void main(String[] args) throws IOException, URISyntaxException, InterruptedException, ExecutionException {
String livyUrl = "http://myserverIp:8998";
String piJar = "C:/path/to/the/pijob.jar";
int samples = 10000;
LivyClient client = new LivyClientBuilder().setURI(new URI(livyUrl)).build();
try {
System.err.printf("Uploading %s to the Spark context...\n", piJar);
client.uploadJar(new File(piJar)).get();
System.err.printf("Running PiJob with %d samples...\n", samples);
double pi = client.submit(new PiJob(samples)).get();
System.out.println("Pi is roughly: " + pi);
} finally {
client.stop(true);
}
}
}
This application works perfectly in an unsecured Hadoop cluster from outside (started from my client). But when I try to run it against a Kerberos-enabled Cluster, it fails.
I tried to set the corresponding Kerberos properties in the LivyClientBuilder class:
Properties props = new Properties();
props.put("livy.environment", "production");
props.put("livy.impersonation.enabled", "true");
props.put("livy.server.auth.kerberos.keytab", "/etc/security/keytabs/spnego.service.keytab");
props.put("livy.server.auth.kerberos.principal", "HTTP/_HOST#MYCLUSTER.DE");
props.put("livy.server.auth.type", "kerberos");
props.put("livy.server.csrf_protection.enabled", "true");
props.put("livy.server.kerberos.keytab", "/etc/security/keytabs/livy.service.keytab");
props.put("livy.server.kerberos.principal", "livy/_HOST#MYCLUSTER.DE");
props.put("livy.server.port", "8998");
props.put("livy.server.session.timeout", "3600000");
props.put("livy.superusers", "zeppelin-MyCluster");
LivyClient client = new LivyClientBuilder().setAll(props).setURI(new URI(livyUrl)).build();
But I still get an exception saying that authentication is required:
Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Authentication required: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 </title>
</head>
<body>
<h2>HTTP ERROR: 401</h2>
<p>Problem accessing /sessions/. Reason:
<pre> Authentication required</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at com.cloudera.livy.client.http.HttpClient.propagate(HttpClient.java:185)
at com.cloudera.livy.client.http.HttpClient.<init>(HttpClient.java:85)
at com.cloudera.livy.client.http.HttpClientFactory.createClient(HttpClientFactory.java:38)
at com.cloudera.livy.LivyClientBuilder.build(LivyClientBuilder.java:124)
at livy.Main.main(Main.java:34)
Caused by: java.io.IOException: Authentication required: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 </title>
</head>
<body>
<h2>HTTP ERROR: 401</h2>
<p>Problem accessing /sessions/. Reason:
<pre> Authentication required</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at com.cloudera.livy.client.http.LivyConnection.sendRequest(LivyConnection.java:230)
at com.cloudera.livy.client.http.LivyConnection.sendJSONRequest(LivyConnection.java:204)
at com.cloudera.livy.client.http.LivyConnection.post(LivyConnection.java:180)
at com.cloudera.livy.client.http.HttpClient.<init>(HttpClient.java:82)
... 3 more
The questions at this point are for me:
Are these all needed Kerberos settings that I need?
Or do I have to add something more to log-in?
Do I have to provide config files / keytabs on my client machine?
or can I still use the server paths (like I did so far)?
Is there some helpful documentation on the Kerberos stuff for Livy?
Related
Trying to configure a JAX-RS resource with #Path("/"), however, the resource is ignored and the first file found in resources is loaded.
Any idea how to prevent this and allow the resource to work?
When clearing META-INF/resources, the JAX-RS resource loads correctly.
Using:
Quarkus 1.4.2.Final
openjdk version "11.0.6" 2020-01-14 LTS
OpenJDK Runtime Environment Zulu11.37+52-SA (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.37+52-SA (build 11.0.6+10-LTS, mixed mode)
Resource:
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
#Path("/")
public class LandingResource {
#GET
#Produces(MediaType.TEXT_HTML)
public String getLandingPage() {
return "<html><head><title>Hello World</title></head><body>Hello!</body></html>";
}
}
Testing:
curl --location --request GET 'http://localhost:8080/'
Response:
<!doctype html>
<html lang="en">
<head>
<title>Internal Server Error - Error handling cee4cff3-551d-44e1-9102-5c9ada9d8fb2-7, java.nio.file.InvalidPathException: Illegal char <:> at index 97: <tempdir>\vertx-cache\file-cache-71fbfca9-5ba3-4a3e-8020-8501379cbf2b\<project dir>\src\main\resources\META-INF\resources\assets\icons\icon-128x128.png</title>
<meta charset="utf-8">
<style>
html, body {
margin: 0;
padding: 0;
font-family: 'Open Sans', Helvetica, Arial, sans-serif;
font-size: 100%;
font-weight: 100;
line-height: 1.4;
}
...
Achieved the desired outcome by adding a vertx web route:
import io.quarkus.vertx.web.Route;
import io.vertx.core.http.HttpMethod;
import io.vertx.ext.web.RoutingContext;
import javax.enterprise.context.ApplicationScoped;
#ApplicationScoped
public class LandingRoute {
#Route(path = "/", methods = HttpMethod.GET)
public void landing(RoutingContext rc) {
rc.response().end("hello ");
}
}
In order to use #Route annotation you need to add quarkus-reactive-routes extension (io.quarkus:quarkus-reactive-routes) to project.
You can find more information about reactive routes in Quarkus documentation at:
https://quarkus.io/guides/reactive-routes
By default Quarkus will serve static resources from the root context.
That means that the resources inside src/main/resources/META-INF/resources/ are already mapped to root (http://localhost:8080/). This means that you can not map a standard JAX-RS on the root easily.
See the documentation for further information: https://quarkus.io/guides/http-reference
In your case you are returning a fixed HTML landing page. As a solution you could remove the LandingResource class and serve the landing page from the static resources.
This can be achieved by placing the HTML snippet in src/main/resources/META-INF/resources/index.html.
This is also how the default Quarkus default landing page is served.
I have a spring boot application (2.0.5.RELEASE) which is running on port 8989. The application sends the log messages over websocket to the client.
i have created a javascript client that connects to the websocket and appends the messages to the html text area. This client is under the resource\static\js folder of the application.
It works fine, when i dont use servet.context.path.
Is there any way, i can work it out with context path.
Am i missing some configurational parameter?
I would really appreciate your help in this regard.
Application.properties
server.port=8989
server.servlet.contextPath=/DemoService/webresources/test/
The Websocket config is mentioned below:
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {
/**
* Register Stomp endpoints: the url to open the WebSocket connection.
*/
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
// Register the "/websocket" endpoint, enabling the SockJS protocol.
// SockJS is used (both client and server side) to allow alternative
// messaging options if WebSocket is not available.
registry.addEndpoint("/websocket").setAllowedOrigins("*").withSockJS();
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry.enableSimpleBroker("/topic");
registry.setApplicationDestinationPrefixes("/app");
}
}
The client looks like this,
$(window).ready(function() {
connect();
});
function connect() {
// var socket = new SockJS('/websocket');
var socket = new SockJS('/DemoService/webresources/test/websocket');
stompClient = Stomp.over(socket);
stompClient.connect({}, function(frame) {
stompClient.subscribe('/topic/pushlognotification', function(
notification) {
var txt = $('#textArea');
txt.val(txt.val() + "\n" + "\n" + notification);
txt.scrollTop(txt[0].scrollHeight);
});
});
}
HTML page is :
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Messages</title>
</head>
<body>
<textarea type="text" id="textArea" placeholder="Messages..." rows="15" cols="60"></textarea>
<script src='js/jquery.min.js'></script>
<script src="js/index.js"></script>
<script src="/webjars/sockjs-client/sockjs.min.js"></script>
<script src="/webjars/stomp-websocket/stomp.min.js"></script>
</body>
</html>
found out the solution after examining the console output.
The HTML code could not find the registered files through src such as 'js/jquery.min.js' etc. because of the configured context paths.
The solution is either append the context path to all the src like below.
or give a "." sign
this worked for me.
I get this infamous error:
cannot be context relative (/) or page relative unless you implement
the IWebContext
I have a spring boot application (without the web module) that creates pdf files.
I am planning to use an HTML file as a template, but I could not link the css file nor the image properly due to these url issues.
Html :
<!DOCTYPE html SYSTEM "http://www.thymeleaf.org/dtd/xhtml1-strict-thymeleaf-4.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:th="http://www.thymeleaf.org">
<head>
<title>Company Invoice</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="stylesheet" type="text/css" media="all"
href="./css/company.css" th:href="#{./css/company.css}"/>
</head>
<body>
<p th:utext="#{home.welcome}">Welcome !</p>
<img src="/images/gtvglogo.png" th:src="#{/images/gtvglogo.png}"/>
</body>
</html>
folder structure:
src/main/resources/templates/sample.html
src/main/resources/templates/css/sample.css
I googled a bit but I donT want to solve this via IWebContext.
Is there another way?
Thanks in advance.
org.thymeleaf.exceptions.TemplateProcessingException: Link base
"/a/relative/link" cannot be context
relative (/...) unless the context used for executing the engine
implements the org.thymeleaf.context.IWebContext interface (template:
"templates/a-template" - line 6, col 13)
1 at org.thymeleaf.linkbuilder.StandardLinkBuilder.computeContextPath
(StandardLinkBuilder.java:493)
...
The exception is thrown by the org.thymeleaf.linkbuilder.StandardLinkBuilder. By providing a different implementation of org.thymeleaf.linkbuilder.ILinkBuilder to the TemplateEngine we can avoid this expception
TemplateEngine templateEngine = new TemplateEngine();
templateEngine.setLinkBuilder(new ILinkBuilder() {
#Override
public String getName() {
return null;
}
#Override
public Integer getOrder() {
return null;
}
#Override
public String buildLink(IExpressionContext context, String base, Map<String, Object> parameters) {
return null;
}
});
I tried reading many topics but I found no answer.
Plz, some illuminated soul can help me?
The code are just 3 files, a very simple query using jsp.
NetBeans 8.1
Maven Web Application
Java EE 7 WEB
GlassFish 4.1.1
Source/Binary 1.8
index.html
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Exercício 01 JDBC</title>
</head>
<body>
<h1>Listagem de Usuário:</h1>
<FORM METHOD="POST" ACTION="App">
<P> Clique em <INPUT TYPE="SUBMIT" VALUE="LISTAR">
para obter a relação do nome do primeiro usuário.</p>
</FORM>
</body>
</html>
App.java
package br.com.yonathan.faculdades.jdbc;
import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.sql.*;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.servlet.ServletConfig;
#WebServlet(name = "App", urlPatterns = {"/App"})
public class App extends HttpServlet {
private Connection con;
#Override
public void init(ServletConfig config) throws ServletException {
super.init(config);
final String url = "jdbc:oracle:thin:#yyy.inf.poa.ifrs.edu.br:1521:XE";
final String us = "xxx";
try {
Class.forName("oracle.jdbc.OracleDriver");
} catch (ClassNotFoundException ex) {
Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
}
try {
con = DriverManager.getConnection(url, us, us);
} catch (SQLException ex) {
throw new ServletException(ex);
}
}
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String sql = "SELECT nome FROM JDBC_USUARIO where ROWNUM = 1";
String saida = "";
try (PreparedStatement stm = con.prepareStatement(sql);
ResultSet rs = stm.executeQuery()) {
while (rs.next()) {
saida = rs.getString(1);
}
rs.close();
} catch (SQLException ex) {
Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
}
request.setAttribute("resultado", saida);
response.setContentType("text/html;charset=UTF-8");
request.getRequestDispatcher("resposta.jsp").forward(request, response);
}
}
resposta.jsp
<%#page contentType="text/html" pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Servlet Exibe Resultado</title>
</head>
<body>
<h1>JDBC Connection</h1>
<h2>Primeiro nome cadastrado:</h2>
${resultado}.
</body>
</html>
The user and passwd are the same.
I really tried to fix before posting.
Look for ojdbc6.jar or ojdbc7.jar in your oracle install and put that file into WEB-INF/lib. Then restart tomcat and try again.
In the file "pom.xml" into Project Files, I had to add:
<dependencies>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>11.2.0.3</version>
</dependency>
</dependencies>
and
<repositories>
<repository>
<id>codelds</id>
<url>https://code.lds.org/nexus/content/groups/main-repo</url>
</repository>
</repositories>
and it works.
I have no idea why doesn't works from the beginning. It's a Maven project, but I had to include "manually" this things and then when I click to run he download the missing driver and seems working well until now.
Thank you guys!
I have a Spring application and need to build support for PDF generation. I'm thinking of using Flying-saucer together with Thymeleaf to render the PDF. However, I cannot find that much information about using Flying-saucer together with Thymeleaf. Have anyone else used those to technologies together?
I'm using Flyingsaucer-R8 with Thymeleaf 2.0.14 without problems (and I'm sure current version of Thymeleaf works as well).
I have separate TemplateEngine with classpath template resolver configured for this purpose. Using it to produce XHTML as String. Flyingsaucer creates PDF document from result then. Check example below.
Code below is example - NOT PRODUCTION ready code use it with NO WARRANTY. For sake of clarity there's no try-catch handling and no resources caching (creating PDF is quite expensive operation). Consider that.
Code
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import org.springframework.core.io.ClassPathResource;
import org.thymeleaf.TemplateEngine;
import org.thymeleaf.context.Context;
import org.thymeleaf.templateresolver.ClassLoaderTemplateResolver;
import org.xhtmlrenderer.pdf.ITextFontResolver;
import org.xhtmlrenderer.pdf.ITextRenderer;
import com.lowagie.text.DocumentException;
import com.lowagie.text.pdf.BaseFont;
import com.sun.xml.internal.messaging.saaj.util.ByteOutputStream;
public class FlyingSoucerTestService {
public void test() throws DocumentException, IOException {
ClassLoaderTemplateResolver templateResolver = new ClassLoaderTemplateResolver();
templateResolver.setPrefix("META-INF/pdfTemplates/");
templateResolver.setSuffix(".html");
templateResolver.setTemplateMode("XHTML");
templateResolver.setCharacterEncoding("UTF-8");
TemplateEngine templateEngine = new TemplateEngine();
templateEngine.setTemplateResolver(templateResolver);
Context ctx = new Context();
ctx.setVariable("message", "I don't want to live on this planet anymore");
String htmlContent = templateEngine.process("messageTpl", ctx);
ByteOutputStream os = new ByteOutputStream();
ITextRenderer renderer = new ITextRenderer();
ITextFontResolver fontResolver = renderer.getFontResolver();
ClassPathResource regular = new ClassPathResource("/META-INF/fonts/LiberationSerif-Regular.ttf");
fontResolver.addFont(regular.getURL().toString(), BaseFont.IDENTITY_H, true);
renderer.setDocumentFromString(htmlContent);
renderer.layout();
renderer.createPDF(os);
byte[] pdfAsBytes = os.getBytes();
os.close();
FileOutputStream fos = new FileOutputStream(new File("/tmp/message.pdf"));
fos.write(pdfAsBytes);
fos.close();
}
}
Template
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html SYSTEM "http://www.thymeleaf.org/dtd/xhtml1-strict-thymeleaf-spring3-4.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<style>
div.border {
border: solid;
border-width: 1px 1px 0px 1px;
padding: 5px 20px 5px 20px;
}
</style>
</head>
<body style="font-family: Liberation Serif;">
<div class="border">
<h1 th:text="${message}">message</h1>
</div>
</body>
</html>