How to set ENCRYPTION_AES_128? - iText 7.1.0 for Java - itext7

At the moment I'm writing a couple of evaluatuation programs with iText.
I have an issue with AES Encryption.
STANDARD_ENCRYPTION_128 is working fine but ENCRYPTION_AES_128 produces a runtime error.
I tried a lot but nothing worked. Has anybody a clue what's wrong here?
Thanks, Dirk
import java.awt.Desktop;
import java.io.File;
import java.io.IOException;
import com.itextpdf.kernel.pdf.EncryptionConstants;
import com.itextpdf.kernel.pdf.PdfDocument;
import com.itextpdf.kernel.pdf.PdfWriter;
import com.itextpdf.kernel.pdf.WriterProperties;
import com.itextpdf.layout.Document;
import com.itextpdf.layout.element.Paragraph;
public class problem3 {
public static void main(String[] args) throws IOException {
String fnPdf = "results/problem3.pdf";
WriterProperties properties = new WriterProperties();
// ENCRYPTION_AES_128 produces an runtime error, STANDARD_ENCRYPTION_128 is working.
properties.setStandardEncryption("Hello".getBytes(), "World".getBytes(), EncryptionConstants.ALLOW_PRINTING,
EncryptionConstants.ENCRYPTION_AES_128);
PdfWriter writer = new PdfWriter(fnPdf, properties);
PdfDocument pdf = new PdfDocument(writer);
Document document = new Document(pdf);
Paragraph paragraph = new Paragraph("Hello AES-128!");
document.add(paragraph);
document.close();
pdf.close();
Desktop.getDesktop().open(new File(fnPdf));
}
}

Your error at runtime is probably
java.lang.NoClassDefFoundError: org/bouncycastle/crypto/BlockCipher
This is because iText uses BouncyCastle library for providing some of the encryption capabilities. The dependency is optional which means you have to add it manually if you need it.
If you use Maven for building, then make sure that you have the following dependencies:
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<version>1.49</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.49</version>
</dependency>
If you are adding jars to the classpath manually, which is not recommended, then you can go to Maven Central and download the necessary artifact jars manually.

Related

Spring boot and Swagger url and startup questions

I come from programming in c# and now I have to create a couple of Rest Apis in Spring Boot.
Everything is working ok and I can show the API in Swagger with springfox-swagger-ui
But I have two questions that I could not find in Internet
Is there any way to show the url ui in the console app with server, port, etc?
Is there any way to open the swagger url everytime I run the app in the localhost?
Thanks
Spring boot version
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.5</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-boot-starter</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>3.0.0</version>
</dependency>
I know it's not the question you're asking, but springfox is currently having issues with newer versions of spring. The spring version you're using is still working but as of 2.6 there are bugs, and it looks like the project is not well maintained. Since you're at the beginning of the project, switching is not too hard. You could move to springdocs for example (for migration: https://springdoc.org/#migrating-from-springfox).
With respect to opening a url, there are some good solutions mentioned here: How to open the default webbrowser using java . You could make your swagger url a property and have swagger configure it accordingly, then you can reuse the property to call the url on run-time. If you want to differentiate between environments I'd suggest use profiles. Only open the url in the browser if you start the app on dev environment, and not on prod is then specified by using #Profile("dev"). Create a commandline/application runner with the profile annotation (https://www.tutorialspoint.com/spring_boot/spring_boot_runners.htm), and call the url from there.
That said, combining it gives:
package com.example.demo;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;
import java.awt.*;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
#Profile("dev")
#Component
public class SwaggerRunner implements ApplicationRunner {
#Value("${springdoc.swagger-ui.path}")
private String swaggerPath;
#Override
public void run(ApplicationArguments args) throws Exception {
log("\nWelcome to Multi Brow Pop.\nThis aims to popup a browsers in multiple operating systems.\nGood luck!\n");
final String swaggerUrl = "http://localhost:8000/" + swaggerPath;
log("We're going to this page: " + swaggerUrl);
String myOS = System.getProperty("os.name").toLowerCase();
log("(Your operating system is: " + myOS + ")\n");
try {
if (Desktop.isDesktopSupported()) { // Probably Windows
log(" -- Going with Desktop.browse ...");
Desktop desktop = Desktop.getDesktop();
desktop.browse(new URI(swaggerUrl));
} else { // Definitely Non-windows
Runtime runtime = Runtime.getRuntime();
if (myOS.contains("mac")) { // Apples
log(" -- Going on Apple with 'open'...");
runtime.exec("open " + swaggerUrl);
} else if (myOS.contains("nix") || myOS.contains("nux")) { // Linux flavours
log(" -- Going on Linux with 'xdg-open'...");
runtime.exec("xdg-open " + swaggerUrl);
} else
log("I was unable/unwilling to launch a browser in your OS :( #SadFace");
}
log("\nThings have finished.\nI hope you're OK.");
} catch (IOException | URISyntaxException eek) {
log("**Stuff wrongly: " + eek.getMessage());
}
}
private static void log(String log) {
System.out.println(log);
}
}
put springdoc.swagger-ui.path=/custom/path in your application.properties to change the path to your swagger-ui

Unable to create file upload service - Receiving ModelValidationException (No injection source ) during startup [duplicate]

I am using Jersey based restful Service implementation strategy to build a service which will be used to upload files.
My service class name is : UploadFileService.java (See Code below)
package com.jerser.service;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import javax.ws.rs.Consumes;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import com.sun.jersey.core.header.FormDataContentDisposition;
import com.sun.jersey.multipart.FormDataParam;
#Path("/fileUpload")
public class UploadFileService {
#POST
#Path("/upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(
#FormDataParam("file") InputStream uploadedInputStream,
#FormDataParam("file") FormDataContentDisposition fileDetail) {
String uploadedFileLocation = "d://uploaded/" + fileDetail.getFileName();
// save it
writeToFile(uploadedInputStream, uploadedFileLocation);
String output = "File uploaded to : " + uploadedFileLocation;
return Response.status(200).entity(output).build();
}
// save uploaded file to new location
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(
uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
These are the JAR files I have inside my lib:
aopalliance-repackaged-2.4.0-b10.jar
asm-debug-all-5.0.2.jar
hk2-api-2.4.0-b10.jar
hk2-locator-2.4.0-b10.jar
hk2-utils-2.4.0-b10.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-2.4.0-b10.jar
javax.servlet-api-3.0.1.jar
javax.ws.rs-api-2.0.1.jar
jaxb-api-2.2.7.jar
jersey-client.jar
jersey-common.jar
jersey-container-servlet-core.jar
jersey-container-servlet.jar
jersey-core-1.11.jar
jersey-guava-2.17.jar
jersey-media-jaxb.jar
jersey-multipart-1.18.jar
jersey-server.jar
org.osgi.core-4.2.0.jar
osgi-resource-locator-1.0.1.jar
persistence-api-1.0.jar
validation-api-1.1.0.Final.jar
I am getting the following error when I am trying to up my tomcat server :
org.glassfish.jersey.server.model.ModelValidationException: Validation of the application resource model has failed during application initialization.
[[FATAL] No injection source found for a parameter of type public javax.ws.rs.core.Response com.jerser.service.UploadFileService.uploadFile(java.io.InputStream,com.sun.jersey.core.header.FormDataContentDisposition) at index 0.; source='ResourceMethod{httpMethod=POST, consumedTypes=[multipart/form-data], producedTypes=[], suspended=false, suspendTimeout=0, suspendTimeoutUnit=MILLISECONDS, invocable=Invocable{handler=ClassBasedMethodHandler{handlerClass=class com.jerser.service.UploadFileService, handlerConstructors=[org.glassfish.jersey.server.model.HandlerConstructor#d3e2d4]}, definitionMethod=public javax.ws.rs.core.Response com.jerser.service.UploadFileService.uploadFile(java.io.InputStream,com.sun.jersey.core.header.FormDataContentDisposition), parameters=[Parameter [type=class java.io.InputStream, source=file, defaultValue=null], Parameter [type=class com.sun.jersey.core.header.FormDataContentDisposition, source=file, defaultValue=null]], responseType=class javax.ws.rs.core.Response}, nameBindings=[]}']
at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:528)
at org.glassfish.jersey.server.ApplicationHandler.access$500(ApplicationHandler.java:166)
at org.glassfish.jersey.server.ApplicationHandler$3.run(ApplicationHandler.java:327)
at org.glassfish.jersey.internal.Errors$2.call(Errors.java:289)
at org.glassfish.jersey.internal.Errors$2.call(Errors.java:286)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:286)
at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:324)
at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:338)
at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:171)
at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:363)
at javax.servlet.GenericServlet.init(GenericServlet.java:160)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1176)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1102)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1009)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4885)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5212)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5207)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Over the internet I found there are plenty of example which shows How to upload MULTIPART file using RESTFul API. But with same solution. I am not able to run those code as well.
I think I am doing something wrong with the JAR files. Could anyone please help me on this?
Get rid of jersey-multipart-1.18.jar. That is for Jersey 1.x. Add these two
jersey-media-multipart-2.17
mimepull-1.9.3
For Maven you would use the following dependency (you don't need to explicitly add the mimepull dependency, as this one will pull it in).
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-multipart</artifactId>
<version>2.17</version> <!-- Make sure the Jersey version matches
the one you are currently using -->
</dependency>
Then you need to register the MultiPartFeature. If you are using a ResourceConfig for configuration, you can simply do
register(MultiPartFeature.class);
If you are using web.xml, then you can add the class as an <init-param> to the Jersey servlet
<init-param>
<param-name>jersey.config.server.provider.classnames</param-name>
<param-value>org.glassfish.jersey.media.multipart.MultiPartFeature</param-value>
</init-param>
Note that if you have multiple providers that you want to register, then you can delimit each provider class with a comma, semicolon, or space/newline. You cannot use this same param-name twice. See Suarabh's answer
UPDATE
Also, once you get rid of jersey-multipart-1.18.jar you will have compile errors for the missing imported classes. For the most part, the class names are still the same, just the packages have changed, i.e.
org.glassfish.jersey.media.multipart.FormDataParam
org.glassfish.jersey.media.multipart.FormDataContentDisposition
For Dropwizard
If you're using Dropwizard, instead of adding the jersey-media-multipart, they document for your to add dropwizard-forms instead. And instead of registering the MultiPartFeature, you should register the MultiPartBundle
#Override
public void initialize(Bootstrap<ExampleConfiguration> bootstrap) {
bootstrap.addBundle(new MultiPartBundle());
}
Really doesn't make much difference though as all the Dropwizard bundle does is register the MultiPartFeature with the ResourceConfig.
Aside
If you are here for a different ModelValidationException, here are some links for information on other causes of the exception.
1
2
3
Yet another possible cause for this very generic error is that Jersey only searches for factories associated with the last annotation when multiple ones are declared on a param. (See bug report)
Until this is fixed, if you are using any other annotations besides #FormDataParam, it has to come last.
This works:
#NotEmpty #FormDataParam("myParam") String myParam
This does not:
#FormDataParam("myParam") #NotEmpty String myParam
I too got the same exception.I did the following changes in web.xml
<init-param>
<param-name>jersey.config.server.provider.classnames</param-name>
<param-value>org.glassfish.jersey.filter.LoggingFilter;org.glassfish.jersey.moxy.json.MoxyFeature;org.glassfish.jersey.media.multipart.MultiPartFeature</param-value>
</init-param>
and changed jersey 2.7 to 2.9 .I do not know what change of this 2 has solved the issue.
Register MultiPartFeature.
In web.xml add to the Jersey servlet:
<init-param>
<param-name>jersey.config.server.provider.classnames</param-name>
<param-value>org.glassfish.jersey.media.multipart.MultiPartFeature</param-value>
</init-param>
Below code worked for me:
Class ->>> add it
Class Property --->> add it
Public Class userREST () {
#POST
#Path("upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces(MediaType.APPLICATION_JSON)
public Response uploadImageFile(#FormDataParam("uploadFile") InputStream fileInputStream,
#FormDataParam("uploadFile") FormDataContentDisposition fileFormDataContentDisposition,
#FormDataParam("FIR_REG_NUM") String FIR_REG_NUM, #FormDataParam("LOGIN_ID") String LOGIN_ID) {
final_json_result = WriteFileInFolder.fileAnalysis(fileInputStream, fileFormDataContentDisposition, FIR_REG_NUM,
LOGIN_ID);
return Response.ok(final_json_result).build();
}// uploadImageFile
Public Class FileJAXRSConfig () {
package ####.jaxrs.jwt;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;
import ####.helper.Common###;
import ####.jaxrs.jwt.filters.JWTRequestFilter;
import ####.jaxrs.jwt.filters.JWTResponseFilter;
import ####.service.FileServicesREST;
#ApplicationPath("fileservice")
public class FileJAXRSConfig extends Application {
#Override
public Set<Class<?>> getClasses() {
Common###.logging("#ApplicationPath#FileServicesREST...");
Set<Class<?>> clazzes = new HashSet<Class<?>>();
clazzes.add(JWTRequestFilter.class);
clazzes.add(FileServicesREST.class);
clazzes.add(JWTResponseFilter.class);
return clazzes;
}
#Override
public Map<String, Object> getProperties() {
Map<String, Object> properties = new HashMap<String, Object>();
properties.put("jersey.config.server.provider.packages", "####.service");
properties.put("jersey.config.server.provider.classnames", "org.glassfish.jersey.media.multipart.MultiPartFeature");
return properties;
}
}
Don't need to add following in web.xml
<init-param>
<param-name>jersey.config.server.provider.packages</param-name>
<param-value>mha.###.service</param-value>
</init-param>
<init-param>
<param-name>jersey.config.server.provider.classnames</param-name>
<param-value>org.glassfish.jersey.media.multipart.MultiPartFeature</param-value>
</init-param>
If someone is using #FormDataParam with #ApiOperation swagger annotation, it won't work(as per swagger latest version at this time) as mentioned here:
https://github.com/swagger-api/swagger-ui/issues/169
I had this same problem with Scala and this helped me solve it. Just want to add some Scala specific details to help anyone using Dropwizard with Scala. Here is an example of how to "register" the MultiPartFeature in a Scala and Dropwizard project.
package org.research.s3.service
import io.dropwizard.Application
import io.dropwizard.setup.Environment
import org.research.s3.service.resource._
import org.research.service.s3.resource.UploadResource
import org.glassfish.jersey.media.multipart.{FormDataParam,MultiPartFeature}
class CmdaaApp() extends Application[CmdaaAppConfig] {
override def run(t: CmdaaAppConfig, env: Environment): Unit = {
env.jersey().register(new RootResource)
//Need this to make the file upload code work in
env.jersey().register(new MultiPartFeature)
env.jersey().register(new UploadResource(curBucket))
}
}
object CmdaaApp {
def main(args: Array[String]): Unit = new CmdaaApp().run(args: _*)
}
and here is the code for the UploadResource that does the upload:
package org.research.service.s3.resource
import java.io.{FileInputStream, InputStream}
import com.google.gson.{Gson, GsonBuilder}
import javax.ws.rs.core.MediaType.APPLICATION_JSON
import javax.ws.rs._
import javax.ws.rs.core.Response
import javax.ws.rs.core.MediaType
import org.research.util.OptionSerializer
import org.research.s3.service.resource.s3Bucket
import org.glassfish.jersey.media.multipart.{FormDataParam,MultiPartFeature}
#Path("/file")
class UploadResource(currentBucket: s3Bucket) {
val gsonb = new GsonBuilder()
gsonb.registerTypeAdapter(classOf[Option[Any]], new OptionSerializer)
val gson = gsonb.create
#POST
#Path("upload")
#Produces(Array(APPLICATION_JSON))
#Consumes(Array(MediaType.MULTIPART_FORM_DATA))
// def uploadFile(): Response = {
def uploadFile(#FormDataParam("file") uploadedInputStream: InputStream): Response = {
/* Need code here to get a uuid for the file name
Then return the uuid if we have success and of course 200
*/
Response.ok.entity(currentBucket.upload("testName",uploadedInputStream,false)).build()
//Response.ok().build()
}
}
This code refers to an s3 bucket but you don't need that. You can just replace that call with code do download your incoming file data to a regular file.
scala
I had the same problem when I tried to upload the file.
I spent a lot of time until I found a solution to the problem.
1.If you changed version of your JARs files you may have a version conflicts!
Clean up your artifacts/libs and rebuild project.
2.You need to register your UploadFileService class too:
register(MultiPartFeature.class);
register(UploadFileService.class);
Hope it will help someone and save your time.
in case you are getting this error while writing Dropwizard tests for the upload Resource this is the solution:
add dependency on dropwizard-forms
2.add this in the application file:
#Override
public void initialize(Bootstrap<ExampleConfiguration> bootstrap) {
bootstrap.addBundle(new MultiPartBundle());
}
in the test file add:
ResourceExtension.builder()
.addResource(new FileResource())
.addProvider(new MultiPartFeature())
.build();
I had a very similar problem and the answer that helped me was this https://stackoverflow.com/a/30407999/6801721
I was trying to use a user defined object as a query parameter and from the answer that's usually not allowed unless it meets some conditions.
In case someone comes across this in the future and is running into the same problem I was running into. Make sure that the annotations you are importing are from the correct packages. In my case I was importing javax.websocket.server.PathParam instead of javax.ws.rs.PathParam.

Sample client for accessing sonar Qube issues of a project

I need sample java client code for accessing the issues of a project in sonar Qube?
I will pass the project key as input and get the list of issues in output.
you can use sonarqube client lib :
Builder builder = HttpConnector.newBuilder();
builder.url("http://xxxx:9000/sonar/");
builder.connectTimeoutMilliseconds(10000);
HttpConnector httpConnector = builder.build();
SearchWsRequest searchWsRequest = new org.sonarqube.ws.client.issue.SearchWsRequest();
List<String> projectKeys = new ArrayList<String>(1);
projectKeys.add("project_key");
searchWsRequest.setProjectKeys(projectKeys);
final WsClient wsClient = WsClientFactories.getDefault().newClient(httpConnector);
List<Issue> issues = wsClient.issues().search(searchWsRequest).getIssuesList();
and use
<dependency>
<groupId>org.sonarsource.sonarqube</groupId>
<artifactId>sonar-ws</artifactId>
<version>5.6</version>
</dependency>
Updated Sourcecode for Sonarqube 8.2.blarghminorversion. Only minor changes were needed to use the fine example delivered by Maxence Lecointe...
19.08.2022 -- Updated for Sonarqube 9.6.0.blarghminorversion. No changes needed
import java.util.ArrayList;
import java.util.List;
import org.sonarqube.ws.Issues.Issue;
import org.sonarqube.ws.Issues.SearchWsResponse;
import org.sonarqube.ws.client.HttpConnector;
import org.sonarqube.ws.client.HttpConnector.Builder;
import org.sonarqube.ws.client.WsClient;
import org.sonarqube.ws.client.WsClientFactories;
import org.sonarqube.ws.client.issues.SearchRequest;
public class SimpleClient {
public static void main(String[] args) {
Builder builder = HttpConnector.newBuilder();
builder.url("http://xxxx:9000/");
builder.token("sUp3rZ3cr37-t0k3n");
builder.connectTimeoutMilliseconds(10000);
HttpConnector httpConnector = builder.build();
SearchRequest searchWsRequest = new org.sonarqube.ws.client.issues.SearchRequest();
List<String> componentKeys = new ArrayList<String>(1);
componentKeys.add("project_key");
searchWsRequest.setComponentKeys(componentKeys);
final WsClient wsClient = WsClientFactories.getDefault().newClient(httpConnector);
SearchWsResponse response = wsClient.issues().search(searchWsRequest);
System.out.println("Total found: " + response.getTotal());
List<Issue> issues = wsClient.issues().search(searchWsRequest).getIssuesList();
for (Issue issue : issues) {
System.out.println("===================================");
System.out.println(issue.toString());
}
}
}
<dependency>
<groupId>org.sonarsource.sonarqube</groupId>
<artifactId>sonar-ws</artifactId>
<version>9.6.0.59041</version>
</dependency>

How to make webjars version agnostic in spring mvc

I have followed the documentation on their website as described here
First of all I added the required path
<mvc:resources mapping="/webjars/**" location="/webjars/"/>
then I created a controller with the following
#ResponseBody
#RequestMapping("/webjarslocator/{webjar}/**")
public ResponseEntity locateWebjarAsset(#PathVariable String webjar, HttpServletRequest request) {
try {
String mvcPrefix = "/webjarslocator/" + webjar + "/"; // This prefix must match the mapping path!
String mvcPath = (String) request.getAttribute(HandlerMapping.PATH_WITHIN_HANDLER_MAPPING_ATTRIBUTE);
String fullPath = assetLocator.getFullPath(webjar, mvcPath.substring(mvcPrefix.length()));
return new ResponseEntity(new ClassPathResource(fullPath), HttpStatus.OK);
} catch (Exception e) {
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
}
}
Few dependencies were missing so I added in maven the following pom
<dependency>
<groupId>org.webjars</groupId>
<artifactId>webjars-locator</artifactId>
<version>0.28</version>
</dependency>
The above will import the following
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.servlet.HandlerMapping;
import org.springframework.core.io.ClassPathResource;
None of these has been imported from the external jar.
The error is: assetLocator cannot be resolved
EDIT: It could be that I need to create a filter rather than put it in a controller. Any thoughts on this?
The documentation is quite sparse, but you can create an instance of an asset locator with new WebJarAssetLocator().
Here's another solution, considering more recent versions of WebJar, in a similar vein for anyone else who stumbles upon this question. Using org.webjars.play.WebJarsUtil#locate(java.lang.String, java.lang.String) (which, in-turn, uses org.webjars.WebJarAssetLocator#getFullPath(java.lang.String, java.lang.String)) accepts the WebJar name (which acts as a scope) and the file without needing to provide a full path that necessitates including the version.

MRUnit with Avro NullPointerException in Serialization

I'm trying to test a Hadoop .mapreduce Avro job using MRUnit. I am receiving a NullPointerException as seen below. I've attached a portion of the pom and source code. Any assistance would be appreciated.
Thanks
The error I'm getting is :
java.lang.NullPointerException
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:73)
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:91)
at org.apache.hadoop.mrunit.internal.io.Serialization.copyWithConf(Serialization.java:104)
at org.apache.hadoop.mrunit.TestDriver.copy(TestDriver.java:608)
at org.apache.hadoop.mrunit.MapDriverBase.setInputKey(MapDriverBase.java:64)
at org.apache.hadoop.mrunit.MapDriverBase.setInput(MapDriverBase.java:104)
at org.apache.hadoop.mrunit.MapDriverBase.withInput(MapDriverBase.java:218)
at org.lab41.project.mapreduce.ParseMetadataAsTextIntoAvroTest.testMap(ParseMetadataAsTextIntoAvroTest.java:115)
.....
pom snippet:
<dependency>
<groupId>org.apache.mrunit</groupId>
<artifactId>mrunit</artifactId>
<version>0.9.0-incubating</version>
<classifier>hadoop2</classifier>
<scope>test</scope>
</dependency>
<avro.version>1.7.4</avro.version>
<hadoop.version>2.0.0-mr1-cdh4.1.3</hadoop.version>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>${avro.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>${hadoop.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>${hadoop.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
<version>${avro.version}</version>
<classifier>hadoop2</classifier>
</dependency>
Here is an excerpt of the test :
import static org.junit.Assert.*;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import java.nio.charset.CharsetEncoder;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import org.apache.avro.mapred.AvroKey;
import org.apache.avro.hadoop.io.AvroSerialization;
import org.apache.avro.mapred.AvroValue;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mrunit.mapreduce.MapDriver;
import org.apache.hadoop.mrunit.types.Pair;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.lab41.project.domain.DataRecord;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ParseMetadataAsTextIntoAvroTest {
Logger logger = LoggerFactory
.getLogger(ParseMetadataAsTextIntoAvroTest.class);
private MapDriver<LongWritable, Text, AvroKey<Long>, AvroValue<DataRecord>> mapDriver;
#BeforeClass
public static void setUpClass() {
}
#AfterClass
public static void tearDownClass() {
}
#Before
public void setUp() throws IOException {
ParseMetadataAsTextIntoAvroMapper mapper = new ParseMetadataAsTextIntoAvroMapper();
mapDriver = new MapDriver<LongWritable, Text, AvroKey<Long>, AvroValue<DataRecord>>();
mapDriver.setMapper(mapper);
mapDriver.getConfiguration().setStrings("io.serializations", new String[]{
AvroSerialization.class.getName()
});
}
#Test
public void testMap() throws ParseException, IOException {
Text testInputText = new Text(test0);
DataRecord record = new DataRecord();
….
AvroKey<Long> expectedPivot = new AvroKey<Long>(1L);
AvroValue<DataRecord> expectedRecord = new AvroValue<DataRecord>(record);
mapDriver.withInput(new Pair<LongWritable, Text>(new LongWritable(1), testInputText));
mapDriver.withOutput(new Pair<AvroKey<Long>, AvroValue<DataRecord>>(expectedPivot, expectedRecord));
mapDriver.runTest();
}
}
In order to get this to work you have add the AvroSerializatio to the default serailizations. You also have to configure AvroSerializationn.
#Before
public void setUp() throws IOException {
ParseMetadataAsTextIntoAvroMapper mapper = new ParseMetadataAsTextIntoAvroMapper();
mapDriver = new MapDriver<LongWritable, Text, AvroKey<Long>, AvroValue<NetworkRecord>>();
mapDriver.setMapper(mapper);
//Copy over the default io.serializations. If you don't do this then you will
//not be able to deserialize the inputs to the mapper
String[] strings = mapDriver.getConfiguration().getStrings("io.serializations");
String[] newStrings = new String[strings.length +1];
System.arraycopy( strings, 0, newStrings, 0, strings.length );
newStrings[newStrings.length-1] = AvroSerialization.class.getName();
//Now you have to configure AvroSerialization by sepecifying the key
//writer Schema and the value writer schema.
mapDriver.getConfiguration().setStrings("io.serializations", newStrings);
mapDriver.getConfiguration().setStrings("avro.serialization.key.writer.schema", Schema.create(Schema.Type.LONG).toString(true));
mapDriver.getConfiguration().setStrings("avro.serialization.value.writer.schema", NetworkRecord.SCHEMA$.toString(true));
}
This also solve the problem, with merits of shorter and more clear code.
MapDriver driver = MapDriver.newMapDriver(your mapper);
Configuration conf = driver.getConfiguration();
AvroSerialization.addToConfiguration(conf);
AvroSerialization.setKeyWriterSchema(conf, your schema);
AvroSerialization.setKeyReaderSchema(conf, your schema);
Job job = new Job(conf);
job.set... your job settings;
AvroJob.set... your avro job settings;
It may be bug of mrunit, that don't set the io.serializations right
Instead it should have been set by job.setInputFormatClass(AvroKeyInputFormat.class) I think.
You have to add AvroSerialization to the default serializations and configure AvroSerialization.
#Before
public void setUp() throws IOException {
ParseMetadataAsTextIntoAvroMapper mapper = new ParseMetadataAsTextIntoAvroMapper();
mapDriver = new MapDriver<LongWritable, Text, AvroKey<Long>, AvroValue<NetworkRecord>>();
mapDriver.setMapper(mapper);
Configuration configuration = mapDriver.getConfiguration();
// Add AvroSerialization to the configuration
// (copy over the default serializations for deserializing the mapper inputs)
String[] serializations = configuration.getStrings(CommonConfigurationKeysPublic.IO_SERIALIZATIONS_KEY);
String[] newSerializations = Arrays.copyOf(serializations, serializations.length + 1);
newSerializations[serializations.length] = AvroSerialization.class.getName();
configuration.setStrings(CommonConfigurationKeysPublic.IO_SERIALIZATIONS_KEY, newSerializations);
//Configure AvroSerialization by specifying the key writer and value writer schemas
AvroSerialization.setKeyWriterSchema(configuration, Schema.create(Schema.Type.LONG));
AvroSerialization.setValueWriterSchema(configuration, NetworkRecord.SCHEMA$)
}
Answered here: https://issues.apache.org/jira/browse/MRUNIT-181 specifically: https://cwiki.apache.org/confluence/display/MRUNIT/MRUnit+with+Avro

Resources