I am trying to map one list to another using streams.
Some elements of the original list fail to map. That is, the mapping function may not be able to find an appropriate new value.
I want to know if any of the mappings has failed. Ideally I would also like to stop the processing once a failure happened.
What I am currently doing is:
The mapping function returns null if there's no mapped value
I filter() to remove nulls from the stream
I collect(), and then
I compare the size of the result to the size of the original list.
For example:
List<String> func(List<String> old, Map<String, String> oldToNew)
{
List<String> holger = old.stream()
.map(oldToNew::get)
.filter(Objects::nonNull)
.collect(Collectors.toList);
if (holger.size() < old.size()) {
// ... appropriate error handling code ...
}
else {
return holger;
}
}
This is not very elegant. Also, everything is processed even when the whole thing should fail.
Suggestions for a better way of doing it?
Or maybe I should ditch streams altogether and use good old loops?
There is no best solution because that heavily depends on the use case. E.g. if lookup failures are expected to be unlikely or the error handling implies throwing an exception anyway, just throwing an exception at the first failed lookup within the mapping function might indeed be a good choice. Then, no follow-up code has to care about error conditions.
Another way of handling it might be:
List<String> func(List<String> old, Map<String, String> oldToNew) {
Map<Boolean,List<String>> map=old.stream()
.map(oldToNew::get)
.collect(Collectors.partitioningBy(Objects::nonNull));
List<String> failed=map.get(false);
if(!failed.isEmpty())
throw new IllegalStateException(failed.size()+" lookups failed");
return map.get(true);
}
This can still be considered being optimized for the successful case as it collects a mostly meaningless list containing null values for the failures. But it has the point of being able to tell the number of failures (unlike using a throwing map function).
If a detailed error analysis has a high priority, you may use a solution like this:
List<String> func(List<String> old, Map<String, String> oldToNew) {
Map<Boolean,List<String>> map=old.stream()
.map(s -> new AbstractMap.SimpleImmutableEntry<>(s, oldToNew.get(s)))
.collect(Collectors.partitioningBy(e -> e.getValue()!=null,
Collectors.mapping(e -> Optional.ofNullable(e.getValue()).orElse(e.getKey()),
Collectors.toList())));
List<String> failed=map.get(false);
if(!failed.isEmpty())
throw new IllegalStateException("The following key(s) failed: "+failed);
return map.get(true);
}
It collects two meaningful lists, containing the failed keys for failed lookups and a list of successfully mapped values. Note that both lists could be returned.
You could change your filter to Objects::requireNonNull and catch a NullPointerException outside the stream
This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
What are Null Pointer Exceptions (java.lang.NullPointerException) and what causes them?
What methods/tools can be used to determine the cause so that you stop the exception from causing the program to terminate prematurely?
There are two overarching types of variables in Java:
Primitives: variables that contain data. If you want to manipulate the data in a primitive variable you can manipulate that variable directly. By convention primitive types start with a lowercase letter. For example variables of type int or char are primitives.
References: variables that contain the memory address of an Object i.e. variables that refer to an Object. If you want to manipulate the Object that a reference variable refers to you must dereference it. Dereferencing usually entails using . to access a method or field, or using [ to index an array. By convention reference types are usually denoted with a type that starts in uppercase. For example variables of type Object are references.
Consider the following code where you declare a variable of primitive type int and don't initialize it:
int x;
int y = x + x;
These two lines will crash the program because no value is specified for x and we are trying to use x's value to specify y. All primitives have to be initialized to a usable value before they are manipulated.
Now here is where things get interesting. Reference variables can be set to null which means "I am referencing nothing". You can get a null value in a reference variable if you explicitly set it that way, or a reference variable is uninitialized and the compiler does not catch it (Java will automatically set the variable to null).
If a reference variable is set to null either explicitly by you or through Java automatically, and you attempt to dereference it you get a NullPointerException.
The NullPointerException (NPE) typically occurs when you declare a variable but did not create an object and assign it to the variable before trying to use the contents of the variable. So you have a reference to something that does not actually exist.
Take the following code:
Integer num;
num = new Integer(10);
The first line declares a variable named num, but it does not actually contain a reference value yet. Since you have not yet said what to point to, Java sets it to null.
In the second line, the new keyword is used to instantiate (or create) an object of type Integer, and the reference variable num is assigned to that Integer object.
If you attempt to dereference num before creating the object you get a NullPointerException. In the most trivial cases, the compiler will catch the problem and let you know that "num may not have been initialized," but sometimes you may write code that does not directly create the object.
For instance, you may have a method as follows:
public void doSomething(SomeObject obj) {
// Do something to obj, assumes obj is not null
obj.myMethod();
}
In which case, you are not creating the object obj, but rather assuming that it was created before the doSomething() method was called. Note, it is possible to call the method like this:
doSomething(null);
In which case, obj is null, and the statement obj.myMethod() will throw a NullPointerException.
If the method is intended to do something to the passed-in object as the above method does, it is appropriate to throw the NullPointerException because it's a programmer error and the programmer will need that information for debugging purposes.
In addition to NullPointerExceptions thrown as a result of the method's logic, you can also check the method arguments for null values and throw NPEs explicitly by adding something like the following near the beginning of a method:
// Throws an NPE with a custom error message if obj is null
Objects.requireNonNull(obj, "obj must not be null");
Note that it's helpful to say in your error message clearly which object cannot be null. The advantage of validating this is that 1) you can return your own clearer error messages and 2) for the rest of the method you know that unless obj is reassigned, it is not null and can be dereferenced safely.
Alternatively, there may be cases where the purpose of the method is not solely to operate on the passed in object, and therefore a null parameter may be acceptable. In this case, you would need to check for a null parameter and behave differently. You should also explain this in the documentation. For example, doSomething() could be written as:
/**
* #param obj An optional foo for ____. May be null, in which case
* the result will be ____.
*/
public void doSomething(SomeObject obj) {
if(obj == null) {
// Do something
} else {
// Do something else
}
}
Finally, How to pinpoint the exception & cause using Stack Trace
What methods/tools can be used to determine the cause so that you stop
the exception from causing the program to terminate prematurely?
Sonar with find bugs can detect NPE.
Can sonar catch null pointer exceptions caused by JVM Dynamically
Now Java 14 has added a new language feature to show the root cause of NullPointerException. This language feature has been part of SAP commercial JVM since 2006.
In Java 14, the following is a sample NullPointerException Exception message:
in thread "main" java.lang.NullPointerException: Cannot invoke "java.util.List.size()" because "list" is null
List of situations that cause a NullPointerException to occur
Here are all the situations in which a NullPointerException occurs, that are directly* mentioned by the Java Language Specification:
Accessing (i.e. getting or setting) an instance field of a null reference. (static fields don't count!)
Calling an instance method of a null reference. (static methods don't count!)
throw null;
Accessing elements of a null array.
Synchronising on null - synchronized (someNullReference) { ... }
Any integer/floating point operator can throw a NullPointerException if one of its operands is a boxed null reference
An unboxing conversion throws a NullPointerException if the boxed value is null.
Calling super on a null reference throws a NullPointerException. If you are confused, this is talking about qualified superclass constructor invocations:
class Outer {
class Inner {}
}
class ChildOfInner extends Outer.Inner {
ChildOfInner(Outer o) {
o.super(); // if o is null, NPE gets thrown
}
}
Using a for (element : iterable) loop to loop through a null collection/array.
switch (foo) { ... } (whether its an expression or statement) can throw a NullPointerException when foo is null.
foo.new SomeInnerClass() throws a NullPointerException when foo is null.
Method references of the form name1::name2 or primaryExpression::name throws a NullPointerException when evaluated when name1 or primaryExpression evaluates to null.
a note from the JLS here says that, someInstance.someStaticMethod() doesn't throw an NPE, because someStaticMethod is static, but someInstance::someStaticMethod still throw an NPE!
* Note that the JLS probably also says a lot about NPEs indirectly.
NullPointerExceptions are exceptions that occur when you try to use a reference that points to no location in memory (null) as though it were referencing an object. Calling a method on a null reference or trying to access a field of a null reference will trigger a NullPointerException. These are the most common, but other ways are listed on the NullPointerException javadoc page.
Probably the quickest example code I could come up with to illustrate a NullPointerException would be:
public class Example {
public static void main(String[] args) {
Object obj = null;
obj.hashCode();
}
}
On the first line inside main, I'm explicitly setting the Object reference obj equal to null. This means I have a reference, but it isn't pointing to any object. After that, I try to treat the reference as though it points to an object by calling a method on it. This results in a NullPointerException because there is no code to execute in the location that the reference is pointing.
(This is a technicality, but I think it bears mentioning: A reference that points to null isn't the same as a C pointer that points to an invalid memory location. A null pointer is literally not pointing anywhere, which is subtly different than pointing to a location that happens to be invalid.)
What is a NullPointerException?
A good place to start is the JavaDocs. They have this covered:
Thrown when an application attempts to use null in a case where an
object is required. These include:
Calling the instance method of a null object.
Accessing or modifying the field of a null object.
Taking the length of null as if it were an array.
Accessing or modifying the slots of null as if it were an array.
Throwing null as if it were a Throwable value.
Applications should throw instances of this class to indicate other
illegal uses of the null object.
It is also the case that if you attempt to use a null reference with synchronized, that will also throw this exception, per the JLS:
SynchronizedStatement:
synchronized ( Expression ) Block
Otherwise, if the value of the Expression is null, a NullPointerException is thrown.
How do I fix it?
So you have a NullPointerException. How do you fix it? Let's take a simple example which throws a NullPointerException:
public class Printer {
private String name;
public void setName(String name) {
this.name = name;
}
public void print() {
printString(name);
}
private void printString(String s) {
System.out.println(s + " (" + s.length() + ")");
}
public static void main(String[] args) {
Printer printer = new Printer();
printer.print();
}
}
Identify the null values
The first step is identifying exactly which values are causing the exception. For this, we need to do some debugging. It's important to learn to read a stacktrace. This will show you where the exception was thrown:
Exception in thread "main" java.lang.NullPointerException
at Printer.printString(Printer.java:13)
at Printer.print(Printer.java:9)
at Printer.main(Printer.java:19)
Here, we see that the exception is thrown on line 13 (in the printString method). Look at the line and check which values are null by
adding logging statements or using a debugger. We find out that s is null, and calling the length method on it throws the exception. We can see that the program stops throwing the exception when s.length() is removed from the method.
Trace where these values come from
Next check where this value comes from. By following the callers of the method, we see that s is passed in with printString(name) in the print() method, and this.name is null.
Trace where these values should be set
Where is this.name set? In the setName(String) method. With some more debugging, we can see that this method isn't called at all. If the method was called, make sure to check the order that these methods are called, and the set method isn't called after the print method.
This is enough to give us a solution: add a call to printer.setName() before calling printer.print().
Other fixes
The variable can have a default value (and setName can prevent it being set to null):
private String name = "";
Either the print or printString method can check for null, for example:
printString((name == null) ? "" : name);
Or you can design the class so that name always has a non-null value:
public class Printer {
private final String name;
public Printer(String name) {
this.name = Objects.requireNonNull(name);
}
public void print() {
printString(name);
}
private void printString(String s) {
System.out.println(s + " (" + s.length() + ")");
}
public static void main(String[] args) {
Printer printer = new Printer("123");
printer.print();
}
}
See also:
Avoiding “!= null” statements in Java?
I still can't find the problem
If you tried to debug the problem and still don't have a solution, you can post a question for more help, but make sure to include what you've tried so far. At a minimum, include the stacktrace in the question, and mark the important line numbers in the code. Also, try simplifying the code first (see SSCCE).
Question: What causes a NullPointerException (NPE)?
As you should know, Java types are divided into primitive types (boolean, int, etc.) and reference types. Reference types in Java allow you to use the special value null which is the Java way of saying "no object".
A NullPointerException is thrown at runtime whenever your program attempts to use a null as if it was a real reference. For example, if you write this:
public class Test {
public static void main(String[] args) {
String foo = null;
int length = foo.length(); // HERE
}
}
the statement labeled "HERE" is going to attempt to run the length() method on a null reference, and this will throw a NullPointerException.
There are many ways that you could use a null value that will result in a NullPointerException. In fact, the only things that you can do with a null without causing an NPE are:
assign it to a reference variable or read it from a reference variable,
assign it to an array element or read it from an array element (provided that array reference itself is non-null!),
pass it as a parameter or return it as a result, or
test it using the == or != operators, or instanceof.
Question: How do I read the NPE stacktrace?
Suppose that I compile and run the program above:
$ javac Test.java
$ java Test
Exception in thread "main" java.lang.NullPointerException
at Test.main(Test.java:4)
$
First observation: the compilation succeeds! The problem in the program is NOT a compilation error. It is a runtime error. (Some IDEs may warn your program will always throw an exception ... but the standard javac compiler doesn't.)
Second observation: when I run the program, it outputs two lines of "gobbledy-gook". WRONG!! That's not gobbledy-gook. It is a stacktrace ... and it provides vital information that will help you track down the error in your code if you take the time to read it carefully.
So let's look at what it says:
Exception in thread "main" java.lang.NullPointerException
The first line of the stack trace tells you a number of things:
It tells you the name of the Java thread in which the exception was thrown. For a simple program with one thread (like this one), it will be "main". Let's move on ...
It tells you the full name of the exception that was thrown; i.e. java.lang.NullPointerException.
If the exception has an associated error message, that will be output after the exception name. NullPointerException is unusual in this respect, because it rarely has an error message.
The second line is the most important one in diagnosing an NPE.
at Test.main(Test.java:4)
This tells us a number of things:
"at Test.main" says that we were in the main method of the Test class.
"Test.java:4" gives the source filename of the class, AND it tells us that the statement where this occurred is in line 4 of the file.
If you count the lines in the file above, line 4 is the one that I labeled with the "HERE" comment.
Note that in a more complicated example, there will be lots of lines in the NPE stack trace. But you can be sure that the second line (the first "at" line) will tell you where the NPE was thrown1.
In short, the stack trace will tell us unambiguously which statement of the program has thrown the NPE.
See also: What is a stack trace, and how can I use it to debug my application errors?
1 - Not quite true. There are things called nested exceptions...
Question: How do I track down the cause of the NPE exception in my code?
This is the hard part. The short answer is to apply logical inference to the evidence provided by the stack trace, the source code, and the relevant API documentation.
Let's illustrate with the simple example (above) first. We start by looking at the line that the stack trace has told us is where the NPE happened:
int length = foo.length(); // HERE
How can that throw an NPE?
In fact, there is only one way: it can only happen if foo has the value null. We then try to run the length() method on null and... BANG!
But (I hear you say) what if the NPE was thrown inside the length() method call?
Well, if that happened, the stack trace would look different. The first "at" line would say that the exception was thrown in some line in the java.lang.String class and line 4 of Test.java would be the second "at" line.
So where did that null come from? In this case, it is obvious, and it is obvious what we need to do to fix it. (Assign a non-null value to foo.)
OK, so let's try a slightly more tricky example. This will require some logical deduction.
public class Test {
private static String[] foo = new String[2];
private static int test(String[] bar, int pos) {
return bar[pos].length();
}
public static void main(String[] args) {
int length = test(foo, 1);
}
}
$ javac Test.java
$ java Test
Exception in thread "main" java.lang.NullPointerException
at Test.test(Test.java:6)
at Test.main(Test.java:10)
$
So now we have two "at" lines. The first one is for this line:
return args[pos].length();
and the second one is for this line:
int length = test(foo, 1);
Looking at the first line, how could that throw an NPE? There are two ways:
If the value of bar is null then bar[pos] will throw an NPE.
If the value of bar[pos] is null then calling length() on it will throw an NPE.
Next, we need to figure out which of those scenarios explains what is actually happening. We will start by exploring the first one:
Where does bar come from? It is a parameter to the test method call, and if we look at how test was called, we can see that it comes from the foo static variable. In addition, we can see clearly that we initialized foo to a non-null value. That is sufficient to tentatively dismiss this explanation. (In theory, something else could change foo to null ... but that is not happening here.)
So what about our second scenario? Well, we can see that pos is 1, so that means that foo[1] must be null. Is this possible?
Indeed it is! And that is the problem. When we initialize like this:
private static String[] foo = new String[2];
we allocate a String[] with two elements that are initialized to null. After that, we have not changed the contents of foo ... so foo[1] will still be null.
What about on Android?
On Android, tracking down the immediate cause of an NPE is a bit simpler. The exception message will typically tell you the (compile time) type of the null reference you are using and the method you were attempting to call when the NPE was thrown. This simplifies the process of pinpointing the immediate cause.
But on the flipside, Android has some common platform-specific causes for NPEs. A very common is when getViewById unexpectedly returns a null. My advice would be to search for Q&As about the cause of the unexpected null return value.
It's like you are trying to access an object which is null. Consider below example:
TypeA objA;
At this time you have just declared this object but not initialized or instantiated. And whenever you try to access any property or method in it, it will throw NullPointerException which makes sense.
See this below example as well:
String a = null;
System.out.println(a.toString()); // NullPointerException will be thrown
A null pointer exception is thrown when an application attempts to use null in a case where an object is required. These include:
Calling the instance method of a null object.
Accessing or modifying the field of a null object.
Taking the length of null as if it were an array.
Accessing or modifying the slots of null as if it were an array.
Throwing null as if it were a Throwable value.
Applications should throw instances of this class to indicate other illegal uses of the null object.
Reference: http://docs.oracle.com/javase/8/docs/api/java/lang/NullPointerException.html
A null pointer is one that points to nowhere. When you dereference a pointer p, you say "give me the data at the location stored in "p". When p is a null pointer, the location stored in p is nowhere, you're saying "give me the data at the location 'nowhere'". Obviously, it can't do this, so it throws a null pointer exception.
In general, it's because something hasn't been initialized properly.
A lot of explanations are already present to explain how it happens and how to fix it, but you should also follow best practices to avoid NullPointerExceptions at all.
See also:
A good list of best practices
I would add, very important, make a good use of the final modifier.
Using the "final" modifier whenever applicable in Java
Summary:
Use the final modifier to enforce good initialization.
Avoid returning null in methods, for example returning empty collections when applicable.
Use annotations #NotNull and #Nullable
Fail fast and use asserts to avoid propagation of null objects through the whole application when they shouldn't be null.
Use equals with a known object first: if("knownObject".equals(unknownObject)
Prefer valueOf() over toString().
Use null safe StringUtils methods StringUtils.isEmpty(null).
Use Java 8 Optional as return value in methods, Optional class provide a solution for representing optional values instead of null references.
A null pointer exception is an indicator that you are using an object without initializing it.
For example, below is a student class which will use it in our code.
public class Student {
private int id;
public int getId() {
return this.id;
}
public setId(int newId) {
this.id = newId;
}
}
The below code gives you a null pointer exception.
public class School {
Student student;
public School() {
try {
student.getId();
}
catch(Exception e) {
System.out.println("Null pointer exception");
}
}
}
Because you are using student, but you forgot to initialize it like in the
correct code shown below:
public class School {
Student student;
public School() {
try {
student = new Student();
student.setId(12);
student.getId();
}
catch(Exception e) {
System.out.println("Null pointer exception");
}
}
}
In Java, everything (excluding primitive types) is in the form of a class.
If you want to use any object then you have two phases:
Declare
Initialization
Example:
Declaration: Object object;
Initialization: object = new Object();
Same for the array concept:
Declaration: Item item[] = new Item[5];
Initialization: item[0] = new Item();
If you are not giving the initialization section then the NullPointerException arise.
In Java all the variables you declare are actually "references" to the objects (or primitives) and not the objects themselves.
When you attempt to execute one object method, the reference asks the living object to execute that method. But if the reference is referencing NULL (nothing, zero, void, nada) then there is no way the method gets executed. Then the runtime let you know this by throwing a NullPointerException.
Your reference is "pointing" to null, thus "Null -> Pointer".
The object lives in the VM memory space and the only way to access it is using this references. Take this example:
public class Some {
private int id;
public int getId(){
return this.id;
}
public setId( int newId ) {
this.id = newId;
}
}
And on another place in your code:
Some reference = new Some(); // Point to a new object of type Some()
Some otherReference = null; // Initiallly this points to NULL
reference.setId( 1 ); // Execute setId method, now private var id is 1
System.out.println( reference.getId() ); // Prints 1 to the console
otherReference = reference // Now they both point to the only object.
reference = null; // "reference" now point to null.
// But "otherReference" still point to the "real" object so this print 1 too...
System.out.println( otherReference.getId() );
// Guess what will happen
System.out.println( reference.getId() ); // :S Throws NullPointerException because "reference" is pointing to NULL remember...
This an important thing to know - when there are no more references to an object (in the example above when reference and otherReference both point to null) then the object is "unreachable". There is no way we can work with it, so this object is ready to be garbage collected, and at some point, the VM will free the memory used by this object and will allocate another.
Another occurrence of a NullPointerException occurs when one declares an object array, then immediately tries to dereference elements inside of it.
String[] phrases = new String[10];
String keyPhrase = "Bird";
for(String phrase : phrases) {
System.out.println(phrase.equals(keyPhrase));
}
This particular NPE can be avoided if the comparison order is reversed; namely, use .equals on a guaranteed non-null object.
All elements inside of an array are initialized to their common initial value; for any type of object array, that means that all elements are null.
You must initialize the elements in the array before accessing or dereferencing them.
String[] phrases = new String[] {"The bird", "A bird", "My bird", "Bird"};
String keyPhrase = "Bird";
for(String phrase : phrases) {
System.out.println(phrase.equals(keyPhrase));
}
I implemented a Collector for a java 8 stream that will store Entities to a Repository when a given threshold is hit.
public BiConsumer<Tuple2<Integer,List<T>>, T> accumulator() {
return (tuple, e) -> {
List<T> list = tuple._2;
list.add(e);
if (list.size() >= this.threshold) {
this.repository.save(list);
this.repository.flush();
list = new LinkedList<>();
}
tuple = new Tuple2<>(tuple._1 + 1, list);
};
}
This does not work as intended. The the Element e is added to the list but is not reset after the threshold is reached. Also Integer stays at 0 which is to be expected since its a final member.
As it seems my only option is to make my Tuple2 mutable and empty the List :-(
Any suggestions how to solve this using immutable tuples?
A lambda expression can be thought of as a function (or method) that simply doesn't have a name. In this example it would be like a method that has two formal parameters tuple and e and also some local variables within its body, including list.
When you make an assignment to a formal parameter or to a local variable, that assignment is local to the current method (or lambda body). No mutation or side effects will be visible to the outside after the accumulator returns, so these assignments won't affect the data structure you're collecting into.
I'm not entirely sure what you're trying to do, but instead of using a tuple (which presumably is immutable and must be replaced instead of mutated) you might try writing an ordinary, mutable class that contains an integer counter (or whatever) and a list. The accumulator would add to the list, conditionally flush and replace the list, and increment the counter. These mutative operations are allowed in collectors because the collector framework carefully thread-confines these operations, so the object you're mutating doesn't need to be thread-safe.
I want to pass list of id's as one of parameter to Spring batch. Is this possible to achieve?
Thanks in advance.
What you are trying to do is not possible.
From the JobParameter doc:
Domain representation of a parameter to a batch job. Only the
following types can be parameters: String, Long, Date, and Double. The
identifying flag is used to indicate if the parameter is to be used as
part of the identification of a job instance.
You might be tempted write your list of of id's to a comma delimited string and pass that as a single parameter but beware that when stored in the DB it has a length of at most 250 bytes. You'll either have to increase that limit or use another way.
Perhaps you can explain what why you need to pass that list of ids.
If you want to pass the list from ItemReader, then you have to get JobParameters first (you have to declare your reader to be step scoped for that, see this thread also).
You will have to put your list as a parameter to the JobParameters. As JobParameters is immutable, you will have to create a new object then
List yourList = ....
JobParameters jp = (JobParameters) fac.getBean("params");
Map map=params.getParameters();
map.put("yourList", list);
params=new JobParameters(map);
launcher.run(job, params);
You cannot use the List<T> concept itself in spring-batch, but I think you can implement your intentions(listOf(a, b, c, d..)) in the following way.
The job parameter itself receives a comma-separated string of items.
#Nonnull
private List<String> someList = Collections.emptyList();
#Value("#{jobParameters['someList']}")
public void setTableNames(#Nullable final String someList) {
if (StringUtils.isNotBlank(tableNames)) {
this.someList = Arrays.stream(StringUtils.split(someList, ","))
.map(String::trim)
.filter(StringUtils::isNotBlank)
.collect(Collectors.toList());
}
}
Hope it was helpful for using list-type parameters in spring-batch!
Thanks.