Please let me know if there is a possibility of changing the below code in terms of Java 8 using parallel streams?
I am looking for an option to run the "outer for loop" in parallel and finally all the values of stationMap gets collected together?
Map<Integer, Set<Integer>> stationMap = new HashMap<>();
Map<Integer, Set<Integer>> routes = function();
for (Map.Entry<Integer, Set<Integer>> entry : routes.entrySet())
{
Set<Integer> stations = entry.getValue();
for (Integer station : stations) {
Set<Integer> temporaryStations = new HashSet<>(stations);
Set<Integer> stationSet = stationMap.get(station);
if (stationSet == null) {
stationSet = new HashSet<>();
temporaryStations.remove(station);
stationSet.addAll(temporaryStations);
stationMap.put(station, stationSet);
} else {
temporaryStations.remove(station);
stationSet.addAll(temporaryStations);
}
}
}
More shorter version:
routes.forEach((k, stations) -> {
stations.forEach((station) -> {
Set<Integer> stationSet = stationMap.get(station);
if (stationSet == null) {
stationSet = new HashSet<>();
stationSet.addAll(stations);
stationMap.put(station, stationSet);
} else {
stationSet.addAll(stations);
}
});
});
Even the long pre-Java 8 version can be simplified as there is no need to iterate over the entry set, when you are only processing the values and there is no need for code duplication within the two conditional branches:
Map<Integer, Set<Integer>> routes = function();
Map<Integer, Set<Integer>> stationMap = new HashMap<>();
for(Set<Integer> stations: routes.values()) {
for(Integer station: stations) {
Set<Integer> temporaryStations = new HashSet<>(stations);
temporaryStations.remove(station);
Set<Integer> stationSet = stationMap.get(station);
if (stationSet == null) {
stationMap.put(station, temporaryStations);
} else {
stationSet.addAll(temporaryStations);
}
}
}
using Java 8 features, you may get the improved variant:
routes.values().forEach(stations ->
stations.forEach(station -> {
Set<Integer> temporaryStations = new HashSet<>(stations);
temporaryStations.remove(station);
Set<Integer> old = stationMap.putIfAbsent(station, temporaryStations);
if(old!=null) old.addAll(stations);
})
);
though it might be simpler to first merge all values and remove the keys afterwards in one step:
routes.values().forEach(stations ->
stations.forEach(station ->
stationMap.computeIfAbsent(station, key -> new HashSet<>()).addAll(stations)
)
);
stationMap.forEach((k,set) -> set.remove(k));
It’s possible to formulate an equivalent (parallel) Stream operation:
Map<Integer, Set<Integer>> stationMap=routes.values().parallelStream()
.flatMap(stations -> stations.stream().map(station -> {
Set<Integer> temporaryStations = new HashSet<>(stations);
temporaryStations.remove(station);
return new AbstractMap.SimpleImmutableEntry<>(station, temporaryStations);
})
).collect(Collectors.toMap(
Map.Entry::getKey, Map.Entry::getValue, (a,b) -> {a.addAll(b); return a; }));
but this may also be simpler when removing the keys from the value set in a post processing step:
Map<Integer, Set<Integer>> stationMap=routes.values().parallelStream()
.flatMap(stations -> stations.stream().map(station ->
new AbstractMap.SimpleImmutableEntry<>(station, new HashSet<>(stations))
)
).collect(Collectors.toMap(
Map.Entry::getKey, Map.Entry::getValue, (a,b) -> {a.addAll(b); return a; }));
stationMap.entrySet().parallelStream().forEach(e -> e.getValue().remove(e.getKey()));
or you use a custom collector instead of flatMap:
Map<Integer, Set<Integer>> stationMap=routes.values().parallelStream()
.collect(HashMap::new,
(map,stations) -> stations.forEach(station ->
map.computeIfAbsent(station, key -> new HashSet<>()).addAll(stations)
),
(m1,m2) -> m2.forEach((k,v)->m1.merge(k, v, (a,b)->{a.addAll(b); return a;})));
stationMap.entrySet().parallelStream().forEach(e -> e.getValue().remove(e.getKey()));
this might be more efficient as it doesn’t need the temporary Map.Entry instances.
Related
private Map<String, Set<Square>> populateZuloSquare(List<Square> squares) {
if (squares == null || squares.isEmpty()) {
return emptyMap();
}
Map<String, Set<Square>> res = new HashMap<>();
squares.stream()
.filter(square -> {
if (square.getZuloCodes().isEmpty()) {
LOG("Ignored {}", square.id);
return false;
}
return true;
})
.forEach(square -> {
square.getZuloCodes()
.forEach(code -> {
res.putIfAbsent(code, new HashSet<>());
res.get(code).add(square);
}));
});
return Collections.unmodifiableMap(res);
}
The code above receives a list of Squares, and those squares may contain ZuloCodes inside. The output should be a immutable Map zuloCode and value all the squares with that UniquePrefix.
As you can see I cannot figure out a way to remove the auxiliar collection res and make the code easily readable, is there a way to explode that collection into a [zuloCode, square] and then collect.groupBy ? Also that if inside the filter is so unreadable, how would you tackle it?
The standard approach is using flatMap before collecting using groupingBy, but since you need the original Square for each element, you need to map to an object holding both, the Square instance and the zulo code String.
Since there is no standard pair or tuple type in Java (yet), a work-around is to use a Map.Entry instance, like this
private Map<String, Set<Square>> populateZuloSquare0(List<Square> squares) {
if (squares == null || squares.isEmpty()) {
return emptyMap();
}
return squares.stream()
.filter(square -> logMismatch(square, !square.getZuloCodes().isEmpty()))
.flatMap(square -> square.getZuloCodes().stream()
.map(code -> new AbstractMap.SimpleEntry<>(code, square)))
.collect(Collectors.collectingAndThen(
Collectors.groupingBy(Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue, Collectors.toSet())),
Collections::unmodifiableMap));
}
private static boolean logMismatch(Square square, boolean match) {
if(!match) LOG("Ignored {}", square.id);
return match;
}
An alternative is to use a custom collector which will iterate over the keys:
private Map<String, Set<Square>> populateZuloSquare(List<Square> squares) {
if (squares == null || squares.isEmpty()) {
return emptyMap();
}
return squares.stream()
.filter(square -> logMismatch(square, !square.getZuloCodes().isEmpty()))
.collect(Collector.of(
HashMap<String, Set<Square>>::new,
(m,square) -> square.getZuloCodes()
.forEach(code -> m.computeIfAbsent(code, x -> new HashSet<>()).add(square)),
(m1,m2) -> {
if(m1.isEmpty()) return m2;
m2.forEach((key,set) ->
m1.merge(key, set, (s1,s2) -> { s1.addAll(s2); return s1; }));
return m1;
},
Collections::unmodifiableMap)
);
}
Note that this custom collector can be seen as a parallel capable variant of the following looping code:
private Map<String, Set<Square>> populateZuloSquare(List<Square> squares) {
if (squares == null || squares.isEmpty()) {
return emptyMap();
}
Map<String, Set<Square>> res = new HashMap<>();
squares.forEach(square -> {
if(square.getZuloCodes().isEmpty()) LOG("Ignored {}", square.id);
else square.getZuloCodes().forEach(
code -> res.computeIfAbsent(code, x -> new HashSet<>()).add(square));
});
return Collections.unmodifiableMap(res);
}
which might not look so bad now, when you don’t need the code to be parallel capable…
How about this. You may use map merge operation to get this thing done. I have updated the filter and simplified it too.
squares.stream().filter(s -> !s.getZuloCodes().isEmpty())
.forEach(s -> s.getZuloCodes().stream().forEach(z -> res.merge(z, new HashSet<>(Arrays.asList(s)),
(s1, s2) -> Stream.of(s1, s2).flatMap(Collection::stream).collect(Collectors.toSet()))));
Task: we`ve got list of mappers, which must be applied on list of arguments.
How we can do?:
My not really good variant:
public static final Function<List<IntUnaryOperator>, UnaryOperator<List<Integer>>> multifunctionalMapper =
lst -> {
UnaryOperator<List<Integer>> uOp = new UnaryOperator<List<Integer>>() {
#Override
public List<Integer> apply(List<Integer> integers) {
final int[] curFunct = new int[1];
List<Integer> newLst = integers;
for (int i = 0; i < lst.size(); i++) {
curFunct[0] = i;
newLst = newLst.stream().map(curInt -> lst.get(curFunct[0]).applyAsInt(curInt)).collect(Collectors.toList());
}
return newLst;
}
};
return uOp;
};
list of mappers addOneMuxTwoTransformation:
public static final UnaryOperator<List<Integer>> addOneMuxTwoTransformation =
multifunctionalMapper.apply(Arrays.asList(x -> x+1, x -> x*2));
test:
addOneMuxTwoTransformation.apply(Arrays.asList(1,2,3)).stream().forEach(System.out::println);
will print:
4
6
8
How can be multifunctionalMapper's code reduced?
Is this what are you trying to do ?
List<IntUnaryOperator> ops = Arrays.asList(a -> a++, a -> a*2);
IntUnaryOperator reduce = ops.stream().reduce(a -> a, IntUnaryOperator::andThen);
IntStream.of(1, 2, 3).map(reduce).forEach(System.out::println);
next solution is
public static final Function<List<IntUnaryOperator>, UnaryOperator<List<Integer>>> multifunctionalMapper =lstFunc->
lstVals -> lstVals.stream()
.map(curValue -> lstFunc.stream().reduce(IntUnaryOperator::andThen).orElse(x -> x).applyAsInt(curValue))
.collect(Collectors.toList());
I have class Element with a list, my intended output is like this:
Map<String , List<Element>>
{
1 = [Element3, Element1],
2 = [Element2, Element1],
3 = [Element2, Element1], 4=[Element2]
}
And my input is set of element objects, I used forEach to get the desired outcome, but I'm looking for how to collect it using collectors.toMap. Any inputs are much appreciated
Set<Element> changes = new HashSet();
List<String> interesetList = new ArrayList();
interesetList.add("1");
interesetList.add("2");
interesetList.add("3");
Element element = new Element(interesetList);
changes.add(element);
interesetList = new ArrayList();
interesetList.add("2");
interesetList.add("3");
interesetList.add("4");
element = new Element(interesetList);
changes.add(element);
Map<String, List<Element>> collect2 = new HashMap();
changes.forEach(element -> {
element.getInterestedList().forEach(tracker -> {
collect2.compute(tracker, ( key , val) -> {
List<Element> elementList = val == null ? new ArrayList<Element>() : val;
elementList.add(Element);
return elementList;
});
});
});
class Element {
List<String> interestedList;
static AtomicInteger sequencer = new AtomicInteger(0);
String mName;
public Element(List<String> aList) {
interestedList = aList;
mName = "Element" + sequencer.incrementAndGet();
}
public List<String> getInterestedList() {
return interestedList;
}
#Override
public String toString() {
return mName;
}
}
You can do it by using Collectors.groupingBy instead of Collectors.toMap, along with Collectors.mapping, which adapts a collector to another collector:
Map<String, List<Element>> result = changes.stream()
.flatMap(e -> e.getInterestedList().stream().map(t -> Map.entry(t, e)))
.collect(Collectors.groupingBy(
Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue, Collectors.toList())));
You need to use the Stream.flatMap method first and then pair the elements of the inner lists with the current Element instance. I did this via the new Java 9's Map.entry(key, value) method. If you're not on Java 9 yet, you could change it to new AbstractMap.SimpleEntry<>(key, value).
After flatmapping, we need to collect instances of Map.Entry. So I'm using Collectors.groupingBy to classify entries by key (where we had previously stored each element of the inner lists, aka what you call tracker in your code). Then, as we don't want to have instances of List<Map.Entry<String, Element>> as the values of the map, we need to transform each Map.Entry<String, Element> of the stream to just Element (that's why I'm using Map.Entry::getValue as the first argument of Collectors.mapping). We also need to specify a downstream collector (here Collectors.toList()), so that the outer Collectors.groupingBy collector knows where to place all the adapted elements of the stream that belong to each group.
A shorter and surely more efficient way to do the same (similar to your attempt) could be:
Map<String, List<Element>> result = new HashMap<>();
changes.forEach(e ->
e.getInterestedList().forEach(t ->
result.computeIfAbsent(t, k -> new ArrayList<>()).add(e)));
This uses Map.computeIfAbsent, which is a perfect fit for your use case.
here is my problem,
I have a List<A> lista1, what I should do is :
map<key, List<A>> m = groupby(lista1);
m = lista.stream.collect(Collectors.groupingBy);
for every group, return one element with some condition, and make a new list List<A> lista2 = MakeListfromGroup(m)
List<A> lista2;
for (Map.Entry<key, List<A>> entry : m.entrySet()) {
A theOne;
for (A a : entry.getValue()) {
if(condition){
theOne = a;
}
}
lista2.add(theOne);
}
return lista2;
List<B> listb1 = makeListB(lista2); //here, I cant use stream.map.collect
List<Integer> b1Key;
List<Integer> b2Key;
for(A a : lista2){
b1key.add(a.b1key);
b2key.add(a.b2key);
}
mapb1 = gerfromBD(b1key);
mapb2 = gerfromBD(b2key);
List<B> listb1;
for(A a : lista2){
listb1.add(new B(mapb1.get(a.b1key),mapb2.get(a.b2key));
}
return listb1
B has member B1 b1 and B2 b2, create a new List<B> listb2= applyFilter(list<predicate<B1>>,list<predicate<B2>>)
List<B> listb2;
nextb:
for(B b : listb1){
for(Predicate p: filtreB1){
if(!p.accept(b.b1)){
continue nextb;
}
}
for(Predicate p: filtreB2){
if(!p.accept(b.b2)){
continue nextb;
}
}
listb2.add(b);
}
return listb2;
Is it possible to put all in one stream? or do the step 4 in a stream way?
thanks in advance
For the next question, please provide real code instead of incomplete, typo ridden pseudo code.
As you need the complete list of As for step 3, you have to collect any potential stream from step 2 into a list anyway. Step 2 and 4 can be simplified by using streams, though:
public static List<A> makeListfromGroup(Map<Object, List<A>> m, Predicate<A> condition)
{
return m.values()
.stream()
.map(as -> as.stream().filter(condition).findAny().orElse(null))
.filter(Objects::nonNull)
.collect(Collectors.toList());
}
For step 4, you can read how to apply multiple conditions at once here:
public static List<B> applyFilter(List<B> list, List<Predicate<B1>> filtreB1, List<Predicate<B2>> filtreB2)
{
return list.stream()
.filter(b -> filtreB1.stream().allMatch(p -> p.test(b.getB1())))
.filter(b -> filtreB2.stream().allMatch(p -> p.test(b.getB2())))
.collect(Collectors.toList());
}
I have the following component:
private JobInfo aggregateJobInfo() {
final JobsResult jobsResult = restClient().getJobs();
final List<String> jobIds = extractJobIds(jobsResult);
//fetch details, exceptions and config for each job
final List<JobDetails> jobDetails = jobIds.stream().map(jobId -> {
final JobDetailResult jobDetailResult = restClient().getJobDetails(jobId);
final JobExceptionsResult jobExceptionsResult = restClient().getJobExceptions(jobId);
final JobConfigResult jobConfigResult = restClient().getJobConfig(jobId);
return new JobDetails(jobDetailResult, jobExceptionsResult, jobConfigResult);
}).collect(Collectors.toList());
return new JobInfo(jobsResult, jobDetails);
}
private static List<String> extractJobIds(final JobsResult jobsResult) {
final ArrayList<String> jobIds = new ArrayList<>();
jobIds.addAll(jobsResult.getRunning());
jobIds.addAll(jobsResult.getFinished());
jobIds.addAll(jobsResult.getCanceled());
jobIds.addAll(jobsResult.getFailed());
return jobIds;
}
It just calls some ENDPOINTS and aggergates some data. Now I'm trying to make that non-blocking by using CompletableFutures, which I didn't really used before..
private CompletableFuture<JobInfo> aggregateJobInfo() {
final CompletableFuture<JobsResult> jobsResultFuture = restClient().getJobs();
final CompletableFuture<List<String>> jobIdsFuture = jobsResultFuture.thenApply(JobInfoCollector::extractJobIds);
//fetch details, exceptions and config for each job
final CompletableFuture<List<CompletableFuture<JobDetails>>> jobDetailsFuture = jobIdsFuture.thenApply(jobIds -> {
return jobIds.stream().map(jobId -> {
final CompletableFuture<JobDetailResult> jobDetailsResultFuture = restClient().getJobDetails(jobId);
final CompletableFuture<JobExceptionsResult> jobExceptionsFuture = restClient().getJobExceptions(jobId);
final CompletableFuture<JobConfigResult> jobConfigFuture = restClient().getJobConfig(jobId);
return jobDetailsResultFuture.thenCompose(jobDetailResult -> {
return jobExceptionsFuture.thenCombine(jobConfigFuture, (jobExceptionsResult, jobConfigResult) -> {
return new JobDetails(jobDetailResult, jobExceptionsResult, jobConfigResult);
});
});
}).collect(Collectors.toList());
});
return null;
My problem is how to create CompletableFuture here when JobInfo is `new JobInfo(jobsResult, jobDetails)?!
As I said, I'm new to this, maybe my approach is bad and there are better solutions?
Any ideas appreciated, thanks in
First working version:
private CompletableFuture<JobInfo> aggregateJobInfo() {
final CompletableFuture<JobsResult> jobsResultFuture = restClient().getJobs();
final CompletableFuture<List<String>> jobIdsFuture = jobsResultFuture.thenApply(JobInfoCollector::extractJobIds);
final CompletableFuture<List<CompletableFuture<JobDetails>>> jobDetailsFutureListFuture =
jobIdsFuture.thenApply(jobIds -> jobIds.stream().map(jobId -> {
final CompletableFuture<JobDetailResult> jobDetailsResultFuture = restClient().getJobDetails(jobId);
final CompletableFuture<JobExceptionsResult> jobExceptionsFuture = restClient().getJobExceptions(jobId);
final CompletableFuture<JobConfigResult> jobConfigFuture = restClient().getJobConfig(jobId);
return jobDetailsResultFuture.thenCompose(jobDetailResult ->
jobExceptionsFuture.thenCombine(jobConfigFuture, (jobExceptionsResult, jobConfigResult) ->
new JobDetails(jobDetailResult, jobExceptionsResult, jobConfigResult)));
}).collect(Collectors.toList()));
return jobDetailsFutureListFuture.thenCompose(jobDetailsFutures ->
CompletableFuture.allOf(jobDetailsFutures.toArray(
new CompletableFuture[jobDetailsFutures.size()])).thenApply(aVoid ->
jobDetailsFutures.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList())))
.thenApply(jobDetails -> jobsResultFuture.thenApply(jobsResult ->
new JobInfo(jobsResult, jobDetails)))
.join();
}
You have:
CompletableFuture<JobsResult> jobsResultFuture
CompletableFuture<List<CompletableFuture<JobDetails>>> jobDetailsFuture
JobInfo(JobsResult a, List<JobDetails> b)
You want
CompletableFuture<JobInfo>
additional observation: jobDetailsFuture can only be completed when jobsResultFuture has completed.
So you can implement the following:
List<CompletableFuture<JobDetails>> -> Void via allOf in thenCompose
Void + List<CompletableFuture<JobDetails>> (as captured var) -> List<JobDetails> via thenApply
List<JobDetails> + CompletableFuture<JobsResult> (as captured var) -> JobInfo via thenApply
You can simply unwrap the futures via get() inside those mapper functions because the futures are guaranteed to have completed at that point due to the dependencies of their ancestor futures at that point.
Other approaches using thenCombine and stream reduction would be possible, but more verbose and create more intermediate futures.