I'm having trouble determining the most efficient way of doing this in Dart.
If have two lists that in sorted descending order,
List<int> messages = [10, 5, 4, 1];
List<int> newMessages = [5, 3, 2];
How can I add newMessages to messages so that messages now looks like
messages = [10, 5, 5, 4, 3, 2, 1];
If both lists are long, and are using the default list implementation, it might be more efficient to create a new list based on the two other lists. The reason is that inserting an element inside an existing list requires all elements after this insertion index to be moved forward. Also, when the list grows, it needs to allocate a bigger list and move all elements into this.
If we instead creates a new list, we can inform Dart what the size of this list is going to be exactly and we can prevent moving elements:
void main() {
List<int> messages = [10, 5, 4, 1];
List<int> newMessages = [5, 3, 2];
// The compare argument is given since both lists are sorted in reverse order
print(newSortedListBasedOnTwoAlreadySortedLists<int>(
messages, newMessages, (a, b) => b.compareTo(a)));
// [10, 5, 5, 4, 3, 2, 1]
}
List<E> newSortedListBasedOnTwoAlreadySortedLists<E>(
List<E> l1,
List<E> l2, [
int Function(E a, E b)? compare,
]) {
Iterator<E> i1 = l1.iterator;
Iterator<E> i2 = l2.iterator;
if (!i1.moveNext()) {
if (!i2.moveNext()) {
return [];
} else {
return l2.toList();
}
}
if (!i2.moveNext()) {
return l1.toList();
}
bool i1alive = true;
bool i2alive = true;
return List.generate(l1.length + l2.length, (_) {
if (i1alive && i2alive) {
E v1 = i1.current;
E v2 = i2.current;
int compareResult = (compare == null)
? Comparable.compare(v1 as Comparable, v2 as Comparable)
: compare(v1, v2);
if (compareResult > 0) {
i2alive = i2.moveNext();
return v2;
} else {
i1alive = i1.moveNext();
return v1;
}
} else if (i1alive) {
E v1 = i1.current;
i1alive = i1.moveNext();
return v1;
} else {
E v2 = i2.current;
i2alive = i2.moveNext();
return v2;
}
});
}
Note: The method could in theory take two Iterable as argument as long as we are sure that a call to .length does not have any negative consequences like e.g. need to iterate over the full structure (with e.g. mappings). To prevent this issue, I ended up declaring the method to take List as arguments since we know for sure that .length is not problematic here.
This sounds like you need to merge the two lists.
As stated elsewhere, it's more efficient to create a new list than to move elements around inside the existing lists.
The merge can be written fairly simply:
/// Merges two sorted lists.
///
/// The lists must be ordered in increasing order according to [compare].
///
/// Returns a new list containing the elements of both [first] and [second]
/// in increasing order according to [compare].
List<T> merge<T>(List<T> first, List<T> second, int Function(T, T) compare) {
var result = <T>[];
var i = 0;
var j = 0;
while (i < first.length && j < second.length) {
var a = first[i];
var b = second[j];
if (compare(a, b) <= 0) {
result.add(a);
i++;
} else {
result.add(b);
j++;
}
}
while (i < first.length) {
result.add(first[i++]);
}
while (j < second.length) {
result.add(second[j++]);
}
return result;
}
(In this case, the lists are descending, so they'll need a compare function which reverses the order, like (a, b) => b.compareTo(a))
You can use binary search to insert all new messages one by one in a sorted manner while maintaining efficiency.
void main() {
List<int> messages = [10, 5, 4, 1];
List<int> newMessages = [5, 3, 2];
for (final newMessage in newMessages) {
final index = binarySearchIndex(messages, newMessage);
messages.insert(index, newMessage);
}
print(messages); // [10, 5, 5, 4, 3, 2, 1]
}
int binarySearchIndex(
List<int> numList,
int value, [
int? preferredMinIndex,
int? preferredMaxIndex,
]) {
final minIndex = preferredMinIndex ?? 0;
final maxIndex = preferredMaxIndex ?? numList.length - 1;
final middleIndex = ((maxIndex - minIndex) / 2).floor() + minIndex;
final comparator = numList[middleIndex];
if (middleIndex == minIndex) {
return comparator > value ? maxIndex : minIndex;
}
return comparator > value ?
binarySearchIndex(numList, value, middleIndex, maxIndex):
binarySearchIndex(numList, value, minIndex, middleIndex);
}
I wrote myself a utility to break a list into batches of given size. I just wanted to know if there is already any apache commons util for this.
public static <T> List<List<T>> getBatches(List<T> collection,int batchSize){
int i = 0;
List<List<T>> batches = new ArrayList<List<T>>();
while(i<collection.size()){
int nextInc = Math.min(collection.size()-i,batchSize);
List<T> batch = collection.subList(i,i+nextInc);
batches.add(batch);
i = i + nextInc;
}
return batches;
}
Please let me know if there any existing utility already for the same.
Check out Lists.partition(java.util.List, int) from Google Guava:
Returns consecutive sublists of a list, each of the same size (the final list may be smaller). For example, partitioning a list containing [a, b, c, d, e] with a partition size of 3 yields [[a, b, c], [d, e]] -- an outer list containing two inner lists of three and two elements, all in the original order.
In case you want to produce a Java-8 stream of batches, you can try the following code:
public static <T> Stream<List<T>> batches(List<T> source, int length) {
if (length <= 0)
throw new IllegalArgumentException("length = " + length);
int size = source.size();
if (size <= 0)
return Stream.empty();
int fullChunks = (size - 1) / length;
return IntStream.range(0, fullChunks + 1).mapToObj(
n -> source.subList(n * length, n == fullChunks ? size : (n + 1) * length));
}
public static void main(String[] args) {
List<Integer> list = Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14);
System.out.println("By 3:");
batches(list, 3).forEach(System.out::println);
System.out.println("By 4:");
batches(list, 4).forEach(System.out::println);
}
Output:
By 3:
[1, 2, 3]
[4, 5, 6]
[7, 8, 9]
[10, 11, 12]
[13, 14]
By 4:
[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10, 11, 12]
[13, 14]
Use Apache Commons ListUtils.partition.
org.apache.commons.collections4.ListUtils.partition(final List<T> list, final int size)
Another approach is to use Collectors.groupingBy of indices and then map the grouped indices to the actual elements:
final List<Integer> numbers = range(1, 12)
.boxed()
.collect(toList());
System.out.println(numbers);
final List<List<Integer>> groups = range(0, numbers.size())
.boxed()
.collect(groupingBy(index -> index / 4))
.values()
.stream()
.map(indices -> indices
.stream()
.map(numbers::get)
.collect(toList()))
.collect(toList());
System.out.println(groups);
Output:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11]]
With Java 9 you can use IntStream.iterate() with hasNext condition. So you can simplify the code of your method to this:
public static <T> List<List<T>> getBatches(List<T> collection, int batchSize) {
return IntStream.iterate(0, i -> i < collection.size(), i -> i + batchSize)
.mapToObj(i -> collection.subList(i, Math.min(i + batchSize, collection.size())))
.collect(Collectors.toList());
}
Using {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}, the result of getBatches(numbers, 4) will be:
[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9]]
Here is a simple solution for Java 8+:
public static <T> Collection<List<T>> prepareChunks(List<T> inputList, int chunkSize) {
AtomicInteger counter = new AtomicInteger();
return inputList.stream().collect(Collectors.groupingBy(it -> counter.getAndIncrement() / chunkSize)).values();
}
Here an example:
final AtomicInteger counter = new AtomicInteger();
final int partitionSize=3;
final List<Object> list=new ArrayList<>();
list.add("A");
list.add("B");
list.add("C");
list.add("D");
list.add("E");
final Collection<List<Object>> subLists=list.stream().collect(Collectors.groupingBy
(it->counter.getAndIncrement() / partitionSize))
.values();
System.out.println(subLists);
Input:
[A, B, C, D, E]
Output:
[[A, B, C], [D, E]]
You can find examples here:
https://e.printstacktrace.blog/divide-a-list-to-lists-of-n-size-in-Java-8/
I came up with this one:
private static <T> List<List<T>> partition(Collection<T> members, int maxSize)
{
List<List<T>> res = new ArrayList<>();
List<T> internal = new ArrayList<>();
for (T member : members)
{
internal.add(member);
if (internal.size() == maxSize)
{
res.add(internal);
internal = new ArrayList<>();
}
}
if (internal.isEmpty() == false)
{
res.add(internal);
}
return res;
}
The following example demonstrates chunking of a List:
package de.thomasdarimont.labs;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class SplitIntoChunks {
public static void main(String[] args) {
List<Integer> ints = Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11);
List<List<Integer>> chunks = chunk(ints, 4);
System.out.printf("Ints: %s%n", ints);
System.out.printf("Chunks: %s%n", chunks);
}
public static <T> List<List<T>> chunk(List<T> input, int chunkSize) {
int inputSize = input.size();
int chunkCount = (int) Math.ceil(inputSize / (double) chunkSize);
Map<Integer, List<T>> map = new HashMap<>(chunkCount);
List<List<T>> chunks = new ArrayList<>(chunkCount);
for (int i = 0; i < inputSize; i++) {
map.computeIfAbsent(i / chunkSize, (ignore) -> {
List<T> chunk = new ArrayList<>();
chunks.add(chunk);
return chunk;
}).add(input.get(i));
}
return chunks;
}
}
Output:
Ints: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
Chunks: [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11]]
There was another question that was closed as being a duplicate of this one, but if you read it closely, it's subtly different. So in case someone (like me) actually wants to split a list into a given number of almost equally sized sublists, then read on.
I simply ported the algorithm described here to Java.
#Test
public void shouldPartitionListIntoAlmostEquallySizedSublists() {
List<String> list = Arrays.asList("a", "b", "c", "d", "e", "f", "g");
int numberOfPartitions = 3;
List<List<String>> split = IntStream.range(0, numberOfPartitions).boxed()
.map(i -> list.subList(
partitionOffset(list.size(), numberOfPartitions, i),
partitionOffset(list.size(), numberOfPartitions, i + 1)))
.collect(toList());
assertThat(split, hasSize(numberOfPartitions));
assertEquals(list.size(), split.stream().flatMap(Collection::stream).count());
assertThat(split, hasItems(Arrays.asList("a", "b", "c"), Arrays.asList("d", "e"), Arrays.asList("f", "g")));
}
private static int partitionOffset(int length, int numberOfPartitions, int partitionIndex) {
return partitionIndex * (length / numberOfPartitions) + Math.min(partitionIndex, length % numberOfPartitions);
}
Using various cheats from the web, I came to this solution:
int[] count = new int[1];
final int CHUNK_SIZE = 500;
Map<Integer, List<Long>> chunkedUsers = users.stream().collect( Collectors.groupingBy(
user -> {
count[0]++;
return Math.floorDiv( count[0], CHUNK_SIZE );
} )
);
We use count to mimic a normal collection index.
Then, we group the collection elements in buckets, using the algebraic quotient as bucket number.
The final map contains as key the bucket number, as value the bucket itself.
You can then easily do an operation on each of the buckets with:
chunkedUsers.values().forEach( ... );
Similar to OP without streams and libs, but conciser:
public <T> List<List<T>> getBatches(List<T> collection, int batchSize) {
List<List<T>> batches = new ArrayList<>();
for (int i = 0; i < collection.size(); i += batchSize) {
batches.add(collection.subList(i, Math.min(i + batchSize, collection.size())));
}
return batches;
}
You can use below code to get the batch of list.
Iterable<List<T>> batchIds = Iterables.partition(list, batchSize);
You need to import Google Guava library to use above code.
Note that List#subList() returns a view of the underlying collection, which can result in unexpected consequences when editing the smaller lists - the edits will reflect in the original collection or may throw ConcurrentModificationException.
if someone is looking for Kotlin version, here is
list.chunked(size)
or
list.windowed(size)
once had an interview question and I wrote below one =D
fun <T> batch(list: List<T>, limit: Int): List<List<T>> {
val result = ArrayList<List<T>>()
var batch = ArrayList<T>()
for (i in list) {
batch.add(i)
if (batch.size == limit) {
result.add(batch)
batch = ArrayList()
}
}
if (batch.isNotEmpty()) {
result.add(batch)
}
return result
}
List<T> batch = collection.subList(i,i+nextInc);
->
List<T> batch = collection.subList(i, i = i + nextInc);
Here's a solution using vanilla java and the super secret modulo operator :)
Given the content/order of the chunks doesn't matter, this would be the easiest approach. (When preparing stuff for multi-threading it usually doesn't matter, which elements are processed on which thread for example, just need an equal distribution).
public static <T> List<T>[] chunk(List<T> input, int chunkCount) {
List<T>[] chunks = new List[chunkCount];
for (int i = 0; i < chunkCount; i++) {
chunks[i] = new LinkedList<T>();
}
for (int i = 0; i < input.size(); i++) {
chunks[i % chunkCount].add(input.get(i));
}
return chunks;
}
Usage:
List<String> list = Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i", "j");
List<String>[] chunks = chunk(list, 4);
for (List<String> chunk : chunks) {
System.out.println(chunk);
}
Output:
[a, e, i]
[b, f, j]
[c, g]
[d, h]
Below solution using Java 8 Streams:
//Sample Input
List<String> input = new ArrayList<String>();
IntStream.range(1,999).forEach((num) -> {
input.add(""+num);
});
//Identify no. of batches
int BATCH_SIZE = 10;
int multiples = input.size() / BATCH_SIZE;
if(input.size()%BATCH_SIZE!=0) {
multiples = multiples + 1;
}
//Process each batch
IntStream.range(0, multiples).forEach((indx)->{
List<String> batch = input.stream().skip(indx * BATCH_SIZE).limit(BATCH_SIZE).collect(Collectors.toList());
System.out.println("Batch Items:"+batch);
});
Another approach to solve this, question:
public class CollectionUtils {
/**
* Splits the collection into lists with given batch size
* #param collection to split in to batches
* #param batchsize size of the batch
* #param <T> it maintains the input type to output type
* #return nested list
*/
public static <T> List<List<T>> makeBatch(Collection<T> collection, int batchsize) {
List<List<T>> totalArrayList = new ArrayList<>();
List<T> tempItems = new ArrayList<>();
Iterator<T> iterator = collection.iterator();
for (int i = 0; i < collection.size(); i++) {
tempItems.add(iterator.next());
if ((i+1) % batchsize == 0) {
totalArrayList.add(tempItems);
tempItems = new ArrayList<>();
}
}
if (tempItems.size() > 0) {
totalArrayList.add(tempItems);
}
return totalArrayList;
}
}
A one-liner in Java 8 would be:
import static java.util.function.Function.identity;
import static java.util.stream.Collectors.*;
private static <T> Collection<List<T>> partition(List<T> xs, int size) {
return IntStream.range(0, xs.size())
.boxed()
.collect(collectingAndThen(toMap(identity(), xs::get), Map::entrySet))
.stream()
.collect(groupingBy(x -> x.getKey() / size, mapping(Map.Entry::getValue, toList())))
.values();
}
import com.google.common.collect.Lists;
List<List<T>> batches = Lists.partition(List<T>,batchSize)
Use Lists.partition(List,batchSize). You need to import Lists from google common package (com.google.common.collect.Lists)
It will return List of List<T> with and the size of every element equal to your batchSize.
I'm trying to use LINQ to transform the following list. LINQ should multiply each element against the next as long as the product is less than 15. Additionally we should save the number of elements used to form the product.
int[] values = { 1, 3, 4, 2, 7, 14 }; //assume Largest value will never be >= 15
1x3x4 = 12
2x7 = 14
14 = 14
{ {12,3}, {14,2}, {14,1} }
My ultimate goal is to take the geometric average of a very large list of numbers. This is normally done by multiplying each element in the list together (1x3x4x2x7x14) then taking the nth root (in this case 1/6).
The obvious problem in using the "normal" method is that you will quickly find yourself using numbers beyond the maximum allowable number. You can workaround this by using the old divide and conquer method and with a little help from the natural log function.
I don't think there is something like that build into standard LINQ method library. But you can easily create your own extension method. I called it AggregateUntil:
public static class EnumerableExtensions
{
public static IEnumerable<TResult> AggregateUntil<TSource, TAccumulate, TResult>(
this IEnumerable<TSource> source,
TAccumulate seed,
Func<TAccumulate, TSource, TAccumulate> func,
Func<TAccumulate, bool> condition,
Func<TAccumulate, TResult> resultSelector
)
{
TAccumulate acc = seed;
TAccumulate newAcc;
foreach(var item in source)
{
newAcc = func(acc, item);
if(!condition(newAcc))
{
yield return resultSelector(acc);
acc = func(seed, item);
}
else
{
acc = newAcc;
}
}
yield return resultSelector(acc);
}
}
And now let's use it. First, take multiplications only, as long as they met < 15 condition:
var grouped
= values.AggregateUntil(1, (a,i) => a * i, a => a < 15, a => a).ToList();
Returns List<int> with 3 items: 12, 14, 14. That's what you need. But now lets take number of items which were aggregated into each multiplication. That's easy using anonymous type::
int[] values = { 1, 3, 4, 2, 7, 14 };
var grouped
= values.AggregateUntil(
new { v = 1, c = 0 },
(a, i) => new { v = a.v * i, c = a.c + 1 },
a => a.v < 15,
a => a).ToList(); ;
Returns exactly what you need:
My ultimate goal is to take the geometric average of a very large list of numbers.
Then just take the nth root of each number and multiply afterwards. Then you don't need to worry about splitting the list into groups:
double mean = 1.0;
foreach(int i in values)
{
mean *= Math.Pow(i, 1.0 / values.Length);
}
Which could also be done in Linq with Aggregate:
mean = values.Aggregate(1.0, (prev, i) => prev * Math.Pow(i, 1.0 / values.Length ));
Well my solution is not quite as elegant as #MarcinJuraszek, but it's fast and it works within your constraints.
int[] values = {1, 3, 4, 2, 7, 14};
int product = 1;
int elementsMultiplied = 0;
List<Tuple<int,int>> allElements = new List<Tuple<int,int>>();
for(int i = 0; i < values.Length ; i++)
{
product = product * values[i];
elementsMultiplied++;
if(i == values.Length - 1 || product * values[i+1] >= 15)
{
allElements.Add(new Tuple<int,int>(product, elementsMultiplied));
product = 1;
elementsMultiplied = 0;
}
}
foreach(Tuple<int,int> pair in allElements)
{
Console.WriteLine(pair.Item1 + "," + pair.Item2);
}