I have two implementations of the same algorithm. I would like to verify that non of them uses more memory than necessary, or, in other words, that they allocate exactly the same number of objects.
My current solution is to measure the number of allocated bytes before and after the procedures via threadMXBean.getThreadAllocatedBytes(threadId) and use that as an approximation of the memory footprint.
The problem is that this method is unstable, e.i. sometimes it returns a much greater number than it should. It especially shows on algorithms that don't allocate objects. One problematic example is a method that sums an int[].
Actual code (Kotlin):
class MemAllocationTest {
private val threadMXBean = (ManagementFactory.getThreadMXBean() as? com.sun.management.ThreadMXBean)
?: throw RuntimeException("Runtime does not support com.sun.management.ThreadMXBean")
/**
* May run [block] several times
* */
private inline fun measureAllocatedBytes(block: () -> Unit): Long {
val threadId = Thread.currentThread().id
val before = threadMXBean.getThreadAllocatedBytes(threadId)
block()
val after = threadMXBean.getThreadAllocatedBytes(threadId)
return after - before
}
....
Is there a better solution?
(I don't know how to do that with JMH, but IMHO this is a very close topic)
JMH has -prof gc profiler, which is supposed to be accurate with allocation profiling. Although it uses the same ThreadMXBean under cover, it can filter out warmup effects, and average the hiccups out over multiple #Benchmark invocations. The typical errors are within 0.001 byte/op there.
My current solution is to collect statistics with several runs:
private inline fun stabiliseMeasureAllocatedBytes(block: () -> Unit): Long {
val runs = List(7) { measureAllocatedBytes(block) }
val results = runs.drop(2) // skip warm-up
val counts = results.groupingBy { it }.eachCount()
val (commonResult, commonCount) = counts.entries.maxBy { (result, count) -> count }!!
if (commonCount >= results.size / 2)
return commonResult
else
throw RuntimeException("Allocation measurements vary too much: $runs")
}
Related
I have a function like this:
fun randomWalk(numSteps: Int): Int {
var n = 0
repeat(numSteps) { n += (-1 + 2 * Random.nextInt(2)) }
return n.absoluteValue
}
This works fine, except that it uses a mutable variable, and I would like to make everything immutable when possible, for better safety and readability. So I came up with an equivalent version that doesn't use any mutable variables:
fun randomWalk_seq(numSteps: Int): Int =
generateSequence(0) { it + (-1 + 2 * Random.nextInt(2)) }
.elementAt(numSteps)
.absoluteValue
This also works fine and produces the same results, but it takes 3 times longer.
I used the following way to measure it:
#OptIn(ExperimentalTime::class)
fun main() {
val numSamples = 100000
val numSteps = 15708
repeat(5) {
val randomWalkSamples: IntArray
val duration = measureTime {
randomWalkSamples = IntArray(numSamples) { randomWalk(numSteps) }
}
println(duration)
}
}
I know it's a bit hacky (I could have used JMH but this is just a quick test - at least I know that measureTime uses a monotonic clock). The results for the iterative (mutable) version:
2.965358406s
2.560777033s
2.554363661s
2.564279403s
2.608323586s
As expected, the first line shows it took a bit longer on the first run due to the warming up of the JIT, but the next 4 lines have fairly small variation.
After replacing randomWalk with randomWalk_seq:
6.636866719s
6.980840906s
6.993998111s
6.994038706s
7.018054467s
Somewhat surprisingly, I don't see any warmup time - the first line is always lesser duration than the following 4 lines, every time I run this. And also, every time I run it, the duration keeps increasing, with line 5 always being the greatest duration.
Can someone explain the findings, and also is there any way of making this function not use any mutable variables but still have performance that is close to the mutable version?
Your solution is slower for two main reasons: boxing and the complexity of the iterator used by generateSequence()'s Sequence implementation.
Boxing happens because a Sequence uses its types generically, so it cannot use primitive 32-bit Ints directly, but must wrap them in classes and unwrap them when retrieving the items.
You can see the complexity of the iterator by Ctrl+clicking the generateSequence function to view the source code.
#Михаил Нафталь's suggestion is faster because it avoids the complex iterator of the sequence, but it still has boxing.
I tried writing an overload of sumOf that uses IntProgression directly instead of Iterable<T>, so it won't use boxing, and that resulted in equivalent performance to your imperative code with the var. As you can see, it's inline and when put together with the { -1 + 2 * Random.nextInt(2) } lambda suggested by #Михаил Нафталь, then the resulting compiled code will be equivalent to your imperative code.
inline fun IntProgression.sumOf(selector: (Int) -> Int): Int {
var sum: Int = 0.toInt()
for (element in this) {
sum += selector(element)
}
return sum
}
Ultimately, I don't think you're buying yourself much in the way of code clarity by removing a single var in such a small function. I would say the sequence code is arguably harder to read. vars may add to code complexity in complex algorithms, but I don't think they do in such simple algorithms, especially when there's only one of them and it's local to the function.
Equivalent immutable one-liner is:
fun randomWalk2(numSteps: Int) =
(1..numSteps).sumOf { -1 + 2 * Random.nextInt(2) }.absoluteValue
Probably, even more performant would be to replace
with
so that you'll have one multiplication and n additions instead of n multiplications and (2*n-1) additions:
fun randomWalk3(numSteps: Int) =
(-numSteps + 2 * (1..numSteps).sumOf { Random.nextInt(2) }).absoluteValue
Update
As #Tenfour04 noted, there is no specific stdlib implementation for IntProgression.sumOf, so it's resolved to Iterable<T>.sumOf, which will add unnecessary overhead for int boxing.
So, it's better to use IntArray here instead of IntProgression:
fun randomWalk4(numSteps: Int) =
(-numSteps + 2 * IntArray(numSteps).sumOf { Random.nextInt(2) }).absoluteValue
Still encourage you to check this all with JMH
I think:"Removing mutability without losing speed" is wrong title .because
mutability thing comes to deal with the flow that program want to achieve .
you are using var inside function.... and 100% this var will not ever change from outside this function and that is mutability concept.
if we git rid off from var everywhere why we need it in programming ?
Kotlin for competitive programming suggests the following code for reading console input.
readLine()!!.split(" ").map{ str -> str.toInt() } //read space separated Integer from console
Until now for every competitive problem I've used the same approach and to be honest, it has never disappointed.
But for certain problems where the count of input integers is very large (close to 2 * 10^6) this method is just too slow and results in TLE (Time Limit Exceeded).
Is there even a faster way to read input from console?
If you suspect that the .split() call is the bottleneck, you could explore some of the alternatives in this thread.
If you suspect that the toInt() call is the bottleneck, perhaps you could try parallelizing the streams using the Java 8 stream API. For example:
readLine()!!.split(" ").parallelStream().map { str -> str.toInt() }...
For best performance, you could probably combine the two methods.
I believe, toInt() convertions are more expensive, than split(" ").
Are you sure you need to convert to Int all strings of the input in the very beginning?
It depends on a task, but sometimes part of this convertions could be avoided.
For instance, if you have a task "check if there is no negative numbers in the input", you may convert strings to Int one by one, and if you met a negative one, no need to convert others.
I think that JMH could be useful here. You can run the benchmark similar to the one bellow and try to identify your bottlenecks.
Note that this is in Mode.SingleShotTime, and so emulates the scenarion where JIT has little opprotunity to do it's thing.
import org.openjdk.jmh.annotations.*
import java.util.concurrent.TimeUnit
import kotlin.random.Random
//#BenchmarkMode(Mode.AverageTime)
#BenchmarkMode(Mode.SingleShotTime)
#OutputTimeUnit(TimeUnit.MILLISECONDS)
#State(Scope.Benchmark)
open class SplitToInt {
val count = 2 * 1_000_000
lateinit var stringToParse: String
lateinit var tokensToParse: List<String>
#Setup(Level.Invocation)
fun setup() {
stringToParse = (0..count).map { Random.nextInt(0, 100) }.joinToString(separator = " ")
tokensToParse = (0..count).map { Random.nextInt(0, 100) }.map { it.toString() }
}
#Benchmark
open fun split() =
stringToParse.split(" ")
#Benchmark
open fun map_toInt() =
tokensToParse.map { it.toInt() }
#Benchmark
open fun split_map_toInt() =
stringToParse.split(" ").map { it.toInt() }
}
The stats on my machine are:
Benchmark Mode Cnt Score Error Units
SplitToInt.map_toInt ss 48.666 ms/op
SplitToInt.split ss 124.899 ms/op
SplitToInt.split_map_toInt ss 186.981 ms/op
So splitting the string and mapping to list of Ints takes ~ 187 ms. Allowing JIT to warm up (Mode.AverageTime) gives me:
Benchmark Mode Cnt Score Error Units
SplitToInt.map_toInt avgt 5 30.670 ± 6.979 ms/op
SplitToInt.split avgt 5 108.989 ± 23.569 ms/op
SplitToInt.split_map_toInt avgt 5 120.628 ± 27.052 ms/op
Whether this is fast or slow depends on the curmstances but are you sure that the input transformation here is the reason you get TLE?
Edit: If you do think that split(" ").map{ str -> str.toInt() } is too slow, you could replace creating the two lists (one from split and one from map) with a single list by splitting and transforming in one go. I wrote a quick hack off kotlin.text.Regex.split that does that and it is about 20% faster.
If in your use case you need to examine only part of the input, splitToSequence is probably a better option.
I would like to know if an update operation on a mutable map is better in performance than reassignment.
Lets assume I have the following Map
val m=Map(1 -> Set("apple", "banana"),
2 -> Set("banana", "cabbage"),
3 -> Set("cabbage", "dumplings"))
which I would like to reverse into this map:
Map("apple" -> Set(1),
"banana" -> Set(1, 2),
"cabbage" -> Set(2, 3),
"dumplings" -> Set(3))
The code to do so is:
def reverse(m:Map[Int,Set[String]])={
var rm = Map[String,Set[Int]]()
m.keySet foreach { k=>
m(k) foreach { e =>
rm = rm + (e -> (rm.getOrElse(e, Set()) + k))
}
}
rm
}
Would it be more efficient to use the update operator on a map if it is very large in size?
The code using the update on map is as follows:
def reverse(m:Map[Int,Set[String]])={
var rm = scala.collection.mutable.Map[String,Set[Int]]()
m.keySet foreach { k=>
m(k) foreach { e =>
rm.update(e,(rm.getOrElse(e, Set()) + k))
}
}
rm
}
I ran some tests using Rex Kerr's Thyme utility.
First I created some test data.
val rndm = new util.Random
val dna = Seq('A','C','G','T')
val m = (1 to 4000).map(_ -> Set(rndm.shuffle(dna).mkString
,rndm.shuffle(dna).mkString)).toMap
Then I timed some runs with both the immutable.Map and mutable.Map versions. Here's an example result:
Time: 2.417 ms 95% CI 2.337 ms - 2.498 ms (n=19) // immutable
Time: 1.618 ms 95% CI 1.579 ms - 1.657 ms (n=19) // mutable
Time 2.278 ms 95% CI 2.238 ms - 2.319 ms (n=19) // functional version
As you can see, using a mutable Map with update() has a significant performance advantage.
Just for fun I also compared these results with a more functional version of a Map reverse (or what I call a Map inverter). No var or any mutable type involved.
m.flatten{case(k, vs) => vs.map((_, k))}
.groupBy(_._1)
.mapValues(_.map(_._2).toSet)
This version consistently beat your immutable version but still doesn't come close to the mutable timings.
The trade-of between mutable and immutable collections usually narrows down to this:
immutable collections are safer to share and allows to use structural sharing
mutable collections have better performance
Some time ago I did comparison of performance between mutable and immutable Maps in Scala and the difference was about 2 to 3 times in favor of mutable ones.
So, when performance is not critical I usually go with immutable collections for safety and readability.
For example, in your case functional "scala way" of performing this transformation would be something like this:
m.view
.flatMap(x => x._2.map(_ -> x._1)) // flatten map to lazy view of String->Int pairs
.groupBy(_._1) // group pairs by String part
.mapValues(_.map(_._2).toSet) // extract all Int parts into Set
Although I used lazy view to avoid creating intermediate collections, groupBy still internally creates mutable map (you may want to check it's sources, the logic is pretty similar to what you have wrote), which in turn gets converted to immutable Map which then gets discarded by mapValues.
Now, if you want to squeeze every bit of performance you want to use mutable collections and do as little updates of immutable collections as possible.
For your case is means having Map of mutable Sets as you intermediate buffer:
def transform(m:Map[Int, Set[String]]):Map[String, Set[Int]] = {
val accum:Map[String, mutable.Set[Int]] =
m.valuesIterator.flatten.map(_ -> mutable.Set[Int]()).toMap
for ((k, vals) <- m; v <- vals) {
accum(v) += k
}
accum.mapValues(_.toSet)
}
Note, I'm not updating accum once it's created: I'm doing exactly one map lookup and one set update for each value, while in both your examples there was additional map update.
I believe this code is reasonably optimal performance wise. I didn't perform any tests myself, but I highly encourage you to do that on your real data and post results here.
Also, if you want to go even further, you might want to try mutable BitSet instead of Set[Int]. If ints in your data are fairly small it might yield some minor performance increase.
Just using #Aivean method in a functional way:
def transform(mp :Map[Int, Set[String]]) = {
val accum = mp.values.flatten
.toSet.map( (_-> scala.collection.mutable.Set[Int]())).toMap
mp.map {case(k,vals) => vals.map( v => accum(v)+=k)}
accum.mapValues(_.toSet)
}
It is common in C/C++ programming to use function pointers to optimize branching in the main data path. So I wrote a test program to find out if similar performance savings can be gotten in Scala using functional programming techniques. The usecase is that of a function which is invoked millions of times and has a branching statement based on a global flag. The code using if() statement -
val b = true
def test() = {
if(b) // do something
else // do something else
}
for(i <- 0 to 100000) test()
And trying to get rid of the if() I did this -
def onTrue() = { // do something }
def onFalse() = { // do something else }
lazy val callback: () => Unit = if(b) onTrue else onFalse
def test() = callback()
for(i <- 0 to 100000) test()
I did a comparison of both these programs by running them for large counters (in the for loop) and running them many times and using the System.nanoTime() differential to measure the time taken.
And my tests seem to suggest that the callback method is actually SLOWER than using if() in the loop. The reason for this could be that a function call requires the params and returns to be pushed on the stack and a new stack frame created etc. Given this finding wanted to know -
Is there a functional way one could code which will better the performance of using the if() in the loop with Scala?
#inline works with compiler. Is there a runtime equivalent to avoid the stack activities? (similar to tail call optimization)
Could my test or results be inaccurate/erroneous in some way?
3) It's very easy to get your methodology wrong when testing this way. Use something like JMH if you want quasi-trustable microbenchmarks!
2) The JVM does inlining at runtime.
1) You aren't measuring a difference in whether something is "functional". You're measuring the difference between using a lazy val and not. If you don't have the lazy val in there, the JVM will probably be able to optimize your code (depending on what "do something" is).
If you remove the lazy val, the second one optimizes to the same speed in my hands. (It has an extra mandatory check for every access that it isn't being initialized in a multi-threaded context.)
I'm reading a tutorial about Swift (http://www.raywenderlich.com/74438/swift-tutorial-a-quick-start) and it preconized to not set type explicitly because it's more readable this way.
I do not really agree about this point but that's not the question. My question is : Is it more efficient, in terms of performance (compiler...) to set type explicitly ?
For example, would this : var hello: Int = 56 be more efficient than this : var tutorialTeam = 56
There is no difference in performance between code that uses explicit types and code which uses type inference. The compiled output is identical in each case.
When you omit the type the compiler simply infers it.
The very small differences observed in the accepted answer are just your usual micro benchmarking artefacts, and cannot be trusted!
Whether or not you include the explicit type is a matter of taste. In some contexts it might make your code more readable.
The only time it makes a difference to your code is when you want to specify a different type to the one which the compiler would infer. As an example:
var num = 2
The above code infers that num is an Int, due to it being initialised with an integer literal. However you can 'force' it to be a Double as follows:
var num: Double = 2
From my experience, there's been a huge performance impact in terms of compilation speed when using explicit vs inferred types. A majority of my slow compiling code has been resolved by explicitly typing variables.
It seems like the Swift compiler still has room for improvement in this area. Try benchmarking some of your projects and you'll see a big difference.
Here's an article I wrote on how to speed up slow Swift compile times and how to find out what is causing it.
Type inference will not affect performance in your given example. However, I did find that being specific about the Type in your Swift array does impact performance significantly.
For example, the method below shuffles an array of type Any.
class func shuffleAny(inout array: [Any]) {
for (var i = 0; i < array.count; i++) {
let currentObject: Any = array[i]
let randomIndex = Int(arc4random()) % array.count
let randomObject: Any = array[randomIndex]
array[i] = randomObject;
array[randomIndex] = currentObject
}
}
The above function is actually much slower than if I were to make this function take an array of Int instead like this
class func shuffleIntObjects(inout array: [Int]) {
for (var i = 0; i < array.count; i++) {
let currentObject: Int = array[i]
let randomIndex = Int(arc4random()) % array.count
let randomObject: Int = array[randomIndex]
array[i] = randomObject;
array[randomIndex] = currentObject
}
}
The function that uses [Any] clocked in at 0.537 seconds 3% STDEV for 1 million Int objects. And the function that uses [Int] clocked in at 0.181 seconds 2% STDEV for 1 million Int objects.
You can check out this repo (https://github.com/vsco/swift-benchmarks) which details a lot more interesting benchmarks in Swift. One of my favorite ones is that Swift generics perform very poorly with the test conditions mentioned above