I wonder how this can be done in Clojure idiomatically and efficiently:
1) Given a vector containing n integers in it: [A0 A1 A2 A3 ... An]
2) Increase the last x items by 1 (let's say x is 100) so the vector will become: [A0 A1 A2 A3 ... (An-99 + 1) (An-98 + 1)... (An-1 + 1) (An + 1)]
One naive implementation looks like:
(defn inc-last [x nums]
(let [n (count nums)]
(map #(if (>= % (- n x)) (inc %2) %2)
(range n)
nums)))
(inc-last 2 [1 2 3 4])
;=> [1 2 4 5]
In this implementation, basically you just map the entire vector to another vector by examine each item to see if it needs to be increased.
However, this is an O(n) operation while I only want to change the last x items in the vector. Ideally, this should be done in O(x) instead of O(n).
I am considering using some functions like split-at/concat to implement it like below:
(defn inc-last [x nums]
(let [[nums1 nums2] (split-at x nums)]
(concat nums1 (map inc nums2))))
However, I am not sure if this implementation is O(n) or O(x). I am new to Clojure and not really sure what the time complexity will be for operations like concat/split-at on persistent data structures in Clojure.
So my questions are:
1) What the time complexity here in second implementation?
2) If it is still O(n), is there any idiomatic and efficient implementation that takes only O(x) in Clojure for solving this problem?
Any comment is appreciated. Thanks.
Update:
noisesmith's answer told me that split-at will convert the vector into a list, which was a fact I did not realised previously. Since I will do random access for the result (call nth after processing the vector), I would like to have an efficient solution (O(x) time) while keeping the vector instead of list otherwise nth will slow down my program as well.
Concat and split-at both turn the input into a seq, effectively a linked-list representation, O(x) time. Here is how to do it with a vector for O(n) performance.
user> (defn inc-last-n
[n x]
(let [count (count x)
update (fn [x i] (update-in x [i] inc))]
(reduce update x (range (- count n) count))))
#'user/inc-last-n
user> (inc-last-n 3 [0 1 2 3 4 5 6])
[0 1 2 3 5 6 7]
This will fail on input that is not associative (like seq / lazy-seq) because there is no O(1) access time in non-associative types.
inc-last is an implementation using a transient, which allows to get a modifiable "in place" vector in constant time and return a persistent! vector also in constant time, which allows to make the updates in O(x). The original implementation used an imperative doseq loop but, as mentioned in the comments, transient operations can return a new object, so it's better to keep doing things in a functional way.
I added a doall to the call to inc-last-2 since it returns a lazy seq, but inc-last and inc-last-3 returns a vector so the doall is needed to be able to compare them all.
According to some quick tests I made, inc-last and inc-last-3 don't actually differ much in performance, not even for huge vectors (10000000 elements). For the inc-last-2 implementation though, there's quite a difference even for a vector of 1000 elements, modifying only the last 10, it's ~100x slower. For smaller vectors or when the n is close to (count nums) the difference is not really that much.
(Thanks to Michał Marczyk for his useful comments)
(def x (vec (range 1000)))
(defn inc-last [n x]
(let [x (transient x)
l (count x)]
(->>
(range (- l n) l)
(reduce #(assoc! %1 %2 (inc (%1 %2))) x)
persistent!)))
(defn inc-last-2 [x nums]
(let [n (count nums)]
(map #(if (>= % (- n x)) (inc %2) %2)
(range n)
nums)))
(defn inc-last-3 [n x]
(let [l (count x)]
(reduce #(assoc %1 %2 (inc (%1 %2))) x (range (- l n) l))))
(time
(dotimes [i 100]
(inc-last 50 x)))
(time
(dotimes [i 100]
(doall (inc-last-2 10 x))))
(time
(dotimes [i 100]
(inc-last-3 50 x)))
;=> "Elapsed time: 49.7965 msecs"
;=> "Elapsed time: 1751.964501 msecs"
;=> "Elapsed time: 67.651 msecs"
Related
So I have an exercise where I should count the ways of climbing a ladder with n amount of steps, with the following restriction: You can only go up by 1, 3 or 5 steps.
The way I could solve the problem was with this code.
(define (climb n)
(cond [(< n 0) 0]
[(<= n 2) 1]
[(= n 3) 2]
[(> n 1) (+ (climb (- n 1)) (climb (- n 3)) (climb (- n 5)))]
)
)
But the problem now is that it is too difficult to get the result, you have to do a lot of calculations.
Is it possible to optimize this on the racket? What would the code be like? If I could save the previous 2 results and add it up, it would work better, but I don't know how.
The trick with dynamic programming is to store previous values that have already been computed, in this way we don't have to redo expensive recursive calls. Usually we store the values in an array, a matrix or a hash table - but for this case we can simply use a bunch of parameters and tail recursion. This works, and it's pretty fast!
(define (climb n)
(if (negative? n)
0
(let loop ([n n] [a 1] [b 1] [c 1] [d 2] [e 3] [f 5])
(if (zero? n)
a
(loop (sub1 n) b c d e f (+ a b c d e))))))
For example:
(climb 100)
=> 20285172757012753619
Below, I have 2 functions computing the sum of squares of their arguments. The first one is nice and functional, but 20x slower than the second one. I presume that the r/map is not taking advantage of aget to retrieve elements from the double-array, whereas I'm explicitly doing this in function 2.
Is there any way I can further typehint or help r/map r/fold to perform faster?
(defn sum-of-squares
"Given a vector v, compute the sum of the squares of elements."
^double [^doubles v]
(r/fold + (r/map #(* % %) v)))
(defn sum-of-squares2
"This is much faster than above. Post to stack-overflow to see."
^double [^doubles v]
(loop [val 0.0
i (dec (alength v))]
(if (neg? i)
val
(let [x (aget v i)]
(recur (+ val (* x x)) (dec i))))))
(def a (double-array (range 10)))
(quick-bench (sum-of-squares a))
800 ns
(quick-bench (sum-of-squares2 a))
40 ns
Before experiments I've added next line in project.clj:
:jvm-opts ^:replace [] ; Makes measurements more accurate
Basic measurements:
(def a (double-array (range 1000000))) ; 10 is too small for performance measurements
(quick-bench (sum-of-squares a)) ; ... Execution time mean : 27.617748 ms ...
(quick-bench (sum-of-squares2 a)) ; ... Execution time mean : 1.259175 ms ...
This is more or less consistent with time difference in the question. Let's try to not use Java arrays (which are not really idiomatic for Clojure):
(def b (mapv (partial * 1.0) (range 1000000))) ; Persistent vector
(quick-bench (sum-of-squares b)) ; ... Execution time mean : 14.808644 ms ...
Almost 2 times faster. Now let's remove type hints:
(defn sum-of-squares3
"Given a vector v, compute the sum of the squares of elements."
[v]
(r/fold + (r/map #(* % %) v)))
(quick-bench (sum-of-squares3 a)) ; Execution time mean : 30.392206 ms
(quick-bench (sum-of-squares3 b)) ; Execution time mean : 15.583379 ms
Execution time increased only marginally comparing to version with type hints. By the way, version with transducers has very similar performance and is much cleaner:
(defn sum-of-squares3 [v]
(transduce (map #(* % %)) + v))
Now about additional type hinting. We can indeed optimize first sum-of-squares implementation:
(defn square ^double [^double x] (* x x))
(defn sum-of-squares4
"Given a vector v, compute the sum of the squares of elements."
[v]
(r/fold + (r/map square v)))
(quick-bench (sum-of-squares4 b)) ; ... Execution time mean : 12.891831 ms ...
(defn pl
(^double [] 0.0)
(^double [^double x] (+ x))
(^double [^double x ^double y] (+ x y)))
(defn sum-of-squares5
"Given a vector v, compute the sum of the squares of elements."
[v]
(r/fold pl (r/map square v)))
(quick-bench (sum-of-squares5 b)) ; ... Execution time mean : 9.441748 ms ...
Note #1: type hints on arguments and return value of sum-of-squares4 and sum-of-squares5 have no additional performance benefits.
Note #2: It's generally bad practice to start with optimizations. Straight-forward version (apply + (map square v)) will have good enough performance for most situations. sum-of-squares2 is very far from idiomatic and uses literally no Clojure concepts. If this is really performance critical code - better to implement it in Java and use interop. Code will be much cleaner despite of having 2 languages. Or even implement it in unmanaged code (C, C++) and use JNI (not really maintainable but if properly implemented, can give the best possible performance).
Why not use areduce:
(def sum-of-squares3 ^double [^doubles v]
(areduce v idx ret 0.0
(let [item (aget v idx)]
(+ ret (* item item)))))
On my machine running:
(criterium/bench (sum-of-squares3 (double-array (range 100000))))
Gives a mean execution time of 1.809103 ms, your sum-of-squares2 executes the same calculation in 1.455775 ms. I think this version using areduce is more idiomatic than your version.
For squeezing a little bit more performance you can try using unchecked math (add-unchecked and multiply-unchecked). But beware, you need to be sure that your calculation cannot overflow:
(defn sum-of-squares4 ^double [^doubles v]
(areduce v idx ret 0.0
(let [item (aget v idx)]
(unchecked-add ret (unchecked-multiply item item)))))
Running the same benchmark gives a mean execution time of 1.144197 ms. Your sum-of-squares2 can also benefit from unchecked math with a 1.126001 ms mean execution time.
In clojure[script], how to write a function nearest that receives two sorted vectors a, b and returns for each element of a the nearest element of b?
As an example,
(nearest [1 2 3 101 102 103] [0 100 1000]); [0 0 0 100 100 100]
I would like the solution to be both idiomatic and with good performances: O(n^2) is not acceptable!
Using a binary search or a sorted-set incurs a O(n*log m) time complexity where n is (count a) and m (count b).
However leveraging the fact that a and b are sorted the time complexity can be O(max(n, m)).
(defn nearest [a b]
(if-let [more-b (next b)]
(let [[x y] b
m (/ (+ x y) 2)
[<=m >m] (split-with #(<= % m) a)]
(lazy-cat (repeat (count <=m) x)
(nearest >m more-b)))
(repeat (count a) (first b))))
=> (nearest [1 2 3 101 102 103 501 601] [0 100 1000])
(0 0 0 100 100 100 100 1000)
Let n be (count a) and m be (count b). Then, if a and b are both ordered, then this can be done in what I believe ought to be O(n log(log m)) time, in other words, very close to linear in n.
First, let's re-implement abs and a binary-search (improvements here) to be independent of host (leveraging a native, e.g. Java's, version ought to be significantly faster)
(defn abs [x]
(if (neg? x) (- 0 x) x))
(defn binary-search-nearest-index [v x]
(if (> (count v) 1)
(loop [lo 0 hi (dec (count v))]
(if (== hi (inc lo))
(min-key #(abs (- x (v %))) lo hi)
(let [m (quot (+ hi lo) 2)]
(case (compare (v m) x)
1 (recur lo m)
-1 (recur m hi)
0 m))))
0))
If b is sorted, a binary search in b takes log m steps. So, mapping this over a is a O(n log m) solution, which for the pragmatist is likely good enough.
(defn nearest* [a b]
(map #(b (binary-search-nearest-index b %)) a))
However, we can also use the fact that a is sorted to divide and conquer a.
(defn nearest [a b]
(if (< (count a) 3)
(nearest* a b)
(let [m (quot (count a) 2)
i (binary-search-nearest-index b (a m))]
(lazy-cat
(nearest (subvec a 0 m) (subvec b 0 (inc i)))
[(b i)]
(nearest (subvec a (inc m)) (subvec b i))))))
I believe this ought to be O(n log(log m)). We start with the median of a and find nearest in b in log m time. Then we recurse on each half of a with split portions of b. If a m-proportional factor of b are split each time, you have O(n log log m). If only a constant portion is split off then the half of a working on that portion is linear time. If that continues (iterative halves of a work on constant size portions of b) then you have O(n).
Inspired by #amalloy, I have found this interesting idea by Chouser and wrote this solution:
(defn abs[x]
(max x (- x)))
(defn nearest-of-ss [ss x]
(let [greater (first (subseq ss >= x))
smaller (first (rsubseq ss <= x))]
(apply min-key #(abs (- % x)) (remove nil? [greater smaller]))))
(defn nearest[a b]
(map (partial nearest-of-ss (apply sorted-set a)) b))
Remark: It's important to create the sorted-set only once, in order to avoid performance penalty!
I was wondering if someone could help me with the performance of this code snippet in Clojure 1.3. I am trying to implement a simple function that takes two vectors and does a sum of products.
So let's say the vectors are X (size 10,000 elements) and B (size 3 elements), and the sum of products are stored in a vector Y, mathematically it looks like this:
Y0 = B0*X2 + B1*X1 + B2*X0
Y1 = B0*X3 + B1*X2 + B2*X1
Y2 = B0*X4 + B1*X3 + B2*X2
and so on ...
For this example, the size of Y will end up being 9997, which corresponds to (10,000 - 3). I've set up the function to accept any size of X and B.
Here's the code: It basically takes (count b) elements at a time from X, reverses it, maps * onto B and sums the contents of the resulting sequence to produce an element of Y.
(defn filt [b-vec x-vec]
(loop [n 0 sig x-vec result []]
(if (= n (- (count x-vec) (count b-vec)))
result
(recur (inc n) (rest sig) (conj result (->> sig
(take (count b-vec))
(reverse)
(map * b-vec)
(apply +)))))))
Upon letting X be (vec (range 1 10001)) and B being [1 2 3], this function takes approximately 6 seconds to run. I was hoping someone could suggest improvements to the run time, whether it be algorithmic, or perhaps a language detail I might be abusing.
Thanks!
P.S. I have done (set! *warn-on-reflection* true) but don't get any reflection warning messages.
You are using count many times unnecessary. Below code calculate count one time only
(defn filt [b-vec x-vec]
(let [bc (count b-vec) xc (count x-vec)]
(loop [n 0 sig x-vec result []]
(if (= n (- xc bc))
result
(recur (inc n) (rest sig) (conj result (->> sig
(take bc)
(reverse)
(map * b-vec)
(apply +))))))))
(time (def b (filt [1 2 3] (range 10000))))
=> "Elapsed time: 50.892536 msecs"
If you really want top performance for this kind of calculation, you should use arrays rather than vectors. Arrays have a number of performance advantages:
They support O(1) indexed lookup and writes - marginally better than vectors which are O(log32 n)
They are mutable, so you don't need to construct new arrays all the time - you can just create a single array to serve as the output buffer
They are represented as Java arrays under the hood, so benefit from the various array optimisations built into the JVM
You can use primitive arrays (e.g. of Java doubles) which are much faster than if you use boxed number objects
Code would be something like:
(defn filt [^doubles b-arr
^doubles x-arr]
(let [bc (count b-arr)
xc (count x-arr)
rc (inc (- xc bc))
result ^doubles (double-array rc)]
(dotimes [i rc]
(dotimes [j bc]
(aset result i (+ (aget result i) (* (aget x-arr (+ i j)) (aget b-arr j))))))
result))
To follow on to Ankur's excellent answer, you can also avoid repeated calls to the reverse function, which gets us even a little more performance.
(defn filt [b-vec x-vec]
(let [bc (count b-vec) xc (count x-vec) bb-vec (reverse b-vec)]
(loop [n 0 sig x-vec result []]
(if (= n (- xc bc))
result
(recur (inc n) (rest sig) (conj result (->> sig
(take bc)
(map * bb-vec)
(apply +))))))))
I decided to work through the CLRS Introduction to Algorithms text, and picked the printing neatly problem here.
I worked through the problem and came up with an imperative solution which was straightforward to implement in Python, but somewhat less so in Clojure.
I'm completely stumped on translating the compute-matrix function from my solution into idiomatic Clojure. Any suggestions? Here is the pseudocode for the compute-matrix function:
// n is the dimension of the square matrix.
// c is the matrix.
function compute-matrix(c, n):
// Traverse through the left-lower triangular matrix and calculate values.
for i=2 to n:
for j=i to n:
// This is our minimum value sentinal.
// If we encounter a value lower than this, then we store the new
// lowest value.
optimal-cost = INF
// Index in previous column representing the row we want to point to.
// Whenever we update 't' with a new lowest value, we need to change
// 'row' to point to the row we're getting that value from.
row = 0
// This iterates through each entry in the previous column.
// Note: we have a lower triangular matrix, meaning data only
// exists in the left-lower half.
// We are on column 'i', but because we're in a left-lower triangular
// matrix, data doesn't start until row (i-1).
//
// Similarly, we go to (j-1) because we can't choose a configuration
// where the previous column ended on a word who's index is larger
// than the word index this column starts on - the case which occurs
// when we go for k=(i-1) to greater than (j-1)
for k=(i-1) to (j-1):
// When 'j' is equal to 'n', we are at the last cell and we
// don't care how much whitespace we have. Just take the total
// from the previous cell.
// Note: if 'j' < 'n', then compute normally.
if (j < n):
z = cost(k + 1, j) + c[i-1, k]
else:
z = c[i-1, k]
if z < optimal-cost:
row = k
optimal-cost = z
c[i,j] = optimal-cost
c[i,j].row = row
Additionally, I would very much appreciate feedback on the rest of my Clojure source, specifically with regards to how idiomatic it is. Have I managed to think sufficiently outside of the imperative paradigm for the Clojure code I've written thus far? Here it is:
(ns print-neatly)
;-----------------------------------------------------------------------------
; High-order function which returns a function that computes the cost
; for i and j where i is the starting word index and j is the ending word
; index for the word list "word-list."
;
(defn make-cost [word-list max-length]
(fn [i j]
(let [total (reduce + (map #(count %1) (subvec word-list i j)))
result (- max-length (+ (- j i) total))]
(if (< result 0)
nil
(* result result result)))))
;-----------------------------------------------------------------------------
; initialization function for nxn matrix
;
(defn matrix-construct [n cost-func]
(let [; Prepend nil to our collection.
append-empty
(fn [v]
(cons nil v))
; Like append-empty; append cost-func for first column.
append-cost
(fn [v, index]
(cons (cost-func 0 index) v))
; Define an internal helper which calls append-empty N times to create
; a new vector consisting of N nil values.
; ie., [nil[0] nil[1] nil[2] ... nil[N]]
construct-empty-vec
(fn [n]
(loop [cnt n coll ()]
(if (neg? cnt)
(vec coll)
(recur (dec cnt) (append-empty coll)))))
; Construct the base level where each entry is the basic cost function
; calculated for the base level. (ie., starting and ending at the
; same word)
construct-base
(fn [n]
(loop [cnt n coll ()]
(if (neg? cnt)
(vec coll)
(recur (dec cnt) (append-cost coll cnt)))))]
; The main matrix-construct logic, which just creates a new Nx1 vector
; via construct-empty-vec, then prepends that to coll.
; We end up with a vector of N entries where each entry is a Nx1 vector.
(loop [cnt n coll ()]
(cond
(zero? cnt) (vec coll)
(= cnt 1) (recur (dec cnt) (cons (construct-base n) coll))
:else (recur (dec cnt) (cons (construct-empty-vec n) coll))))))
;-----------------------------------------------------------------------------
; Return the value at a given index in a matrix.
;
(defn matrix-lookup [matrix row col]
(nth (nth matrix row) col))
;-----------------------------------------------------------------------------
; Return a new matrix M with M[row,col] = value
; but otherwise M[i,j] = matrix[i,j]
;
(defn matrix-set [matrix row col value]
(let [my-row (nth matrix row)
my-cel (assoc my-row col value)]
(assoc matrix row my-cel)))
;-----------------------------------------------------------------------------
; Print the matrix out in a nicely formatted fashion.
;
(defn matrix-print [matrix]
(doseq [j (range (count matrix))]
(doseq [i (range (count matrix))]
(let [el (nth (nth matrix i) j)]
(print (format "%1$8.8s" el)))) ; 1st item max 8 and min 8 chars
(println)))
;-----------------------------------------------------------------------------
; Main
;-----------------------------------------------------------------------------
;-----------------------------------------------------------------------------
; Grab all arguments from the command line.
;
(let [line-length (Integer. (first *command-line-args*))
words (vec (rest *command-line-args*))
cost (make-cost words line-length)
matrix (matrix-construct (count words) cost)]
(matrix-print matrix))
EDIT: I've updated my matrix-construct function with the feedback given, so now it's actually one line shorter than my Python implementation.
;-----------------------------------------------------------------------------
; Initialization function for nxn matrix
;
(defn matrix-construct [n cost-func]
(letfn [; Build an n-length vector of nil
(construct-empty-vec [n]
(vec (repeat n nil)))
; Short-cut so we can use 'map' to apply the cost-func to each
; element in a range.
(my-cost [j]
(cost-func 0 j))
; Construct the base level where each entry is the basic cost function
; calculated for the base level. (ie., starting and ending at the
; same word)
(construct-base-vec [n]
(vec (map my-cost (range n))))]
; The main matrix-construct logic, which just creates a new Nx1 vector
; via construct-empty-vec, then prepends that to coll.
; We end up with a vector of N entries where each entry is a Nx1 vector.
(let [m (repeat (- n 1) (construct-empty-vec n))]
(vec (cons (construct-base-vec n) m)))))
Instead of using let with fn's in it, try letfn.
doseq doseq -> looks like it would likely be better as a for comprehension
Your cond / zero? / = 1 code would be easier to read (and faster) with case.
My spidey-sense tells me that the loop/recurs here should be some kind of map call instead
I strongly suspect that this would be far faster with primitive arrays (and possibly cleaner in some places)
You might like to use or look at the source for Incanter
I climbed the wall and was able to think in a sufficiently Clojure-like way to translate the core compute-matrix algorithm into a workable program.
It's just one line longer than my Python implementation, although it appears to be more densely written. For sure, concepts like 'map' and 'reduce' are higher-level functions that require you to put your thinking cap on.
I believe this implementation also fixes a bug in my Python one. :)
;-----------------------------------------------------------------------------
; Compute all table entries so we can compute the optimal cost path and
; reconstruct an optimal solution.
;
(defn compute-matrix [m cost]
(letfn [; Return a function that computes 'cost(k+1,j) + c[i-1,k]'
; OR just 'c[i-1,k]' if we're on the last row.
(make-min-func [matrix i j]
(if (< j (- (count matrix) 1))
(fn [k]
(+ (cost (+ k 1) j) (get-in matrix [(- i 1) k])))
(fn [k]
(get-in matrix [(- i 1) k]))))
; Find the minimum cost for the new cost: 'cost(k+1,j)'
; added to the previous entry's cost: 'c[i-1,k]'
(min-cost [matrix i j]
(let [this-cost (make-min-func matrix i j)
rang (range (- i 1) (- j 1))
cnt (if (= rang ()) (list (- i 1)) rang)]
(apply min (map this-cost cnt))))
; Takes a matrix and indices, returns an updated matrix.
(combine [matrix indices]
(let [i (first indices)
j (nth indices 1)
opt (min-cost matrix i j)]
(assoc-in matrix [i j] opt)))]
(reduce combine m
(for [i (range 1 (count m)) j (range i (count m))] [i j]))))
Thank you Alex and Jake for your comments. They were both very helpful and have helped me on my way toward idiomatic Clojure.
My general approach to dynamic programs in Clojure is not to mess with construction of the matrix of values directly, but rather to use memorization in tandem with a fixed point combinator. Here's my example for computing edit distance:
(defn edit-distance-fp
"Computes the edit distance between two collections"
[fp coll1 coll2]
(cond
(and (empty? coll1) (empty? coll2)) 0
(empty? coll2) (count coll1)
(empty? coll1) (count coll2)
:else (let [x1 (first coll1)
xs (rest coll1)
y1 (first coll2)
ys (rest coll2)]
(min
(+ (fp fp xs ys) (if (= x1 y1) 0 1))
(inc (fp fp coll1 ys))
(inc (fp fp xs coll2))))))
The only difference from the naive recursive solution here is simply to replace the recursive calls with calls to fp.
And then I create a memoized fixed point with:
(defn memoize-recursive [f] (let [g (memoize f)] (partial g g)))
(defn mk-edit-distance [] (memoize-recursive edit-distance-fp))
And then call it with:
> (time ((mk-edit-distance)
"the quick brown fox jumped over the tawdry moon"
"quickly brown foxes moonjumped the tawdriness"))
"Elapsed time: 45.758 msecs"
23
I find memoization easier to wrap my brain around than mutating tables.
Your matrix-lookup and matrix-set functions can be simplified. You can use assoc-in and get-in for manipulating nested associative structures.
(defn matrix-lookup [matrix row col]
(get-in matrix [row col]))
(defn matrix-set [matrix row col value]
(assoc-in matrix [row col] value))
Alex Miller mentioned using primitive arrays. If you end up needing to go that direction you can start by looking at int-array, aset-int, and aget. Look at the clojure.core documentation to find out more.