I am trying to solve the Maximum subarray problem on hacker rank. This is a standard DP problem and I write an O(n) solution:
(defn dp
[v]
(let [n (count v)]
(loop [i 1 f (v 0) best f]
(if (< i n)
(let [fi (max (v i) (+ f (v i)))]
(recur (inc i) fi (max fi best)))
best))))
(defn positive-only
[v]
(reduce + (filterv #(> % 0) v)))
(defn line->ints
[line]
(->>
(clojure.string/split line #" ")
(map #(Integer. %))
(into [])
))
(let [T (Integer. (read-line))]
(loop [test 0]
(when (< test T)
(let [_ (read-line)
x (read-line)
v (line->ints x)
a (dp-array v)
b (let [p (positive-only v)]
(if (= p 0) (reduce max v) p))]
(printf "%d %d\n" a b))
(recur (inc test)))))
To my surprise, I got time-limited-exceed for a large test case. I downloaded the input file, and found that the above version needs about 3 seconds to run.
I thought the bottleneck is in (v i) (getting the i-th element in vector v). So I changed the data structure from vector to an array:
(defn dp-array
[v0]
(let [v (into-array v0)
n (int (alength v))]
(loop [i 1
f (aget v 0)
best f]
(if (< i n)
(let [fi (max (aget v i) (+ f (aget v i)))]
(recur (inc i) fi (max fi best)))
best))))
This array version is even slower. On the same input, it costs 33 seconds, much slower than the vector version. I think the slowness is due to boxing and unboxing. I tried to add type hints, but encountered run-time errors. Could anyone help me improve dp-array function? Thanks!
Also, great appreciate if anyone knows how to improve the vector version.
UPDATE:
Finally I managed to get my clojure program accepted, not by optimizing over the dynamic programming function, but by changing (Integer. str) to (Integer/parseInt str). In this way, reflection is avoided in converting from string to integer.
I also replace into-array by int-array. But the speed of both versions are still on par with each other. I would expect the array version be faster than the vector version.
The Clojure compiler can't infer the type of v in the array version of the dp-array function whose argument v0 has unknown type. This causes costs to reflections when evaluating the following alength and aget. In order to avoid these unnecessary reflections, you have to replace into-array with long-array.
Related
Below, I have 2 functions computing the sum of squares of their arguments. The first one is nice and functional, but 20x slower than the second one. I presume that the r/map is not taking advantage of aget to retrieve elements from the double-array, whereas I'm explicitly doing this in function 2.
Is there any way I can further typehint or help r/map r/fold to perform faster?
(defn sum-of-squares
"Given a vector v, compute the sum of the squares of elements."
^double [^doubles v]
(r/fold + (r/map #(* % %) v)))
(defn sum-of-squares2
"This is much faster than above. Post to stack-overflow to see."
^double [^doubles v]
(loop [val 0.0
i (dec (alength v))]
(if (neg? i)
val
(let [x (aget v i)]
(recur (+ val (* x x)) (dec i))))))
(def a (double-array (range 10)))
(quick-bench (sum-of-squares a))
800 ns
(quick-bench (sum-of-squares2 a))
40 ns
Before experiments I've added next line in project.clj:
:jvm-opts ^:replace [] ; Makes measurements more accurate
Basic measurements:
(def a (double-array (range 1000000))) ; 10 is too small for performance measurements
(quick-bench (sum-of-squares a)) ; ... Execution time mean : 27.617748 ms ...
(quick-bench (sum-of-squares2 a)) ; ... Execution time mean : 1.259175 ms ...
This is more or less consistent with time difference in the question. Let's try to not use Java arrays (which are not really idiomatic for Clojure):
(def b (mapv (partial * 1.0) (range 1000000))) ; Persistent vector
(quick-bench (sum-of-squares b)) ; ... Execution time mean : 14.808644 ms ...
Almost 2 times faster. Now let's remove type hints:
(defn sum-of-squares3
"Given a vector v, compute the sum of the squares of elements."
[v]
(r/fold + (r/map #(* % %) v)))
(quick-bench (sum-of-squares3 a)) ; Execution time mean : 30.392206 ms
(quick-bench (sum-of-squares3 b)) ; Execution time mean : 15.583379 ms
Execution time increased only marginally comparing to version with type hints. By the way, version with transducers has very similar performance and is much cleaner:
(defn sum-of-squares3 [v]
(transduce (map #(* % %)) + v))
Now about additional type hinting. We can indeed optimize first sum-of-squares implementation:
(defn square ^double [^double x] (* x x))
(defn sum-of-squares4
"Given a vector v, compute the sum of the squares of elements."
[v]
(r/fold + (r/map square v)))
(quick-bench (sum-of-squares4 b)) ; ... Execution time mean : 12.891831 ms ...
(defn pl
(^double [] 0.0)
(^double [^double x] (+ x))
(^double [^double x ^double y] (+ x y)))
(defn sum-of-squares5
"Given a vector v, compute the sum of the squares of elements."
[v]
(r/fold pl (r/map square v)))
(quick-bench (sum-of-squares5 b)) ; ... Execution time mean : 9.441748 ms ...
Note #1: type hints on arguments and return value of sum-of-squares4 and sum-of-squares5 have no additional performance benefits.
Note #2: It's generally bad practice to start with optimizations. Straight-forward version (apply + (map square v)) will have good enough performance for most situations. sum-of-squares2 is very far from idiomatic and uses literally no Clojure concepts. If this is really performance critical code - better to implement it in Java and use interop. Code will be much cleaner despite of having 2 languages. Or even implement it in unmanaged code (C, C++) and use JNI (not really maintainable but if properly implemented, can give the best possible performance).
Why not use areduce:
(def sum-of-squares3 ^double [^doubles v]
(areduce v idx ret 0.0
(let [item (aget v idx)]
(+ ret (* item item)))))
On my machine running:
(criterium/bench (sum-of-squares3 (double-array (range 100000))))
Gives a mean execution time of 1.809103 ms, your sum-of-squares2 executes the same calculation in 1.455775 ms. I think this version using areduce is more idiomatic than your version.
For squeezing a little bit more performance you can try using unchecked math (add-unchecked and multiply-unchecked). But beware, you need to be sure that your calculation cannot overflow:
(defn sum-of-squares4 ^double [^doubles v]
(areduce v idx ret 0.0
(let [item (aget v idx)]
(unchecked-add ret (unchecked-multiply item item)))))
Running the same benchmark gives a mean execution time of 1.144197 ms. Your sum-of-squares2 can also benefit from unchecked math with a 1.126001 ms mean execution time.
I have translated this code, the snippet below, from Python to Clojure. I replaced Python's while construct with Clojure's loop-recur here. But this doesn't look idiomatic.
(loop [d 2 [n & more] (list 256)]
(if (> n 1)
(recur (inc d)
(loop [x n sublist more]
(if (= (rem x d) 0)
(recur (/ x d) (conj sublist d))
(conj sublist x))))
(sort more)))
This routine gives me (3 3 31), that is prime factors of 279. For 256, it gives, (2 2 2 2 2 2 2 2), that means, 2^8.
Moreover, it performs worse for large values, say 987654123987546 instead of 279; whereas Python's counterpart works like charm.
How to start composing core functions, rather then translating imperative code as is? And specifically, how to improve this bit?
Thanks.
[Edited]
Here is the python code, I referred above,
def prime_factors(n):
factors = []
d = 2
while n > 1:
while n % d == 0:
factors.append(d)
n /= d
d = d + 1
return factors
A straight translation of the Python code in Clojure would be:
(defn prime-factors [n]
(let [n (atom n) ;; The Python code makes use of mutability which
factors (atom []) ;; isn't idiomatic in Clojure, but can be emulated
d (atom 2)] ;; using atoms
(loop []
(when (< 1 #n)
(loop []
(when (== (rem #n #d) 0)
(swap! factors conj #d)
(swap! n quot #d)
(recur)))
(swap! d inc)
(recur)))
#factors))
(prime-factors 279) ;; => [3 3 31]
(prime-factors 987654123987546) ;; => [2 3 41 14389 279022459]
(time (prime-factors 987654123987546)) ;; "Elapsed time: 13993.984 msecs"
;; same performance on my machine
;; as the Rosetta Code solution
You can improve this code to make it more idiomatic:
from nested loops to a single loop:
(loop []
(cond
(<= #n 1) #factors
(not= (rem #n #d) 0) (do (swap! d inc)
(recur))
:else (do (swap! factors conj #d)
(swap! n quot #d)
(recur))))))
get rid of the atoms:
(defn prime-factors [n]
(loop [n n
factors []
d 2]
(cond
(<= n 1) factors
(not= (rem n d) 0) (recur n factors (inc d))
:else (recur (quot n d) (conj factors d) d))))
replace == 0 by zero?:
(not (zero? (rem n d))) (recur n factors (inc d))
You can also overhaul it completely to make a lazy version of it:
(defn prime-factors [n]
((fn step [n d]
(lazy-seq
(when (< 1 n)
(cond
(zero? (rem n d)) (cons d (step (quot n d) d))
:else (recur n (inc d)))))
n 2))
I planned to have a section on optimization here, but I'm no specialist. The only thing I can say is that you can trivially make this code faster by interrupting the loop when d is greater than the square root of n:
(defn prime-factors [n]
(if (< 1 n)
(loop [n n
factors []
d 2]
(let [q (quot n d)]
(cond
(< q d) (conj factors n)
(zero? (rem n d)) (recur q (conj factors d) d)
:else (recur n factors (inc d)))))
[]))
(time (prime-factors 987654123987546)) ;; "Elapsed time: 7.124 msecs"
Not every loop unrolls cleanly into an elegant "functional" decomposition.
The Rosetta Code solution suggested by #edbond is pretty simple and concise; I would say it's idiomatic since no obvious "functional" solution is apparent. That solution runs noticeably faster on my machine than your Python version for 987654123987546.
More generally, if you're looking to expand your understanding of functional idioms, Bedra and Halloway's "Programming Clojure" (pp.90-95) presents an excellent comparison of different versions of the Fibonacci sequence, using loop, lazy seqs, and an elegant "functional" version. Chouser and Fogus's "Joy of Clojure" (MEAP version) also has a nice section on function composition.
I wrote the code below for game I am working on. But it seems a little slow. If you have not checked the code yet, it's the A* search/pathfinding algorithm. It takes about 100-600 ms for a 100x100 grid, depending on the heuristic used (and consequently the number of tiles visited).
There are no reflection warnings. However, I suspect boxing might be an issue. But I don't know how to get rid of boxing in this case, because the computation is split among several functions. Also, I save tiles/coordinates as vectors of two numbers, like this: [x y]. But then the numbers will be boxed, right? A typical piece of code, if you don't want to read through it all, is: (def add-pos (partial mapv + pos)) where pos is the aforementioned kind of two-number vector. There are sereval of places where the numbers are manipulated in a way similar to add-pos above, and put back in a vector afterwards. Is there any way to optimize code like this? Any other tips is welcome too, performance-related or other.
EDIT: Thinking some more about it, I came up with a few follow-up questions: Can a Clojure function ever return primitives? Can a Clojure function ever take primitives (without any boxing)? Can I put primitives in a type/record without boxing?
(ns game.server.pathfinding
(:use game.utils)
(:require [clojure.math.numeric-tower :as math]
[game.math :as gmath]
[clojure.data.priority-map :as pm]))
(defn walkable? [x]
(and x (= 1 x)))
(defn point->tile
([p] (apply point->tile p))
([x y] [(int x) (int y)]))
(defn get-tile [m v]
"Gets the type of the tile at the point v in
the grid m. v is a point in R^2, not grid indices."
(get-in m (point->tile v)))
(defn integer-points
"Given an equation: x = start + t * step, returns a list of the
values for t that make x an integer between start and stop,
or nil if there is no such value for t."
[start stop step]
(if-not (zero? step)
(let [first-t (-> start ((if (neg? step) math/floor math/ceil))
(- start) (/ step))
t-step (/ 1 (math/abs step))]
(take-while #((if (neg? step) > <) (+ start (* step %)) stop)
(iterate (partial + t-step) first-t)))))
(defn crossed-tiles [[x y :as p] p2 m]
(let [[dx dy :as diff-vec] (map - p2 p)
ipf (fn [getter]
(integer-points (getter p) (getter p2) (getter diff-vec)))
x-int-ps (ipf first)
y-int-ps (ipf second)
get-tiles (fn [[x-indent y-indent] t]
(->> [(+ x-indent x (* t dx)) (+ y-indent y (* t dy))]
(get-tile m)))]
(concat (map (partial get-tiles [0.5 0]) x-int-ps)
(map (partial get-tiles [0 0.5]) y-int-ps))))
(defn clear-line?
"Returns true if the line between p and p2 passes over only
walkable? tiles in m, otherwise false."
[p p2 m]
(every? walkable? (crossed-tiles p p2 m)))
(defn clear-path?
"Returns true if a circular object with radius r can move
between p and p2, passing over only walkable? tiles in m,
otherwise false.
Note: Does not currently work for objects with a radius >= 0.5."
[p p2 r m]
(let [diff-vec (map (partial * r) (gmath/normalize (map - p2 p)))
ortho1 ((fn [[x y]] (list (- y) x)) diff-vec)
ortho2 ((fn [[x y]] (list y (- x))) diff-vec)]
(and (clear-line? (map + ortho1 p) (map + ortho1 p2) m)
(clear-line? (map + ortho2 p) (map + ortho2 p2) m))))
(defn straighten-path
"Given a path in the map m, remove unnecessary nodes of
the path. A node is removed if one can pass freely
between the previous and the next node."
([m path]
(if (> (count path) 2) (straighten-path m path nil) path))
([m [from mid to & tail] acc]
(if to
(if (clear-path? from to 0.49 m)
(recur m (list* from to tail) acc)
(recur m (list* mid to tail) (conj acc from)))
(reverse (conj acc from mid)))))
(defn to-mid-points [path]
(map (partial map (partial + 0.5)) path))
(defn to-tiles [path]
(map (partial map int) path))
(defn a*
"A* search for a grid of squares, mat. Tries to find a
path from start to goal using only walkable? tiles.
start and goal are vectors of indices into the grid,
not points in R^2."
[mat start goal factor]
(let [width (count mat)
height (count (first mat))]
(letfn [(h [{pos :pos}] (* factor (gmath/distance pos goal)))
(g [{:keys [pos parent]}]
(if parent
(+ (:g parent) (gmath/distance pos (parent :pos)))
0))
(make-node [parent pos]
(let [node {:pos pos :parent parent}
g (g node) h (h node)
f (+ g h)]
(assoc node :f f :g g :h h)))
(get-path
([node] (get-path node ()))
([{:keys [pos parent]} path]
(if parent
(recur parent (conj path pos))
(conj path pos))))
(free-tile? [tile]
(let [type (get-in mat (vec tile))]
(and type (walkable? type))))
(expand [closed pos]
(let [adj [[1 0] [0 1] [-1 0] [0 -1]]
add-pos (partial mapv + pos)]
(->> (take 4 (partition 2 1 (cycle adj)))
(map (fn [[t t2]]
(list* (map + t t2) (map add-pos [t t2]))))
(map (fn [[d t t2]]
(if (every? free-tile? [t t2]) d nil)))
(remove nil?)
(concat adj)
(map add-pos)
(remove (fn [[x y :as tile]]
(or (closed tile) (neg? x) (neg? y)
(>= x width) (>= y height)
(not (walkable? (get-in mat tile)))))))))
(add-to-open [open tile->node [{:keys [pos f] :as node} & more]]
(if node
(if (or (not (contains? open pos))
(< f (open pos)))
(recur (assoc open pos f)
(assoc tile->node pos node)
more)
(recur open tile->node more))
{:open open :tile->node tile->node}))]
(let [start-node (make-node nil start)]
(loop [closed #{}
open (pm/priority-map start (:f start-node))
tile->node {start start-node}]
(let [[curr _] (peek open) curr-node (tile->node curr)]
(when curr
(if (= curr goal)
(get-path curr-node)
(let [exp-tiles (expand closed curr)
exp-nodes (map (partial make-node curr-node) exp-tiles)
{:keys [open tile->node]}
(add-to-open (pop open) tile->node exp-nodes)]
(recur (conj closed curr) open tile->node))))))))))
(defn find-path [mat start goal]
(let [start-tile (point->tile start)
goal-tile (point->tile goal)
path (a* mat start-tile goal-tile)
point-path (to-mid-points path)
full-path (concat [start] point-path [goal])
final-path (rest (straighten-path mat full-path))]
final-path))
I recommend the Clojure High Performance Programming book for addressing questions like yours.
There are functions to unbox primitives (byte, short, int, long, float, double).
Warn-on-reflection does not apply to numeric type reflection / failure to optimize numeric code. There is a lib to force warnings for numeric reflection - primitive-math.
You can declare the types of function arguments and function return values (defn ^Integer foo [^Integer x ^Integer y] (+ x y)).
Avoid apply if you want performance.
Avoid varargs (a common reason to need apply) if you want performance. Varargs functions create garbage on every invocation (in order to construct the args map, which usually is not used outside the function body). partial always constructs a varargs function. Consider replacing the varargs (partial * x) with #(* x %), the latter can be optimized much more aggressively.
There is a tradeoff with using primitive jvm single-type arrays (they are mutible and fixed in length, which can lead to more complex and brittle code), but they will perform better than the standard clojure sequential types, and are available if all else fails to get the performance you need.
Also, use criterium to compare various implementations of your code, it has a bunch of tricks to help rule out the random things that affect execution time so you can see what really performs best in a tight loop.
Also, regarding your representation of a point as [x y] - you can reduce the space and lookup overhead of the collection holding them with (defrecord point [x y]) (as long as you know they will remain two elements only, and you don't mind changing your code to ask for (:x point) or (:y point)). You could further optimize by making or using a simple two-number java class (with the tradeoff of losing immutibility).
I was wondering if someone could help me with the performance of this code snippet in Clojure 1.3. I am trying to implement a simple function that takes two vectors and does a sum of products.
So let's say the vectors are X (size 10,000 elements) and B (size 3 elements), and the sum of products are stored in a vector Y, mathematically it looks like this:
Y0 = B0*X2 + B1*X1 + B2*X0
Y1 = B0*X3 + B1*X2 + B2*X1
Y2 = B0*X4 + B1*X3 + B2*X2
and so on ...
For this example, the size of Y will end up being 9997, which corresponds to (10,000 - 3). I've set up the function to accept any size of X and B.
Here's the code: It basically takes (count b) elements at a time from X, reverses it, maps * onto B and sums the contents of the resulting sequence to produce an element of Y.
(defn filt [b-vec x-vec]
(loop [n 0 sig x-vec result []]
(if (= n (- (count x-vec) (count b-vec)))
result
(recur (inc n) (rest sig) (conj result (->> sig
(take (count b-vec))
(reverse)
(map * b-vec)
(apply +)))))))
Upon letting X be (vec (range 1 10001)) and B being [1 2 3], this function takes approximately 6 seconds to run. I was hoping someone could suggest improvements to the run time, whether it be algorithmic, or perhaps a language detail I might be abusing.
Thanks!
P.S. I have done (set! *warn-on-reflection* true) but don't get any reflection warning messages.
You are using count many times unnecessary. Below code calculate count one time only
(defn filt [b-vec x-vec]
(let [bc (count b-vec) xc (count x-vec)]
(loop [n 0 sig x-vec result []]
(if (= n (- xc bc))
result
(recur (inc n) (rest sig) (conj result (->> sig
(take bc)
(reverse)
(map * b-vec)
(apply +))))))))
(time (def b (filt [1 2 3] (range 10000))))
=> "Elapsed time: 50.892536 msecs"
If you really want top performance for this kind of calculation, you should use arrays rather than vectors. Arrays have a number of performance advantages:
They support O(1) indexed lookup and writes - marginally better than vectors which are O(log32 n)
They are mutable, so you don't need to construct new arrays all the time - you can just create a single array to serve as the output buffer
They are represented as Java arrays under the hood, so benefit from the various array optimisations built into the JVM
You can use primitive arrays (e.g. of Java doubles) which are much faster than if you use boxed number objects
Code would be something like:
(defn filt [^doubles b-arr
^doubles x-arr]
(let [bc (count b-arr)
xc (count x-arr)
rc (inc (- xc bc))
result ^doubles (double-array rc)]
(dotimes [i rc]
(dotimes [j bc]
(aset result i (+ (aget result i) (* (aget x-arr (+ i j)) (aget b-arr j))))))
result))
To follow on to Ankur's excellent answer, you can also avoid repeated calls to the reverse function, which gets us even a little more performance.
(defn filt [b-vec x-vec]
(let [bc (count b-vec) xc (count x-vec) bb-vec (reverse b-vec)]
(loop [n 0 sig x-vec result []]
(if (= n (- xc bc))
result
(recur (inc n) (rest sig) (conj result (->> sig
(take bc)
(map * bb-vec)
(apply +))))))))
I decided to work through the CLRS Introduction to Algorithms text, and picked the printing neatly problem here.
I worked through the problem and came up with an imperative solution which was straightforward to implement in Python, but somewhat less so in Clojure.
I'm completely stumped on translating the compute-matrix function from my solution into idiomatic Clojure. Any suggestions? Here is the pseudocode for the compute-matrix function:
// n is the dimension of the square matrix.
// c is the matrix.
function compute-matrix(c, n):
// Traverse through the left-lower triangular matrix and calculate values.
for i=2 to n:
for j=i to n:
// This is our minimum value sentinal.
// If we encounter a value lower than this, then we store the new
// lowest value.
optimal-cost = INF
// Index in previous column representing the row we want to point to.
// Whenever we update 't' with a new lowest value, we need to change
// 'row' to point to the row we're getting that value from.
row = 0
// This iterates through each entry in the previous column.
// Note: we have a lower triangular matrix, meaning data only
// exists in the left-lower half.
// We are on column 'i', but because we're in a left-lower triangular
// matrix, data doesn't start until row (i-1).
//
// Similarly, we go to (j-1) because we can't choose a configuration
// where the previous column ended on a word who's index is larger
// than the word index this column starts on - the case which occurs
// when we go for k=(i-1) to greater than (j-1)
for k=(i-1) to (j-1):
// When 'j' is equal to 'n', we are at the last cell and we
// don't care how much whitespace we have. Just take the total
// from the previous cell.
// Note: if 'j' < 'n', then compute normally.
if (j < n):
z = cost(k + 1, j) + c[i-1, k]
else:
z = c[i-1, k]
if z < optimal-cost:
row = k
optimal-cost = z
c[i,j] = optimal-cost
c[i,j].row = row
Additionally, I would very much appreciate feedback on the rest of my Clojure source, specifically with regards to how idiomatic it is. Have I managed to think sufficiently outside of the imperative paradigm for the Clojure code I've written thus far? Here it is:
(ns print-neatly)
;-----------------------------------------------------------------------------
; High-order function which returns a function that computes the cost
; for i and j where i is the starting word index and j is the ending word
; index for the word list "word-list."
;
(defn make-cost [word-list max-length]
(fn [i j]
(let [total (reduce + (map #(count %1) (subvec word-list i j)))
result (- max-length (+ (- j i) total))]
(if (< result 0)
nil
(* result result result)))))
;-----------------------------------------------------------------------------
; initialization function for nxn matrix
;
(defn matrix-construct [n cost-func]
(let [; Prepend nil to our collection.
append-empty
(fn [v]
(cons nil v))
; Like append-empty; append cost-func for first column.
append-cost
(fn [v, index]
(cons (cost-func 0 index) v))
; Define an internal helper which calls append-empty N times to create
; a new vector consisting of N nil values.
; ie., [nil[0] nil[1] nil[2] ... nil[N]]
construct-empty-vec
(fn [n]
(loop [cnt n coll ()]
(if (neg? cnt)
(vec coll)
(recur (dec cnt) (append-empty coll)))))
; Construct the base level where each entry is the basic cost function
; calculated for the base level. (ie., starting and ending at the
; same word)
construct-base
(fn [n]
(loop [cnt n coll ()]
(if (neg? cnt)
(vec coll)
(recur (dec cnt) (append-cost coll cnt)))))]
; The main matrix-construct logic, which just creates a new Nx1 vector
; via construct-empty-vec, then prepends that to coll.
; We end up with a vector of N entries where each entry is a Nx1 vector.
(loop [cnt n coll ()]
(cond
(zero? cnt) (vec coll)
(= cnt 1) (recur (dec cnt) (cons (construct-base n) coll))
:else (recur (dec cnt) (cons (construct-empty-vec n) coll))))))
;-----------------------------------------------------------------------------
; Return the value at a given index in a matrix.
;
(defn matrix-lookup [matrix row col]
(nth (nth matrix row) col))
;-----------------------------------------------------------------------------
; Return a new matrix M with M[row,col] = value
; but otherwise M[i,j] = matrix[i,j]
;
(defn matrix-set [matrix row col value]
(let [my-row (nth matrix row)
my-cel (assoc my-row col value)]
(assoc matrix row my-cel)))
;-----------------------------------------------------------------------------
; Print the matrix out in a nicely formatted fashion.
;
(defn matrix-print [matrix]
(doseq [j (range (count matrix))]
(doseq [i (range (count matrix))]
(let [el (nth (nth matrix i) j)]
(print (format "%1$8.8s" el)))) ; 1st item max 8 and min 8 chars
(println)))
;-----------------------------------------------------------------------------
; Main
;-----------------------------------------------------------------------------
;-----------------------------------------------------------------------------
; Grab all arguments from the command line.
;
(let [line-length (Integer. (first *command-line-args*))
words (vec (rest *command-line-args*))
cost (make-cost words line-length)
matrix (matrix-construct (count words) cost)]
(matrix-print matrix))
EDIT: I've updated my matrix-construct function with the feedback given, so now it's actually one line shorter than my Python implementation.
;-----------------------------------------------------------------------------
; Initialization function for nxn matrix
;
(defn matrix-construct [n cost-func]
(letfn [; Build an n-length vector of nil
(construct-empty-vec [n]
(vec (repeat n nil)))
; Short-cut so we can use 'map' to apply the cost-func to each
; element in a range.
(my-cost [j]
(cost-func 0 j))
; Construct the base level where each entry is the basic cost function
; calculated for the base level. (ie., starting and ending at the
; same word)
(construct-base-vec [n]
(vec (map my-cost (range n))))]
; The main matrix-construct logic, which just creates a new Nx1 vector
; via construct-empty-vec, then prepends that to coll.
; We end up with a vector of N entries where each entry is a Nx1 vector.
(let [m (repeat (- n 1) (construct-empty-vec n))]
(vec (cons (construct-base-vec n) m)))))
Instead of using let with fn's in it, try letfn.
doseq doseq -> looks like it would likely be better as a for comprehension
Your cond / zero? / = 1 code would be easier to read (and faster) with case.
My spidey-sense tells me that the loop/recurs here should be some kind of map call instead
I strongly suspect that this would be far faster with primitive arrays (and possibly cleaner in some places)
You might like to use or look at the source for Incanter
I climbed the wall and was able to think in a sufficiently Clojure-like way to translate the core compute-matrix algorithm into a workable program.
It's just one line longer than my Python implementation, although it appears to be more densely written. For sure, concepts like 'map' and 'reduce' are higher-level functions that require you to put your thinking cap on.
I believe this implementation also fixes a bug in my Python one. :)
;-----------------------------------------------------------------------------
; Compute all table entries so we can compute the optimal cost path and
; reconstruct an optimal solution.
;
(defn compute-matrix [m cost]
(letfn [; Return a function that computes 'cost(k+1,j) + c[i-1,k]'
; OR just 'c[i-1,k]' if we're on the last row.
(make-min-func [matrix i j]
(if (< j (- (count matrix) 1))
(fn [k]
(+ (cost (+ k 1) j) (get-in matrix [(- i 1) k])))
(fn [k]
(get-in matrix [(- i 1) k]))))
; Find the minimum cost for the new cost: 'cost(k+1,j)'
; added to the previous entry's cost: 'c[i-1,k]'
(min-cost [matrix i j]
(let [this-cost (make-min-func matrix i j)
rang (range (- i 1) (- j 1))
cnt (if (= rang ()) (list (- i 1)) rang)]
(apply min (map this-cost cnt))))
; Takes a matrix and indices, returns an updated matrix.
(combine [matrix indices]
(let [i (first indices)
j (nth indices 1)
opt (min-cost matrix i j)]
(assoc-in matrix [i j] opt)))]
(reduce combine m
(for [i (range 1 (count m)) j (range i (count m))] [i j]))))
Thank you Alex and Jake for your comments. They were both very helpful and have helped me on my way toward idiomatic Clojure.
My general approach to dynamic programs in Clojure is not to mess with construction of the matrix of values directly, but rather to use memorization in tandem with a fixed point combinator. Here's my example for computing edit distance:
(defn edit-distance-fp
"Computes the edit distance between two collections"
[fp coll1 coll2]
(cond
(and (empty? coll1) (empty? coll2)) 0
(empty? coll2) (count coll1)
(empty? coll1) (count coll2)
:else (let [x1 (first coll1)
xs (rest coll1)
y1 (first coll2)
ys (rest coll2)]
(min
(+ (fp fp xs ys) (if (= x1 y1) 0 1))
(inc (fp fp coll1 ys))
(inc (fp fp xs coll2))))))
The only difference from the naive recursive solution here is simply to replace the recursive calls with calls to fp.
And then I create a memoized fixed point with:
(defn memoize-recursive [f] (let [g (memoize f)] (partial g g)))
(defn mk-edit-distance [] (memoize-recursive edit-distance-fp))
And then call it with:
> (time ((mk-edit-distance)
"the quick brown fox jumped over the tawdry moon"
"quickly brown foxes moonjumped the tawdriness"))
"Elapsed time: 45.758 msecs"
23
I find memoization easier to wrap my brain around than mutating tables.
Your matrix-lookup and matrix-set functions can be simplified. You can use assoc-in and get-in for manipulating nested associative structures.
(defn matrix-lookup [matrix row col]
(get-in matrix [row col]))
(defn matrix-set [matrix row col value]
(assoc-in matrix [row col] value))
Alex Miller mentioned using primitive arrays. If you end up needing to go that direction you can start by looking at int-array, aset-int, and aget. Look at the clojure.core documentation to find out more.