I am looking for an example of using python multiprocessing (i.e. a process-pool/threadpool, job queue etc.) with hylang.
The first example from the multiprocessing documentation can be literally translated to Hy like so:
(import multiprocessing [Pool])
(defn f [x]
(* x x))
(when (= __name__ "__main__")
(with [p (Pool 5)]
(print (.map p f [1 2 3]))))
Note that a straightforward translation runs into a problem on macOS (which is not officially supported, but mostly works anyway): Hy sets sys.executable to the Hy interpreter, and multiprocessing relies on that value to start up new processes. You can work around that particular problem by calling (multiprocessing.set_executable hy.sys_executable), but then it will fail to parse the file containing the Hy code itself, which it does again for some reason in the child process. So there doesn't seem to be a good solution for using multiprocessing with Hy running natively on a Mac.
Which is why we have Docker, I suppose.
Related
I'm writing a loop that only works on single-character inputs. Depending on what the user pressed (without pressing ENTER), I want to display what key the user typed in, and then repeat. If the user presses "q", then the loop must exit.
Constraints:
I don't care about Unicode (only supporting US ASCII character set is desirable).
I only care about Unixy systems (only Linux is fine).
I am using Leiningen.
Can this be done? Some searching led me to jline2 which had ConsoleReader class but it seems to have disappeared in jline3.
I saw https://gist.github.com/mikeananev/f5138eeee12144a3ca82136184e7a742 and using the linked duplicate answer, came up with this:
; Need `(:import [org.jline.terminal TerminalBuilder Terminal])`
(defn new-terminal
"creates new JLine3 Terminal.
returns terminal object"
^Terminal [term-name]
(let [terminal (-> (TerminalBuilder/builder)
(.jna true)
(.system true)
(.name term-name)
(.build))]
terminal))
(defn interactive-loop []
(let [t (new-terminal "xxx")]
(.enterRawMode t)
(let [reader (.reader t)]
(loop [char-int (.read reader)]
(case (char char-int)
\q (do
(println "Bye!")
(.close reader)
(.close t))
(do (println (char char-int))
(recur (.read reader))))))))
I'm on NixOS and apparently the jline3 library depends on the infocmp binary being installed, which is part of the popular ncurses package. Unfortunately the Nixpkgs version I use currently does not package this binary so I put in a PR here: https://github.com/NixOS/nixpkgs/pull/72135
Looking on simple example of cython code I wonder if CPython interpreter needed some hard-coded hacks to understand cython syntax (e.g. cdef int N) or if it is implemented using some concepts of standard python syntax (e.g. functions, classes, operators etc.)
I mean, I can roughly imagine how the cython backend can be implemented (c-code generation, compilation etc.), but I don't understand how the frontend syntax can be integrated within standard python interpreter without touching the python interpteter itself (i.e. without extending python language beyond standard).
What is cdef ?
In other words, what cdef actually is? Is it a function, operator, or some new keyword? I would understand N = cedef(int) - that would create instance of some class derived from int. But written like that, I don't see how these 3 tokens even interact (1 cdef, 2 int, 3 N) ?
Does the for-loop actually iterate?
if you write code like this:
cdef int i,N = 1000000
cdef double f = 0
for i in xrange(N):
f += (i*i)
print f
The loop for i in xrange(N): is normal python loop. What prevents python interpreter to uselessly iterate 1000000 iterations before cython compile it into to C-code?
Does it work something like this:
N is an instance of some class cdef. xrange call N.__int__() which returns 1, passing the loop only once. The expression f += (i*i) contains only cdef objects, so cython can redefine __add_(), __set__(), __get__() functions in a way that generate C-code for f+=(i*i)
But then I still don't see how the construct for i in xrange() is send to cython, to generate C code from it.
Anyway, it seems quite complicated and fragiel, so perhaps it must be otherwise
I'm learning the Julia language and followed some tutorials to test OLS (ordinary least squares) estimation in Julia. First, I need to simulate a dataset of dependent variable ("Y"), independent variables ("X") ,error terms (epsilon) and parameters. The script is like:
# ols_simulate ：generate necessary data
using Distributions
N=100000
K=3
genX = MvNormal(eye(K))
X = rand(genX,N)
X = X'
X_noconstant = X
constant = ones(N)
X = [constant X]
genEpsilon = Normal(0, 1)
epsilon = rand(genEpsilon,N)
trueParams = [0.1,0.5,-0.3,0.]
Y = X*trueParams + epsilon
and then I defined an OLS function
function OLSestimator(y,x)
estimate = inv(x'*x)*(x'*y)
return estimate
end
What I planed to do is first to simulate data from terminal with command:
ols_simulate
and hope this step generates and stores data properly, and then I could call olsestimator . But after trying this, when I typed mean(Y) in Julia REPL, it gives me an error message like
Error: UnderdefvarError: Y not defined
it seems the data are not stored properly. More generally, if I have multiple scripts (scripts and function), how can I use the data generated by one from others in the terminal?
Thank you.
Each time you run the Julia REPL (the Julia "command-line"), it begins with a fresh memory workspace. Thus, to define variables and then use them, you should run the interpreter once.
If I understand correctly, you have multiple scripts which do parts of the calculations. To run a script in the REPL and stay in it with all the global variables still defined, you can use
include("scriptname.jl")
(with scriptname changed to appropriate .jl filename).
In this case, the workflow could look like:
include("ols_simulate.jl")
estimate = OLSestimator(Y,X)
mean(Y)
In general, it's best to stay in the REPL, unless you want to clear everything and start fresh and then quitting and restarting is the way to go.
You need to save the script in a separate file and then load it into Julia. Say you already saved it with name "ols_simulate.jl" in directory "dir1", then navigate to that directory in the Terminal, startup Julia (you might want to see this). Once in Julia, you have to load "ols_simulate.jl", after which you can calculate the mean of Y and do whatever you want:
include("ols_simulate.jl")
mean(Y)
OLSestimator(Y, X)
For the kind of stuff that I think you are doing, I think you could find useful using a notebook interface like Jupyter.
Is it possible to update the program code while it is being interpreted by csi, the Chicken Scheme Interpreter? If so, how?
So that I can interactively change part of the code and immediately see the effects of that changes.
For example, suppose I have written the following program:
(define (loop)
(print "Ciao")
(rest 1)
(loop))
(loop)
(assume (rest 1) has the effect of pausing the program for a second).
If I run this program, trough csi, it prints the string "Ciao" every second. If I change the string "Ciao" into something else, for example into "else", and I save the program code file, then csi continue interpreting the old program code, so I continuously see the string "Ciao". I'd like, in this case, that when I save the modified code with the string "Ciao" replaced by "else", csi continue its interpretation job by looking into the modified file, instead of the old file.
So that I obtain as output some "Ciao" followed by some "else": the "else" start to appear when I replace "Ciao" by "else" in the source code.
In general, the answer is "don't". The way you're supposed to use the REPL is by evaluating piecemeal changes against it, then evaluating a function or two to make sure everything went as expected. Part of this approach is structuring your program so that it can be easily tested in pieces, which implies not automatically starting any infinite loops. In the specific case you're asking about, you could instead write
(define (do-stuff)
(print "Ciao"))
(define (main-loop)
(do-stuff)
(rest 1)
(main-loop))
(define (start) (main-loop))
You can now incrementally develop do-stuff, periodically evaluating new versions to your interpreter and calling them to make sure they work, then eventually calling start once you're confident that it's doing the right thing.
You may get mileage out of this similar question asked about Common Lisp.
As an aside, if you were using Common Lisp and SLIME, you could now do more or less what you proposed:
(defun do-stuff ()
(format t "Ciao~%"))
(defun main-loop ()
(loop (progn (do-stuff)
(sleep 1))))
(main-loop)
Starting that up would start printing Ciao on separate lines in your SLIME REPL. If you changed do-stuff to
(defun do-stuff ()
(format t "else~%"))
then hit it with C-c C-c (the default binding for slime-compile-defun), you'd see your SLIME REPL start printing else lines in-flight.
CL-USER> (main-loop)
Ciao
Ciao
Ciao
; compiling (DEFUN DO-STUFF ...)else
else
else
else
else
else
; Evaluation aborted on NIL. User break.
CL-USER>
I'm not sure how to accomplish the same thing in Scheme, but I'm reasonably sure it's possible.
All that being said, you sometimes want to run a program part of which is an infinite loop. A real world example would be while testing a TCP server of some sort. If you're in such a situation, and your desired workflow is
Write file
Run csi my-file.scm
Edit file
Kill csi
Run csi my-file.scm
goto 3
and you basically just want to automate steps 4 through 6, you'll need an external tool to do it for you. Take a look at entr or hsandbox (the latter doesn't have Scheme support out of the box, but it doesn't look like it would be too hard to add).
There is no common way to make a running program check its source for changes but you there seems to be enough feratures available in Chicken to roll your own:
(use posix)
(use srfi-18)
(define (watch-reload! file)
(define (tsleep n)
(thread-sleep! (seconds->time (+ n (time->seconds (current-time))))))
(define (get-time)
(file-modification-time file))
(thread-start!
(lambda ()
(let loop ((filetime '()))
(let ((newtime (get-time)))
(when (not (equal? filetime newtime))
(load file))
(tsleep 10)
(loop newtime))))))
Now all you have to do is to use watch-reload! instead of load and it will check and reload every 10 seconds if the file has been modified.
If you save when the file is not valid scheme it stops working until you call watch-reload! on it again.
It may be that chicken programmers might have a better solution.
I would like to create a command line application that generates jpg images using Quil. I plan to write a couple of generic image processing functions to mix and match (some fn drawing shapes and some manipulating the pixel array).
A simple example of what I would like to do is create one function that draws a circle then a second function that applies a dither algorithm to the pixel array.
(defn draw-circle [x y] ...) ;; e.g. internally uses Quil draw functions.
(defn apply-dither [pixels] ...) ;; e.g. internally uses Quil color functions on the pixels array.
(defn draw []
(draw-circle 100 100)
(apply-dither (pixels))
...)
(defsketch sketch
:draw draw)
What is causing me a bit of grief is all Quil functions seems to only run within a sketch macro. This means that my own functions using the Quil functions internally in turn cannot be directly called (must be called from a draw function triggered by a sketch macro), making debugging and running them individually from the repl harder/impossible.
How do I go about creating and debugging such generic functions? Am I stuck having sketch call functions on my behalf or is there another way?
There is also the possibility that Quil isn't the right tool for my project. I'm considering directly using java / processing classes.
My development environment is Emacs + Cider.
Thanks
I've created a series of quil sketches that run from the command line at https://github.com/rogerallen/qeom
I developed & iterated within emacs+cider and just recompiled the draw function to see the results update live in the sketch window.
I used an atom (defonce dump-image-count (atom 1)) to control when the draw function would save an image:
(defn draw []
...
(when (> #dump-image-count 0)
(save-frame "q007-dump-####.png")
(swap! dump-image-count dec)))
Hope this helps.