I/O performance in mzscheme - performance

Being a Linux administrator, I used to write my scripts in Bash, TCL and, less often, in Perl. Just out of curiosity, I tried to write something in mzscheme, but what I found out was that the performance was so much worse. I cut the script to simply reading a 500MB log file:
#lang scheme
(require rnrs/programs-6)
(call-with-input-file (vector-ref (current-command-line-arguments) 0)
(lambda (in)
(let loop ((line (read-line in)))
(unless (eof-object? line)
(loop (read-line in))))))
This simple process takes about 40 seconds. The same script, adapted for Guile, executes in 10 seconds. TCL version runs for 5 seconds. Chicken Scheme takes only 3.8 seconds, ten time less than MZScheme:
#!/usr/bin/csi -script
(call-with-input-file (list-ref (command-line-arguments) 0)
(lambda (in)
(let loop ((line (read-line in)))
(if (not (eof-object? line))
(loop (read-line in))))))
What am I doing wrong? Are there any recommendations on writing faster MZScheme programs?
Some more tests:
minaev#minaev:~/1$ time ./t.tcl blog.log
real 0m8.907s
user 0m8.417s
sys 0m0.468s
Mon Oct 31 13:15:19 MSK 2011
minaev#minaev:~/1$ time ./t.scm blog.log # Chicken 4.2.0
real 0m7.678s
user 0m6.896s
sys 0m0.580s
Mon Oct 31 13:15:29 MSK 2011
minaev#minaev:~/1$ time /usr/bin/mzscheme t.ss blog.log # mzscheme 4.2.1
real 0m44.047s
user 0m41.803s
sys 0m0.948s
Mon Oct 31 13:17:03 MSK 2011
minaev#minaev:~/1$ time racket t.ss blog.log # racket 5.1.3
real 0m25.287s
user 0m23.189s
sys 0m0.828s
Mon Oct 31 13:17:39 MSK 2011
minaev#minaev:~/1$ raco make t.ss
Mon Oct 31 13:17:47 MSK 2011
minaev#minaev:~/1$ time racket t.ss blog.log # racket 5.1.3 byte-compiled
real 0m23.237s
user 0m22.469s
sys 0m0.688s

This has now been substantially improved in the development version of Racket. See the commit here and the message from Racket maintainer Matthew Flatt here. It should now be only about 50% slower than Chicken. The remaining slowdown is primarily due to the additional features Racket provides, such as events, line-counting, unicode encoding, and others.

I tried repeating your results, and I got about 3.4s for Chicken 4.5.0 and about 10s for various versions of mzscheme/racket. (I tried mzscheme from PLT Scheme 4.2 and racket from both Racket 5.1.1 and the development tree, all compiled in 64-bit mode.)
What version of mzscheme are you using? What platform?
Are your timings repeatable? (I wonder about block cache effects.)

Related

Why is package user not an environment in MIT Scheme 10.1?

Here is my MIT Scheme code:
(display (name->package '(user)))
(newline)
(->environment (name->package '(user)))
(%exit)
This runs fine on MIT/GNU Scheme Release 9.1.1. Here is the output:
$ scheme --quiet --load foo.scm
#[package 13 (user)]
But when I run this with MIT/GNU Scheme Release 10.1.5, I get this error:
$ mit-scheme --quiet --load foo.scm
#[package 12 (user)]
;The object #[package 12 (user)], passed as an argument to ->environment, is not an environment.
;To continue, call RESTART with an option number:
; (RESTART 1) => Return to read-eval-print level 1.
2 error>
On debugging I get this,
2 error> (debug)
There are 7 subproblems on the stack.
Subproblem level: 0 (this is the lowest subproblem level)
Compiled code expression unknown
#[compiled-return-address 13 ("rep" #x2f) #xd8 #x95ed10]
There is no current environment.
There is no execution history for this subproblem.
You are now in the debugger. Type q to quit, ? for commands.
3 debug>
Why does this error occur in 10.1.5 but not in 9.1.1?

Is there a way to turn down the verbosity of MIT Scheme?

I recently decided to start playing with MIT Scheme by following along with the examples in SICP. I installed scheme from the Ubuntu repository.
sudo apt-get install mit-scheme
Given an input file that looks like this:
486
(+ 137 349)
(- 1000 334)
(* 5 99)
(/ 10 5)
(* 25 4 12)
I run scheme as follows.
scheme < Numbers.scm
It produces the following output.
MIT/GNU Scheme running under GNU/Linux
Type `^C' (control-C) followed by `H' to obtain information about interrupts.
Copyright (C) 2011 Massachusetts Institute of Technology
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Image saved on Sunday February 7, 2016 at 10:35:34 AM
Release 9.1.1 || Microcode 15.3 || Runtime 15.7 || SF 4.41 || LIAR/x86-64 4.118 || Edwin 3.116
1 ]=> 486
;Value: 486
1 ]=> (+ 137 349)
;Value: 486
1 ]=> (- 1000 334)
;Value: 666
1 ]=> (* 5 99)
;Value: 495
1 ]=> (/ 10 5)
;Value: 2
1 ]=> (* 25 4 12)
;Value: 1200
1 ]=>
End of input stream reached.
Moriturus te saluto.
This output feels excessive, so I'm currently paring it down like so.
scheme < Numbers.scm | awk '/Value/ {print $2}
486
486
666
495
2
1200
Is there a native way to reduce the verbosity of scheme, so I can get something resembling the above output without resorting to an external process?
I have examined the output of scheme --help but did not find any obvious options.
Note that passing the filename as an argument does not appear to work in MIT-Scheme.
scheme Numbers.scm
MIT/GNU Scheme running under GNU/Linux
Type `^C' (control-C) followed by `H' to obtain information about interrupts.
Copyright (C) 2011 Massachusetts Institute of Technology
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Image saved on Sunday February 7, 2016 at 10:35:34 AM
Release 9.1.1 || Microcode 15.3 || Runtime 15.7 || SF 4.41 || LIAR/x86-64 4.118 || Edwin 3.116
;Warning: Invalid keyword: "Numbers.scm"
;Warning: Unhandled command line options: ("Numbers.scm")
1 ]=>
Here you go:
scheme --quiet < Numbers.scm
Now this will suppress the REPL entirely except when errors occur so that what is not explicitly displayed will not be displayed. eg. evaluatiing (+ 2 3) returns 5, but does not print since you have not told it to print. You need to use procedures like display to get the information printed or go back to using a REPL which sole purpose is to display your results.
I was originally hoping you could do:
scheme --quiet --load Numbers.scm
But it does not exit after the file and adding --eval (exit) has the REPL ask if you want to quit.
EDIT
(define (displayln v)
(display v)
(newline)
v)
(displayln (+ 4 5))
; ==> 9, in addition you get the side effect that "9\n" is written to current output port
You could also perhaps make a macro to do this:
(define-syntax begin-display
(syntax-rules ()
((_ form ...) (begin (displayln form) ...))))
(begin-display
486
(+ 137 349)
(- 1000 334)
(* 5 99)
(/ 10 5)
(* 25 4 12))
; ==> 1200. In addition you get the side effect that "486\n486\n666\n49\n2\n1200\n" is written to current output port
As a workaround,
scheme < Numbers.scm | gawk '/^;Value: / { sub(/^;Value: /, ""); print }'
But maybe you'd run it as a script file rather than an stdin stream? Not sure about MIT Scheme invocation, something like
scheme Numbers.scm
Though this way you'll have to print out the results explicitly, with (display) or something, otherwise they'll go unnoticed.

How to use sbcl to use shell command

I want to programing a function about open the file directly. Like python code:
os.system("ls")
For example, when I use this function (fun_open "/path/to/file"), the system will open the file use the default app. If file is a .txt, open it with textedit.
How to make it?
----UPDATE 9/24/2015-----
My code is:
(defun open_by_system (dir)
(sb-ext:run-program "/usr/bin/open" (list "-a" "Preview" dir)))
and I use it:
CL-USER> (open_by_system "~/Desktop/ML.pdf")
#<SB-IMPL::PROCESS :EXITED 1>
Nothing else happen
I'd recommend using UIOP, which provides portable interface to the OS and is universally available as a part of ASDF3:
(uiop:run-program "ls")
See the docstrings in run-program.lisp for details.
If you need more convenience functions, you could take a look at inferior-shell.
I recommend you to take a look at the available libraries on quickdocs:
link
I recommend you to use inferior-shell available on quicklisp
link
loading:
CL-USER> (ql:quickload 'inferior-shell)
To load "inferior-shell":
Load 5 ASDF systems:
alexandria asdf closer-mop named-readtables optima
Install 4 Quicklisp releases:
fare-mop fare-quasiquote fare-utils inferior-shell
; Fetching #<URL "http://beta.quicklisp.org/archive/fare-quasiquote/2015-06-08/fare-quasiquote-20150608-git.tgz">
; 15.08KB
==================================================
15,437 bytes in 0.10 seconds (157.03KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/archive/fare-utils/2015-06-08/fare-utils-20150608-git.tgz">
; 31.51KB
==================================================
32,264 bytes in 0.14 seconds (218.80KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/archive/fare-mop/2015-06-08/fare-mop-20150608-git.tgz">
; 2.67KB
==================================================
2,738 bytes in 0.00 seconds (0.00KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/archive/inferior-shell/2015-06-08/inferior-shell-20150608-git.tgz">
; 12.87KB
==================================================
13,182 bytes in 0.00 seconds (12873.05KB/sec)
; Loading "inferior-shell"
[package fare-utils]..............................
[package fare-stateful]...........................
[package fare-quasiquote].........................
[package fare-mop].............
(INFERIOR-SHELL)
a simple sample:
CL-USER> (inferior-shell:run/ss '(echo (1) "2" (+ 3)))
"1 2 3"
NIL
0
a sample with pipes:
CL-USER> (inferior-shell:run/ss `(inferior-shell:pipe (echo (+ hel "lo,") world) (tr "hw" "HW") (sed -e "s/$/!/")))
"Hello, World!"
NIL
0
Mac OS X and SBCL:
Open a text file in the default text editor application TextEdit:
Lisp Machine:~ lispm$ touch /tmp/test.text
Lisp Machine:~ lispm$ sbcl
This is SBCL 1.2.14, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (sb-ext:run-program "/usr/bin/open" '("/tmp/test.text"))
#<SB-IMPL::PROCESS :EXITED 0>
Open the file with LispWorks as the text editor:
* (sb-ext:run-program
"/usr/bin/open"
'("-a"
"/Applications/LispWorks 7.0 (64-bit)/LispWorks (64-bit).app"
"/tmp/test.text"))
You might want to consult the SBCL manual for such questions. For example the chapter on Running External Programs.

Platform (OS) detection in scheme

That must be something like that :
(if (= system-type 'gnu/linux)
(system "make"))
To be honest I think my scheme implementation even can't do it in anyways but I'm free to add realization for it. What is usual scheme syntax for Platform detection?
thank you
I can't speak for any other Schemes, but Racket has a procedure called system-type:
> (system-type)
'unix
> (system-type 'machine)
"Linux ... x86_64 GNU/Linux" ;; ellipses mine, output is same as `uname -a`
And guile has a uname function, which returns a description as a scheme vector object:
scheme#(guile-user)> (uname)
$2 = #("Linux" "gblaptop" "2.6.39-gentoo-r3" "#4 SMP Fri Oct 21 08:12:17 PDT 2011" "i686")

Does there exist standard way to run external program in Common Lisp?

In clisp, the following code works:
(defun hit-history () (shell "tail ssqHitNum.txt"))
However, in Clozure CL, the shell function is not supported!
No, there is no standard way, but there are libraries which provide this functionality for the important implementations. For example, there's trivial-shell available in Quicklisp, which provides shell-command. (I didn't actually test it, but its among the recommended libraries on CLiki.) There is also external-program. Update: inferior-shell seems to be prefered these days, as Ehvince points out in a comment and his own answer.
You could also use read-time conditionals to make different implementations use their respective functionality to do this.
CCL has ccl:run-program, for example:
CL-USER> (run-program "whoami" '() :output *standard-output*)
foobar
#<EXTERNAL-PROCESS (whoami)[NIL] (EXITED : 0) #xC695EA6>
Yes, with UIOP, part of ASDF, that should be included in all modern implementations.
synchronous commands: uiop:run-program
asynchronous commands: uiop:launch-program
So for example
(uiop:run-program (list "firefox" "http:url") :output t)
or
(defparameter *shell* (uiop:launch-program "bash" :input :stream :output :stream))
where you can send input and read output.
They are more explained here: https://lispcookbook.github.io/cl-cookbook/os.html#running-external-programs
trivial-shell is deprecated and replaced by inferior-shell, which internally uses the portable uiop's run-program (synchronous), so we can use just that.
(defun dot->png (fname thunk)
(with-open-file (*standard-output*
fname
:direction :output
:if-exists :superseded)
(funcall thunk))
(ccl:run-program "dot" (list "-Tpng -O" fname))
)
i run success in ccl(clozure),when study land of lisp p123
The following shows an example of calling wget from within common lisp:
https://diasp.eu/posts/1742240
Here's the code:
(sb-ext:run-program "/usr/bin/wget" '("-O" "<path-to-output-file>" "<url-link>") :output *standard-output*)
Have a look at the inferior-shell package.
(Get it via the almighty quicklisp package manager.)
This works in the interpreter, if you have internet:
(require 'inferior-shell)
(inferior-shell:run/s '(curl icanhazip.com))
CL21 defines simple methods:
(in-package :cl21-user)
(use-package :cl21.process)
Then either with run-process or with the #` reader macro:
(run-process '("ls" "-l"))
;-> total 0
; drwxrwxrwt 5 root wheel 170 Nov 1 18:00 Shared
; drwxr-xr-x+ 174 nitro_idiot staff 5916 Mar 5 21:41 nitro_idiot
;=> #<PROCESS /bin/sh -c ls -l /Users (76468) EXITED 0>
or
#`ls -l /Users`
;=> "total 0
; drwxrwxrwt 5 root wheel 170 Nov 1 18:00 Shared
; drwxr-xr-x+ 174 nitro_idiot staff 5916 Mar 5 21:41 nitro_idiot
; "
; ""
; 0
The source shows implementation for SBCL and CCL.
http://cl21.org/
https://github.com/cl21/cl21/wiki
https://lispcookbook.github.io/cl-cookbook/cl21.html

Resources