I am trying to use an external program from within Scala that accepts its input from the standard input stream.
Code roughly the equivalent to the following runs on Linux but raises an exception when run on Windows (tested on Windows 7). The exception's description states that the echo command cannot be found.
def invokeProgram(data : String) {
import scala.sys.process._
val cmdEcho = Seq("echo", data)
val cmdProgram = Seq("program")
println((cmdEcho #| cmdProgram).!!)
}
The code works correctly on Linux but fails as described on Windows, yet the echo command is common to both platforms and its usage is syntactically the same for my purposes. Is this as simple as echo not being in PATH? Is there even a separate echo.exe on Windows or is it bundled into something else? The invoked program can be made to accept its input from a temporary file which is what I will fall back to if I cannot resolve this issue.
The difference is this:
dcs#shadowfax:~$ which echo
/bin/echo
That is, on Unix the echo command is actually a binary, though most shells implement it as a builtin. On Windows, on the other hand, there's no binary called echo.exe (or echo.com, etc). It's solely a builtin command of the shell.
You don't need to use echo at all. Instead, use the #< method of ProcessBuilder. Here is an example:
import java.io.ByteArrayInputStream
import scala.sys.process._
val data = "hello"
val is = new ByteArrayInputStream(data.getBytes)
"cat" #< is ! //complicated way to print hello
Related
Is it possible to have (Rakudo) Perl6 execute some code before dropping you into the REPL? Like python does with "python -i ".
For instance, I want to load up some modules and maybe read a side file and build some data structures from that side file before dropping into the REPL and letting the user do the things they need to do on the data structure, using the REPL as a user interface.
This is similar but different than Start REPL with definitions loaded from file though answers to this question might satisfy that one. The basic case is that, at the end of execution of any program, instead of exiting, the interpreter leaves the user at the REPL. Aside from providing a nifty, built-in, Perl6-based user interface for interactive programs, it also provides a good tool from which to debug code that otherwise exits with an error.
edit:
Selecting Zoffix's solution as the correct (so far) one as it is the only one that satisfies all requirements as stated. Here's hoping this capability gets added to the compiler or language spec.
You can load modules with the -M switch.
$ perl6 -MJSON::Tiny
To exit type 'exit' or '^D'
> to-json Array.new: 1,2,3.Str
[ 1, 2, "3" ]
>
If you want to run other code, currently you have to put it into a module first.
$ mkdir lib
$ echo 'our $bar = 42' > lib/foo.pm6
$ perl6 -Ilib -Mfoo
To exit type 'exit' or '^D'
> $bar
42
>
I'd like to provide an answer that Zoffix gave on IRC. It satisfies the basic requirement but is far from pretty and it uses NQP for which there is no user support nor is the NQP API ( "nqp::*" calls ) guaranteed for the future and can change without warning.
replify ｢
say 'Hello to your custom REPL! Type `say $a` to print the secret variable';
my $a = "The value is {rand}";
｣;
sub replify (Str:D \pre-code = '') {
use nqp;
my %adverbs; # command line args like --MFoo
my \r := REPL.new: nqp::getcomp('perl6'), %adverbs;
my \enc := %adverbs<encoding>:v.Str;
enc && enc ne 'fixed_8' && $*IN.set-encoding: enc;
my $*CTXSAVE := r;
my $*MAIN_CTX;
pre-code and r.repl-eval: pre-code, $, :outer_ctx(nqp::getattr(r, REPL, '$!save_ctx')),
|%adverbs;
$*MAIN_CTX and nqp::bindattr(r, REPL, '$!save_ctx', $*MAIN_CTX);
r.repl-loop: :interactive, |%adverbs;
}
I'm just learning Python. One thing I hope to use Python for is Linux shell scripting. I've been studying some examples and practicing by converting my bash scripts to Python.
I just spent a few hours converting one of my bash scripts to Python and I was very disappointed with my resulting code. It was much more verbose and syntactically cluttered. It was certainly not concise and tight like the bash script I started with.
Then I stumbled across this (unrelated) Ruby script. Wow. It seems to give direct access to the shell, unlike Python which has a layer or two between the language and the shell.
#! /usr/bin/ruby
MAX_TIME_SINCE_START = 1800
begin
curr_time = Time.new.to_i
kslideshow_processes_list = `pgrep kslideshow.kss`.split("\n")
kslideshow_processes_list.each { |kslideshow_process|
kslideshow_process.strip!
ps_result = `ps -p #{kslideshow_process} -o lstart`
process_start_date = `ps -p #{kslideshow_process} -o
lstart`.split("\n")[1].strip
get_date_command = "date -d \"#{process_start_date}\" +\"%s\""
kslideshow_start_time = `#{get_date_command}`.to_i
time_since_started = curr_time - kslideshow_start_time
if time_since_started MAX_TIME_SINCE_START
system( "kill #{kslideshow_process}" )
end
}
sleep MAX_TIME_SINCE_START
end while true
This is the kind of code I was hoping would result from my switch from bash script to Python. Is it possible in Python to write shell scripts so cleanly?
It would be very educational for me to see the above code converted to Python by someone who knows what they are doing.
I don't intend to start a Ruby vs. Python discussion. I simply want to see how cleanly the above task can be implemented in Python by someone who knows more Python than I do. Is that a fair question for this site? Thanks
You can IMO do this at least as elegant as in Ruby by using the plumbum package. There are ways to optimize away system calls (e.g. using dateutils to convert from a date format, or using psutils), but that would no longer be a comparison of calling system utilities (I hope I got the logic from the Ruby example correct):
from time import time, sleep
from plumbum.cmd import pgrep, ps, date, kill
MAX_TIME_SINCE_START = 5 # 1800
while True:
curr_time = time()
for kslideshow_process in pgrep("gimp").split():
print kslideshow_process
process_start_date = ps("-p", kslideshow_process, "-o", "lstart"
).split("\n")[1].strip()
kslideshow_start_time = int(date("-d", process_start_date, '+%s'))
print kslideshow_start_time
time_since_started = curr_time - kslideshow_start_time
if time_since_started > MAX_TIME_SINCE_START:
kill([kslideshow_process])
sleep(MAX_TIME_SINCE_START)
You can import any commandline program using from plumbum.cmd import ... and it is automatically available as a function that takes the commandline arguments as parameters.
You can also make chains of commands:
chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"]
result = chain()
so there is far less often need for intermediate variables to carry results from one subshell to another.
Barring the bad ideas and ways you could poke your own eye out.. an example of the more efficient way to open a another process in python is as follows...
process = subprocess.Popen([MyCommand, parameter1, "parameter 2"], shell=False, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, stdin=subprocess.PIPE)
stdout,stderr = process.communicate()
return_code = process.poll()
Doing things with popen is a little bit more overhead than some other languages, but it offers a lot of versatility that no other scripting language I know of offers like getting output from many many processes live.
I would use programming-quote like this in Bash
$ export b=`ls /`
$ echo $b
Applications Library Network System Users Volumes tmp usr var
and now I want to find similar functionality in Matlab. Also, I want to find a command that outputs relative paths, not absolute paths like Matlab's ls -- I feel like reinventing the wheel if I am parsing this with a regex. I need to find this command to debug what is wrong with namespaces here. Familiar-Bash-style functionatilities would be so cool.
For your first question, I think you can get that behavior with anonymous functions:
b = #() ls('C:\'); %I'm on windows
b()
The expression b() now returns the contents of my C drive.
The Matlab equivalent of bash backticks is calling the system() function and using the second output argument. It will run an external command and capture the output.
[status,b] = system('ls /');
If it's a string of Matlab code you want to run and capture the console output of, use evalc.
But to just get a listing of files, you want the Matlab dir function. Lots easier than parsing that output string, and you get more info. See Matlab dir documentation or doc dir for more details.
children = dir('/');
childNames = { children.name };
childIsDir = [ children.isdir ];
I have an environment variable set in Windows as TEST=abc£ which uses Windows-1252 code page. Now, when I run a Perl program test.pl this environment value comes properly.
When I call another Perl code - test2.pl from test1.pl either by system(..) or Win32::Process, the environment comes garbled.
Can someone provide information why this could be and way to resolve it?
The version of perl I am using is 5.8.
If my understanding is right, perl internally uses utf-8, so the initial process - test1.pl received it right from Windows-1252 → utf-8. When we call another process, should we convert back to Windows-1252 code page?
This has nothing to do with Perl's internal string encoding, but with the need to properly decode data coming from the outside. I'll provide the test case. This is Strawberry Perl 5.10 on a Western European Windows XP.
test1.pl:
use Devel::Peek;
print Dump $ENV{TEST};
use Encode qw(decode);
my $var = decode 'Windows-1252', $ENV{TEST};
print Dump $var;
system "B:/sperl/perl/bin/perl.exe B:/test2.pl";
test2.pl:
use Devel::Peek;
print Dump $ENV{TEST};
use Encode qw(decode);
my $var = decode 'IBM850', $ENV{TEST};
# using Windows-1252 again is wrong here
print Dump $var;
Execute:
> set TEST=abc£
> B:\sperl\perl\bin\perl.exe B:\test1.pl
Output (shortened):
SV = PVMG(0x982314) at 0x989a24
FLAGS = (SMG, RMG, POK, pPOK)
PV = 0x98de0c "abc\243"\0
SV = PV(0x3d6a64) at 0x989b04
FLAGS = (PADMY, POK, pPOK, UTF8)
PV = 0x9b5be4 "abc\302\243"\0 [UTF8 "abc\x{a3}"]
SV = PVMG(0x982314) at 0x989a24
FLAGS = (SMG, RMG, POK, pPOK)
PV = 0x98de0c "abc\243"\0
SV = PV(0x3d6a4c) at 0x989b04
FLAGS = (PADMY, POK, pPOK, UTF8)
PV = 0x9b587c "abc\302\243"\0 [UTF8 "abc\x{a3}"]
You are bitten by the fact that Windows uses a different encoding for the text environment (IBM850) than for the graphical environment (Windows-1252). An expert has to explain the deeper details of that phenomenon.
Edit:
It is possible to heuristically (meaning it will fail to do the right thing sometimes, especially for such short strings) determine encodings. The best general purpose solution is Encode::Detect/Encode::Detect::Detector which is based on Mozilla nsUniversalDetector.
There are some ways to decode external data implicitely such as the open pragma/IO layers and the -C switch, however they deal with file streams and program arguments only. As of now, from the environment must be decoded explicitely. I like that better anyway, explicite shows the maintainance programmer you have thought the topic through.
I want to (need to) start a sub-process from a perl script that checks certain environment variables. In one instance the environment variable needs to be there but empty.
$ENV{"GREETING"} = "Hello World"; # Valid
$ENV{"GREETING"} = ""; # also valid
I can set $ENV{"GREETING"} = ""; and in that perl script $ENV{"GREETING"} is empty, but in any sub-process that environment variable is not there.
Here is some example code to demonstrate. This script, env_in.pl sets up some environment variables, ZZZ_3 is empty. It then calls env_out.pl to output the environment variables, ZZZ_3 is missing from the output.
#!/usr/bin/perl
# env_in.pl
use strict;`enter code here`
use warnings;
$ENV{ZZZ_1} = "One";
$ENV{ZZZ_2} = "Two";
$ENV{ZZZ_3} = "";
$ENV{ZZZ_4} = "Four";
my (#cmd) = ("perl", "env_out.pl");
system(#cmd) == 0 or die "system #cmd failed: $?";
Here is the env_out.pl script.
#!/usr/bin/perl
use strict;
use warnings;
print ($_," = ", $ENV{$_}, "\n") for (sort keys %ENV);
I'm using ActiveState perl version v5.8.8 on a WinXP box.
I know that this DOES work in python, but I don't have a choice about the implementation language, it has to be Perl.
As far as I know, there's no such thing as an empty environment variable in Windows. Defining them to empty is the same thing as undefining them.
Your Perl scripts does indeed display a ZZZ_3 entry when run on a Unix-like system.
That seems to be some issue with your activestate perl. Testing on windows2000 + cygwin gives:
$ perl --version
This is perl, v5.10.0 built for cygwin-thread-multi-64int
(with 6 registered patches, see perl -V for more detail)
Copyright 1987-2007, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
$ perl env_in.pl | grep ZZZ
ZZZ_1 = One
ZZZ_2 = Two
ZZZ_3 =
ZZZ_4 = Four
$
Using both strawberry perl and cygwin perl on vista results in
ZZZ_1 = One
ZZZ_2 = Two
ZZZ_4 = Four