I would like to start a subprocess in JavaScript for Automation (JXA) and send a string to that subprocess's stdin which might include newlines, shell metas, etc. Previous AppleScript approaches for this used bash's <<< operator, string concatenation, and quoted form of the string. If there was a JavaScript equivalent of quoted form of that I could trust to get all of the edge cases, I could use the same approach; I'm investigating regex methods toward that end.
However, I thought since we have access to unistd.h from JXA, why not try to just call $.pipe, $.fork, and $.execlp directly? $.pipe looks like it should take an array of 2 integers as its parameter, but none of the things that I have tried worked:
ObjC.import('unistd')
$.pipe() // Error: incorrect number of arguments
$.pipe([]) // segfault
$.pipe([3,4]) // segfault
$.pipe([$(), $()]) // segfault
var a = $(), b=$()
$.pipe([a,b]) // segfault
$.pipe($([a,b])) // NSException without a terribly helpful backtrace
$.pipe($([$(3), $(4)])) // segfault
var ref = Ref('int[2]')
$.pipe(ref)
ref[0] // 4, which is close!
Any suggestions?
I found an approach that works, using Cocoa instead of stdio:
ObjC.import('Cocoa')
var stdin = $.NSPipe.pipe
var stdout = $.NSPipe.pipe
var task = $.NSTask.alloc.init
task.launchPath = "/bin/cat"
task.standardInput = stdin
task.standardOutput = stdout
task.launch
var dataIn = $("foo$HOME'|\"").dataUsingEncoding($.NSUTF8StringEncoding)
stdin.fileHandleForWriting.writeData(dataIn)
stdin.fileHandleForWriting.closeFile
var dataOut = stdout.fileHandleForReading.readDataToEndOfFile
var stringOut = $.NSString.alloc.initWithDataEncoding(dataOut, $.NSUTF8StringEncoding).js
console.log(stringOut)
It is indeed curious that there appears to be no JXA equivalent of AppleScript's quoted form of for safely passing script literals to shell commands.
However, it is fairly easy to implement:
// JXA implementation of AppleScript's `quoted form of`
function quotedForm(s) { return "'" + s.replace(/'/g, "'\\''") + "'" }
// Example
app = Application.currentApplication();
app.includeStandardAdditions = true;
console.log(app.doShellScript('cat <<<' + quotedForm("foo$HOME'|\"")))
Credit for quotedForm() goes to this comment.
As far as I can tell, this implementation does the same as quoted form of does:
In the simplest form, if the string contains no embedded single-quotes, it single-quotes the entire string; since POSIX-like shells perform no interpolation whatsoever on a single-quoted string, it is preserved as-is.
If the string does contain embedded single-quotes, it is effectively broken into multiple single-quoted strings, with each embedded single-quote spliced in as \' (backslash-escaped) - this is necessary, because it is not possible to embed single-quotes in single-quoted literal in POSIX-compatible shells.
In a POSIX-compatible shell, this should work for all strings.
The quotedForm function above (below?) is lacking one very important feature, it only quotes/escapes the first in-line apostrophe whereas it needs to deal with however many exist in the string.
I changed it to this which seems to work:-
// JXA implementation of AppleScript's `quoted form of`
function quotedFormOf(s) { return "'" + s.replace(/'/g, "'\\''") + "'" }
Related
I stumbled over this problem using the following simplified example:
line = searchstring.dup
line.gsub!(Regexp.escape(searchstring)) { '' }
My understanding was, that for every String stored in searchstring, the gsub! would cause that line is afterwards empty. Indeed, this is the case for many strings, but not for this case:
searchstring = "D "
line = searchstring.dup
line.gsub!(Regexp.escape(searchstring)) { '' }
p line
It turns out, that line is printed as "D " afterwards, i.e. no replacement had been performed.
This happens to any searchstring containing a space. Indeed, if I do a
p(Regexp.escape(searchstring))
for my example, I see "D\\ " being printed, while I would expect to get "D " instead. Is this a bug in the Ruby core library, or did I misuse the escape function?
Some background: In my concrete application, where this simplified example is derived from, I just want to do a literal string replacement inside a long string, in the following way:
REPLACEMENTS.each do
|from, to|
line.chomp!
line.gsub!(Regexp.escape(from)) { to }
end
. I'm using Regexp.escape just as a safety measure in the case that the string being replaced contains some regex metacharacter.
I'm using the Cygwin port of MRI Ruby 2.6.4.
line.gsub!(Regexp.escape(searchstring)) { '' }
My understanding was, that for every String stored in searchstring, the gsub! would cause that line is afterwards empty.
Your understanding is incorrect. The guarantee in the docs is
For any string, Regexp.new(Regexp.escape(str))=~str will be true.
This does hold for your example
Regexp.new(Regexp.escape("D "))=~"D " # => 0
therefore this is what your code should look like
line.gsub!(Regexp.new(Regexp.escape(searchstring))) { '' }
As for why this is the case, there used to be a bug where Regex.escape would incorrectly handle space characters:
# in Ruby 1.8.4
Regex.escape("D ") # => "D\\s"
My guess is they tried to keep the fix as simple as possible by replacing 's' with ' '. Technically this does add an unnecessary escape character but, again, that does not break the intended use of the method.
This happens to any searchstring containing a space. Indeed, if I do a
p(Regexp.escape(searchstring))
for my example, I see "D\\ " being printed, while I would expect to get "D " instead. Is this a bug in the Ruby core library, or did I misuse the escape function?
This looks to be a bug. In my opinion, whitespace is not a Regexp meta character, there is no need to escape it.
Some background: In my concrete application, where this simplified example is derived from, I just want to do a literal string replacement inside a long string […]
If you want to do literal string replacement, then don't use a Regexp. Just use a literal string:
line.gsub!(from, to)
I need to create a string from a full POSIX path (starting at the root), so that it could be pasted directly into a Unix shell like bash, e.g. in Terminal.app, without the need for quotes around the path.
(I do not actually pass the string to a shell, but instead need it for passing it to another program. That program expects the path in just the form that you get when you drag a file into Terminal.app.)
For that, I need to escape at least any spaces in the string, by prepending them with a backslash. And some more characters as well.
For example, this path:
/directory/-as"<>' *+
Would be escaped as follows:
/directory/-as\"\<\>\'\ \*+
What's a safe algorithm to perform that conversion? I could escape every character, but that would be overkill.
There seems to be no framework function for doing this, so I'll need to do the replacing with string operations.
To be conservative (for the most popular shells), while also avoiding clearly unnecessary escapings, what set of characters should be escaped?
For the record, Terminal.app escapes the following non-control ASCII chars when dropping a file name into its window:
Space
!"#$%&'()*,:;<=>?[]`{|}~
And these are not escaped:
Control codes (00-1F and 7F)
Alphanumerical
+-.#^_
And here's the code that would perform the replacement:
NSString* shellPathFromPOSIXPath (NSString *path)
{
static NSRegularExpression *regex = nil;
if (!regex) {
NSString *pattern =
#"([ !\\\"\\#\\$\\%\\&\\'\\(\\)\\*\\,\\:\\;\\<\\=\\>\\?\\[\\]\\`\\{\\|\\}\\~])";
regex =
[NSRegularExpression regularExpressionWithPattern:pattern options:0 error:nil];
}
NSString *result =
[regex stringByReplacingMatchesInString:path
options:0
range:NSMakeRange(0, path.length)
withTemplate:#"\\\\$1"];
return result;
}
Better to put the whole thing in single quotes, rather than adding backslashes to individual characters; then the only character you need to escape is a single-quote present inside the string.
The Python standard library's implementation, provided as an example which can be easily reimplemented in any other language having only basic primitives, reads as follows:
def quote(s):
"""Return a shell-escaped version of the string *s*."""
if not s:
return "''"
if _find_unsafe(s) is None:
return s
# use single quotes, and put single quotes into double quotes
# the string $'b is then quoted as '$'"'"'b'
return "'" + s.replace("'", "'\"'\"'") + "'"
That is to say, the general algorithm is as follows:
An empty string becomes '' (a pair of literal single-quotes).
A string which is known to be safe (though it's safest to not try to implement a codepath for this at all, particularly as shells often implement their own syntax extensions in undefined space) can be emitted bare/unquoted.
Otherwise, prepend a ', emit your input string with all 's replaced with the literal string '"'"', and then append a final '.
That's it. You don't need to escape backslashes (they're literal inside single quotes), newlines (likewise), or anything else.
I'm trying to process this shell script in applescript, but it keeps saying error EOF, because of the single quote in the folder name.
do shell script "cp -R " & a & " " & "'" & d & "/My 'Folder/java/" & "'"
The folder /My 'Folder/ is a legitimate directory.
The variable a = '/Applications/MyProgram/' (and includes the single quotes)
The variable d = /Folders (with no single quotes)
However, shell is getting stuck processing it, im guessing because the folder is enclosed in quotes.
Is there any way to escape this single quote, so it works in applescript using shell? Ive tried multiple backslashes but its not working.
Cheers
N
Always use quoted form of when using arbitrary data as an argument to an command. This will always quote the value even if it doesn't need to be quoted but it's better safe than sorry. When you have a string single quoted, you can only unquote (turn substitution mode back on) with another quote. Unlike AppleScript strings, you can't escape characters inside single quoted strings. So you need to turn substitution mode on, escape a quote and then turn substitution mode back one. For instance "Joe's car" should be quoted as "'Joe'\\''s car'". It's a quoted string "Joe" + escaped quote character + quoted string "s car". But like I started you should use quoted form of, quoted form of "Joe's car" will return "'Joe'\\''s car'"
Your command using quoted form will look like:
do shell script "cp -R " & quoted form of a & space & quoted form of (d & "/My 'Folder/java/")
The problem arises because you have that tick in the filename of course, on the terminal commandline you would use a tick - double-tick tick - double-tick tick sequence to present it.
(From the terminal below)
729$ echo 'My '"'"'Java Folder'
My 'Java Folder
In Applescript, and your command line, it becomes even more complicated, I recommend you echo your commandline, until you get back what you expect.
set res to (do shell script "echo 'My '\"'\"'Java Folder'")
log res
--> (*My 'Java Folder*)
I think you'll have to start out with that, and reconstruct how you escape the rest of the commandline around it, if it can't be just plugged in as it is.
You should also remove single quotes for entities, that doesn't need them, (no spaces). That way your commandline will become easier to both edit and read.
Adding an answer for JavaScript for AppleScript (JAX) users based on answer from McUsr:
debugger
var app = Application.currentApplication();
app.includeStandardAdditions = true;
var source = "/documents/John's Spreadsheet.xls";
var target = "/documents/John's Spreadsheet.csv";
source = source.replace("'", "'\"'\"'", "g");
target = target.replace("'", "'\"'\"'", "g");
var exportScript = "/excel -i";
exportScript += " '" + source + "'";
exportScript += " '" + target + "'";
exportScript += ";true";
try {
app.doShellScript(exportScript);
}
catch (error) {
console.log(error.message);
}
If you don't know what JAX is it's AppleScript but using JavaScript. Open Script Editor and select JavaScript from the dropdown beneath the record button.
I have a variable such :
export ITEM={countryname}
this can be :
"Albania",
"United States" // with space
"Fs. Artic Land" // dot
"Korea (Rep. Of)" // braket
"Cote d'Ivoir" // '
This variable $(ITEM) is passed to other commands, some needing is as it (fine, I will use $(ITEM)), some MUST HAVE characters replacements, by example, to go with mkdir -p ../folder/{countryname} :
"Albania" // => Albania
"United States" // => United_States
"Fs. Artic Land" // => Fs\._Artic_Land
"Korea (Rep. Of)" // => Korea_\(Rep\._Of\)
"Cote d'Ivoire" // => Cote_d\'Ivoire
So I need a new make variable such
export ITEM={countryname}
export escaped_ITEM=$(ITEM).processed_to_be_fine
How should I do this characters replacements within my makefile ? (to keep things simple and not have to do an external script). I was thinking to use some transclude tr or something.
Note: working on Ubuntu.
You can use the subst function in GNU Make to perform substitutions.
escaped_ITEM := $(subst $e ,_,$(ITEM))
(where $e is an undefined or empty variable; thanks to #EtanReisner for pointing it out).
You will need one subst for each separate substitution, though.
If at all possible, I would advise against this, however -- use single, machine-readable tokens for file names, and map them to human readable only as the very last step. That's also much easier in your makefile:
human_readable_us=United States
human_readable_kr=Korea (Rep. of)
human_readable_ci=Côte d'Ivoire
human_readable_tf=FS. Antarctic Lands
stuff:
echo "$(human_readable_$(ITEM))"
Given the input simply "quoting" the country "names" when using them in the shell will work fine (for the few shown here) but double quoting arbitrary strings is not safe as any number of things can still evaluate inside double quotes (and with the way make operates even double quotes themselves in the string will cause problems).
If you need to pass "random" strings to the shell their is only one safe way to do that: replace every instance of ' (a single quote) in the string with '\'' and then wrap the string in ' (single quotes). (Depending on the consumer of the string replacing each ' with \047 can also work.)
I need to actually print a Dollar sign in Dart, ahead of a variable. For example:
void main()
{
int dollars=42;
print("I have $dollars."); // I have 42.
}
I want the output to be: I have $42. How can I do this? Thanks.
Dart strings can be either raw or ... not raw (normal? cooked? interpreted? There isn't a formal name). I'll go with "interpreted" here, because it describes the problem you have.
In a raw string, "$" and "\" mean nothing special, they are just characters like any other.
In an interpreted string, "$" starts an interpolation and "\" starts an escape.
Since you want the interpolation for "$dollars", you can't use "$" literally, so you need to escape it:
int dollars = 42;
print("I have \$$dollars.");
If you don't want to use an escape, you can combine the string from raw and interpreted parts:
int dollars = 42;
print(r"I have $" "$dollars.");
Two adjacent string literals are combined into one string, even if they are different types of string.
You can use a backslash to escape:
int dollars=42;
print("I have \$$dollars."); // I have $42.
When you are using literals instead of variables you can also use raw strings:
print(r"I have $42."); // I have $42.