There is a file and I would like to prepend at top of the file.
file name is xyz.csv
I am able to run the same command in terminal and getting the result as well but when running through ruby script I get an error
xyz.rb
#Script
file = "/home/sumeet/xyz.csv"
command = "sed -i '1s/^/resource_id,code,value,date\n/'"
full = "#{command} #{file}"
`full`
error
$ ruby xyz.rb
xyz.rb:4:in ``': No such file or directory - full (Errno::ENOENT)
from xyz.rb:4:in `<main>'
The file exists in system
You need to use string interpolation when you want to use a variable in system calls. Try:
#Script
file = "/home/sumeet/xyz.csv"
command = "sed -i '1s/^/resource_id,code,value,date\\n/'"
full = "#{command} #{file}"
`#{full}`
EDIT: remember to escape \ in \n using \\n. Otherwise it'll be interpreted as a newline and terminate sed command to early.
Related
Here
https://stackoverflow.com/a/876267/1579327
I learned how to do that for a single command cmd appending output to file.txt
cmd >>file.txt 2>&1
But my script contains many statements and commands.
And I would like to avoid appending >>file.txt 2>&1 to every line.
Is there a directive to let me to do that by default for every subsequent command?
Side note: I'm looking for a solution suitable also for MacOs X's bash
On top of your script you can use exec like this:
#!/bin/bash
# append stdout/stderr to a file
exec >> file.log 2>&1
# script starts here
Struggling with string interpolation while executing a shell command in ruby. Can you someone please help me identify what I'm missing here?
My one-liner ruby code follows redirects of any shortURL and returns the final URL. For example, this ruby code works perfectly fine.
curl -I https://bit.ly/1mJk8X7 | perl -n -e '/^Location: (.*)$/ && print "$1\n"'
It prints out the final URL.
I have a .txt file with a series of short URLs from which I'd like to derive a list of the final URLs. Say, it's called shortURLs.txt. I'm using IO.foreach to loop through each line in the file, but I don't know what I'm doing wrong to bring the variable 'x' into the ruby command. This is my first time working with string interpolation, and I've tried various combinations of it, but no luck yet.
IO.foreach("shortURLs.txt") { |x| system "curl -I #{x} | perl -n -e '/^Location: (.*)$/ && print \"$1\n\"' >> finalURLs.txt" }
I get an error message around the pipe '|' symbol:
sh: -c: line 1: syntax error near unexpected token|'
sh: -c: line 1: | perl -n -e '/^Location: (.*)$/ && print "https://bit.ly/1mJk8X7'
Other threads have been useful about string interpolation and running shell commands through ruby.
In order to pass two commands to a shell, you should run the system command twice (check method 8 in this post)
require 'shell'
sh = Shell.new
IO.foreach("shortURLs.txt") { |x| sh.system("curl -I #{x}") | sh.system("perl -n -e '/^Location: (.*)$/ && print \"$1\n\" ' ") >> "finalURLs.txt" }
IO.foreach yields the lines including the new line at the end so you're actually executing
curl -I https://bit.ly/1mJk8X7
| perl -n -e ...
Which is why you get the syntax error. You could use strip to remove the new line from. I think the \n in the call to print will also get substituted before the string is passed to system. You may be interested in shellwords which has functions for escaping strings before passing them to a shell.
You could of course dodge the issue entirely and use ruby to get the redirect locations
require 'net/http'
require 'uri'
IO.foreach("shortURLs.txt") do |url|
puts Net::HTTP.get_response(URI.parse(url))["Location"]
end
I was trying a simple shell program as below to append data at the end of the file,
path="/root/dir"
secure="*(rw,..)"
echo "$path $secure" >> a.txt
is not appending the string to a.txt
Just a guess but your script may be in DOS format that you're actually trying to write output to a.txt\r instead. Try to run one of the following to your code and try again:
sed -i 's|\r||' file
dos2unix file
As per the title, is it possible to get the raw command line used to invoke a ruby script?
The exact behaviour I'm after is similar to SSH when invoking a command directly:
ssh somehost -- ls -l
SSH will run "ls -l" on the server. It needs to be unparsed because if the shell has already interpreted quotes and performed expansions etc the command may not work correctly (if it contains quotes and such). This is why ARGV is no good; quotes are stripped.
Consider the following example:
my-command -- sed -e"s/something/something else/g"
The ARGV for this contains the following:
--
sed
-es/something/something else/g
The sed command will fail as the quotes will have been stripped and the space in the substitution command means that sed will not see "else/g".
So, to re-iterate, is it possible to get the raw command line used to invoke a ruby script?
No, this is at the OS level.
You could try simply quoting the entire input:
my-command -- "sed -e\"s/something/something else/g\""
In Ruby, this could be used like this:
ruby -e "puts ARGV[0]" -- "sed -e\"s/something/something else/g\""
(output) sed -e"s/something/something else/g"
Or, in a file putsargv1.rb (with the contents puts ARGV[1]):
ruby -- "putsargv1.rb" "sed -e\"s/something/something else/g\""
(output) sed -e"s/something/something else/g"
Your example is misguided. ssh somehost -- ls * will expand * on localhost (into e.g. ls localfile1 localfile2 localfile3), then execute that on the remote host, with the result of lots and lots of ls: cannot access xxx: No such file or directory errors. ssh does not see the uninterpreted command line.
As you said, you would get -es/something/something else/g as a single parameter. That is exactly what sed would get, too. This is, in fact, identical to what you get if you write -e"s/something/something else/g" and to "-es/something/something else/g", and to -es/something/something\ else.
Using this fact, you can use Shellwords.shellescape to "protect" the spaces and other unmentionables before handing them off to an external process. You can't get the original line, but you can make sure that you preserve the semantics.
Shellescape on the args worked but didn't quite mimic SSH. Take the following example (see below for test.rb contents):
ruby test.rb -- ls -l / \| sed -e's/root/a b c/g'
This will fail using the shellescape approach but succeed with SSH. I opted for manually escaping quotes and spaces. There may be some edge cases this doesn't capture but it seems to work for the majority of cases.
require 'shellwords'
unparsed = if ARGV.index('--')
ARGV.slice(ARGV.index('--') + 1, ARGV.length)
end || []
puts "Unparsed args: #{unparsed}"
exit if unparsed.empty?
shellescaped = unparsed.map(&Shellwords.method(:shellescape)).join(" ")
quoted = unparsed.map do |arg|
arg.gsub(/(["' ])/) { '\\' + $1 }
end.join(" ")
puts "Shellescaped: #{shellescaped}"
puts `bash -c #{shellescaped.shellescape}`
puts "Quoted: #{quoted}"
puts `bash -c #{quoted.shellescape}`
Thanks for your answers :)
I have a rake task that performs a shell command. When I run it from bash, the substitution worked, but when I run the rake task, the substitution did not take place.
My rake file:
require 'rake'
namespace 'performance_tests' do
task :test_guests_generation do
%x{sed -e 's/NO_OF_GUESTS = \[.*]/NO_OF_GUESTS = \[400, 10]/' -i '' db/seeds.rb}
end
end
Trying the command in Mac OS Terminal does perform the substitution:
$ sed -e 's/NO_OF_GUESTS = \[.*]/NO_OF_GUESTS = \[400, 10]/' -i '' db/seeds.rb
Um... not sure to be honest. Have you tried escaping the spaces? Can you paste the line you are working on and the desired result?