I have a csv type file. I want to be able to call a function from the command line, and feed it the content of this file - not the file itself.
I tried to call
mix run lib/my_module.ex $(cat tmp.txt)
having in the end of my module file :
IO.puts MyModule.my_func(System.argv)
Content is correctly processed to the func, but System.argv being a list of strings, it lost its format and is not possible to parse correctly.
If I instead try to
mix run -e "MyModule.my_func(:args)"
I can't find how to feed it the content of the file, using cat or something else.
How to make it work ?
Try to put between quotes, like this: mix run lib/my_module.ex "$(cat tmp.txt)"
This looks like XY Problem to me. Pass the name of the file only and do File.read/1 inside your module.
Somewhat like:
mix run lib/my_module.ex tmp.txt
and in your module:
System.argv
|> File.read!()
|> MyModule.my_func()
|> IO.puts()
Related
There was a problem when im creating the CLI. I'm want to give the user the opportunity to insert their data into a text file, for this I created a file and added a heredoc to it
I'm trying to get data from a text document that has a heredoc inside of it with a function that is supposed to interpolate
When I try to display the result of the file, I get the entire contents of the file, including the heredoc
an example will be below
I tried to solve my problem through File class
variable_name = File::open("path_directory/file_with_heredoc.txt", "r+")::read
Next, I decided to give the value of the variable to the terminal via
exec("echo #{variable_name}")
The terminal displays
file = <<-EOM
single text with def result: #{upcase_def("Hello")}
EOM
Tried to give through struct, but result is unchanged
exec("echo #{variable_name.strip}")
What do I need to do to get only data, no HEREDOC syntax?
I want to get this result
"single text with def result: HELLO"
I think this is what you are trying to do but I recommend you to first do some research why 'eval() is evil'. If the file is a user (or hacker) input you definitely want some sanitization there or a completely different approach.
def upcase_def(str)
str.upcase
end
data = File.read('file_with_heredoc.txt')
eval(data)
# => " single text with def result: HELLO\n"
I have a stream of data that I’m writing to a named pipe:
named_pipe = '/tmp/pipe' # Location of named pipe
File.mkfifo(named_pipe) # Create named pipe
File.open(named_pipe, 'w+') # Necessary to not get a broken pipe when ⌃C from another process later on
system('youtube-dl', '--newline', 'https://www.youtube.com/watch?v=aqz-KE-bpKQ', out: named_pipe) # Output download progress, one line at a time
Trouble is, while I can cat /tmp/pipe and get the information, I’m unable to read the file from another Ruby process. I’ve tried File.readlines, File.read with seeking, File.open then reading, and other stuff I no longer remember. Some of those hang, others error out.
How can I get the same result as with cat, in pure Ruby?
Note I don’t have to use system to send to the pipe (Open3 would be acceptable), but any solution requiring external dependencies is a no-go.
it looks like File.readlines/IO.readlines, File.read/IO.read need to load the whole temp file first so you don't see any be printed out.
try File#each/IO.foreach which process a file line by line and it does not require the whole file be loaded into memory
File.foreach("/tmp/pipe") { |line| p line }
# or
File.open('/tmp/pipe','r').each { |line| p line }
I want to run some shell scripts remotely as part of my capistrano setup. To test that functionality, I use this code:
execute <<SHELL
cat <<TEST
something
TEST
SHELL
However, that is actually running /usr/bin/env cat <<TEST; something; TEST which is obviously not going to work. How do I tell capistrano to execute the heredoc as I have written it, without converting the newlines into semicolons?
I have Capistrano Version: 3.2.1 (Rake Version: 10.3.2) and do not know ruby particularly well, so there might be something obvious I missed.
I think it might work to just specify the arguments to cat as a second, er, argument to execute:
cat_args = <<SHELL
<<TEST
something
TEST
SHELL
execute "cat", cat_args
From the code #DavidGrayson posted, it looks like only the command (the first argument to execute) is sanitized.
I agree with David, though, that the simpler way might be to put the data in a file, which is what the SSHKit documentation suggests:
Upload a file from a stream
on hosts do |host|
file = File.open('/config/database.yml')
io = StringIO.new(....)
upload! file, '/opt/my_project/shared/database.yml'
upload! io, '/opt/my_project/shared/io.io.io'
end
The IO streaming is useful for uploading something rather than "cat"ing it, for example
on hosts do |host|
contents = StringIO.new('ALL ALL = (ALL) NOPASSWD: ALL')
upload! contents, '/etc/sudoers.d/yolo'
end
This spares one from having to figure out the correct escaping sequences for something like "echo(:cat, '...?...', '> /etc/sudoers.d/yolo')".
This seems like it would work perfectly for your use case.
The code responsible for this sanitization can be found in SSHKit::Command#sanitize_command!, which is called by that class's initialize method. You can see the source code here:
https://github.com/capistrano/sshkit/blob/9ac8298c6a62582455b1b55b5e742fd9e948cefe/lib/sshkit/command.rb#L216-226
You might consider monkeypatching it to do nothing by adding something like this to the top of your Rakefile:
SSHKit::Command # force the class to load so we can re-open it
class SSHKit::Command
def sanitize_command!
return if some_condition
super
end
end
This is risky and could introduce problems in other places; for example there might be parts of Capistrano that assume that the command has no newlines.
You are probably better off making a shell script that contains the heredoc or putting the heredoc in a file somewhere.
Ok, so this is the solution I figured out myself, in case it's useful for someone else:
str = %x(
base64 <<TEST
some
thing
TEST
).delete("\n")
execute "echo #{str} | base64 -d | cat -"
As you can see, I'm base64 encoding my command, sending it through, then decoding it on the server side where it can be evaluated intact. This works, but it's a real ugly hack - I hope someone can come up with a better solution.
I want to write a ruby script that read from a config file that will have filenames, and then when I run the script it will take the tail of each file and output the console.
What's the best way to go about doing this?
Take a look at File::Tail gem.
You can invoke linux tail -number_of_lines file_name command from your ruby script and let it print on console or capture output and print it yourself (if you need to do something with these lines before you print it)
We have a configuration file that contain a list of the log files; for example, like this:
---
- C:\fe\logs\front_end.log
- C:\mt\logs\middle_tier.log
- C:\be\logs\back_end.log
The format of the configuration file is a yaml simple sequence , therefore suppose we named this file 'settings.yaml'
The ruby script that take the tail of each file and output the console could be like this:
require 'yaml'
require 'file-tail'
logs = YAML::load(File.open('settings.yaml'))
threads = []
logs.each do |the_log|
threads << Thread.new(the_log) { |log_filename|
File.open(log_filename) do |log|
log.extend(File::Tail)
log.interval = 10
log.backward(10)
log.tail { |line| p "#{File.basename(the_log,".log")} - #{line}" }
end
}
end
threads.each { |the_thread| the_thread.join }
Note: displaying each line I wanted to prefix it with the name of the file from which it originates, ...this for me is a good option but you can edit the script to change as you like ; is the same for the tails parameters.
if file-tail is missing in your environment, follow the link as #Mark Thomas posts in his answear; i.e you need to:
> gem install file-tail
I found the file-tail gem to be a bit buggy. I would write to a file and it would read the entire file again instead of just thelines appended. This happened even though I had log.backward set to 0. I ended up writing my own and figured that I would share it here in case any one else is looking for a Ruby alternative to the file-tail gem. You can find the repo here. It uses non_blocking io, so it will catch amendments to the file immediately. There is one caveat that can be easily fixed if you can program in the Ruby programming language; log.backward is hard coded to be -1.
I'm trying to write a very simple markdown-like converter in ruby, then pass the output to PrinceXML (which is awesome). Prince basically converts html to pdf.
Here's my code:
#!/usr/bin/ruby
# USAGE: command source-file.txt target-file.pdf
# read argument 1 as input
text = File.read(ARGV[0])
# wrap paragraphs in paragraph tags
text = text.gsub(/^(.+)/, '<p>\1</p>')
# create a new temp file for processing
htmlFile = File.new('/tmp/sample.html', "w+")
# place the transformed text in the new file
htmlFile.puts text
# run prince
system 'prince /tmp/sample.html #{ARGV[1]}'
But this dumps an empty file to /tmp/sample.html. When I exclude calling prince, the conversion happens just fine.
What am I doing wrong?
It's possible that the file output is being buffered, and not written to disk, because of how you are creating the output file. Try this instead:
# create a new temp file for processing
File.open('/tmp/sample.html', "w+") do |htmlFile|
# place the transformed text in the new file
htmlFile.puts text
end
# run prince
system 'prince /tmp/sample.html #{ARGV[1]}'
This is idiomatic Ruby; We pass a block to File.new and it will automatically be closed when the block exits. As a by-product of closing the file, any buffered output will be flushed to disk, where your code in your system call can find it.
From the fine manual:
prince doc.html -o out.pdf
Convert doc.html to out.pdf.
I think your system call should look like this:
system "prince /tmp/sample.html -o #{ARGV[1]}"
Also note the switch to double quotes so that #{} interpolation will work. Without the double quotes, the shell will see this command:
prince /tmp/sample.html #{ARGV[1]}
and then it will ignore everything after # as a comment. I'm not sure why you end up with an empty /tmp/sample.html, I'd expect a PDF in /tmp/sample.pdf based on my reading of the documentation.