Here is the case;
There is this app called "termux" on android which allows me to use a terminal on android, and one of the addons are androids API's like sensors, tts engines, etc.
I wanted to make a script in ruby using this app, specifically this api, but there is a catch:
The script:
require('json')
JSON.parse(%x'termux-sensor -s "BMI160 Gyro" -n 1')
-s = Name or partially the name of the sensor
-n = Count of times the command will run
returns me:
{
"BMI160 Gyroscope" => {
"values" => [
-0.03...,
0.00...,
1.54...
]
}
}
I didn't copied and pasted the values, but that's not the point, the point is that this command takes almost a full second the load, but there is a way to "make it faster"
If I use the argument "-d" and not use "-n", I can specify the time in milliseconds to delay between data being sent in STDOUT, it also takes a full second to load, but when it loads, the delay works like charm
And since I didn't specify a 'n' number of times, it never stops, and there is the problem
How can I retrieve the data continuously in ruby??
I thought about using another thread so it won't stop my program, but how can I tell ruby to return the last X lines of the STDOUT from a command that hasn't and will not ever stop since "%x'command'" in ruby waits for a return?
If I understood you need to connect to stdout from a long running process.
see if this works for your scenario using IO.popen:
# by running this program
# and open another terminal
# and start writing some data into data.txt
# you will see it appearing in this program output
# $ date >> data.txt
io_obj = IO.popen('tail -f ./data.txt')
while !io_obj.eof?
puts io_obj.readline
end
I found out a built in module that saved me called PTY and the spawn#method plus thread management helped me to keep a variable updated with the command values each time the command outputted new bytes
Related
I've been working with Terraform, v0.15.4, for a few weeks now, and have gotten to grips with most of the lingo. I'm currently trying to create a cluster of RHEL 7 instances dynamically on GCP, and have, for the most part, got it to run okay.
I'm at the point of deploying an instance with certain metadata passed along to it for use in scripts built into the machine image for configuration thereafter. This metadata is typically just passed via an echo into a text file, which the scripts then pickup as required.
It's... very simple. Echo "STUFF" > file... Alas, I am hitting the same issue OVER AND OVER and it's driving me INSANE. I've Google'd around for ages, but all I can find is examples of the exact thing that I'm doing, the only difference is that theirs works, mine doesn't... So hopefully I can get some help here.
My 'makes it half-way' code is as follows:
resource "google_compute_instance" "GP_Master_Node" {
...
metadata_startup_script = <<-EOF
echo "hello
you" > /test.txt
echo "help
me" > /test2.txt
EOF
Now the instance with this does create successfully, although when I look onto the instance, I get one file called ' /test.txt? ' (or if I 'ls' the file, it shows as ' /test.txt^M ') and no second file.. I can run any command instead of echo, and whilst the first finishes, the second+ does not. Why?? What on earth is causing that??
The following code I found also, but it doesn't work for me at all, with the error, 'Blocks of type "metadata" are not expected here.'
resource "google_compute_instance" "GP_Master_Node" {
...
metadata {
startup-script = "echo test > /test.txt"
}
Okaaaaay! Simple answer for a, in hindsight, silly question (sort of). The file was somehow formmated in DOS, meaning the script required a line continuation character to run correctly (specifically \ at the end of each individual command). Code as follows:
resource "google_compute_instance" "GP_Master_Node" {
...
metadata_startup_script = <<-EOF
echo "hello
you" > /test.txt \
echo "help
me" > /test2.txt \
echo "example1" > /test3.txt \
echo "and so on..." > /final.txt
EOF
However, what also fixed my issue was just 'refreshing' the file (probably a word for this, I don't know). I created a brand new file using touch, 'more'd the original file contents to screen, and then copy pasted them into the new one. On save, it is no longer DOS, as expected, and then when I run terraform the code runs as expected without requiring the line continuation characters at the end of commands.
Thank you to commentors for the help :)
What I'm trying to achieve:
From a Ruby process, spawning a subprocess
The subprocess should print as normal back to the terminal. By "normal", I mean the process shouldn't miss out color output, or ignore user input (STDIN).
For that subprocess, capturing STDOUT/STDERR (jointly) e.g. into a String variable that can be accessed after the subprocess is dead. Escape characters and all.
Capturing STDOUT/STDERR is possible by passing a different IO pipe, however the subprocess can then detect that it's not in a tty. For example git log will not print characters that influence text color, nor use it's pager.
Using a pty to launch the process essentially "tricks" the subprocess into thinking it's being launched by a user. As far as I can tell, this is exactly what I want, and the result of this essentially ticks all the boxes.
My general tests to test if a solution fits my needs is:
Does it run ls -al normally?
Does it run vim normally?
Does it run irb normally?
The following Ruby code is able to check all the above:
to_execute = "vim"
output = ""
require 'pty'
require 'io/console'
master, slave = PTY.open
slave.raw!
pid = ::Process.spawn(to_execute, :in => STDIN, [:out, :err] => slave)
slave.close
master.winsize = $stdout.winsize
Signal.trap(:WINCH) { master.winsize = $stdout.winsize }
Signal.trap(:SIGINT) { ::Process.kill("INT", pid) }
master.each_char do |char|
STDOUT.print char
output.concat(char)
end
::Process.wait(pid)
master.close
This works for the most part but it turns out it's not perfect. For some reason, certain applications seem to fail to switch into a raw state. Even though vim works perfectly fine, it turned out neovim did not. At first I thought it was a bug in neovim but I have since been able to reproduce the problem using the Termion crate for the Rust language.
By setting to raw manually (IO.console.raw!) before executing, applications like neovim behave as expected, but then applications like irb do not.
Oddly spawning another pty in Python, within this pty, allows the application to work as expected (using python -c 'import pty; pty.spawn("/usr/local/bin/nvim")'). This obviously isn't a real solution, but interesting nonetheless.
For my actual question I guess I'm looking towards any help to resolving the weird raw issue or, say if I've completely misunderstood tty/pty, any different direction to where/how I should look at the problem.
[edited: see the bottom for the amended update]
Figured it out :)
To really understand the problem I read up a lot on how a PTY works. I don't think I really understood it properly until I drew it out. Basically PTY could be used for a Terminal emulator, and that was the simplest way to think of the data flow for it:
keyboard -> OS -> terminal -> master pty -> termios -> slave pty -> shell
|
v
monitor <- OS <- terminal <- master pty <- termios
(note: this might not be 100% correct, I'm definitely no expert on the subject, just posting it incase it helps anybody else understand it)
So the important bit in the diagram that I hadn't really realised was that when you type, the only reason you see your input on screen is because it's passed back (left-wards) to the master.
So first thing's first - this ruby script should first set the tty to raw (IO.console.raw!), it can restore it after execution is finished (IO.console.cooked!). This'll make sure the keyboard inputs aren't printed by this parent Ruby script.
Second thing is the slave itself should not be raw, so the slave.raw! call is removed. To explain this, I originally added this because it removes extra return carriages from the output: running echo hello results in "hello\r\n". What I missed was that this return carriage is a key instruction to the terminal emulator (whoops).
Third thing, the process should only be talking to the slave. Passing STDIN felt convenient, but it upsets the flow shown in the diagram.
This brings up a new problem on how to pass user input through, so I tried this. So we basically pass STDIN to the master:
input_thread = Thread.new do
STDIN.each_char do |char|
master.putc(char) rescue nil
end
end
that kind of worked, but it has its own issues in terms of some interactive processes weren't receiving a key some of the time. Time will tell, but using IO.copy_stream instead appears to solve that issue (and reads much nicer of course).
input_thread = Thread.new { IO.copy_stream(STDIN, master) }
update 21st Aug:
So the above example mostly worked, but for some reason keys like CTRL+c still wouldn't behave correctly. I even looked up other people's approach to see what I could be doing wrong, and effectively it seemed the same approach - as IO.copy_stream(STDIN, master) was successfully sending 3 to the master. None of the following seemed to help at all:
master.putc 3
master.putc "\x03"
master.putc "\003"
Before I went and delved into trying to achieve this in a lower level language I tried out 1 more thing - the block syntax. Apparently the block syntax magically fixes this problem.
To prevent this answer getting a bit too verbose, the following appears to work:
require 'pty'
require 'io/console'
def run
output = ""
IO.console.raw!
input_thread = nil
PTY.spawn('bash') do |read, write, pid|
Signal.trap(:WINCH) { write.winsize = STDOUT.winsize }
input_thread = Thread.new { IO.copy_stream(STDIN, write) }
read.each_char do |char|
STDOUT.print char
output.concat(char)
end
Process.wait(pid)
end
input_thread.kill if input_thread
IO.console.cooked!
end
Bundler.send(:with_env, Bundler.clean_env) do
run
end
I have been trying to run a shell script at boot time of freebsd. I have read all simmilar questions in stackoverflow and tried. But nothing is worked. This is the sample code that i tried is dummy.
#!/bin/sh
. /etc/rc.subr
name="dummy"
start_cmd="${name}_start"
stop_cmd=":"
dummy_start()
{
echo "Nothing started."
}
load_rc_config $name
run_rc_command "$1"
Saved with name of dummy.
Permissions are -r-xr-xr-x.
in rc.conf file made dummy_enable="YES".
The problem is, when i rebooted my system to test, dummy file is not there. So script is not executing. what else need to do run my dummy script.
SRC:http://www.freebsd.org/doc/en/articles/rc-scripting/article.html#rc-flags
You need to add rcvar="dummy_enable" to your script. At least for FreeBSD 9.1.
Call your script with parameter rcvar to get the enabled status:
# /etc/rc.d/dummy rcvar
# dummy
#
dummy_enable="YES"
# (default: "")
And finally start it with parameter start - this won't start the service/script unless dummy_enable is set in /etc/rc.conf (or /etc/rc.conf.local, or /etc/defaults/rc.conf)
# /etc/rc.d/dummy start
Nothing started.
One possible explanation is that rcorder(8) says:
Within each file, a block containing a series of "REQUIRE", "PROVIDE",
"BEFORE" and "KEYWORD" lines must appear.
Though elsewhere I recall that if a file doesn't have "REQUIRE", "PROVIDE" or "BEFORE", then it will be arbitrarily placed in the dependency ordering. And, it could be that the arbitrary placement differs between the first run up to $early_late_divider and in the second run of those after $early_late_divider.
OTOH, is this a stock FreeBSD, or some variant? I recall reading that FreeNAS saves its configuration somewhere else and recreates its system files on every boot. And, quite possibly that /etc is actually on a ramdisk.
Also, /usr/local/etc/rc.d doesn't come into existence until the first port installing an rc file is installed.
I have a Capistrano deploy file (Capfile) that is rather large, contains a few namespaces and generally has a lot of information already in it. My ultimate goal is, using the Tinder gem, paste the output of the entire deployment into Campfire. I have Tinder setup properly already.
I looked into using the Capistrano capture method, but that only works for the first host. Additionally that would be a lot of work to go through and add something like:
output << capture 'foocommand'
Specifically, I am looking to capture the output of any deployment from that file into a variable (in addition to putting it to STDOUT so I can see it), then pass that output in the variable into a function called notify_campfire. Since the notify_campfire function is getting called at the end of a task (every task regardless of the namespace), it should have the task name available to it and the output (which is stored in that output variable). Any thoughts on how to accomplish this would be greatly appreciated.
I recommend not messing with the Capistrano logger, Instead use what unix gives you and use pipes:
cap deploy | my_logger.rb
Where your logger reads STDIN and STDOUT and both records, and pipes it back to the appropriate stream.
For an alternative, the Engineyard cap recipies have a logger – this might be a useful reference if you do need to edit the code, but I recommend not doing.
It's sort of a hackish means of solving your problem, but you could try running the deploy task in a Rake task and capturing the output using %x.
# ...in your Rakefile...
task :deploy_and_notify do
output = %x[ cap deploy ] # Run your deploy task here.
notify_campfire(output)
puts output # Echo the output.
end
I have a process that runs on cron every five minutes. Usually, it takes only a few seconds to run, but sometimes it takes several minutes. I want to ensure that only one version of this is running at a time.
I tried an obvious way...
File.open("/tmp/indexer_lock.tmp",'w') do |f|
exit unless f.flock(File::LOCK_EX)
end
...but it's not testing to see if it can get the lock, it's blocking until the lock is released.
Any idea what I'm missing? I'd rather not hack something using ps, but that's an alternative.
I know this is old, but for anyone interested, there's a non-blocking constant that you can pass to flock so that it returns instead of blocking.
File.new("/tmp/foo.lock").flock( File::LOCK_NB | File::LOCK_EX )
Update for slhck
flock will return true if this process received the lock, false otherwise. So to ensure just one process is running at a time you just want to try to get the lock, and exit if you weren't able to. It's as simple as putting an exit unless in front of the line of code I have above:
exit unless File.new("/tmp/foo.lock").flock( File::LOCK_NB | File::LOCK_EX )
Depending on your needs, this should work just fine and doesn't require creating another file anywhere.
exit unless DATA.flock(File::LOCK_NB | File::LOCK_EX)
# your script here
__END__
DO NOT REMOVE: required for the DATA object above.
Although this isn't directly answering your question, if I were you I'd probably write a daemon script (you could use http://daemons.rubyforge.org/)
You could have your indexer (assuming its indexer.rb) be run through a wrapper script named script/index for example:
require 'rubygems'
require 'daemons'
Daemons.run('indexer.rb')
And your indexer can do almost the same thing, except you specify a sleep interval
loop do
# code executing your indexing
sleep INDEXING_INTERVAL
end
This is how job processors in tandem with a queue server usually function.
You could create and delete a temporary file and check for existence of this file.
Please check the answer to this question :
one instance shell script
There's a lockfile gem for exactly this situation. I've used it before and it's dead simple.
If your using cron it might be easier to do something like this in the shell script that cron calls:
#!/usr/local/bin/bash
#
if ps -C $PROGRAM_NAME &> /dev/null ; then
: #Program is already running.. appropriate action can be performed here (kill it?)
else
#Program is not running.. launch it.
$PROGRAM_NAME
fi
Here's a one-liner that should work at the top of any Ruby script:
exit unless File.new(__FILE__)).tap {|f| f.autoclose = false}.flock(File::LOCK_NB | File::LOCK_EX)
There are two issues with the original code.
First, the reason it's blocking is that the call to #flock is missing File::LOCK_NB:
Don't block when locking. May be combined
with other lock options using logical or.
Second, if a File object is closed (whether at the end of an #open block as in the code above, via explicit #close, or implicitly auto-closed when the File is garbage-collected), the underlying file descriptor is closed and the lock is released. To prevent this you can set #autoclose =false.
Ok, working off notes from #shodanex's pointer, here's what I have. I rubied it up a little bit (though I don't know of a touch analogue in Ruby).
tmp_file = File.expand_path(File.dirname(__FILE__)) + "/indexer.lock"
if File.exists?(tmp_file)
puts "quitting"
exit
else
`touch #{tmp_file}`
end
.. do stuff ..
File.delete(tmp_file)
Can you not add File::LOCK_NB to your lock, to make it non-blocking (i.e. it fails if it can't get the lock)
That would work in C, Perl etc.
At a higher level, you might find the lock_method gem useful:
def the_method_my_cron_job_calls
# something really expensive
end
lock_method :the_method_my_cron_job_calls
It uses lockfiles stored on the local filesystem (what was being discussed above) by default, but you can also configure remote lock storage:
LockMethod.config.storage = Redis.new([...]) # a remote RedisToGo instance, perhaps?
Also...
def the_method_my_cron_job_calls
# something really expensive
end
lock_method :the_method_my_cron_job_calls, (60*60) # automatically expire lock after an hour