Is there a shell command to delay a buffer? - bash

Question

I am looking for a shell command X such as, when I execute:
command_a | X 5000 | command_b
the stdout of command_a is written in stdin of command_b (at least) 5 seconds later.
A kind of delaying buffer.
As far as I know, buffer/mbuffer can write at constant rate (a fixed number of bytes per second). Instead, I would like a constant delay in time (t=0 is when X read a command_a output chunk, at t=5000 it must write this chunk to command_b).
[edit] I've implemented it: https://github.com/rom1v/delay

Answer 1

I know you said you're looking for a shell command, but what about using a subshell to your advantage? Something like:
command_a | (sleep 5; command_b)
So to grep a file cat-ed through (I know, I know, bad use of cat, but just an example):
cat filename | (sleep 5; grep pattern)
A more complete example:
$ cat testfile
The
quick
brown
fox
$ cat testfile | (sleep 5; grep brown)
# A 5-second sleep occurs here
brown
Or even, as Michale Kropat recommends, a group command with sleep would also work (and is arguably more correct). Like so:
$ cat testfile | { sleep 5; grep brown; }
Note: don't forget the semicolon after your command (here, the grep brown), as it is necessary!

Answer 2

As it seemed such a command dit not exist, I implemented it in C:
https://github.com/rom1v/delay
delay [-b <dtbufsize>] <delay>

Answer 3

Something like this?
#!/bin/bash
while :
do
read line
sleep 5
echo $line
done
Save the file as "slowboy", then do
chmod +x slowboy
and run as
command_a | ./slowboy | command_b

Answer 4

This might work
time_buffered () {
delay=$1
while read line; do
printf "%d %s\n" "$(date +%s)" "$line"
done | while read ts line; do
now=$(date +%s)
if (( now - ts < delay)); then
sleep $(( now - ts ))
fi
printf "%s\n" "$line"
done
}
commandA | time_buffered 5 | commandB
The first loop tags each line of its input with a timestamp and immediately feeds it to the second loop. The second loop checks the timestamp of each line, and will sleep if necessary until $delay seconds after it was first read before outputting the line.

Answer 5

Your question intrigued me, and I decided to come back and play with it. Here is a basic implementation in Perl. It's probably not portable (ioctl), tested on Linux only.
The basic idea is:
read available input every X microseconds
store each input chunk in a hash, with current timestamp as key
also push current timestamp on a queue (array)
lookup oldest timestamps on queue and write + discard data from the hash if delayed long enough
repeat
Max buffer size
There is a max size for stored data. If reached, additional data will not be read until space becomes available after writing.
Performance
It is probably not fast enough for your requirements (several Mb/s). My max throughput was 639 Kb/s, see below.
Testing
# Measure max throughput:
$ pv < /dev/zero | ./buffer_delay.pl > /dev/null
# Interactive manual test, use two terminal windows:
$ mkfifo data_fifo
terminal-one $ cat > data_fifo
terminal-two $ ./buffer_delay.pl < data_fifo
# now type in terminal-one and see it appear delayed in terminal-two.
# It will be line-buffered because of the terminals, not a limitation
# of buffer_delay.pl
buffer_delay.pl
#!/usr/bin/perl
use strict;
use warnings;
use IO::Select;
use Time::HiRes qw(gettimeofday usleep);
require 'sys/ioctl.ph';
$|++;
my $delay_usec = 3 * 1000000; # (3s) delay in microseconds
my $buffer_size_max = 10 * 1024 * 1024 ; # (10 Mb) max bytes our buffer is allowed to contain.
# When buffer is full, incoming data will not be read
# until space becomes available after writing
my $read_frequency = 10; # Approximate read frequency in Hz (will not be exact)
my %buffer; # the data we are delaying, saved in chunks by timestamp
my #timestamps; # keys to %buffer, used as a queue
my $buffer_size = 0; # num bytes currently in %buffer, compare to $buffer_size_max
my $time_slice = 1000000 / $read_frequency; # microseconds, min time for each discrete read-step
my $sel = IO::Select->new([\*STDIN]);
my $overflow_unread = 0; # Num bytes waiting when $buffer_size_max is reached
while (1) {
my $now = sprintf "%d%06d", gettimeofday; # timestamp, used to label incoming chunks
# input available?
if ($overflow_unread || $sel->can_read($time_slice / 1000000)) {
# how much?
my $available_bytes;
if ($overflow_unread) {
$available_bytes = $overflow_unread;
}
else {
$available_bytes = pack("L", 0);
ioctl (STDIN, FIONREAD(), $available_bytes);
$available_bytes = unpack("L", $available_bytes);
}
# will it fit?
my $remaining_space = $buffer_size_max - $buffer_size;
my $try_to_read_bytes = $available_bytes;
if ($try_to_read_bytes > $remaining_space) {
$try_to_read_bytes = $remaining_space;
}
# read input
if ($try_to_read_bytes > 0) {
my $input_data;
my $num_read = read (STDIN, $input_data, $try_to_read_bytes);
die "read error: $!" unless defined $num_read;
exit if $num_read == 0; # EOF
$buffer{$now} = $input_data; # save input
push #timestamps, $now; # save the timestamp
$buffer_size += length $input_data;
if ($overflow_unread) {
$overflow_unread -= length $input_data;
}
elsif (length $input_data < $available_bytes) {
$overflow_unread = $available_bytes - length $input_data;
}
}
}
# write + delete any data old enough
my $then = $now - $delay_usec; # when data is old enough
while (scalar #timestamps && $timestamps[0] < $then) {
my $ts = shift #timestamps;
print $buffer{$ts} if defined $buffer{$ts};
$buffer_size -= length $buffer{$ts};
die "Serious problem\n" unless $buffer_size >= 0;
delete $buffer{$ts};
}
# usleep any remaining time up to $time_slice
my $time_left = (sprintf "%d%06d", gettimeofday) - $now;
usleep ($time_slice - $time_left) if $time_slice > $time_left;
}
Feel free to post comments and suggestions below!