Shell script pipeline: different behavior with or without xclip - bash

I have a script named password-for-object which I normally run like that:
$ password-for-object example.com
sOtzC0UY1K3EDYp8a6ltfA
I.e. it does an intricate hash calculation and outputs a password that I should use when accessing an object (for example, website) named example.com. I'll just double click the whole password, it gets copied into my buffer and I'll paste it into the form.
I've also learnt a trick on how to use such a script without making my password visible:
$ password-for-object example.com | xclip
This way output of a script ends up in X's primary buffer and I can insert it right into password field in the form and it's not shown on the screen.
The only problem with this way is that password-for-object outputs a string with trailing newline and thus "xclip" always catches up an extra symbol - this newline. If I omit output of newline in password-for-object, then I'll end up with messed up string without xclip, i.e. when I'm just putting it on the stdout. I use 2 shells: zsh and bash, and I'll get the following in zsh (note the extra % sign):
$ password-for-object example.com
sOtzC0UY1K3EDYp8a6ltfA%
$
Or the following in bash (note that prompt would be started on the same line):
$ password-for-object example.com
sOtzC0UY1K3EDYp8a6ltfA$
Any ideas on how to work around this issue? Is it possible to modify the script in a way so it will detect that xclip is in the pipeline and only output newline if it isn't?

If you change password-for-object so that it doesn't output a newline, you can call it with a script like:
#!/bin/bash
password-for-object "$1"
if [ -t 1 ]
then
echo
fi
The -t condition is described in the bash manual as:
-t fd
True if file descriptor fd is open and refers to a terminal.
See the following question:
How to detect if my shell script is running through a pipe?

Give this a try:
$ password-for-object example.com | tr -d '\n' | xclip
tr -d '\n' deletes the newline

Related

Bash get the command that is piping into a script

Take the following example:
ls -l | grep -i readme | ./myscript.sh
What I am trying to do is get ls -l | grep -i readme as a string variable in myscript.sh. So essentially I am trying to get the whole command before the last pipe to use inside myscript.sh.
Is this possible?
No, it's not possible.
At the OS level, pipelines are implemented with the mkfifo(), dup2(), fork() and execve() syscalls. This doesn't provide a way to tell a program what the commands connected to its stdin are. Indeed, there's not guaranteed to be a string representing a pipeline of programs being used to generate stdin at all, even if your stdin really is a FIFO connected to another program's stdout; it could be that that pipeline was generated by programs calling execve() and friends directly.
The best available workaround is to invert your process flow.
It's not what you asked for, but it's what you can get.
#!/usr/bin/env bash
printf -v cmd_str '%q ' "$#" # generate a shell command representing our arguments
while IFS= read -r line; do
printf 'Output from %s: %s\n' "$cmd_str" "$line"
done < <("$#") # actually run those arguments as a command, and read from it
...and then have your script start the things it reads input from, rather than receiving them on stdin.
...thereafter, ./yourscript ls -l, or ./yourscript sh -c 'ls -l | grep -i readme'. (Of course, never use this except as an example; see ParsingLs).
It can't be done generally, but using the history command in bash it can maybe sort of be done, provided certain conditions are met:
history has to be turned on.
Only one shell has been running, or accepting new commands, (or failing that, running myscript.sh), since the start of myscript.sh.
Since command lines with leading spaces are, by default, not saved to the history, the invoking command for myscript.sh must have no leading spaces; or that default must be changed -- see Get bash history to remember only the commands run with space prefixed.
The invoking command needs to end with a &, because without it the new command line wouldn't be added to the history until after myscript.sh was completed.
The script needs to be a bash script, (it won't work with /bin/dash), and the calling shell needs a little prep work. Sometime before the script is run first do:
shopt -s histappend
PROMPT_COMMAND="history -a; history -n"
...this makes the bash history heritable. (Code swiped from unutbu's answer to a related question.)
Then myscript.sh might go:
#!/bin/bash
history -w
printf 'calling command was: %s\n' \
"$(history | rev |
grep "$0" ~/.bash_history | tail -1)"
Test run:
echo googa | ./myscript.sh &
Output, (minus the "&" associated cruft):
calling command was: echo googa | ./myscript.sh &
The cruft can be halved by changing "&" to "& fg", but the resulting output won't include the "fg" suffix.
I think you should pass it as one string parameter like this
./myscript.sh "$(ls -l | grep -i readme)"
I think that it is possible, have a look at this example:
#!/bin/bash
result=""
while read line; do
result=$result"${line}"
done
echo $result
Now run this script using a pipe, for example:
ls -l /etc | ./script.sh
I hope that will be helpful for you :)

Why doesn't LIMIT=\`ulimit -u\` work in bash?

In my program I need to know the maximum number of process I can run. So I write a script. It works when I run it in shell but but when in program using system("./limit.sh"). I work in bash.
Here is my code:
#/bin/bash
LIMIT=\`ulimit -u\`
ACTIVE=\`ps -u | wc -l \`
echo $LIMIT > limit.txt
echo $ACTIVE >> limit.txt
Anyone can help?
Why The Original Fails
Command substitution syntax doesn't work if escaped. When you run:
LIMIT=\`ulimit -u\`
...what you're doing is running a command named
-u`
...with the environment variable named LIMIT containing the value
`ulimit
...and unless you actually have a command that starts with -u and contains a backtick in its name, this can be expected to fail.
This is because using backticks makes characters which would otherwise be syntax into literals, and running a command with one or more var=value pairs preceding it treats those pairs as variables to export in the environment for the duration of that single command.
Doing It Better
#!/bin/bash
limit=$(ulimit -u)
active=$(ps -u | wc -l)
printf '%s\n' "$limit" "$active" >limit.txt
Leave off the backticks.
Use modern $() command substitution syntax.
Avoid multiple redirections.
Avoid all-caps names for your own variables (these names are used for variables with meaning to the OS or system; lowercase names are reserved for application use).
Doing It Right
#!/bin/bash
exec >limit.txt # open limit.txt as output for the rest of the script
ulimit -u # run ulimit -u, inheriting that FD for output
ps -u | wc -l # run your pipeline, likewise with output to the existing FD
You have a typo on the very first line: #/bin/bash should be #!/bin/bash - this is often known as a "shebang" line, for "hash" (#) + "bang" (!)
Without that syntax written correctly, the script is run through the system's default shell, which will see that line as just a comment.
As pointed out in comments, that also means only the standardised options available to the builtin ulimit command, which doesn't include -u.

fgrep with string containing spaces inside ksh script

I am trying to write an fgrep statement removing records with a full record match from a file. I can do this on the command line, but not inside a ksh script. The code I am using boils down to these 4 lines of code:
Header='abc def|ghi jkl' #I use the head command to populate this variable
workfile=abc.txt
command="fgrep -Fxv \'$Header\' $workfile" >$outfile
$command
When I echo $command to STDIN the command is exactly what I would type on the command line (with the single quotes) and that works on the command line. When I execute it within the ksh script (file) the single quotes seem not to be recognized because the errors show it is parsing on spaces.
I have tried back ticks, exec, eval, double quotes instead of single quotes, and not using the $command variable. The problem remains.
I can do this on the command line, but not inside a ksh script
Here's a simple, portable, reliable solution using a heredoc.
#!/usr/bin/env ksh
workfile=abc.txt
outfile=out.txt
IFS= read -r Header <<'EOF'
abc def|ghi jul
EOF
IFS= read -r command <<'EOF'
grep -Fxv "$Header" "$workfile" > "$outfile"
EOF
eval "$command"
Explanation :
(Comments can't be added to the script above because they would affect the lines in the heredoc)
IFS= read -r Header <<'EOF' # Line separated literal strings
abc def|ghi jul # Set into the $Header variable
EOF # As if it were a text file
IFS= read -r command <<'EOF' # Command to execute
grep -Fxv "$Header" "$workfile" > "$outfile" # As if it were typed into
EOF # the shell command line
eval "$command" # Execute the command
The above example is the same as having a text file called header.txt, which contains the contents: abc def|ghi jul and typing the following command:
grep -Fxvf header.txt abc.txt
The heredoc addresses the problem of the script operating differently than the command line as a result of quoting/expansions/escaping issues.
A Word of caution regarding eval:
The use of eval in this example is specific. Please see Eval command and security issues for information on how eval can be misused and cause potentially very damaging results.
More Detail / Alternate Example:
For the sake of completeness, clarity, and ability to apply this concept to other situations, some notes about the heredoc and an alternative demonstration:
This implementation of the heredoc in this example is specifically designed with the following criteria:
Literal string assignment of contents, to the variables (using 'EOF')
Use of the eval command to evaluate and execute the referenced variables within the heredoc itself.
File or heredoc ?
One strength of using a heredoc combined with grep -F (fgrep), is the ability to treat a section of the script as if it were a file.
Case for file:
You want to frequently paste "pattern" lines into the file, and remove them as necessary, without having to modify the script file.
Case for heredoc:
You apply the script in an environment where specific files already exist, and you want to match specific exact literal patterns against it.
Example:
Scenario: I have 5 VPS Servers, and I want a script to produce a new fstab file but to ensure it doesn't contain the exact line:
/dev/xvda1 / ext3 errors=remount-ro,noatime,barrier=0 0 1
This scenario fits the type of situation addressed in this question. I could use the boilerplate from the above code in this answer and modify it as following:
#!/usr/bin/env ksh
workfile=/etc/fstab
IFS= read -r Header <<'EOF'
/dev/xvda1 / ext3 errors=remount-ro,noatime,barrier=0 0 1
EOF
IFS= read -r command <<'EOF'
grep -Fxv "$Header" "$workfile"
EOF
eval "$command"
This would give me a new fstab file, without the line contained in the heredoc.
Bash FAQ #50: I'm trying to put a command in a variable, but the complex cases always fail! provides comprehensive guidance - while it is written for Bash, most of it applies to Ksh as well.[1]
If you want to stick with storing your command in a variable (defining a function is the better choice), use an array, which bypasses the quoting issues:
#!/usr/bin/env ksh
Header='abc def|ghi jkl'
workfile=abc.txt
# Store command and arguments as elements of an array
command=( 'fgrep' '-Fxv' "$Header" "$workfile" )
# Invoke the array as a command.
"${command[#]}" > "$outfile"
Note: only a simple command can be stored in an array, and redirections can't be part of it.
[1] The function examples use local to create local variables, which ksh doesn't support. Omit local to make do with shell-global variables instead, or use function <name> {...} syntax with typeset instead of local to declare local variables in ksh.

How to execute lines of text on the clipboard as bash commands

I'm working with Mac OS X's pbpaste command, which returns the clipboard's contents. I'd like to create a shell script that executes each line returned by pbpaste as a separate bash command. For example, let's say that the clipboard's contents consists of the following lines of text:
echo 1234 >~/a.txt
echo 5678 >~/b.txt
I would like a shell script that executes each of those lines, creating the two files a.txt and b.txt in my home folder. After a fair amount of searching and trial and error, I've gotten to the point where I'm able to assign individual lines of text to a variable in a while loop with the following construct:
pbpaste | egrep -o [^$]+ | while read l; do echo $l; done
which sends the following to standard out, as expected:
echo 1234 >~/a.txt
echo 5678 >~/b.txt
Instead of simply echoing each line of text, I then try to execute them with the following construct:
pbpaste | egrep -o [^$]+ | while read l; do $l; done
I thought that this would execute each line (thus creating two text files a.txt and b.txt in my home folder). Instead, the first term (echo) seems to be interpreted as the command, and the remaining terms (nnnn >~/...) seem to get lumped together as if they were a single parameter, resulting in the following being sent to standard out without any files being created:
1234 >~/a.txt
5678 >~/b.txt
I would be grateful for any help in understanding why my construct isn't working and what changes might get it to work.
[…] the remaining terms (nnnn >~/...) seem to get lumped together as if they were a single parameter, […]
Not exactly. The line actually gets split on whitespace (or whatever $IFS specifies), but the problem is that the redirection operator > cannot be taken from a shell variable. For example, this snippet:
gt='>'
echo $gt foo.txt
will print > foo.txt, rather than printing a newline to foo.txt.
And you'll have similar problems with various other shell metacharacters, such as quotation marks.
What you need is the eval builtin, which takes a string, parses it as a shell command, and runs it:
pbpaste | egrep -o [^$]+ | while IFS= read -r LINE; do eval "$LINE"; done
(The IFS= and -r and the double-quotes around $LINE are all to prevent any other processing besides the processing performed by eval, so that e.g. whitespace inside quotation marks will be preserved.)
Another possibility, depending on the details of what you need, is simply to pipe the commands into a new instance of Bash:
pbpaste | egrep -o [^$]+ | bash
Edited to add: For that matter, it occurs to me that you can pass everything to eval in a single batch; just as you can (per your comment) write pbpaste | bash, you can also write eval "$(pbpaste)". That will support multiline while-loops and so on, while still running in the current shell (useful if you want it to be able to reference shell parameters, to set environment variables, etc., etc.).

Copy current command at bash prompt to clipboard

I would like a quick keyboard command sequence to copy the current command at a bash prompt to the clipboard.
So that, for example, to copy the last bash command to the clipboard, I'd press up+[some command sequence] to copy it. Or, for example, to search for a command in bash hisory, I'd use ctrl+r, search, display it on the command prompt, and then [some command sequence] to copy it, etc.
My current solution is using bash pipes: Pipe to/from the clipboard
So, to copy the previous command to clipboard:
echo "!!" | pbcopy
Which isn't too terrible, but what if the command to copy isn't the last command, etc.
What's the proper way to achieve what I'm trying to achieve here?
Taking #Lauri's post for inspiration, here's a solution using the bind command:
bind '"\C-]":"\C-e\C-u pbcopy <<"EOF"\n\C-y\nEOF\n"'
ctrl-] then will copy whatever is on the current bash prompt to the clipboard.
To make it persistent, you can add the bind command as above to your ~/.bashrc, or you can strip off the outer quotes and remove the 'bind' part of the call and add the result to your ~/.inputrc.
Non-OS-X users will have to swap pbcopy out with the appropriate command, probably xclip.
A quoted heredoc was used instead of a an echo+pipe technique so that both single and double quotes in the command at the bash prompt are preserved. With this technique, for example, I was able to hit ctrl-], copy the actual bind command from the terminal prompt, and paste it here in the answer. So the heredoc technique handles all of the special characters in the bind command here.
You can use READLINE_LINE with bind -x in bash 4:
copyline() { printf %s "$READLINE_LINE"|pbcopy; }
bind -x '"\C-xc":copyline'
You can install bash 4 and make it the default login shell by running brew install bash;echo /usr/local/bin/bash|sudo tee -a /etc/shells;chsh -s /usr/local/bin/bash.
I also use this function to copy the last command:
cl() { history -p '!!'|tr -d \\n|pbcopy; }
I spent a decent amount of time today writing a simple zsh implementation for macOS; usage is as follows:
example command: git commit -m "Changed a few things"
command that copies: c git commit -m "Changed a few things"
# The second command does not actually execute the command, it just copies it.
# Using zsh, this should reduce the whole process to about 3 keystrokes:
#
# 1) CTRL + A (to go to the beginning of the line)
# 2) 'c' + ' '
# 3) ENTER
preexec() is a zsh hook function that gets called right when you press enter, but before the command actually executes.
Since zsh strips arguments of certain characters like ' " ', we will want to use preexec(), which allows you to access the unprocessed, original command.
Pseudocode goes like this:
1) Make sure the command has 'c ' in the beginning
2) If it does, copy the whole command, char by char, to a temp variable
3) Pipe the temp variable into pbcopy, macOS's copy buffer
Real code:
c() {} # you'll want this so that you don't get a command unrecognized error
preexec() {
tmp="";
if [ "${1:0:1}" = "c" ] && [ "${1:1:1}" = " " ] && [ "${1:2:1}" != " " ]; then
for (( i=2; i<${#1}; i++ )); do
tmp="${tmp}${1:$i:1}";
done
echo "$tmp" | pbcopy;
fi
}
Go ahead and stick the two aforementioned functions in your .zshrc file, or wherever you want (I put mine in a file in my .oh-my-zsh/custom directory).
If anyone has a more elegant solution, plz speak up.
Anything to avoid using the mouse.
If xsel is installed on your system you can add this in .inputrc :
C-]: '\C-e\C-ucat <<"EOF" | tr -d "\\n" | xsel -ib\n\C-y\nEOF\n'
Alternatively, if xclip is installed you could add this:
C-]: '\C-e\C-ucat <<"EOF" | tr -d "\\n" | xclip -se c\n\C-y\nEOF\n'
Notice: Used code from #Clayton's answer.
I use history to find the command number that I am looking for, then I do:
echo "!command_number" | xclip -in
$ history | cut -c 8- | tail -1 | pbcopy
or in .zshrc file add an alias
alias copy='history | cut -c 8- | tail -1 | pbcopy'

Resources