Stale file descriptor with /dev/stdin - bash

I'm attempting to write a script to loop over entries in .ssh/authorized_keys and do things with them, namely print their fingerprint and append them to a new place. This is what I have so far:
echo "$SSH_KEYS" | while read key ; do
ssh-keygen -lf /dev/stdin <<< "$key"
echo "$key" >> newplace
done
This unfortunately gives me the following error:
/dev/stdin: Stale file handle
I'm running Bash 4.3.11 on Ubuntu 14.04 kernel 3.13.0-24-generic.
On the same kernel running Bash 4.3.8, it works fine. Changing my version of Bash doesn't look to be an option at this point, this is an automated script for something in production.
I found this solution in another question here on StackOverflow but it seems to not work with this later version of Bash.

I think you're on the right track, but you want something like:
while read key; do
ssh-keygen -lf /dev/stdin <<< "$key"
echo "$key" >> newplace
done < .ssh/authorized_keys
As opposed to:
echo "$SSH_KEYS" | while read key ; do
ssh-keygen -lf /dev/stdin <<< "$key"
echo "$key" >> newplace
done
Note that instead of piping the output of echo, simply feed the file directly into the stdin of the while loop.
This worked for me on:
$ bash --version
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
I'm also on Ubuntu 14.04, but it seems that someone has also maybe seen this problem: https://github.com/docker/docker/issues/2067
A work-around is to write each key to a tempfile, process it, then remove the file.

Related

Calling another variable inside a variable assignment [duplicate]

This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed last month.
My line of code is :
while IFS= read -r line
do
echo "$line"
echo "**"
CURL_REQ=$(curl -is "${line}" -H 'Pragma: akamai-x-get-extracted-values' | grep 'AKA_PM_PROPERTY_NAME')
echo "$line"
echo "$CURL_REQ"
After doing some trial an error, I now understood that inside variable declaration of "CURL_REQ"
${line} is not being recognized.
However the echo "$line" is working perfectly without any issues outside the variable declaration.
When I replace "${line}" with a static hostname, the curl works fine and I can see the value on echo "$CURL_REQ"
Reproducible example :
#!/bin/bash
INPUT_FILE="/domain.txt"
EXPECTED_HEADER="AKA_PM_PROPERTY_NAME"
declare -i COUNT=1
echo "*************************************************************************"
while IFS= read -r line
do
CURL_REQ=$(curl -is ${line} -H 'Pragma: akamai-x-get-extracted-values' | grep 'AKA_PM_PROPERTY_NAME')
echo "$line"
echo $CURL_REQ
PROP_VALUE=$(echo ${CURL_REQ} | cut -c 51-)
echo $PROP_VALUE
done < "$INPUT_FILE"
Input file domain.txt should contain the lines below :
myntra.com
ndtv.com
#mhs Based on your provided script, seems it is working for me - I have only changed the INPUT_FILE="/domain.txt" to INPUT_FILE="./domain.txt" to read the file from the same directory that I am running the script.
Here are the details, please compare them with yours and you'd find the issue on your side.
IMHO the domain.txt file's context or permissions (accessibility) should be the problem.
Note: The script.sh and the domain.txt files are both in the same directory (my user's home directory) and they have proper permissions relevant to user (my user) that I'm running the script
If you didn't find the issue, I'll suggest re-format your question and provide step by step description and details as I have in the following.
The script.sh file content I'm running:
#!/bin/bash
INPUT_FILE="./domain.txt"
EXPECTED_HEADER="AKA_PM_PROPERTY_NAME"
declare -i COUNT=1
echo "*************************************************************************"
while IFS= read -r line
do
CURL_REQ=$(curl -is ${line} -H 'Pragma: akamai-x-get-extracted-values' | grep 'AKA_PM_PROPERTY_NAME')
echo "$line"
echo $CURL_REQ
PROP_VALUE=$(echo ${CURL_REQ} | cut -c 51-)
echo $PROP_VALUE
done < "$INPUT_FILE"
The domain.txt file content:
myntra.com
ndtv.com
Here are the files permissions:
-rw-r--r-- 1 vrej vrej 20 Jan 5 17:18 domain.txt
-rwxr-xr-x 1 vrej vrej 440 Jan 5 17:18 script.sh
The output that I get is:
*************************************************************************
myntra.com
X-Akamai-Session-Info: name=AKA_PM_PROPERTY_NAME; value=www.myntra.com_ssl
value=www.myntra.com_ssl
ndtv.com
X-Akamai-Session-Info: name=AKA_PM_PROPERTY_NAME; value=www.ndtv.com-EON-1501
value=www.ndtv.com-EON-1501
My BASH version is:
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Bash Process Substitution with WSL

I am trying to understand the following (odd?) behavior on my setup. I am running Windows 10, with WSL.
Here is what I see from my powershell+wsl session:
$ ./run.sh
Windows IP Configuration
<cut>
1
with:
$ cat run.sh
#!/bin/bash
mkdir -p dir/1
mkdir -p dir/2
mkdir -p dir/3
var=0
while read i;
do
((var++))
ipconfig.exe
done < <(find dir -type d)
echo $var
If I now comment out the ipconfig.exe line:
...
#ipconfig.exe
...
Here is now what I get:
$ ./run.sh
4
Why calling a window executable (eg. ipconfig.exe) seems to interfere with my while loop using process substitution) ?
For reference:
$ uname -a
Linux FR-L0002146 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
and
$ bash -version
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Most likely ipconfig.exe is reading from stdin, so it's sucking up the input from the process substitution. Redirect its input to /dev/null to prevent this.
while read i;
do
((var++))
ipconfig.exe </dev/null
done < <(find dir -type d)

Why can't I pass these parameters to my command?

B=master U='my.email#email.com' curl -u "${U}" "https://bitbucket.org/myteam/pod-dev/raw/${B}/install.bash"
I get asked:
Enter host password for user '':
See, the "U" is missing.
And also curl is performed:
GET /myteam/pod-dev/raw//install.bash HTTP/1.1
Also here, the B is missing (branch)
macOS Sierra 10.12.2
$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
Copyright (C) 2007 Free Software Foundation, Inc.
Variables assignments like you are using them are part of the command. In your case the shell will try to expand "${A}", "${B}" before they have been actually set - during parsing the command.
You can separate the variable assignments and the actual command by ;:
B=master; U='my.email#email.com'; curl -u "${U}" "https://bitbucket.org/myteam/pod-dev/raw/${B}/install.bash"
That way they are 3 separate commands.
bash will try to expand the variables first, but you haven't set them at this point. See for example:
$ x=foo echo "$x"
>
$ x=foo; echo "$x"
> foo
Try like this instead of your command:
export B=master; export U='my.email#email.com' ; curl -u "$U" "https://bitbucket.org/myteam/pod-dev/raw/$B/install.bash"

Pipes inside backticks fail under MinGW

When I try to run some Bash scripts on a Windows XP machine using MinGW, I get the following error:
./autogen.sh: pipe error: No such file or directory
I have localised the problem to Bash lines such as the following, which have a pipe inside backticks:
__copyright="`grep Copyright ./autogen.sh | head -1`"
However, pipes not inside backticks work just fine:
grep Copyright ./autogen.sh | head -1
All the programs you expect (sh, head, grep) are available and run happily on the command line and in Bash.
What should I do to resolve this error? I cannot run most Bash build tools without it.
$ msysinfo | head -3
msysinfo-1.3: Send this to the MSYS support list:
MSYS 1.0.11(0.46/3/2) 2004-04-30 18:55 i686 unknown; targ=MINGW32
$ echo '__copyright="`grep Copyright ./autogen.sh | head -1`" && echo $__copyright' >test.sh
$ cat test.sh
__copyright="`grep Copyright ./autogen.sh | head -1`" && echo $__copyright
$ sh test.sh
# Copyright 2005, 2006, 2007, 2008 by
For this test, I copied autogen.sh from some place like... randomly... http://svn.ghostscript.com/ghostscript/tags/freetype-2.3.7/autogen.sh
Probably this means your question needs more information.
But... sometimes when I run into snarly scenarios, it can help to enclose the breaking code in ( ). Technically, also, you do not need " " around backticks, and that complicates some use, but your example does not seem problematic.
I really hate to propose the following as it seems completely unnecessary:
$ for item in `grep Copyright ./autogen.sh`;
do
__copyright="$__copyright $item";
done;
echo $__copyright
What is more weird is that your error message seems to imply autogen.sh itself generated the error as though autogen.sh was grepping itself so I did:
$ sh test.sh
__copyright="`grep Copyright ./test.sh | head -1`" && echo $__copyright
after modifying test.sh to grep itself, and even that worked.
It therefore sounds like a sequencing issue and not a backtick issue, you know, which came first, the chicken or the egg?
Is autogen.sh trying to read its own Copyright comment? One way to let comments serve as data is to wrap them in here documents:
_Copyright ()
{
cat <<-END_OF_TEXT
# Copyright 2012, me
END_OF_TEXT
}
_Copyright

How to check the ls version

This topic is about the util 'ls'
The BSD version uses the parameter '-G' to color up the output,
while the Linux version uses parameter '--color'
Also the environment variable to set the colors is different:
BSD: $LSCOLORS
Linux: $LS_COLORS
But now the problem is: I want to determine which version is installed (using a small Shell script), so I can set alias ls and the environment appropriate in my .bachrc file.
As I mentioned above this seems to me to be the handiest method
if ls --color -d . >/dev/null 2>&1; then
GNU_LS=1
elif ls -G -d . >/dev/null 2>&1; then
BSD_LS=1
else
SOLARIS_LS=1
fi
I've essentially this in my l script, which I use on various platforms to tweak ls output as I like
Just run 'ls' and see whether it throws an error, e.g. on my mac:
$ ls --color 1>/dev/null 2>&1
$ echo $?
1
Whereas
$ ls -G 1>/dev/null 2>&1
$ echo $?
0
Indicating -G is supported, but --color is not.
Ironically, the --version switch Kimmo mentions is not supported on most BSD systems :-)
Writing a portable configuration file for your particular setup can be a Herculean task. In your case, if you're sure your .bashrc is going to be used only on GNU/Linux and on a BSD system, you can check for switches that exist in one of the ls' but not in the other: for example, -D doesn't seem to be an accepted switch by ls on my BSD machines (FreeBSD and Mac OS X), whereas it is for GNU ls. Conversely, -P is accepted on BSD, but not on GNU/Linux. Knowing this, you can distinguish between the two ls' and set up environment variables accordingly.
$ ls --version
ls (GNU coreutils) 6.10
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Richard Stallman and David MacKenzie.
combining the methods described, here's an easy way to use a bash function instead of an alias in order to make colors work regardless of if you are using BSD or GNU ls.
ll () {
if ls --version &>/dev/null; then
ls --color=auto -lahtr
else
ls -Glahtr
fi
}
inspired by a particular conda env recipe pulling in GNU ls on my macOS system where my ls aliases were all hard-coded for stock BSD ls only.

Resources