The ash shell in busybox doesn't seem to implement any of the standard ways to get the filename that's being sourced. For instance:
testo:
#!/usr/bin/env -S busybox ash
echo hello whorl
echo using source
source ./sourceme
echo using .
. ./sourceme
sourceme:
echo underscore $_
echo bs $BASH_SOURCE
echo zero $0
# ./testo
hello whorl
using source
underscore ./testo
bs
zero ./testo
using .
underscore ./testo
bs
zero ./testo
I need something to put in sourceme that will get its own name/path.
This excellent answer contains a very clever way to accomplish this. I've adapted their solution here - insert this line into sourceme:
echo lsof `lsof -p $$ -Fn | tail -n1 | sed 's!^[^/]*!!g'``
And you get:
lsof /absolute/path/to/sourceme
Note: since we're talking about busybox here, this is an implementation of the above using busybox's lsof:
lsof | grep '^'$$ | tail -n1 | awk '{print $3}'
Note: if anyone finds a way to do this using some builtin mechanism from busybox ash, post an answer and I'll change the accepted answer to yours.
Related
Disclaimer: this happens on macOS (Big Sur); more info about the context below.
I have to write (almost did) a script which will replace images URLs in big text (xml) files by their Base64-encoded value.
The script should run the same way with single filenames or patterns, or both, e.g.:
./replace-encode single.xml
./replace-encode pattern*.xml
./replace-encode single.xml pattern*.xml
./replace-encode folder/*.xml
Note: it should properly handle files\ with\ spaces.xml
So I ended up with this script:
#!/bin/bash
#needed for `ls` command
IFS=$'\n'
ls -1 $* | xargs -I % sed -nr 's/.*>(https?:\/\/[^<]+)<.*/\1/p' % | xargs -tI % sh -c 'sed -i "" "s#%#`curl -s % | base64`#" $0' "$*"
What it does: ls all files, pipe the list to xargs then search all URLs surrounded by anchors (hence the > and < in the search expr. - also had to use sed because grep is limited on macOS), then pipe again to a sh script which runs the sed search & replace, where the remplacement is the big Base64 string.
This works perfectly fine... but only for fileswithoutspaces.xml
I tried to play with $0 vs $1, $* vs $#, w/ or w/o " but to no avail.
I don't understand exactly how does the variable substitution (is it how it's called? - not a native English speaker, and above all, not a script-writer at all!!! just a Java dev. all day long...) work between xargs, sh or even bash with arguments like filenames.
The xargs -t is here to let me check out how the substitution works, and that's how I noticed that using a pattern worked but I have to let the " around the last $*, otherwise only the 1st file is searched & replaced; output is like:
user#host % ./replace-encode pattern*.xml
sh -c sed -i "" "s#https://www.some.com/public/123456.jpg#`curl -s https://www.some.com/public/123456.jpg | base64`#" $0 pattern_123.xml
pattern_456.xml
Both pattern_123.xml and pattern_456.xml are handled here; w/ $* instead of "$*" in the end of the command, only pattern_123.xml is handled.
So is there a simple way to "fix" this?
Thank you.
Note: macOS commands have some limitations (I know) but as this script is intended to non-technical users, I can't ask them to install (or have the IT team installed on their behalf) some alternate GNU-versions installed e.g. pcregrep or 'ggrep' like I've read many times...
Also: I don't intend to change from xargs to for loops or so because, 1/ don't have the time, 2/ might want to optimize the 2nd step where some URLs might be duplicate or so.
There's no reason for your software to use ls or xargs, and certainly not $*.
./replace-encode single.xml
./replace-encode pattern*.xml
./replace-encode single.xml pattern*.xml
./replace-encode folder/*.xml
...will all work fine with:
#!/usr/bin/env bash
while IFS= read -r line; do
replacement=$(curl -s "$line" | base64)
in="$line" out="$replacement" perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' "$#"
done < <(sed -nr 's/.*>(https?:\/\/[^<]+)<.*/\1/p' "$#" | sort | uniq)
Finally ended up with this single-line script:
sed -nr 's/.*>(https?:\/\/[^<]+)<.*/\1/p' "$#" | xargs -I% sh -c 'sed -i "" "s#%#`curl -s % | base64`#" "$#"' _ "$#"
which does properly support filenames with or without spaces.
I want to write a script that loops through the output (array possibly?) of a shell command, ps.
Here is the command and the output:
$ ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh
3089 python /var/www/atm_securit 37:02
17116 python /var/www/atm_securit 00:01
17119 python /var/www/atm_securit 00:01
17122 python /var/www/atm_securit 00:01
17125 python /var/www/atm_securit 00:00
Convert it into bash script (snippet):
for tbl in $(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh)
do
echo $tbl
done
But the output becomes:
3089
python
/var/www/atm_securit
38:06
17438
python
/var/www/atm_securit
00:02
17448
python
/var/www/atm_securit
00:01
How do I loop through every row like in the shell output, but in a bash script?
Never for loop over the results of a shell command if you want to process it line by line unless you are changing the value of the internal field separator $IFS to \n. This is because the lines will get subject of word splitting which leads to the actual results you are seeing. Meaning if you for example have a file like this:
foo bar
hello world
The following for loop
for i in $(cat file); do
echo "$i"
done
gives you:
foo
bar
hello
world
Even if you use IFS='\n' the lines might still get subject of Filename expansion
I recommend to use while + read instead because read reads line by line.
Furthermore I would use pgrep if you are searching for pids belonging to a certain binary. However, since python might appear as different binaries, like python2.7 or python3.4 I suggest to pass -f to pgrep which makes it search the whole command line rather than just searching for binaries called python. But this will also find processes which have been started like cat foo.py. You have been warned! At the end you can refine the regex passed to pgrep like you wish.
Example:
pgrep -f python | while read -r pid ; do
echo "$pid"
done
or if you also want the process name:
pgrep -af python | while read -r line ; do
echo "$line"
done
If you want the process name and the pid in separate variables:
pgrep -af python | while read -r pid cmd ; do
echo "pid: $pid, cmd: $cmd"
done
You see, read offers a flexible and stable way to process the output of a command line-by-line.
Btw, if you prefer your ps .. | grep command line over pgrep use the following loop:
ps -ewo pid,etime,cmd | grep python | grep -v grep | grep -v sh \
| while read -r pid etime cmd ; do
echo "$pid $cmd $etime"
done
Note how I changed the order of etime and cmd. Thus to be able to read cmd, which can contain whitespace, into a single variable. This works because read will break down the line into variables, as many times as you specified variables. The remaining part of the line - possibly including whitespace - will get assigned to the last variable which has been specified in the command line.
I found you can do this just use double quotes:
while read -r proc; do
#do work
done <<< "$(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh)"
This will save each line to the array rather than each item.
When using for loops in bash it splits the given list by default by whitespaces, this can be adapted by using the so called Internal Field Seperator, or IFS in short .
IFS The Internal Field Separator that is used for word splitting after
expansion and to split lines into words with the read builtin command.
The default value is "".
For your example we would need to tell IFS to use new-lines as break point.
IFS=$'\n'
for tbl in $(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh)
do
echo $tbl
done
This example returns the following output on my machine.
668 /usr/bin/python /usr/bin/ud 03:05:54
27892 python 00:01
Here is another bash-based solution, inspired by comment of #Gordon Davisson.
For this we need (atleast bash v1.13.5 (1992) or later verison), because Process-Substitution2,3,4 while read var; do { ... }; done < <(...);, etc are used.
#!/bin/bash
while IFS= read -a oL ; do { # reads single/one line
echo "${oL}"; # prints that single/one line
};
done < <(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh);
unset oL;
Note: You can use any simple or complex command/command-set inside the <(...) which may have multiple output lines.
And what code does what function is shown here.
And here is a single/one-liner way:
while IFS= read -a oL ; do { echo "${oL}"; }; done < <(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh); unset oL;
( As Process-Substitution is not part of POSIX yet So its not supported in many POSIX compliant shell or in POSIX shell mode of bash-shell. Process-Substitution existed in bash since 1992 (so that is 28yrs ago from now/2020), & existed in ksh86 (before 1985)1. So POSIX should've included it. )
If you or any user wants to use something similar as Process-Substitution in POSIX compliant shell (i.e: sh, ash, dash, pdksh/mksh, etc), then look into NamedPipes.
I want to write a script that loops through the output (array possibly?) of a shell command, ps.
Here is the command and the output:
$ ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh
3089 python /var/www/atm_securit 37:02
17116 python /var/www/atm_securit 00:01
17119 python /var/www/atm_securit 00:01
17122 python /var/www/atm_securit 00:01
17125 python /var/www/atm_securit 00:00
Convert it into bash script (snippet):
for tbl in $(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh)
do
echo $tbl
done
But the output becomes:
3089
python
/var/www/atm_securit
38:06
17438
python
/var/www/atm_securit
00:02
17448
python
/var/www/atm_securit
00:01
How do I loop through every row like in the shell output, but in a bash script?
Never for loop over the results of a shell command if you want to process it line by line unless you are changing the value of the internal field separator $IFS to \n. This is because the lines will get subject of word splitting which leads to the actual results you are seeing. Meaning if you for example have a file like this:
foo bar
hello world
The following for loop
for i in $(cat file); do
echo "$i"
done
gives you:
foo
bar
hello
world
Even if you use IFS='\n' the lines might still get subject of Filename expansion
I recommend to use while + read instead because read reads line by line.
Furthermore I would use pgrep if you are searching for pids belonging to a certain binary. However, since python might appear as different binaries, like python2.7 or python3.4 I suggest to pass -f to pgrep which makes it search the whole command line rather than just searching for binaries called python. But this will also find processes which have been started like cat foo.py. You have been warned! At the end you can refine the regex passed to pgrep like you wish.
Example:
pgrep -f python | while read -r pid ; do
echo "$pid"
done
or if you also want the process name:
pgrep -af python | while read -r line ; do
echo "$line"
done
If you want the process name and the pid in separate variables:
pgrep -af python | while read -r pid cmd ; do
echo "pid: $pid, cmd: $cmd"
done
You see, read offers a flexible and stable way to process the output of a command line-by-line.
Btw, if you prefer your ps .. | grep command line over pgrep use the following loop:
ps -ewo pid,etime,cmd | grep python | grep -v grep | grep -v sh \
| while read -r pid etime cmd ; do
echo "$pid $cmd $etime"
done
Note how I changed the order of etime and cmd. Thus to be able to read cmd, which can contain whitespace, into a single variable. This works because read will break down the line into variables, as many times as you specified variables. The remaining part of the line - possibly including whitespace - will get assigned to the last variable which has been specified in the command line.
I found you can do this just use double quotes:
while read -r proc; do
#do work
done <<< "$(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh)"
This will save each line to the array rather than each item.
When using for loops in bash it splits the given list by default by whitespaces, this can be adapted by using the so called Internal Field Seperator, or IFS in short .
IFS The Internal Field Separator that is used for word splitting after
expansion and to split lines into words with the read builtin command.
The default value is "".
For your example we would need to tell IFS to use new-lines as break point.
IFS=$'\n'
for tbl in $(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh)
do
echo $tbl
done
This example returns the following output on my machine.
668 /usr/bin/python /usr/bin/ud 03:05:54
27892 python 00:01
Here is another bash-based solution, inspired by comment of #Gordon Davisson.
For this we need (atleast bash v1.13.5 (1992) or later verison), because Process-Substitution2,3,4 while read var; do { ... }; done < <(...);, etc are used.
#!/bin/bash
while IFS= read -a oL ; do { # reads single/one line
echo "${oL}"; # prints that single/one line
};
done < <(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh);
unset oL;
Note: You can use any simple or complex command/command-set inside the <(...) which may have multiple output lines.
And what code does what function is shown here.
And here is a single/one-liner way:
while IFS= read -a oL ; do { echo "${oL}"; }; done < <(ps -ewo pid,cmd,etime | grep python | grep -v grep | grep -v sh); unset oL;
( As Process-Substitution is not part of POSIX yet So its not supported in many POSIX compliant shell or in POSIX shell mode of bash-shell. Process-Substitution existed in bash since 1992 (so that is 28yrs ago from now/2020), & existed in ksh86 (before 1985)1. So POSIX should've included it. )
If you or any user wants to use something similar as Process-Substitution in POSIX compliant shell (i.e: sh, ash, dash, pdksh/mksh, etc), then look into NamedPipes.
I wanted to print the name from the entire address by shell scripting. So user1#12.12.23.234 should give output "user1" and similarly 11234#12.123.12.23 should give output 11234
Reading from the terminal:
$ IFS=# read user host && echo "$user"
<user1#12.12.23.234>
user1
Reading from a variable:
$ address='user1#12.12.23.234'
$ cut -d# -f1 <<< "$address"
user1
$ sed 's/#.*//' <<< "$address"
user1
$ awk -F# '{print $1}' <<< "$address"
user1
Using bash in place editing:
EMAIL='user#server.com'
echo "${EMAIL%#*}
This is a Bash built-in, so it might not be very portable (it won't run with sh if it's not linked to /bin/bash for example), but it is probably faster since it doesn't fork a process to handle the editing.
Using sed:
echo "$EMAIL" | sed -e 's/#.*//'
This tells sed to replace the # character and as many characters that it can find after it up to the end of line with nothing, ie. removing everything after the #.
This option is probably better if you have multiple emails stored in a file, then you can do something like
sed -e 's/#.*//' emails.txt > users.txt
Hope this helps =)
I tend to use expr for this kind of thing:
address='user1#12.12.23.234'
expr "$address" : '\([^#]*\)'
This is a use of expr for its pattern matching and extraction abilities. Translated, the above says: Please print out the longest prefix of $address that doesn't contain an #.
The expr tool is covered by Posix, so this should be pretty portable.
As a note, some historical versions of expr will interpret an argument with a leading - as an option. If you care about guarding against that, you can add an extra letter to the beginning of the string, and just avoid matching it, like so:
expr "x$address" : 'x\([^#]*\)'
I'm looking for a way to use the ouput of a command (say command1) as an argument for another command (say command2).
I encountered this problem when trying to grep the output of who command but using a pattern given by another set of command (actually tty piped to sed).
Context:
If tty displays:
/dev/pts/5
And who displays:
root pts/4 2012-01-15 16:01 (xxxx)
root pts/5 2012-02-25 10:02 (yyyy)
root pts/2 2012-03-09 12:03 (zzzz)
Goal:
I want only the line(s) regarding "pts/5"
So I piped tty to sed as follows:
$ tty | sed 's/\/dev\///'
pts/5
Test:
The attempted following command doesn't work:
$ who | grep $(echo $(tty) | sed 's/\/dev\///')"
Possible solution:
I've found out that the following works just fine:
$ eval "who | grep $(echo $(tty) | sed 's/\/dev\///')"
But I'm sure the use of eval could be avoided.
As a final side node: I've noticed that the "-m" argument to who gives me exactly what I want (get only the line of who that is linked to current user). But I'm still curious on how I could make this combination of pipes and command nesting to work...
One usually uses xargs to make the output of one command an option to another command. For example:
$ cat command1
#!/bin/sh
echo "one"
echo "two"
echo "three"
$ cat command2
#!/bin/sh
printf '1 = %s\n' "$1"
$ ./command1 | xargs -n 1 ./command2
1 = one
1 = two
1 = three
$
But ... while that was your question, it's not what you really want to know.
If you don't mind storing your tty in a variable, you can use bash variable mangling to do your substitution:
$ tty=`tty`; who | grep -w "${tty#/dev/}"
ghoti pts/198 Mar 8 17:01 (:0.0)
(You want the -w because if you're on pts/6 you shouldn't see pts/60's logins.)
You're limited to doing this in a variable, because if you try to put the tty command into a pipe, it thinks that it's not running associated with a terminal anymore.
$ true | echo `tty | sed 's:/dev/::'`
not a tty
$
Note that nothing in this answer so far is specific to bash. Since you're using bash, another way around this problem is to use process substitution. For example, while this does not work:
$ who | grep "$(tty | sed 's:/dev/::')"
This does:
$ grep $(tty | sed 's:/dev/::') < <(who)
You can do this without resorting to sed with the help of Bash variable mangling, although as #ruakh points out this won't work in the single line version (without the semicolon separating the commands). I'm leaving this first approach up because I think it's interesting that it doesn't work in a single line:
TTY=$(tty); who | grep "${TTY#/dev/}"
This first puts the output of tty into a variable, then erases the leading /dev/ on grep's use of it. But without the semicolon TTY is not in the environment by the moment bash does the variable expansion/mangling for grep.
Here's a version that does work because it spawns a subshell with the already modified environment (that has TTY):
TTY=$(tty) WHOLINE=$(who | grep "${TTY#/dev/}")
The result is left in $WHOLINE.
#Eduardo's answer is correct (and as I was writing this, a couple of other good answers have appeared), but I'd like to explain why the original command is failing. As usual, set -x is very useful to see what's actually happening:
$ set -x
$ who | grep $(echo $(tty) | sed 's/\/dev\///')
+ who
++ sed 's/\/dev\///'
+++ tty
++ echo not a tty
+ grep not a tty
grep: a: No such file or directory
grep: tty: No such file or directory
It's not completely explicit in the above, but what's happening is that tty is outputting "not a tty". This is because it's part of the pipeline being fed the output of who, so its stdin is indeed not a tty. This is the real reason everyone else's answers work: they get tty out of the pipeline, so it can see your actual terminal.
BTW, your proposed command is basically correct (except for the pipeline issue), but unnecessarily complex. Don't use echo $(tty), it's essentially the same as just tty.
You can do it like this:
tid=$(tty | sed 's#/dev/##') && who | grep "$tid"