ok, i'm working on a different kinda of script but the problem comes down to something like this: assume the following "for loop":
for i in $(ls -l); do echo $i; done
the problem is that "for loop" separate values by space, so each "i" equals to each word separated by space in 'ls -l'. Hence the output is something like this:
total
24
drwxrwxr-x.
2
james
james
4096
Oct
26
16:56
bg
.
.
.
but I want throughout each irritation variable "i" be equal to ENTIRE line of 'ls -a' instead of each word. In other word "i" be equal to entire line
"drwxrwxr-x. 2 james james 4096 Oct 26 16:56 bg"
instead of irritating through each word. I've tried many workarounds, none of them has worked and kinda freaks me out.
Is there a way to tell "for loop" to separate by new line instead of space
P.S. The above example is just for illustration (you might argue that it's a bit pointless) but my problem is something similar to that.
Instead you can
ls -l | while IFS= read -r l ; do echo "This is it: $l" ; done
or do
IFS=\\n
before running your for, but I'd avoid that due to possible side effects.
Related
I wrote a simple script, the whole purpose of which is to simply create a link between two different cygwin directories. It should be very simple, but since $LOCALAPPDATA can contain spaces, it wound up being far more difficult than I originally envisioned.
Here is the script:
#!/bin/sh
echo "Unlinking any existing data link..."
unlink /usr/local/astrometry/shared_data 2>/dev/null
echo "Generating link between astrometry shared_data..."
my_dir=`cygpath -u $LOCALAPPDATA/cygwin_ansvr/usr/share/astrometry/data`
echo $my_dir
my_test=`echo $my_dir`
echo $my_test
# Note here, if I use $my_dir rather than $my_test, it introduces a LINE BREAK!
ln -s "$my_test" /usr/local/astrometry/shared_data
exit 0
So, if I run the above script, here is the output:
Unlinking any existing data link...
Generating link between astrometry shared_data...
/cygdrive/c/Users/Dwight Towler/AppData/Local/cygwin_ansvr/usr/share/astrometry/data
/cygdrive/c/Users/Dwight Towler/AppData/Local/cygwin_ansvr/usr/share/astrometry/data
And the link is formed as such:
lrwxrwxrwx 1 Dwight Towler None 84 Sep 22 02:56 shared_data -> /cygdrive/c/Users/Dwight Towler/AppData/Local/cygwin_ansvr/usr/share/astrometry/data
drwx------+ 1 Dwight Towler None 0 Sep 22 02:56 .
The above is the desired link (no line break).
Now, if I replace $my_test with $my_dir in the ln -s call, I instead wind up with this:
lrwxrwxrwx 1 Dwight Towler None 84 Sep 22 02:55 shared_data -> /cygdrive/c/Users/Dwight
Towler/AppData/Local/cygwin_ansvr/usr/share/astrometry/data
drwx------+ 1 Dwight Towler None 0 Sep 22 02:55 .
Notice the line break? I cannot figure out where that is coming from, especially since I put quotes around the variables in the ln -s call.
It is especially puzzling since the output of the echo command seems to indicate that both variables have the same content:
echo $my_dir
/cygdrive/c/Users/Dwight Towler/AppData/Local/cygwin_ansvr/usr/share/astrometry/data
echo $my_test
/cygdrive/c/Users/Dwight Towler/AppData/Local/cygwin_ansvr/usr/share/astrometry/data
Any ideas on what is going on?
That difference in values between 'my_dir' and 'my_test' is the result of using command substitution (my_test=echo $my_dir) to copy the my_dir to my_test. This construct will result in any consecutive white spaces (newline included) replaced with a single space.
As per man page, this command substitution, will result in the value of 'my_dir' being split by the value of IFS (by default - white spaces - spaces, tabs and new line) into words, and than the individual words are printed with a single space between them. Assuming original string contained new lines (or multiple spaces between words) - those will all get converted into a single space.
Consider the following assignment, which will result in embedded newline (between the 'first' and 'second). Using the unquoted "echo" will replace the newline with a space.
A="first
second"
echo "NO QUOTE"
echo $A
echo "QUOTED"
echo "$A"
echo "----"
The output will be
NO QUOTE
first second
QUOTED
first
second
----
Bottom line, the new line is presented in the original string ('my_dir'), and is replaced by space in the echo statement, because of the shell word/command substitution.
Issue
I have been experiencing issues with Linux commands run in folders that contain numerically numbered files and folders; e.g., files sequentially numbered 1, 2, 3 ...
For example, if I am in a folder that contains a file or folder with a numeric name that appears in my command, the output from that command output might be truncated.
Here are some examples:
$ ls -l
total 8
drwxr-xr-x 2 victoria victoria 4096 May 7 18:34 1
drwxr-xr-x 2 victoria victoria 4096 May 7 18:14 2
-rw-r--r-- 1 victoria victoria 0 May 7 18:34 3
## fail
$ a="[CPT1A] A Selective"; echo $a
1 A Selective
$ b="[CPT2A] A Selective"; echo $b
2 A Selective
$ c="[CPT3A] A Selective"; echo $c
2 A Selective
...
## pass
$ d="[CPT4A] A Selective"; echo $d
[CPT4A] A Selective
Update/solution
... per accepted answer: quote the BASH variable, when used.
$ a="[CPT1A] A Selective"; echo $a
1 A Selective
$ a="[CPT1A] A Selective"; echo "$a"
[CPT1A] A Selective
The problem is that you aren't quoting the variable when you use it -- that is, you're using echo $a instead of echo "$a". When a variable is referenced without quotes, it gets split into words (so "[CPT1A] A Selective" becomes "[CPT1A]" "A" "Selective"), and then each of those words that contains anything that looks like a filename wildcard gets expanded into a list of matching filenames.
Square-bracket expressions like [CPT1A] are actually valid wildcard expressions that match any single character within them, so if there are files named "A", "C", "P", "T", or "1", it would expand to the matching names. If there aren't any, the wildcard expression just gets passed through intact.
Solution: double-quote variable references to avoid surprises like this. The same goes for command substitutions with $( ) (or backticks, but don't use those). There are a few places where it's safe to leave them off, like in a direct assignment, but IMO it's safer to just use them everywhere than to try to keep track of the exceptions. For example, a=$b is ok, but so is a="$b". On the other hand, export a=$b might or might not work (depending on which shell you're using), but export a="$b" will work.
BTW, shellcheck.net is good at pointing these out (along with some other common mistakes).
So, I am building a bash script which iterates through folders named by numbers from 1 to 9. The script depends on getting the folder names by user input. My intention is to use a for loop using read input to get a folder name or a range of folder names and then do some stuff.
Example:
Let's assume I want to make a backup with rsync -a of a certain range of folders. Usually I would do:
for p in {1..7}; do
rsync -a $p/* backup.$p
done
The above would recursively backup all content in the directories 1 2 3 4 5 6 and 7 and put them into folders named as 'backup.{index-number}'. It wouldn't catch folders/files with a leading . but that is not important right now.
Now I have a similar loop in an interactive bash script. I am using select and case statements for this task. One of the options in case is this loop and it shall somehow get a range of numbers from user input. This now becomes a problem.
Problem:
If I use read to get the range then it fails when using {1..7} as input. The input is taken literally and the output is just:
{1..7}
I really would like to know why this happens. Let me use a more descriptive example with a simple echo command.
var={1..7} # fails and just outputs {1..7}
for p in $var; do echo $p;done
read var # Same result as above. Just outputs {1..7}
for p in $var; do echo $p;done
for p in {1..7}; do echo $p;done # works fine and outputs the numbers 1-7 seperated with a newline.
I've found a workaround by storing the numbers in an array. The user can then input folder names seperated by a space character like this: 1 2 3 4 5 6 7
read -a var # In this case the output is similar to the 3rd loop above
for p in ${var[#]}; do echo $p; done
This could be a way to go but when backing up 40 folders ranging from 1-40 then adding all the numbers one-by-one completely makes my script redundant. One could find a solution to one of the millennium problems in the same time.
Is there any way to read a range of numbers like {1..9} or could there be another way to get input from terminal into the script so I can iterate through the range within a for-loop?
This sounds like a question for google but I am obviously using the wrong patterns to get a useful answer. Most of similar looking issues on SO refer to brace and parameter expansion issues but this is not exactly the problem I have. However, to me it feels like the answer to this problem is going in a similar direction. I fail to understand why when a for-loop for assigning {1..7} to a variable works but doing the same like var={1..7} doesn't. Plz help -.-
EDIT: My bash version:
$ echo $BASH_VERSION
4.2.25(1)-release
EDIT2: The versatility of a brace expansion is very important to me. A possible solution should include the ability to define as many ranges as possible. Like I would like to be able to choose between backing up just 1 folder or a fixed range between f.ex 4-22 and even multiple options like folders 1,2,5,6-7
Brace expansion is not performed on the right-hand side of a variable, or on parameter expansion. Use a C-style for loop, with the user inputing the upper end of the range if necessary.
read upper
for ((i=1; i<=$upper; i++)); do
To input both a lower and upper bound separated by whitespace
read lower upper
for (i=$lower; i <= $upper; i++)); do
For an arbitrary set of values, just push the burden to the user to generate the appropriate list; don't try to implement your own parser to process something like 1,2,20-22:
while read p; do
rsync -a $p/* backup.$p
done
The input is one value per line, such as
1
2
20
21
22
Even if the user is using the shell, they can call your script with something like
printf '%s\n' 1 2 20..22 | backup.sh
It's easier for the user to generate the list than it is for you to safely parse a string describing the list.
The evil eval
$ var={1..7}
$ for i in $(eval echo $var); do echo $i; done
this also works,
$ var="1 2 {5..9}"
$ for i in $(eval echo $var); do echo $i; done
1
2
5
6
7
8
9
evil eval was a joke, that is, as long as you know what you're evaluating.
Or, with awk
$ echo "1 2 5-9 22-25" |
awk -v RS=' ' '/-/{split($0,a,"-"); while(a[1]<=a[2]) print a[1]++; next}1'
1
2
5
6
7
8
9
22
23
24
25
I want to find the median for each column, however it doesn't work like what I want.
1 2 3
3 2 1
2 1 5
I'm expecting for
2 2 3
for the result, however turns out it just give sum error and some "sum" of the column. Below is a snippet of the code for "median in column"
while read -r line; do
read -a array <<< "$line"
for i in "${!array[#]}"
do
column[${i}]=${array[$i]}
((length[${i}]++))
result=${column[*]} | sort -n
done < file
for i in ${!column[#]}
do
#some median calculation.....
Notes: I want to practice bash, that's why I hard-coded using bash.
I really appreciate if someone could help me, especially in BASH. Thank you.
Bash is really not suitable for low-level text processing like this: the read command does a system call for each character that it reads, which means that it's slow, and it's a CPU hog. It's ok for processing interactive input, but using it for general text processing is madness. It would be much better to use awk (Python, Perl, etc) for this.
As an exercise in learning about Bash I guess it's ok, but please try to avoid using read for bulk text processing in real programs. For further information, please see Why is using a shell loop to process text considered bad practice? on the Unix & Linux Stack Exchange site, especially the answer written by
Stéphane Chazelas (the discoverer of the Shellshock Bash bug).
Anyway, to get back to your question... :)
Most of your code is ok, but
result=${column[*]} | sort -n
doesn't do what you want it to.
Here's one way to get the column medians in pure Bash:
#!/usr/bin/env bash
# Find medians of columns of numeric data
# See http://stackoverflow.com/q/33095764/4014959
# Written by PM 2Ring 2015.10.13
fname=$1
echo "input data:"
cat "$fname"
echo
#Read rows, saving into columns
numrows=1
while read -r -a array; do
((numrows++))
for i in "${!array[#]}"; do
#Separate column items with a newline
column[i]+="${array[i]}"$'\n'
done
done < "$fname"
#Calculate line number of middle value; which must be 1-based to use as `head`
#argument, and must compensate for extra newline added by 'here' string, `<<<`
midrow=$((1+numrows/2))
echo "midrow: $midrow"
#Get median of each column
result=''
for i in "${!column[#]}"; do
median=$(sort -n <<<"${column[i]}" | head -n "$midrow" | tail -n 1)
result+="$median "
done
echo "result: $result"
output
input data:
1 2 3
3 2 1
2 1 5
midrow: 3
result: 2 2 3
I picked up a copy of the book 10 PRINT CHR$(205.5+RND(1)); : GOTO 10
http://www.amazon.com/10-PRINT-CHR-205-5-RND/dp/0262018462
This book discusses the art produced by the single line of Commodore 64 BASIC:
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
This just repeatedly prints randomly character 205 or 206 to the screen from the PETSCII set:
http://en.wikipedia.org/wiki/PETSCII
https://vimeo.com/26472518
I'm not sure why the original uses the characters 205 and 206 instead of the identical 109 and 110. Also, I prefer to add a clear at the beginning. This is what I usually type into the C64:
1?CHR$(147)
2?CHR$(109.5+RND(1));:GOTO2
RUN
You can try this all for yourself in an emulator, such as this one using Flash or JavaScript:
http://codeazur.com.br/stuff/fc64_final/
http://www.kingsquare.nl/jsc64
When inputting the above code into the emulators listed, you'll need to realize that
( is *
) is (
+ is ]
I decided it would be amusing to write a bash line to do something similar.
I currently have:
clear; while :; do [ $(($RANDOM%2)) -eq 0 ] && (printf "\\") || (printf "/"); done;
Two questions:
Any suggestions for making this more concise?
Any suggestions
for a better output character? The forward and backward slash are
not nearly as beautiful since their points don't line up. The characters used from PETSCII are special characters, not slashes. I didn't see anything in ASCII that could work as well, but maybe you can suggest a way to pull in a character from UTF-8 or something else?
Best ANSWERS So Far
Shortest for bash (40 characters):
yes 'c=(╱ ╲);printf ${c[RANDOM%2]}'|bash
Here is a short one for zsh (53 characters):
c=(╱ ╲);clear;while :;do printf ${c[RANDOM%2+1]};done
Here is an alias I like to put in my .bashrc or .profile
alias art='c=(╱ ╲);while :;do printf "%s" ${c[RANDOM%2]};done'
Funny comparing this to the shortest I can do for C64 BASIC (23 characters):
1?C_(109.5+R_(1));:G_1
The underscores are shift+H, shift+N, and shift+O respectively. I can't paste the character here since they are specific to PETSCII. Also, the C64 output looks prettier ;)
You can read about the C64 BASIC abbreviations here:
http://www.commodore.ca/manuals/c64_programmers_reference/c64-programmers_reference_guide-02-basic_language_vocabulary.pdf
How about this?
# The characters you want to use
chars=( $'\xe2\x95\xb1' $'\xe2\x95\xb2' )
# Precompute the size of the array chars
nchars=${#chars[#]}
# clear screen
clear
# The loop that prints it:
while :; do
printf -- "${chars[RANDOM%nchars]}"
done
As a one-liner with shorter variable names to make it more concise:
c=($'\xe2\x95\xb1' $'\xe2\x95\xb2'); n=${#c[#]}; clear; while :; do printf -- "${c[RANDOM%n]}"; done
You can get rid of the loop if you know in advance how many characters to print (here 80*24=1920)
c=($'\xe2\x95\xb1' $'\xe2\x95\xb2'); n=${#c[#]}; clear; printf "%s" "${c[RANDOM%n]"{1..1920}"}"
Or, if you want to include the characters directly instead of their code:
c=(╱ ╲); n=${#c[#]}; clear; while :; do printf "${c[RANDOM%n]}"; done
Finally, with the size of the array c precomputed and removing unnecessary spaces and quotes (and I can't get shorter than this):
c=(╱ ╲);clear;while :;do printf ${c[RANDOM%2]};done
Number of bytes used for this line:
$ wc -c <<< 'c=(╱ ╲);clear;while :;do printf ${c[RANDOM%2]};done'
59
Edit. A funny way using the command yes:
clear;yes 'c=(╱ ╲);printf ${c[RANDOM%2]}'|bash
It uses 50 bytes:
$ wc -c <<< "clear;yes 'c=(╱ ╲);printf \${c[RANDOM%2]}'|bash"
51
or 46 characters:
$ wc -m <<< "clear;yes 'c=(╱ ╲);printf \${c[RANDOM%2]}'|bash"
47
After looking at some UTF stuff:
2571 BOX DRAWINGS LIGHT DIAGONAL UPPER RIGHT TO LOWER LEFT
2572 BOX DRAWINGS LIGHT DIAGONAL UPPER LEFT TO LOWER RIGHT
(╱ and ╲) seem best.
f="╱╲";while :;do print -n ${f[(RANDOM % 2) + 1]};done
also works in zsh (thanks Clint on OFTC for giving me bits of that)
Here is my 39 character command line solution I just posted to #climagic:
grep -ao "[/\\]" /dev/urandom|tr -d \\n
In bash, you can remove the double quotes around the [/\] match expression and make it even shorter than the C64 solution, but I've included them for good measure and cross shell compatibility. If there was a 1 character option to grep to make grep trim newlines, then you could make this 27 characters.
I know this doesn't use the Unicode characters so maybe it doesn't count. It is possible to grep for the Unicode characters in /dev/urandom, but that will take a long time because that sequence comes up less often and if you pipe it the command pipeline will probably "stick" for quite a while before producing anything due to line buffering.
Bash supports Unicode now, so we don't need to use UTF-8 character sequences such as $'\xe2\x95\xb1'.
This is my most-correct version: it loops, prints either / or \ based on a random number as others do.
for((;;x=RANDOM%2+2571)){ printf "\U$x";}
41
My previous best was:
while :;do printf "\U257"$((RANDOM%2+1));done
45
And this one 'cheats' using embedded Unicode (I think for obviousness, maintainability, and simplicity, this is my favourite).
Z=╱╲;for((;;)){ printf ${Z:RANDOM&1:1};}
40
My previous best was:
while Z=╱╲;do printf ${Z:RANDOM&1:1};done
41
And here are some more.
while :;do ((RANDOM&1))&&printf "\U2571"||printf "\U2572";done
while printf -v X "\\\U%d" $((2571+RANDOM%2));do printf $X;done
while :;do printf -v X "\\\U%d" $((2571+RANDOM%2));printf $X;done
while printf -v X '\\U%d' $((2571+RANDOM%2));do printf $X;done
c=('\U2571' '\U2572');while :;do printf ${c[RANDOM&1]};done
X="\U257";while :;do printf $X$((RANDOM%2+1));done
Now, this one runs until we get a stack overflow (not another one!) since bash does not seem to support tail-call elimination yet.
f(){ printf "\U257"$((RANDOM%2+1));f;};f
40
And this is my attempt to implement a crude form of tail-process elimination. But when you have had enough and press ctrl-c, your terminal will vanish.
f(){ printf "\U257"$((RANDOM%2+1));exec bash -c f;};export -f f;f
UPDATE:
And a few more.
X=(╱ ╲);echo -e "\b${X[RANDOM&1]"{1..1000}"}" 46
X=("\U2571" "\U2572");echo -e "\b${X[RANDOM&1]"{1..1000}"}" 60
X=(╱ ╲);while :;do echo -n ${X[RANDOM&1]};done 46
Z=╱╲;while :;do echo -n ${Z:RANDOM&1:1};done 44
Sorry for necroposting, but here's bash version in 38 characters.
yes 'printf \\u$[2571+RANDOM%2]'|bash
using for instead of yes inflates this to 40 characters:
for((;;)){ printf \\u$[2571+RANDOM%2];}
109 chr for Python 3
Which was the smallest I could get it.
#!/usr/bin/python3
import random
while True:
if random.randrange(2)==1:print('\u2572',end='')
else:print('\u2571',end='')
#!/usr/bin/python3
import random
import sys
while True:
if random.randrange(2)==1:sys.stdout.write("\u2571")
else:sys.stdout.write("\u2572")
sys.stdout.flush()
Here's a version for Batch which fits in 127 characters:
cmd /v:on /c "for /l %a in (0,0,0) do #set /a "a=!random!%2" >nul & if "!a!"=="0" (set /p ".=/" <nul) else (set /p ".=\" <nul)"