I have a problem with tables being sorted in a different ways on my two ubuntu servers. I'm executing the following command in the redis-cli tool. This is the server 1:
127.0.0.1:6379> eval "local a = {'_mcat:banner','for_meta:1','_size:300x250','_mtype:html', 'axx:1'};table.sort(a);return a;" 0
1) "_mcat:banner"
2) "_mtype:html"
3) "_size:300x250"
4) "axx:1"
5) "for_meta:1"
And the server 2:
127.0.0.1:6379> eval "local a = {'_mcat:banner','for_meta:1','_size:300x250','_mtype:html', 'axx:1'};table.sort(a);return a;" 0
1) "axx:1"
2) "for_meta:1"
3) "_mcat:banner"
4) "_mtype:html"
5) "_size:300x250"
As you can see the problem lays in how the sort is treating the '_' character. I have tried to upgrade versions of redis & gcc and the behavior is not changed between the servers. Redis versions tested: 2.8.4, 2.8.19, 4.0.2, gcc version: 4.8.2. Running locale returns the same values on both servers:
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
Executing the same code in Python, works the same on both machines:
>>> sorted(['_mcat:banner','for_meta:1','_size:300x250','_mtype:html', 'axx:1'])
['_mcat:banner', '_mtype:html', '_size:300x250', 'axx:1', 'for_meta:1']
Maybe somebody has a suggestion on what to try next? Thanks.
EDITED:
On both servers running:
eval "local a = {'_mcat:banner','for_meta:1','_size:300x250','_mtype:html', 'axxx:2'};table.sort(a, function(a,b) return a<b end);return a;" 0
Produces results consistent with the table.sort(a) (so different between servers), but take a look at this:
On server 1:
127.0.0.1:6379> eval "local a;if 'axxx:2'>'_mcat:banner' then a=1 else a=0 end;return a;" 0
(integer) 1
127.0.0.1:6379> eval "local a;if 'a'>'_' then a=1 else a=0 end;return a;" 0
(integer) 1
On server 2:
127.0.0.1:6379> eval "local a;if 'axxx:2'>'_mcat:banner' then a=1 else a=0 end;return a;" 0
(integer) 0
127.0.0.1:6379> eval "local a;if 'a'>'_' then a=1 else a=0 end;return a;" 0
(integer) 1
Given that table.sort is implemented in Lua itself using quicksort (src/ltablib.c) and for strings the comparison boils down to strcoll call, it's difficult to say where it could go wrong for underscores. I would try adding a custom sort function to see if this changes the outcome (table.sort(a, function(a,b) return a<b end)) and if it still produces the incorrect results, then you can print a, b, a<b values to see where it goes wrong.
Thanks for all the comments and suggestions, I still don't know what exactly happened but rebuilding the server worked.
Related
I’ve got this bash script to download 52k files:
for i in {1..52000};
do wget -c "download.hebrewbooks.org/downloadhandler.ashx?req=$i" ;
done
However the server gives me an 429 error.
How can I pause the loop for X amount of time after every 5 files that are downloaded?
If i is a multiple of five, sleep.
for i in {1..52000}; do
wget -c "download.hebrewbooks.org/downloadhandler.ashx?req=$i"
((i % 5)) || sleep $X
done
Note that ((expr)) returns the Boolean value of expr, where false=0 and true=1, which is the opposite of normal Bash return codes. That's why you have to use OR || instead of AND &&. If that's too confusing, use this instead: ((i % 5 == 0)) && ...
I know the title is a bit weird, but I'm not sure how to better word it.
I'm using $(return &> /dev/null) to detect if a file is sourced or not.
I know that this method is not 100% reliable, but it has worked for me without issues before now. I've figured out how to work around the problem, but I can't figure out why this is happening.
Edit: This is for an internal company project and is not intended to be POSIX compliant or portable to other shells.
I've tried this on three different systems and had the same results:
Redhat 6.9; $BASH_VERSION=4.1.2(1)-release
Mint 18.2; $BASH_VERSION=4.3.48(1)-release
Arch; $BASE_VERSION=4.4.23(1)-release
If $? is 1 (false) when $(return &> /dev/null) the return code is reversed.
$ hr | cat test_0source - test_0source.sh; hr; ./test_0source
#!/bin/bash
# test_0source
false
test_0source.sh
false
source test_0source.sh
true
test_0source.sh
true
source test_0source.sh
========================================
#!/bin/bash
# test_0source.sh
# shellcheck disable=SC2091
$(return &> /dev/null)
echo "$?"
========================================
1
1
1
0
I expected to be seeing
1
0
1
0
My workaround is to add true right before the return check. While this gives me the correct results, adding false instead of true cause the return check to always return 1.
I realize I'm missing something basic here, but I'm not seeing it. Why is this doing this?
Edit: I had an incorrect value for $? in my initial explanation above.
Update:
First, using $(return 0 &> /dev/null) resolves the problem, thanks to #ondre-k.
Ondre also pointed out that comparing BASH_SOURCE against $0 is a more idiomatic way to do this:
$ hr | cat test_0bs - test_0bs.sh; hr; test_0bs | less
#!/bin/bash
# test_0bs
false
test_0bs.sh
hr
false
source test_0bs.sh
hr
true
test_0bs.sh
hr
true
source test_0bs.sh
========================================
#!/bin/bash
# test_0bs.sh
if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
echo "We are not being sourced."
else
echo "We are being sourced."
fi
========================================
We are not being sourced.
========================================
We are being sourced.
========================================
We are not being sourced.
========================================
We are being sourced.
I suspect there may be an edge case or two around this, but it also seems to me that these cases will be less of a concern.
What you are doing is basically "abusing" the fact that return can only happen from function or a sourced script and assuming it yields 0 when a return was possible (we're sourced) and 1 if not, while suppressing the error output. You also do that in a subshell. Oddly enough, bash is still OK with placement of the return but does not return from a sourced script and keeps going. So far, so god.
The problem is, that "naked" (w/o explicit value specified) return, just like exit propagate last seen return code (return code of the command immediately preceding it).
In other words false; return would be the same thing as return 1. You can also try this out by replacing true and false by (exit 255) and see what happens. The 1 you are seeing for return following a false is not the one of "error: you cannot say return now" as your test is looking for, but just an ordinary return that has returned the last from the command preceding it (false). This difference would also become obvious if you've dropped the stderr redirection.
TL;DR for this construct to work as you expected, change it to return 0.
I hope I have not missed some corner case, but this should work as an alternative with bash by comparing name of the executed script and a source file. [[ "${BASH_SOURCE}" = ${0} ]] evaluates to 0 if file has not been sourced and 1 if it has. Or replace = with != to get same meaning of values as in the return case above.
I have the following problem with a bash script:
validParameters=0
argumentLength=${#1}
argumentLength==$((argumentLength - 1))
#[code to increment validParameters]
if [[ $validParameters != argumentLength ]]
then
diff=$((argumentLength - validParameters))
printf "Attention:\n$diff invalid argument(s) found!\n"
fi
exit 1
The error happens in the line: diff=$((argumentLength - validParameters))
=3: syntax error: operand expected (error token is "=3")
with the command script.sh abc
If I set diff to a fixed value (e.g. diff=1) instead of the subtraction, the script works perfectly.
Is my subtraction syntax somehow wrong?
argumentLength==$((argumentLength - 1))
You've got two =s here. It's equivalent to:
argumentLength="=$((argumentLength - 1))"
That's why the error message says =3.
Sounds like one of the variables argumentLength and validParameters did not store a number, but something including the string =3.
For debugging, try to print both variables before subtracting them.
By the way, you can write ((diff = argumentLength - validParameters)).
Edit after your edit: Found the Bug
There is one = too much in
argumentLength==$((argumentLength - 1))
write
argumentLength=$((argumentLength - 1))
or
(( argumentLength-- ))
instead.
I'm looking to set the value of a variable to one thing if it was already set or another if it was not. Here's an example of what I mean
export RESULT=${VALID:+Yes:-No}
Where the value of ${VALID:+Yes:-No} would be Yes if the variable was set or No if it was not.
One way I can do it now:
if [ -n "${VALID}" ]; then
export RESULT=Yes
else
export RESULT=No
fi
I could do it like this, but it would be nice to have a "one-liner".
Is it possible to do this in one line?
There's no way to do multiple parameter expansions within one variable assignement.
Without using an if statement, you can just use a couple of parameter expansions.
$ VALID="aaa"
$ RESULT="${VALID:+Yes}"; RESULT="${RESULT:-No}"; echo $RESULT
Yes
$ VALID=""
$ RESULT="${VALID:+Yes}"; RESULT="${RESULT:-No}"; echo $RESULT
No
It's not a single statement, but it's short enough that it fits on one line without the complexity of a subshell and if.
You can use an array with 2 elements (no yes) and based on variable's length decide which value to be returned:
# variable set case
>>> VALID=foo
>>> arr=(no yes) && echo "${arr[$((${#VALID} > 0))]}"
yes
# variable not set
>>> unset VALID
>>> arr=(no yes) && echo "${arr[$((${#VALID} > 0))]}"
no
The -v VAR printf option is a bash extension. Without it, you could use command substitution. (Variable names deliberately down-cased.)
printf -v result %.3s ${valid:+YES}NO
One line, requirement met:
export RESULT=$([ -n "$VALID" ] && echo Yes || echo No)
I have the following txt document:
deviceIDs.txt
UDID0=LGH811dec0bfd6
UDID1=41006289e4b2a179
UDID2=d9a7aa45
PORT0=4567
PORT1=4568
PORT2=4569
BOOTPORT0=5556
BOOTPORT1=5557
BOOTPORT2=5558
I want to be able to write the following bash script:
#!/bin/bash
source /path/deviceIDs.txt
for ((i=0;i<=2;i++))
do
echo $UDID$i
echo $PORT$i
echo $BOOTPORT$i
done
however this doesn't work. Presumable because the dollar sign is being used twice in one expression. I've tried formatting the variables using {} too but still no success.
You can use bash's form of indirection:
#!/bin/bash
source deviceIDs.txt
for ((i=0;i<=2;i++))
do
for x in UDID PORT BOOTPORT
do
y=$x$i
echo ${!y}
done
done
The key here is that ${!y} returns the value of the variable that is named by y.
This produces the output:
LGH811dec0bfd6
4567
5556
41006289e4b2a179
4568
5557
d9a7aa45
4569
5558
Alternative
If we are allowed to change the data file format, consider defining your variables as arrays this way:
$ cat IDarrays.txt
UDID=(LGH811dec0bfd6 41006289e4b2a179 d9a7aa45)
PORT=(4567 4568 4569)
BOOTPORT=(5556 5557 5558)
With that, the script can be written:
#!/bin/bash
source IDarrays.txt
for ((i=0;i<=2;i++))
do
echo ${UDID[$i]}
echo ${PORT[$i]}
echo ${BOOTPORT[$i]}
done
I didn't knew John1024's solution with !y.
If you can't modify deviceIDs.txt because it is generated by someone out of your control, you could use arrays like this too:
#!/bin/bash
source /path/deviceIDs.txt
u=($UDID{0..2});p=($PORT{0..2});b=($BOOTPORT{0..2})
for i in {0..2}; do echo -e ${u[i]}"\n"${p[i]}"\n"${b[i]}"\n"; done
LGH811dec0bfd6
4567
5556
41006289e4b2a179
4568
5557
d9a7aa45
4569
5558