I have the following code in my script:
RESULT=$(cora_cmd --input={connect hrdwtst01.campbellsci.com';' file-control $STATION_NAME stop-program';' bye';'})
declare y
IFS=$'\n' y=($RESULT)
echo ${y[2]}
if [ ${y[2]} == '-file-control,loggernet datalogger locked' ]; then
echo -e "\t\E[31;1m.. ERROR - Not able to stop the datalogger's program. ..\E[37;0m"
fi
echo "$RESULT"
The compare is not working and it never goes into the if statement. Any ideas?
Results of the set -x:
'++ cora_cmd '--input={connect' 'hrdwtst01.campbellsci.com;' file-control TS_CR850_PB_801 'stop-program;' 'bye;}'
+ RESULT='CoraScript 1, 13, 06 Beta
+connect,"coralib3.dll version 1, 7, 18 Beta"
'file-control,loggernet datalogger locked
+ declare y
+ IFS='
'
+ y=($RESULT)
' echo '-file-control,loggernet datalogger locked
-file-control,loggernet datalogger locked
' '!=' '-file-control,loggernet datalogger locked' ']'
+ echo -e '\t\E[31;1m.. ERROR - Not able to stop the datalogger'\''s program. ..\E[37;0m'
.. ERROR - Not able to stop the datalogger's program. ..'
Always quote your variables unless you know you want word splitting and globbing to be done on the expansion.
if [ "${y[2]}" = '-file-control,loggernet datalogger locked' ]; then
You can also use the built-in [[ syntax, which doesn't do word splitting of variables.
if [[ ${y[2]} = '-file-control,loggernet datalogger locked' ]]; then
So there's a stray ^M on there indicating that your cora_cmd output
includes DOS-style \r\n line endings, and you're only stripping off
the \n, leaving the carriage return character. That's why you're not
getting a match... – twalberg
twalberg correctly diagnosed the problem. The simple solution is of course to include \r in IFS:
IFS=$'\r\n' y=($RESULT)
Including the \r in the test == $'-file-control,loggernet datalogger locked\r' is also possible.
Related
I have looked and looked for a solution to this, but I can't find anything that directly addresses my problem. I'm trying to add up the lines added and removed by certain authors in a git repo. I'm using git log piped to sed piped to awk which I am now trying to pipe to a subshell to add up the numbers. The issue is that the piped input isn't getting interpreted properly in the subshell, and I can't figure out why. I suspect it's in the while-loop, because of the nature of subshell syntax and its pickiness with semicolons.
I've moved around the code inside the subshell, added and removed semicolons, used backslashes for line separation to see if that was the issue, and none of it has worked. I'm not well-versed in shell so it could be a glaringly obvious issue to someone who is more experienced. "$author" is just the nth positional parameter from the command-line.
for author; do
echo "Listing file and line changes for $author"
git log --shortstat --author="$author" ${date:+--since="$date"} \
| sed -n -e '/files\? changed/s/, /\n/gp' \
| awk '
$3=="changed" {changed+=$1}
$2=="deletions(-)" {deletions+=$1}
$2=="insertions(+)" {insertions+=$1}
END{
print "files changed:", changed,
" lines removed:", deletions,
" lines added:", insertions,
" net change:", insertions-deletions
}'
done | {
total_changed=0
total_added=0
total_removed=0
while read changed insertions deletions; do
let total_changed+=changed
let total_added+=insertions
let total_removed+=deletions
done
echo "totals:"
echo "files changed: $total_changed"
echo "lines added: $total_added"
echo "lines removed: $total_removed" ;
}
The last part should output the totals but instead they output 0. I also get some weird syntax errors. Here's the output (input is "Benjamin Hills):
/home/bhills/./git-log-lines-removed.sh: line 65: let: and line changes for Benjamin Hills: syntax error in expression (error token is "line changes for Benjamin Hills")
/home/bhills/./git-log-lines-removed.sh: line 64: let: changed:: syntax error in expression (error token is ":")
/home/bhills/./git-log-lines-removed.sh: line 65: let: 61 lines removed: 1345 lines added: 246 net change: -1099: syntax error in expression (error token is "lines removed: 1345 lines added: 246 net change: -1099")
totals:
files changed: 0
lines added: 0
lines removed: 0
Your code is trying to generate human-readable output twice: Once in awk, another time in bash. Since the output awk generates is input to bash, that means you're trying to pipe an output format meant for humans and read it as if it were an input format meant for machines; obviously, it's not.
There's no good reason to take that approach at all: Generate all your human-readable output in the same process.
#!/usr/bin/env bash
# ^^^^- NOT /bin/sh
total_changed=0
total_deletions=0
total_insertions=0
for author; do
changed=0; deletions=0; insertions=0
# Loop over output from "git log" for a single author
while read added deleted _; do
(( ++changed )) # each line is a file changed
{ [[ $added = - ]] || [[ $deleted = - ]]; } && continue # skip binary files
(( insertions += added ))
(( deletions += deleted ))
done < <(git log --numstat --format='' --author="$author" ${date:+--since="$date"})
# Print results from that author
printf '%s\n' "For author: $author" \
" Files changed: $changed" \
" Deletions: $deletions" \
" Insertions: $insertions"
# Add to totals
(( total_changed+=changed ))
(( total_deletions+=deletions ))
(( total_insertions+=insertions ))
done
# Print those totals
printf '%s\n' "Totals:" \
" Files changed: $total_changed" \
" Deletions: $total_deletions" \
" Insertions: $total_insertions"
What does "#?" mean in ksh script? e.g:
tt=03
while [ "$tt" !=' ' ];
do
tt=${tt#"?}
done
echo $tt
I will get nothing here. So what "#?" means in this scipt? Thank you.
Assuming corrected syntax on the while as downtheroad suggested (need blank after [ and before ]) and also omitting the " in tt=${tt#"?} (the omitted version is what you cite).
Also the test in the while condition needs to be against an empty string'', otherwise the loop does not terminate.
tt=${tt#?}
chops off the first character from the string.
See this test program (I added the 2 echos and the parens to be able to see the exact contents of tt in the loop before and after chopping:
tt=03
while [ "$tt" != '' ]
do
echo "A: (tt=$tt)"
tt=${tt#?}
echo "B: (tt=$tt)"
done
echo $tt
gives this result (note the empty last line from the last echo):
A: (tt=03)
B: (tt=3)
A: (tt=3)
B: (tt=)
Following output consisting of several devices needs to be parsed:
0 interface=ether1 address=172.16.127.2 address4=172.16.127.2
address6=fe80::ce2d:e0ff:fe00:05 mac-address=CC:2D:E0:00:00:08
identity="myrouter1" platform="MikroTik" version="6.43.8 (stable)"
1 interface=ether2 address=10.5.44.100 address4=10.5.44.100
address6=fe80::ce2d:e0ff:fe00:07 mac-address=CC:2D:E0:00:00:05
identity="myrouter4" platform="MikroTik" version="6.43.8 (stable)"
3 interface=ether4 address=fe80::ba69:f4ff:fe00:0017
address6=fe80::ba69:f4ff:fe00:0017 mac-address=B8:69:F4:00:00:07
identity="myrouter2" platform="MikroTik" version="6.43.8 (stable)"
...
10 interface=ether5 address=10.26.51.24 address4=10.26.51.24
address6=fe80::ba69:f4ff:fe00:0039 mac-address=B8:69:F4:00:00:04
identity="myrouter3" platform="MikroTik" version="6.43.8 (stable)"
11 interface=ether3 address=10.26.51.100 address4=10.26.51.100
address6=fe80::ce2d:e0ff:fe00:f00 mac-address=CC:2D:E0:00:00:09
identity="myrouter5" platform="MikroTik" version="6.43.8 (stable)"
edit: for ease of things I shortened and anonymized the output, first block has 7 lines, second block has 5 lines, third block has 7 lines, fourth block 4 lines, so the number of lines is inconsistent.
Basically its the output from a Mikrotik device: "/ip neighbor print detail"
Optimal would be to access every device(=number) on its own, then further access all setting=value (of one device) seperately to finally access settings like $device[0][identity] or similar.
I tried to set IFS='\d{1,2} ' but seems IFS only works for single character seperation.
Looking on the web I didn't find a way to accomplish this, am I looking for the wrong way and there is another way to solve this?
Thanks in advance!
edit: Found this solution Split file by multiple line breaks which helped me to get:
devices=()
COUNT=0;
while read LINE
do
[ "$LINE" ] && devices[$COUNT]+="$LINE " || { (( ++COUNT )); }
done < devices.txt
then i could use #Kamil's solution to easily access values.
While your precise output format is a bit unclear, bash offers an efficient way to parse the data making use of process substitution. Similar to command substitution, process substitution allows redirecting the output of commands to stdin. This allows you to read the result of a set of commands that reformat your mikrotik file into a single line for each device.
While there are a number of ways to do it, one of the ways to handle the multiple gymnastics needed to reformat the multi-line information for each device into a single line is by using tr and sed. tr to first replace each '\n' with an '_' (or pick your favorite character not used elsewhere), and then again to "squeeze" the leading spaces to a single space (technically not required, but for completeness). After replacing the '\n' with '_' and squeezing spaces, you simply use two sed expressions to change the "__" (resulting from the blank line) back into a '\n' and then to remove all '_'.
With that you can read your device number n and the remainder of the line holing your setting=value pairs. To ease locating your "identity=" line, simply converting the line into an array and looping using parameter expansions (for substring removal), you can save and store the "identity" value as id (trimming the double-quotes is left to you)
Now it is simply a matter of outputting the value (or doing whatever you wish with them). While you can loop again and output the array values, it is just a easy to pass the intentionally unquoted line to printf and let the printf-trick handle separating the setting=value pairs for output. Lastly, you form your $device[0][identity] identifier and output as the final line in the device block.
Putting it altogether, you could do something like the following:
#!/bin/bash
id=
while read n line; do ## read each line from process substitution
a=( $line ) ## split line into array
for i in ${a[#]}; do ## search array, set id
[ "${i%=*}" = "identity" ] && id="${i##*=}"
done
echo "device=$n" ## output device=
printf " %s\n" ${line[#]} ## output setting=value (unquoted on purpose)
printf " \$device[%s][%s]\n" "$n" "$id" ## $device[0][identity]
done < <(tr '\n' '_' < "$1" | tr -s ' ' | sed -e 's/__/\n/g' -e 's/_//g')
Example Use/Output
Note, the script takes the filename to parse as the first input.
$ bash mikrotik_parse.sh mikrotik
device=0
interface=ether1
address=172.16.127.2
address4=172.16.127.2
address6=fe80::ce2d:e0ff:fe00:05
mac-address=CC:2D:E0:00:00:08
identity="myrouter1"
platform="MikroTik"
version="6.43.8
(stable)"
$device[0]["myrouter1"]
device=1
interface=ether2
address=10.5.44.100
address4=10.5.44.100
address6=fe80::ce2d:e0ff:fe00:07
mac-address=CC:2D:E0:00:00:05
identity="myrouter4"
platform="MikroTik"
version="6.43.8
(stable)"
$device[1]["myrouter4"]
device=3
interface=ether4
address=fe80::ba69:f4ff:fe00:0017
address6=fe80::ba69:f4ff:fe00:0017
mac-address=B8:69:F4:00:00:07
identity="myrouter2"
platform="MikroTik"
version="6.43.8
(stable)"
$device[3]["myrouter2"]
Look things over and let me know if you have further questions. As mentioned at the beginning, you haven't defined an explicit output format you are looking for, but gleaning what information was in the question, this should be close.
I think you're on the right track with IFS.
Try piping IFS=$'\n\n' (to break apart the line groups by interface) through cut (to extract the specific field(s) you want for each interface).
Bash likes single long rows with delimter separated values. So first we need to convert your file to such format.
Below I read 4 lines at a time from input. I notices that the output spans over 4 lines only - I just concatenate the 4 lines and act as if it is a single line.
while
IFS= read -r line1 &&
IFS= read -r line2 &&
IFS= read -r line3 &&
IFS= read -r line4 &&
line="$line1 $line2 $line3 $line4"
do
if [ -n "$line4" ]; then
echo "ERR: 4th line should be empt - $line4 !" >&2
exit 4
fi
if ! num=$(printf "%d" ${line:0:3}); then
echo "ERR: reading number" >&2
exit 1
fi
line=${line:3}
# bash variables can't have `-`
line=${line/mac-address=/mac_address=}
# unsafe magic
vars=(interface address address4
address6 mac_address identity platform version)
for v in "${vars[#]}"; do
unset "$v"
if ! <<<"$line" grep -q "$v="; then
echo "ERR: line does not have $v= part!" >&2
exit 1
fi
done
# eval call
if ! eval "$line"; then
echo "ERR: eval line=$line" >&2
exit 1
fi
for v in "${vars[#]}"; do
if [ -z "${!v}" ]; then
echo "ERR: variable $v was not set in eval!" >&2
exit 1;
fi
done
echo "$num: $interface $address $address4 $address6 $mac_address $identity $platform $version"
done < file
then I retrieve the leading number from the line, which I suspect was printed with printf "%3d" so I just slice the line ${line:0:3}
for the rest of the line I indent to use eval. In this case I trust upstream, but I try to assert some cases (variable not defined in the line, some syntax error and similar)
then the magic eval "$line" happens, which assigns all the variables in my shell
after that I can use variables from the line like normal variables
live example at tutorialspoint
Eval command and security issues
I am trying to write a bash script which will read two letter variables (startletter/stopletter) and after that I need to print from the start letter to the stop letter with a for or something else. How can I do that?
I tried to do
#! /bin/bash
echo "give start letter"
read start
echo "give stop letter" read stop
But none of the for constructs work
#for value in {a..z}
#for value in {$start..$stop}
#for (( i=$start; i<=$stop; i++)) do echo "Letter: $c" done
This question is very well explained in BashFAQ/071 How do I convert an ASCII character to its decimal (or hexadecimal) value and back?
# POSIX
# chr() - converts decimal value to its ASCII character representation
# ord() - converts ASCII character to its decimal value
chr () {
local val
[ "$1" -lt 256 ] || return 1
printf -v val %o "$1"; printf "\\$val "
# That one requires bash 3.1 or above.
}
ord() {
# POSIX
LC_CTYPE=C printf %d "'$1"
}
Re-using them for your requirement, a proper script would be written as
read -p "Input two variables: " startLetter stopLetter
[[ -z "$startLetter" || -z "$stopLetter" ]] && { printf 'one of the inputs is empty\n' >&2 ; }
asciiStart=$(ord "$startLetter")
asciiStop=$(ord "$stopLetter")
for ((i=asciiStart; i<=asciiStop; i++)); do
chr "$i"
done
Would print the letters as expected.
Adding it to community-wiki since this is also a cross-site duplicate from Unix.SE - Bash script to get ASCII values for alphabet
In case you feel adventurous and want to use zsh instead of bash, you can use the following:
For zsh versions below 5.0.7 you can use the BRACE_CCL option:
(snip man zshall) If a brace expression matches none of the above forms, it is left
unchanged, unless the option BRACE_CCL (an abbreviation for 'brace character class') is set. In that case, it is expanded to a list of the individual characters between the braces sorted into the order of the characters in the ASCII character set (multibyte characters are not currently handled). The syntax is similar to a [...] expression in filename generation: - is treated specially to denote a range of characters, but ^ or ! as the first character is treated normally. For example, {abcdef0-9}
expands to 16 words 0 1 2 3 4 5 6 7 8 9 a b c d e f.
#!/usr/bin/env zsh
setopt brace_ccl
echo "give start letter"
read cstart
echo "give stop letter"
read cstop
for char in {${cstart}-${cstop}}; do echo $char; done
For zsh versions from 5.0.7 onwards you can use the default brace expansion :
An expression of the form {c1..c2}, where c1 and c2 are single characters (which may be multibyte characters), is expanded to every character in the range from c1 to c2 in whatever character sequence is used internally. For characters with code points below 128 this is US ASCII (this is the only case most users will need). If any intervening character is not printable, appropriate quotation is used to render it printable. If the character sequence is reversed, the output is in reverse order, e.g. {d..a} is substituted as d c b a.
#!/usr/bin/env zsh
echo "give start letter"
read cstart
echo "give stop letter"
read cstop
for char in {${cstart}..${cend}; do echo $char; done
More information on zsh can be found here and the quick reference
I am trying to do a bash script that:
loop over some files : OK
check if the first line matches this pattern (#!f:\test\python.exe) : OK
create a new path by changing the unix style to windows style : KO
Precisely,
From: \c\tata\development\tools\virtualenvs\test2\Scripts\python.exe
I want to get: c:\tata\development\tools\virtualenvs\test2\Scripts\python.exe
insert the new line by appending #! and the new path : KO
Follow is my script but I'm really stuck!
for f in $WORKON_HOME/$env_name/$VIRTUALENVWRAPPER_ENV_BIN_DIR/*.py
do
echo "----"
echo file=$f >&2
FIRSTLINE=`head -n 1 $f`
echo firstline=$FIRSTLINE >&2
unix_path=$WORKON_HOME/$env_name/$VIRTUALENVWRAPPER_ENV_BIN_DIR/python.exe
new_path=`echo $unix_path | awk '{gsub("/","\\\")}1'`
echo new_path=$new_path >&2
# I need to change the new_path by removing the first \ and adding : after the first letter => \c -> c:
new_line="#!"$new_path
echo new_line=$new_line >&2
case "$FIRSTLINE" in
\#!*python.exe* )
# Rewrite first line
sed -i '1s,.*,'"$new_line"',' $f
esac
done
Output:
file=/c/tata/development/tools/virtualenvs/test2/Scripts/pip-script.py
firstline=#!f:\test\python.exe
new_path=\c\tata\development\tools\virtualenvs\test2\Scripts\python.exe
new_line=#!\c\tata\development\tools\virtualenvs\test2\Scripts\python.exe
Line that is written in the file: (some weird characters are written I do not know why...)
#!tatadevelopment oolsirtualenvs est2Scriptspython.exe
Line I am expecting:
#!c:\tata\development\tools\virtualenvs\test2\Scripts\python.exe
sed is interpreting the backslashes and characters following them as escapes, so you're getting, e.g. tab. You need to escape the backslashes.
sed -i "1s,.*,${new_line//\\/\\\\}," "$f"