I am currently trying to use sed to find and replace some text in a file, and for the terms that are being found/replaced, I am trying to use variables, but I cannot get them to work properly for some reason. The variables consist of c1-c9, which can be seen below:
c1=$( echo "start time;project_start_time" )
c2=$( echo "start_time;project_start_time" )
c3=$( echo "end time;project_end_time" )
c4=$( echo "end_time;project_end_time" )
c5=$( echo "total time;project_total_time" )
c6=$( echo "total_time;project_total_time" )
c7=$( echo "project_id;project_ID" )
c8=$( echo "status;project_status" )
c9=$( echo "client_id;client_ID" )
c10=$( echo "employee_id;employee_ID" )
c11=$( echo "employee_name;employee_name" )
c12=$( echo "date created;date_created" )
c13=$( echo "date_created;date_created" )
and the code that contains the sed part is:
while [ "$countc" -le 13 ]; do
real_string=$( eval echo "\$c$countc" | cut -d ';' -f 2 )
nc_string=$( eval echo "\$c$countc" | cut -d ';' -f 1 )
sed -e "s/$nc_string/$real_string/gI" phasef/"$count"p.csv #> phasef/"$count".csv
countc=$(( $countc + 1 ))
done
(there is more code to the script, but it is irrelevant) When I run the script, if i tell it to output real_string/nc_string as the while loop moves along, the variables are correctly outputted, real_string/nc_string variables are actually being defined correctly, but I am not sure why sed is not reading them correctly. If anyone could point out what I am doing wrong I would really appreciate it as I have been trying to figure this out for a few hours now, thanks!
I think #anishsane probably has the right answer. I just want to offer some notes about your code. If you're using bash, take advantage of some of bash's features:
1) just assign a string instead of spawning a subshell to echo the string:
c1="start time;project_start_time"
(unless the echo is just a placeholder here for some more complicated process
2) use an array instead of numerically-indexed variables
c=(
"start time;project_start_time"
"start_time;project_start_time"
"end time;project_end_time"
"end_time;project_end_time"
"total time;project_total_time"
"total_time;project_total_time"
"project_id;project_ID"
"status;project_status"
"client_id;client_ID"
"employee_id;employee_ID"
"employee_name;employee_name"
"date created;date_created"
"date_created;date_created"
)
3) split the strings with read and IFS and avoid eval nastiness:
for (( countc=1; countc <= ${#c[#]}; countc++ )); do
IFS=';' read nc_string real_string <<< "${c[countc]}"
# ...
done
I checked the echo for "s/$nc_string/$real_string/gI" & it is expanding the variables properly. Please let us know, the input file contents.
I suspect that, highlited part in below line has some issue.
sed -e "s/$nc_string/$real_string/gI" phasef/"$count"p.csv > phasef/"$count".csv
If you want to replace all pairs in input file (phasef/${count}p.csv) & save to output file (phasef/$count.csv), use below code:
cp phasef/"$count"p.csv phasef/"$count".csv
while [ "$countc" -le 13 ]; do
real_string=$( eval echo "\$c$countc" | cut -d ';' -f 2 )
nc_string=$( eval echo "\$c$countc" | cut -d ';' -f 1 )
sed -i -e "s/$nc_string/$real_string/gI" phasef/"$count".csv
countc=$(( $countc + 1 ))
done
Note the -i in sed line.
You have a typo in the sed line. Change
sed -e "s/$nc_string/$real_string/gI" phasef/"$count"p.csv
to
sed -e "s/$nc_string/$real_string/gI" phasef/"$countc"p.csv
Related
I have bash script which checks presence of certain files and that the content has a valid format. It uses variable prefixes so i can easily add/remove new files w/o the need of further adjustments.
Problem is that i need to run this on AIX servers where bash is not present. I've adjusted the script except the part with variable prefixes. After some attempts i am lost and have no idea how to properly migrate the following piece of code so it runs under sh ( $(echo ${!ifile_#}) ). Alternatively i have ksh or csh if plain sh is not an option.
Thank you in advance for any help/hints
#!/bin/sh
# Source files
ifile_one="/path/to/file/one.csv"
ifile_two="/path/to/file/two.csv"
ifile_three="/path/to/file/three.csv"
ifile_five="/path/to/file/four.csv"
min_columns='10'
existing_files=""
nonexisting_files=""
valid_files=""
invalid_files=""
# Check that defined input-files exists and can be read.
for input_file in $(echo ${!ifile_#})
do
if [ -r ${!input_file} ]; then
existing_files+="${!input_file} "
else
nonexisting_files+="${!input_file} "
fi
done
echo "$existing_files"
echo "$nonexisting_files"
# Check that defined input files have proper number of columns.
for input_file_a in $(echo "$existing_files")
do
check=$(grep -v "^$" $input_file_a | sed 's/[^;]//g' | awk -v min_columns="$min_columns" '{ if (length == min_columns) {print "OK"} else {print "KO"} }' | grep -i KO)
if [ ! -z "$check" ]; then
invalid_files+="${input_file_a} "
else
valid_files+="${input_file_a} "
fi
done
echo "$invalid_files"
echo "$valid_files"
Bash returns expected output (of the four ECHOes):
/path/to/file/one.csv /path/to/file/two.csv /path/to/file/three.csv
/path/to/file/four.csv
/path/to/file/three.csv
/path/to/file/one.csv /path/to/file/two.csv
ksh/sh throws:
./report.sh[14]: "${!ifile_#}": 0403-011 The specified substitution is not valid for this command.
Thanks #Benjamin W. and #user1934428 , ksh93 arrays are the answer.
So bellow code works for me as desired.
#!/bin/ksh93
typeset -A ifile
ifile[one]="/path/to/file/one.csv"
ifile[two]="/path/to/file/two.csv"
ifile[three]="/path/to/file/three.csv"
ifile[whatever]="/path/to/file/something.csv"
existing_files=""
nonexisting_files=""
for input_file in "${!ifile[#]}"
do
if [ -r ${ifile[$input_file]} ]; then
existing_files+="${ifile[$input_file]} "
else
nonexisting_files+="${ifile[$input_file]} "
fi
done
I have been working on a Bash script to beep when the PC is too hot.
I have removed the beep to try identifying the problem.
What I have so far:
temp=$(sysctl -n hw.acpi.thermal.tz0.temperature | tr -d 'C')
echo $temp
if ["$temp" -gt "30.1"]
then
echo "temp hot"
else
echo "temp ok"
fi
My output is
54.1
temp.sh: line 4: [54.1: command not found
temp ok
Removing the if statement just outputs
54.1
so I think it's the if statement that's not working.
You should use double parenthesis (( )) to do arithmetic expressions, and since Bash cannot handle decimal values, you just have to remove the dot (as if you want to multiply it by ten).
temp=$(sysctl -n hw.acpi.thermal.tz0.temperature | tr -d 'C')
max_temp=50.2
(( ${temp//./} > ${max_temp//./} )) && echo "temp hot" || echo "temp ok"
Be sure to use the same format for both values (especially leading zeros, 54.10 would become 5410).
If the format cannot be guaranteed, there is a second method, as mentioned by Benjamin W, using bc. You can send to this command a logical operation involving floats, it returns 0 if true, 1 otherwise.
temp=$(sysctl -n hw.acpi.thermal.tz0.temperature | tr -d 'C')
max_temp=50.2
(( $(echo "$temp > $max_temp" | bc) )) && echo "temp hot" || echo "temp ok"
Your immediate problem is syntax error -- you omitted the space between the command ([ -- /bin/[) and its arguments ($temp). The last argument of [ must be ] -- your script is missing a blank there too, which makes the last argument "30.1"]. The quotes aren't necessary here, BTW, and only make things harder to read.
This is a generic sh-scripting quation, it has nothing to do with FreeBSD nor with temperature-measuring.
I'm new to bash scripting and I'm asking for a little help !
I've got a little scipt in bash that is not making what I want (but almost) and the behavior of my echo command seems strange to me, look at it :
TST='test'
TEST="${ADDR[3]}"_"$TST"
echo $TEST
#result : _test
echo ${ADDR[3]}
#result : 5
How can you explain these results ? Thanks in advance :)
My ADDR var is defined like this :
#parsing the read line, split on whitespace
IFS=' ' read -ra ADDR <<< "$line"
Here is my complete script :
#!/bin/bash
NUMBER=2
{ read ;
while IFS= read -r line; do
echo "$NUMBER : $line"
IFS=' ' read -ra ADDR <<< "$line"
#If the countdown is set to 0, launch the task ans set it to init value
if [ ${ADDR[0]} == '0' ]; then
#task launching
echo `./${ADDR[1]}.sh ${ADDR[2]} &`
TST='test'
TEST=${ADDR[3]}_$TST
echo $TEST
VAR=$(echo -E "${ADDR[3]}" | tr -d '\n')
#countdown set to init value
sed -i "$NUMBER c $VAR ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
else
sed -i "$NUMBER c $((ADDR-1)) ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
fi
((NUMBER++))
done } < listing.txt
Answer: the following is fine,
TEST="${ADDR[3]}"_"$TST"
Although I would recommend.
TEST="${ADDR[3]}_${TST}"
What you need to do is dump ${ADDR[3]} before this statement and confirm that ADDR holds the expected values. You may as well dump the entire array with indexes and confirm all entries
for ((i=0; i<${#ADDR[#]}; i++)); do
printf "ADDR[%3d] %s\n" "$i" "${ADDR[$i]}"
done
This will help isolate the issue. Sorry for the earlier answer. Lesson [sleep 1st: answer 2nd]
I'd like to understand bash a bit better as I'm apparently horrible at it...
I'm trying to generate a sequence of constant width integers, but then test them to do something exceptional for particular values. Like so:
for n in $(seq -w 1 150)
do
# The next line does not work: doit.sh: line 9: XX: command not found
#decval= $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' )
#if [[ ${decal} -eq 98 ]] ; then
if [[ $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' ) -eq 98 ]] ; then
echo "Do something different for 98"
elif [[ $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' ) -eq 105 ]] ; then
echo "Do something different for 98"
fi
done
This script works for my purposes, but if I try and make the assignment 'decval= $(…' I get an error 'command not found'. I don't understand this, can someone explain?
Also, is there an improvement I can make to this script if I have a large number of exceptions to prevent a long list of if ; then elif … ?
The problem is in the space between = and $:
decval= $(…
You should write without spaces:
decval=$(...
Because, if you write the space, your shell reads decval= as declval="" and treats the result of $(echo...) as the name of a command to execute, and obviously it doesn't find the command.
Also (just a small optimization), you can write:
sed 's/^0\+//'
instead of
sed 's/^0//g' | sed 's/^0//g'
Here:
0\+ means 0 one or more times;
g is removed, because g means replace all occurences in the string, and you have only one occurence (^ can be only one time in a string).
Also, you can check your variable even with leading zeros, without sed:
[[ "$n" =~ "0*98" ]]
I'm trying to write a small script that will count entries in a log file, and I'm incrementing a variable (USCOUNTER) which I'm trying to use after the loop is done.
But at that moment USCOUNTER looks to be 0 instead of the actual value. Any idea what I'm doing wrong? Thanks!
FILE=$1
tail -n10 mylog > $FILE
USCOUNTER=0
cat $FILE | while read line; do
country=$(echo "$line" | cut -d' ' -f1)
if [ "US" = "$country" ]; then
USCOUNTER=`expr $USCOUNTER + 1`
echo "US counter $USCOUNTER"
fi
done
echo "final $USCOUNTER"
It outputs:
US counter 1
US counter 2
US counter 3
..
final 0
You are using USCOUNTER in a subshell, that's why the variable is not showing in the main shell.
Instead of cat FILE | while ..., do just a while ... done < $FILE. This way, you avoid the common problem of I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?:
while read country _; do
if [ "US" = "$country" ]; then
USCOUNTER=$(expr $USCOUNTER + 1)
echo "US counter $USCOUNTER"
fi
done < "$FILE"
Note I also replaced the `` expression with a $().
I also replaced while read line; do country=$(echo "$line" | cut -d' ' -f1) with while read country _. This allows you to say while read var1 var2 ... varN where var1 contains the first word in the line, $var2 and so on, until $varN containing the remaining content.
Always use -r with read.
There is no need to use cut, you can stick with pure bash solutions.
In this case passing read a 2nd var (_) to catch the additional "fields"
Prefer [[ ]] over [ ].
Use arithmetic expressions.
Do not forget to quote variables! Link includes other pitfalls as well
while read -r country _; do
if [[ $country = 'US' ]]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
minimalist
counter=0
((counter++))
echo $counter
You're getting final 0 because your while loop is being executed in a sub (shell) process and any changes made there are not reflected in the current (parent) shell.
Correct script:
while read -r country _; do
if [ "US" = "$country" ]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
I had the same $count variable in a while loop getting lost issue.
#fedorqui's answer (and a few others) are accurate answers to the actual question: the sub-shell is indeed the problem.
But it lead me to another issue: I wasn't piping a file content... but the output of a series of pipes & greps...
my erroring sample code:
count=0
cat /etc/hosts | head | while read line; do
((count++))
echo $count $line
done
echo $count
and my fix thanks to the help of this thread and the process substitution:
count=0
while IFS= read -r line; do
((count++))
echo "$count $line"
done < <(cat /etc/hosts | head)
echo "$count"
USCOUNTER=$(grep -c "^US " "$FILE")
Incrementing a variable can be done like that:
_my_counter=$[$_my_counter + 1]
Counting the number of occurrence of a pattern in a column can be done with grep
grep -cE "^([^ ]* ){2}US"
-c count
([^ ]* ) To detect a colonne
{2} the colonne number
US your pattern
Using the following 1 line command for changing many files name in linux using phrase specificity:
find -type f -name '*.jpg' | rename 's/holiday/honeymoon/'
For all files with the extension ".jpg", if they contain the string "holiday", replace it with "honeymoon". For instance, this command would rename the file "ourholiday001.jpg" to "ourhoneymoon001.jpg".
This example also illustrates how to use the find command to send a list of files (-type f) with the extension .jpg (-name '*.jpg') to rename via a pipe (|). rename then reads its file list from standard input.