I'm looking for the easiest and most readable way to remove a field from a path. So for example, I have /this/is/my/complicated/path/here, and I would like to remove the 5th field ("/complicated") from the string, using bash commands, so that it becomes /this/is/my/path.
I could do this with
echo "/this/is/my/complicated/path/here" | cut -d/ -f-4
echo "/"
echo "/this/is/my/complicated/path/here" | cut -d/ -f6-
but I would like this done in just one easy command, something that would like
echo "/this/is/my/complicated/path" | tee >(cut -d/ -f-4) >(cut -d/ -f6-)
except that this doesn't work.
With cut, you can specify a comma separated list of fields to print:
$ echo "/this/is/my/complicated/path/here" | cut -d/ -f-4,6-
/this/is/my/path/here
So, it's not really necessary to use two commands.
How about using sed?
$ echo "/this/is/my/complicated/path/here" | sed -e "s%complicated/%%"
/this/is/my/path/here
This removes the 5th path element
echo "/this/is/my/complicated/path/here" |
perl -F/ -lane 'splice #F,4,1; print join("/", #F)'
just bash
IFS=/ read -a dirs <<< "/this/is/my/complicated/path/here"
newpath=$(IFS=/; echo "${dirs[*]:0:4} ${dirs[*]:5}")
Anything wrong with a bash script?
#!/bin/bash
if [ -z "$1" ]; then
us=$(echo $0 | sed "s/^\.\///") # Get rid of a starting ./
echo " "Usage: $us StringToParse [delimiterChar] [start] [end]
echo StringToParse: string to remove something from. Required
echo delimiterChar: Character to mark the columns "(default '/')"
echo " "start: starting column to cut "(default 5)"
echo " "end: last column to cut "(default 5)"
exit
fi
# Parse the parameters
theString=$1
if [ -z "$2" ]; then
delim=/
start=4
end=6
else
delim=$2
if [ -z "$3" ]; then
start=4
end=6
else
start=`expr $3 - 1`
if [ -z "$4" ]; then
end=6
else
end=`expr $4 + 1`
fi
fi
fi
result=`echo $theString | cut -d$delim -f-$start`
result=$result$delim
final=`echo $theString | cut -d$delim -f$end-`
result=$result$final
echo $result
Related
I'm trying to get this script working properly. When I'm executing this bash-script with:
./parameters.sh -name="Jonathan" -id="1"
I'm getting the output: -id 1
I'm expecting an output like: Jonathan 1
#!/bin/bash
global=$*
function parameters() {
for arguments in $global; do
name=$(echo "$arguments" | cut -f1 -d=)
value=$(echo "$arguments" | cut -f2 -d=)
case "$name" in
-name) name=$value;;
-id) id=$value;;
*) echo "This parameter is not reconized."
esac
done
echo "$name"
echo "$id"
}
parameters
What do I have to change after the pipe inside the cut to make it working?
name=$(echo "$arguments" | cut -f1 -d=)
value=$(echo "$arguments" | cut -f2 -d=)
I suggest with bash:
#!/bin/bash
global=$*
parameters() {
for arguments in $global; do
[[ "$arguments" =~ (.*)=(.*) ]]
option="${BASH_REMATCH[1]}"
value="${BASH_REMATCH[2]}"
case "$option" in
-name) name="$value";;
-id) id="$value";;
*) echo "This parameter is not reconized."
esac
done
echo "$name"
echo "$id"
}
parameters
Complete novice in Bash. Trying to iterate thru 1000 gzip files, may be GNU parallel is the solution??
#!/bin/bash
ctr=0
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
for f in "$dir"/*.gz; do
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr" >> $1
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym
else
ctr=$((ctr+1))
fi
done
done
Any help to speed the process will be greatly appreciated !!!
#!/bin/bash
ctr=0
export ctr
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
export dir
doit() {
f="$1"
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr"
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym >&2
else
ctr=$((ctr+1))
fi
done
}
export -f doit
parallel doit ::: *gz 2>&1 > $1
The Bash while read loop is probably your main bottleneck here. Calling multiple external processes for simple field splitting will exacerbate the problem. Briefly,
while IFS="|" read -r first second third rest; do ...
leverages the shell's built-in field splitting functionality, but you probably want to convert the whole thing to a simple Awk script anyway.
echo "file_name,symbol,record_count" > "$1"
for f in "/data/myfolder"/*.gz; do
gunzip -c "$f" |
awk -F "\|" -v f="$f" -v OFS="," '
/H/ { if(ctr) print f, sym, ctr
ctr=0; sym=$3;
print sym >"/dev/stderr"
next }
{ ++ctr }'
done >>"$1"
This vaguely assumes that printing the lone sym is just for diagnostics. It should hopefully not be hard to see how this can be refactored if this is an incorrect assumption.
The main issue is that i try to parse ls to do a mock "Compare directories" but when i do so since i use nested fors i cant properly compare the results from it since the comparison of two filenames/strings even if they are the same it always returns false
I tried erasing the white characters but no results.
var1=$(ls -l $1 | grep -v ^d | tail -n +2 | tr -s " "| cut -d " " -f 9)
var2=$(ls -l $2 | grep -v ^d | tail -n +2 | tr -s " "| cut -d " " -f 9)
for i in $var1 ; do
i=$(printf "$i" | tr -d '[:space:]')
flag=0
var3=$(ls -l $1 | grep -v ^d | tail -n +2 | tr -s " " | grep $i | cut -d " " -f 5)
for j in $var2 ; do
j=$(printf $j | tr -d '[:space:]')
var4=$(ls -l $2 | grep -v ^d | tail -n +2 | tr -s " " | grep $j | cut -d " " -f 5)
if [ "$i" == "$j" ] ; then
if [ "$var3" != "$var4" ] ; then
flag=1
fi
else
flag=1
fi
done
if [ $flag -eq 1 ] ; then
printf "$i file does not exist on the $2 catalog\n"
printf "It 's size is :$var3 \n"
let Sum=$Sum+$var3
fi
done
This is not a string comparison problem, it's a logic problem.
I wrote you a MCVE that demonstrates the same problem with less code and fewer dependencies:
flag=0
target="hello"
for candidate in "hello" "world"
do
if [ "$target" != "$candidate" ]
then
flag=1
fi
done
if [ "$flag" -eq 1 ]
then
echo "The string was not found"
fi
This prints The string was not found every time, just like your script, even though it's clearly there.
The problem here is that the script requires that ALL files match. It should only require that ANY file matches. The easiest way to fix this is to:
Set flag=1 when a MATCH is found (not a mismatch)
Make flag=1 signify that a match was found (rather than no match was found)
Here's the version which correctly finds the string:
flag=0
target="hello"
for candidate in "hello" "world"
do
if [ "$target" = "$candidate" ]
then
flag=1
fi
done
if [ "$flag" -eq 1 ]
then
echo "The string was found"
else
echo "The string was not found"
fi
#!/bin/bash
a=2
b=2
COUNTER=0
sam="abcd"
sam1="xyz"
sam2="mno"
for x in ls | grep .rpm
do
`C=rpm -qpR $x | grep -v CompressedFileNames | grep -v PayloadFilesHavePrefix | wc -l`
if [ "sam2"!="$sam1" ]
then
echo "${sam1}"
echo "${sam2}"
if [ $C -eq $a ]
then
COUNTER=$((COUNTER+1))
echo "${x}"
eval sam=$x
#eval sam1=sam | cut -d '-' -f 1
sam1=`echo "${sam}"| cut -d '-' -f 1`
if [ $COUNTER -eq $b ]
then
break
fi
fi
fi
sam2=`echo "${x}"| cut -d '-' -f 1`
done
This is the output I am getting:
xyz
mno
comps-4ES-0.20050107.x86_64.rpm
comps
comps
comps-4ES-0.20050525.x86_64.rpm
My question is: why is the if condition returning true despite sam1 and sam2 being equal? I have checked for non-equality.
Response is the same even if I use
if [ $C -eq $a ] && [ "$sam2" != " $sam1" ]
As Ansgar Wiechers pointed out, you're missing a "$" in front of the sam2 variable. That way, you're comparing the literal string "sam2" with the string value of $sam1 (which initially is set to "xyz"). What you want to do is compare the string values of both variables:
if [ "$sam2" != "$sam1" ]
Regarding $C, you should only include the commands to be evaluated inside backticks, not the evaluation itself. This is called a command substitution - a subshell is created in which the commands are executed, and the backtick expression is substituted by the computed value. The line should look like this:
C=`rpm -qpR $x | grep -v CompressedFileNames | grep -v PayloadFilesHavePrefix | wc -l`
Your for loop also needs a command substitution: for x in ls | grep .rpm makes it look as if you're piping the output of a for command into grep. What you want to do is iterate over the ls | grep part, which you can do with the following command substitution:
for x in `ls | grep .rpm`
Hi Guys Got the solution:
#!/bin/bash
read -p "enter dep number" a
read -p "enter no of rpms" b
COUNTER=0
sam="abcd"
sam1="xyz"
sam2="mno"
for x in `ls | grep .rpm`
do
C=`rpm -qpR $x |grep -v CompressedFileNames | grep -v PayloadFilesHavePrefix | wc -l`
# echo "${C}:c"
if [ $C -eq $a ] && [ "$sam2" != "$sam1" ]
then
COUNTER=$((COUNTER+1))
# echo "${COUNTER}:counter"
# echo "${x}"
eval sam=$x
#eval sam1=sam | cut -d '-' -f 1
sam1=`echo "${sam}"| cut -d '-' -f 1`
if [ $COUNTER -eq $b ]
then
break
fi
fi
sam2=`echo "${x}"| cut -d '-' -f 1`
#echo "${sam2}"
#echo "${sam1}"
done
what am I doing wrong in the code below?
I am replacing the salary data within the text file, but the Telephone number field (the 3rd column) is being updated instead of the salary field (the 5th column) which is 0.
In the example below, the calculated salary for Ruben is 500.
My Desired output is:
Ruben,1223,97707001,Salaried,500
But instead, I get this (replacing zero between 9770 and 7001 with 535):
Ruben,1223,9775007001,Salaried,0
payroll_employee()
{
echo
echo "[Option: $input]"
echo "Enter Payroll of an employee "
echo
echo -en "Enter employee name: "
read Name
#Retrieve current entry into individual fields
line=`grep -i "$Name" $PAYROLL`
Name=`echo $line | cut -d "," -f1`
EmployeeID=`echo $line | cut -d "," -f2`
EmployeeHP=`echo $line | cut -d "," -f3`
EmployeeType=`echo $line | cut -d "," -f4`
Salary=`echo $line | cut -d "," -f5`
#Check if entry exist in records
if [ `count_lines "^${Name},"` -eq 0 ]
then
echo "Error: This particular record does not exist!!"
else
echo "$Name is ${EmployeeType} employee."
if [ "$EmployeeType" = "Salaried" ]
then
echo $EmployeeType
echo -en "Enter Weekly Salary:"
read swages
if [ -z $swages ]
then
swages=$Salary
else
grep -vi "$Name" $PAYROLL > tmpfile #Perform updating to salary field entry
grep -x "$line" $PAYROLL | sed -e "s/$Salary/$swages/" >> tmpfile
mv tmpfile $PAYROLL
echo "$Name's weekly payroll has been updated to \$$swages!!"
fi
echo
}
Sample code:
update_employee()
{
echo
echo "[Option: $input]"
echo "Updating employee record... "
echo "Please enter the name of the employee to update: "
echo -en "[1]Name: "
read update_name
#Retrieve current entry into individual fields
line=`grep -i "$update_name" $PAYROLL`
oldname=`echo $line | cut -d "," -f1`
oldjob=`echo $line | cut -d "," -f2`
olddept=`echo $line | cut -d "," -f3`
oldsal=`echo $line | cut -d "," -f4`
#Check if entry to update exist in records
if [ `count_lines "^${update_name},"` -eq 0 ]
then
echo "Error: This particular record does not exist!!"
else
while [ "$choice" != "6" ]
do
update_menu #Display update menu for user input,allows update of individual field or all at once
read update_choice
case $update_choice in
"1") echo -en "Please enter employee's new name: "
read new_name
if [ -z $new_name ]
then
new_name=$oldname
elif [ `count_lines "^${new_name},"` -ne 0 ] #Check if name already exist in records
then
echo "Error: Employee [$new_name] already exist in records!"
else
grep -vi "$oldname" $PAYROLL > tmpfile #Perform updating to name field entry
grep -x "$line" $PAYROLL | sed -e "s/$oldname/$new_name/" >> tmpfile
mv tmpfile $PAYROLL
echo "Employee's name $oldname has been updated to [$new_name]!!"
fi
break
;; }
All I changed was add one more column.
Salary=`echo $line | cut -d "," -f5`
payroll_employee()
{
echo
echo "[Option: $input]"
echo "Enter Payroll of an employee "
echo
echo -en "Enter employee name: "
read Name
#Retrieve current entry into individual fields
if [ $(grep -ciw "$Name" $PAYROLL) -eq 0 ]
then
echo "No matches found in $PAYROLL";
else
line=$(grep -iw "$Name" $PAYROLL);
Name=$(echo $line | awk -F "," print $1);
EmployeeID=$(echo $line | awk -F "," print $2);
EmployeeHP=$(echo $line | awk -F "," print $3);
EmployeeType=$(echo $line | awk -F "," print $4);
Salary=$(echo $line | awk -F "," print $5);
fi
if [ "$EmployeeType" == "Salaried" ]
then
echo $EmployeeType
echo -en "Enter Weekly Salary:"
read swages
if [ -z $swages ]
then
swages=$Salary
else
sed "/$Name/d" $PAYROLL
echo "$Name $EmployeeID $EmployeeHP $EmployeeType $swages" >> $PAYROLL;
echo "$Name's weekly payroll has been updated to \$$swages!!"
fi
echo
}
I Prefer use of AWK for delimination
also you can delete line using
sed "/$Name/d" $PAYROLL
Why to store it in other file using grep -vi and rename back and replace with original (Waste of memory & resources)
also "`" backtick should be replaced with new method of using $(..)
Also one make sure when you match strings use == sign
also in grep use -w also just to make sure that you select proper complete string
Ruben baruben rubenner all will be searched in grep if -w is not given
Seems like you are replacing every occurrence of the string $Salary in the payroll line, so if salary is 100 and the employee ID number is 2100, it will be replaced.
Instead of using sed at the end, you would probably be better off generating the output using printf and building the fields up that way.
Something like:
printf "%s,%s,%s,%s,%s\n" $Name $EmployeeID $EmployeeHP $EmployeeType $swages >> tmpfile
EDIT: You should fix the = to == as pointed out in the comments.
Also, to illustrate what I think is happening:
line="ABC,5100,DEF,100"
Salary=100
echo $line | sed -e s/${Salary}/XXX/
ABC,5XXX,DEF,100
If you "anchor" the query by putting a $ at the end of the search string, it will only match the last value.
echo $line | sed -e s/${Salary}$/XXX/
ABC,5100,DEF,XXX
Add some echo statements in your code to check the status of variables....