remove contiguous subarray to leave the average minimum - algorithm

Question

This problem appeared in some regional contest for ICPC.
Given n numbers, you have to remove numbers between i to j such that remaining numbers have least average. You can't remove first and last numbers.
2 <= n <= 10^5
We had a discussion about it, and I am still not able to understand it. Some how this problem can be converted to finding contiguous subarray with maximum sum and then it was solved with binary search in O(nlog n).
I couldn't catch that solution while discussion and now after thinking a lot I am not able to understand that solution.
Link to the original problem in case it's not clear: http://programmingteam.cc.gatech.edu/contest/Mercer14/problems/6.pdf

Answer 1

Here is an approach that I think might work:
Compute the partial average from left for all elements, with and updating average, this can be done in O(N): a_L(i) = (a_L(i-1)*(i-1) + a_L(i))/i
Do the same for the partial averages from the right: a_R(i) = (a_R(i+1)*(N-i) + a_R(i))/(N-i+1)
Find the minimum in of both lists.
If the minimum is in the left partial averages (a_L), look for the minimum right to it in the a_R and the other way around if the minimum is found in a_R.
All parts take O(N). Thus, this would result in an O(N) algorithm. Though, it sounds a little bit simple and I might be missing something.
Edit: The original answer stopped in the middle for both lists, which is insufficient on second thought.
Actually, if the minima overlap, I believe, there is no interval to cut out. Here is a little Python implementation of the algorithm:
grades = [5, 5, 1, 7, 8, 2]
N = len(grades)
glob_avg = float(sum(grades))/float(N)
print('total average: {0}'.format(glob_avg))
avg_L = grades[:]
avg_R = grades[:]
minL = 0
minR = N-1
for i in range(1,N):
avg_L[i] = float(avg_L[i-1]*i + grades[i])/float(i+1)
if avg_L[i] <= avg_L[minL]:
minL = i
avg_R[N-i-1] = float(avg_R[N-i]*i + grades[N-i-1])/float(i+1)
if avg_R[N-i-1] <= avg_R[minR]:
minR = N-i-1
opti_avg = glob_avg
if minL < minR:
first = minL+1
last = minR
opti_avg = (avg_L[first-1]*first + avg_R[last]*(N-last)) / float(N + first - last)
print('')
print('Interval to cut: {0} - {1}'.format(first,last))
for pre in grades[:first]:
print('{0}'.format(pre))
for cut in grades[first:last]:
print('X {0} X'.format(cut))
for post in grades[last:]:
print('{0}'.format(post))
else:
print('NO interval found that would reduce the avg!')
print('')
print('--------------------------------------')
print('minimal avg: {0:0.3f}'.format(opti_avg))
print('--------------------------------------')

Answer 2

I would try checking each value above the global minimum, starting with largest.
You can add to left or right (whichever is largest), as long as the average is above the global average.
Keep a note of any minimums to remaining items.
For each item >= global average
While( average( selected) > global average
If average(un selected items) < best so far
Best so far = selected range
End
Add to selection largest of left and right
End while
End for
Only by finding sequences which are above the average will a minimum for unselected work.
Any item which has been considered as a list can be discounted
Had a go at implementing in Python :-
lst = [ -1, -1,1,-90,1,3,-1,-1,1,2,3,1,2,3,4,1, -1,-1];
First solution - look really at an exhausitve test - allow me to verify correctness.
lbound = 0
ubound = len( lst)
print( ubound );
# from http://math.stackexchange.com/questions/106700/incremental-averageing
def Average( lst, lwr, upr, runAvg = 0, runCnt = 0 ):
cnt = runCnt;
avg = runAvg;
for i in range( lwr, upr ):
cnt = cnt + 1
avg = float(avg) + (float(lst[i]) - avg)/cnt
return (avg, cnt )
bestpos_l = 0
bestpos_u = 0
bestpos_avg = 0
best_cnt = 0
######################################################
# solution in O(N^2) - works always
for i in range( 1, len( lst ) - 1 ):
for j in range( i+1, len(lst ) ):
tpl = Average( lst, 0, i ) # get lower end
res = Average( lst, j, len(lst), tpl[0], tpl[1] )
if (best_cnt == 0 or
(best_cnt < res[1] and res[0] == bestpos_avg ) or
res[0] < bestpos_avg ):
bestpos_l = i
bestpos_u = j
bestpos_avg = res[0]
best_cnt = res[1]
print( "better", i,j, res[0], res[1] )
print( "solution 1", bestpos_l, bestpos_u, bestpos_avg, best_cnt )
This came up with valid answers, but I hadn't appreciated, with the current data set, it doesn't really want the right hand side.
########################################################
# O(N)
#
# Try and minimize left/right sides.
#
# This doesn't work - it knows -90 is really good, but can't decide if to
# ignore -90 from the left, or the right, so does neither.
#
lower = []
upper = []
lower_avg = 0
best_lower = lst[0]
lower_i = 0
best_upper = lst[-1]
upper_avg = 0
upper_i = len(lst) -1
cnt = 0
length = len(lst)
for i in range( 0, length ):
cnt = cnt + 1
lower_avg = float( lower_avg) + ( float(lst[i]) - lower_avg)/cnt
upper_avg = float( upper_avg) + ( float(lst[-(i+1)]) - upper_avg)/cnt
upper.append( upper_avg )
lower.append( lower_avg )
if lower_avg <= best_lower:
best_lower = lower_avg
lower_i = i
if upper_avg <= best_upper:
best_upper = upper_avg
upper_i = (len(lst) - (i+1))
if( lower_i + 1 > upper_i ):
sol2 = Average( lst,0, len(lst ))
else:
sol_tmp = Average( lst,0, lower_i+1 )
sol2 = Average( lst, upper_i, len(lst),sol_tmp[0],sol_tmp[1] )
print( "solution 2", lower_i + 1, upper_i, sol2[0],sol2[1] )
The third solution was what I was trying to explain. My implementation is limited because :-
Couldn't find a good way of finding starting points. I wanted to start from the biggest elements, as they are most likely to reduce the average, but haven't got a good way of finding them.
Wasn't sure about the stability of keeping running-averages. Thought about removing items from the average by un-doing each numbers effect. Wasn't sure how this affected precision.
Was fairly sure that any interval which has been checked, can't have a starting item. That would limit further work, but unsure how best to implement such (keeping O(xx) to a minimum.
Solution 3
#################################
## can we remove first / last? if so, this needs adjusting
def ChooseNext( lst, lwr, upr ):
if lwr > 1 and upr < len(lst) -2:
# both sides available.
if lst[lwr-1] > lst[upr]:
return -1
else:
return 1
elif lwr > 1:
return -1
elif upr < len(lst) -2:
return 1
return 0
# Maximize average of data removed.
glbl_average = Average( lst, 0, len(lst) )
found = False
min_pos = 0
max_pos = 0
best_average = glbl_average[0]
for i in range(1, len(lst ) - 1):
# ignore stuff below average.
if lst[i]> glbl_average[0] or (found == False ):
lwr = i
upr = i+1
cnt = 1 # number for average
avg = lst[i]
tmp = Average( lst, 0, lwr)
lcl = Average( lst, upr, len(lst ), tmp[0], tmp[1] )
if found == False or lcl[0] < best_average:
best_average = lcl[0]
min_pos = lwr
max_pos = upr
found = True
# extend from interval (lwr,upr]
choice = ChooseNext( lst, lwr, upr )
while( choice != 0 ):
if( choice == -1):
new_lwr = lwr -1
new_upr = upr
else:
new_lwr = lwr
new_upr = upr + 1
tmp = Average( lst, 0, new_lwr )
lcl_best = Average( lst, new_upr, len(lst), tmp[0], tmp[1] )
if( lcl_best[0] > glbl_average[0]):
choice = 0
else:
lwr = new_lwr
upr = new_upr
if lcl_best[0] < best_average:
min_pos = lwr
max_pos = upr
best_average = lcl_best[0]
choice = ChooseNext( lst, lwr, upr )
print( "solution 3", min_pos, max_pos, best_average )