i have a number
(23452)
and i what my function to return the most frequent num in this number and the highest one.
for the num above it should return '2' and for '225566' it should return '6'
i tried :
def most_popular_digit(num):
pop_dig = {}
c = str(num)
for n in range(len(c)):
count = pop_dig.get(c[n],0)
count += 1
pop_dig[c[n]] = count
list_keys = pop_dig.keys()
sorted_num = sorted(list_keys, key=pop_dig.get)
but i cant figure out how to get also the highest number with the highest apperances.
managed to figure it out:
def most_popular_digit(num):
pop_dig = {}
c = str(num)
for n in range(len(c)):
count = pop_dig.get(c[n],0)
count += 1
pop_dig[c[n]] = count
list_keys = pop_dig.keys()
sorted_num = sorted(list_keys, key=pop_dig.get)
a = pop_dig.keys()
b = pop_dig.values()
if b.index(max(b)) == a.index(max(a)):
return a[a.index(max(a))]
else:
return sorted_num[-1]
I'm trying to implement Smith-Waterman alignment in parallel using Julia (see: Figure 1 of http://www.cs.virginia.edu/~rl6sf/paper_dump/2011:12:33:22.pdf), but the algorithm is running much slower in Julia than the serial version. I'm using shared arrays to do this and figure I am doing something silly that is making the code run slow. Could someone take a look and see if my code is optimized as possible? The parallel version should run faster than in serial….
The basic concept of it is to compute the anti-diagonal elements of a matrix in parallel from the upper left to lower right corner and to update them. I'm trying to use 32 cores on a shared array machine to do this. I have a SharedArray matrix that I am using to do this and am computing the elements of each anti-diagonal in parallel as shown below. The while loops in the spSW function submit tasks to workers in sync for each anti-diagonal using the helper function shared_get_score(). The main goal of this function is to fill in each element in the shared arrays "matrix" and "path".
function spSW(seq1,seq2,p)
indel = -1
match = 2
seq1 = "^$seq1"
seq2 = "^$seq2"
col = length(seq1)
row = length(seq2)
wl = workers()
matrix,path = shared_initialize_path(seq1,seq2)
for j = 2:col
jcol = j
irow = 2
#sync begin
count = 0
while jcol > 1 && irow < row + 1
#println(j," ",irow," ",jcol)
if seq1[jcol] == seq2[irow]
equal = true
else
equal = false
end
w = wl[(count % p) + 1]
#async remotecall_wait(w,shared_get_score!,matrix,path,equal,indel,match,irow,jcol)
jcol -= 1
irow += 1
count += 1
end
end
end
for i = 3:row
jcol = col
irow = i
#sync begin
count = 0
while irow < row+1 && jcol > 1
#println(j," ",irow," ",jcol)
if seq1[jcol] == seq2[irow]
equal = true
else
equal = false
end
w = wl[(count % p) + 1]
#async remotecall_wait(w,shared_get_score!,matrix,path,equal,indel,match,irow,jcol)
jcol -= 1
irow += 1
count += 1
end
end
end
return matrix,path
end
The other helper functions are:
function shared_initialize_path(seq1,seq2)
col = length(seq1)
row = length(seq2)
matrix = convert(SharedArray,fill(0,(row,col)))
path = convert(SharedArray,fill(0,(row,col)))
return matrix,path
end
#everywhere function shared_get_score!(matrix,path,equal,indel,match,i,j)
pathvalscode = ["-","|","M"]
pathvals = [1,2,3]
scores = []
push!(scores,matrix[i,j-1]+indel)
push!(scores,matrix[i-1,j]+indel)
if equal
push!(scores,matrix[i-1,j-1]+match)
else
push!(scores,matrix[i-1,j-1]+indel)
end
val,ind = findmax(scores)
if val < 0
matrix[i,j] = 0
else
matrix[i,j] = val
end
path[i,j] = pathvals[ind]
end
Does anyone see an obvious way to make this run faster? Right now it's about 10 times slower than the serial version.
I am working on implementing "Goal-based vector field pathfinding" (demonstrated in the article at this link) It requires that I label every node in my world graph with a path distance from the goal node and recommends using a brushfire (wavefront) algorithm to do this. This is the area I am having issues in. When I get to the 8th iteration of my while loop and the 6th iteration of my nested for, I get a nil reference error on the marked line.
g is my graph, which has an 8-way adjacency list form.
q is an instance of this FIFO lua library.
rtx and rty are the x and y coords of the root node.
d is an iterator added to keep track of the path distance assigned to each node.
The structure of each node in the graph is not the same as the structure of a node being processed.
Node for processing:
n = {}
n[1] = x coord
n[2] = y coord
n[3] = adjacency list (eight entries)
n.vX = x componant of vector for vector field
n.vY = y componant of vector for vector field
Node stored in graph:
n = {}
n[1] = adjacency list
n.vX = x componant of vector for vector field
n.vY = y componant of vector for vector field
Beneath is my implementation so far. The t in the for loop is just a temporary node used to pass information along to the queue. BTW t is where the distance of all the nodes gets set.
local function brushFire( g, rtx, rty )
local q = q.new()
local s = {}
s[1] = rtx
s[2] = rty
s[3] = g[rtx][rty][3]
s.dist = 0
q:pushRight( s )
s = nil
local d = 0
while( table.getn( q.list[q.first] ) ~= 0 ) do
print( d )
local n = q:popLeft()
setDist( g, n )
print( #n[3] )
for i = 1, #n[3] do
print( ":"..i )
if( g[n[3][i][4]][n[3][i][2]].v ~= true ) then
g[n[3][i][5]][n[3][i][2]].v = true
local t = {}
t[1] = n[3][i][1]
t[2] = n[3][i][2]
t[3] = g[n[3][i][7]][n[3][i][2]][1] <------Error here
t.dist = d
q:pushRight( t )
t = nil
end
end
d = d + 1
end
end
Let me know if you need more information in order to answer my question.
I found the answer to my problem. If anyone wants to use the source, I am posting it below:
Queue module:
local q = {}
local q_mt = { __index = q }
function q.new()
local nq = { first = 0, last = -1, list = {} }
return setmetatable( nq, q_mt )
end
function q:pushLeft( value )
local first = self.first - 1
self.first = first
self.list[first] = value
end
function q:pushRight( value )
local last = self.last + 1
self.last = last
self.list[last] = value
end
function q:popLeft()
local first = self.first
if first > self.last then error( "list is empty" ) end
local value = self.list[first]
self.list[first] = nil
self.first = first + 1
return value
end
function q:popRight()
local last = self.last
if self.first > last then error( "list is empty" ) end
local value = self.list[last]
self.list[last] = nil
self.last = last - 1
return value
end
return q
The following module creates a vector field which points towards a goal when pathFind.findPath is called:
local q = require( "Queue" )
local pathFind = {}
local pathFind_mt = { __index = pathFind }
-- Private Functions
local function genDistMap( g, rtx, rty ) -<-<-<- genDistMap is the brushfire part
local q = q.new()
local g = g
g[rtx][rty].v = true
g[rtx][rty].dist = 0
local s = {}
s[1] = rtx
s[2] = rty
s[3] = g[rtx][rty][1]
s.dist = 0
q:pushRight( s )
s = nil
while( q.list[q.first] ~= nil ) do
local n = q:popLeft()
for i = 1, #n[3] do
local x, y = n[3][i][1], n[3][i][2]
if( g[x][y].v ~= true ) then
g[x][y].v = true
local t = {}
t[1] = x
t[2] = y
t[3] = g[x][y][1]
t.dist = n.dist + 1
g[x][y].dist = n.dist + 1
q:pushRight( t )
t = nil
end
end
end
return g
end
local function genVectorField( nodes )
local nodes = nodes
for i = 2, #nodes - 1 do
for j = 2, #nodes[i] - 1 do
local a = nodes[i - 1][j].dist - nodes[i + 1][j].dist
local b = nodes[i][j - 1].dist - nodes[i][j + 1].dist
local c = math.sqrt( a*a + b*b )
nodes[i][j].vX = a/c*5
nodes[i][j].vY = b/c*5
end
end
return nodes
end
-- Public Functions
function pathFind.new()
local newPathFind = {}
return setmetatable ( newPathFind, pathFind_mt )
end
function pathFind.findPath( nodeSet, rootX, rootY )
local nodes = nodeSet
nodes = genDistMap( nodes, rootX, rootY )
nodes = genVectorField( nodes )
return( nodes )
end
return pathFind
I am using the following lines of code for edge detection using canny edge detector :
I=imread('bradd.tif');
figure,imshow(I);
IDtemp = im2double(I);
[r c]=size(I);
ID(r,c) = 0;
IDx(r,c) = 0;
IDfil(r,c) = 0;
IDxx(r,c) = 0;
IDy(r,c) = 0;
IDyy(r,c) = 0;
mod(r,c) = 0;
for i= 1 : r+4
for j = 1:c+4
if(i<=2 || j<=2 || i>=r+3 || j>=c+3)
ID(i,j) = 0;
else
ID(i,j) = IDtemp(i-2,j-2);
end;
end
end
%figure,imshow(ID);
filter=[2 4 5 4 2;4 9 12 9 4;5 12 15 12 5;4 9 12 9 4;2 4 5 4 2];
for i=1:5
for j=1:5
filter(i,j)=filter(i,j)/159;
end
end
%figure,imshow(filter);
for v = 3 : r
for u = 3 : c
sum = 0;
for i = -2 : 2
for j = -2 : 2
sum = sum + (ID(u+i, v+j) * filter(i+3, j+3));
end
end
IDx(u,v) = sum;
end
end
%figure,imshow(IDx);
IDxtemp = IDx;
for i= 1 : r+2
for j = 1:c+2
if(i<=1 || j<=1 || i>=r || j>=c)
IDfil(i,j) = 0;
else
IDfil(i,j) = IDxtemp(i-1,j-1);
end;
end
end
%figure,imshow(IDfil);
Mx = [-1 0 1; -2 0 2; -1 0 1]; % Sobel Mask in X-Direction
My = [-1 -2 -1; 0 0 0; 1 2 1]; % Sobel Mask in Y-Direction
for u = 2:r
for v = 2:c
sum1 = 0;
for i=-1:1
for j=-1:1
sum1 = sum1 + IDfil(u + i, v + j)* Mx(i + 2,j + 2);
end
end
IDxx(u,v) = sum1;
end;
end
%figure,imshow(IDxx);
for u = 2:r
for v = 2:c
sum2 = 0;
for i=-1:1
for j=-1:1
sum2 = sum2 + IDfil(u + i, v + j)* My(i + 2,j + 2);
end
end
IDyy(u,v) = sum2;
end
end
%figure,imshow(IDyy);
for u = 1:r
for v = 1:c
mod(u,v) = sqrt(IDxx(u,v)^2 + IDyy(u,v)^2) ;
%mod(u,v) = sqrt(IDxx(u,v)^2 + IDyy(u,v)^2);
end
end
%figure,imshow(mod);
modtemp = mod;
for i= 1 : r+2
for j = 1:c+2
if(i<=1 || j<=1 || i>=r || j>=c)
mod(i,j) = 0;
else
mod(i,j) = modtemp(i-1,j-1);
end;
end
end
%figure,imshow(mod);
theta(u,v) = 0;
supimg(u,v) = 0;
ntheta(u,v) = 0;
for u = 2 : r
for v = 2 : c
theta(u,v) = atand(IDyy(u,v)/IDxx(u,v));
if ((theta(u,v) > 0 ) && (theta(u,v) < 22.5) || (theta(u,v) > 157.5) && (theta(u,v) < -157.5))
ntheta(u,v) = 0;
end
if ((theta(u,v) > 22.5) && (theta(u,v) < 67.5) || (theta(u,v) < -112.5) && (theta(u,v) > -157.5))
ntheta(u,v) = 45;
end
if ((theta(u,v) > 67.5 && theta(u,v) < 112.5) || (theta(u,v) < -67.5 && theta(u,v) > 112.5))
ntheta(u,v) = 90;
end
if ((theta(u,v) > 112.5 && theta(u,v) <= 157.5) || (theta(u,v) < -22.5 && theta(u,v) > -67.5))
ntheta(u,v) = 135;
end
if (ntheta(u,v) == 0)
if (mod(u, v) > mod(u, v-1) && mod(u, v) > mod(u, v+1))
supimg(u,v) = mod(u,v);
else supimg(u,v) = 0;
end
end
if (ntheta(u,v) == 45)
if (mod(u, v) > mod(u+1, v-1) && mod(u, v) > mod(u-1, v+1))
supimg(u,v) = mod(u,v);
else supimg(u,v) = 0;
end
end
if (ntheta(u,v) == 90)
if (mod(u, v) > mod(u-1, v) && mod(u, v) > mod(u+1, v))
supimg(u,v) = mod(u,v);
else supimg(u,v) = 0;
end
end
if (ntheta(u,v) == 135)
if (mod(u, v) > mod(u-1, v-1) && mod(u, v) > mod(u+1, v+1))
supimg(u,v) = mod(u,v);
else supimg(u,v) = 0;
end
end
end
end
%figure,imshow(ntheta);
th = 0.2;
tl = 0.1;
resimg(u,v)= 0;
for u = 2 : r-1
for v = 2 : c-1
if(supimg(u,v) > th)
resimg(u,v) = 1;
else
if(supimg(u,v) >= tl && supimg(u,v) <= th )
resimg(u,v) = 1;
else
if (supimg(u,v) < tl)
resimg(u,v) = 0;
end
end
end
if (supimg(u-1,v-1) > th || supimg(u,v-1) > th || supimg(u+1,v-1) > th || supimg(u+1,v) > th || supimg(u+1,v+1) > th || supimg(u,v+1) > th || supimg(u-1,v+1) > th || supimg(u-1,v) > th)
resimg(u,v) = 1;
else
resimg(u,v) = 0;
end
end
end
figure,imshow(supimg);
figure,imshow(resimg);
However, for some of the images it is working fine, while for others it is showing the following error :
Index exceeds matrix dimensions.
Error in canny_edge (line 45)
sum = sum + (ID(u+i, v+j) * filter(i+3, j+3));
Can someone help me sort out this problem ??
Thanks and Regards.
Your loop ranges are in the wrong order leading to the error. If you modify your loop ranges to this
for u = 3 : r
for v = 3 : c
sum = 0;
for i = -2 : 2
for j = -2 : 2
sum = sum + (ID(u+i, v+j) * filter(i+3, j+3));
end
end
IDx(u,v) = sum;
end
end
the problem is solved.
My guess is that the code worked only for square images with c==r.
Note you are not making use of Matlab's vectorization capability, which allows you to shorten the first steps to:
ID = [zeros(2,c+4) ; [zeros(r,2) IDtemp zeros(r,2)]; zeros(2,c+4)];
filter=[2 4 5 4 2;4 9 12 9 4;5 12 15 12 5;4 9 12 9 4;2 4 5 4 2];
filter=filter/159;
for u = 1 : r
for v = 1 : c
IDx(u,v) = sum(reshape(ID(u+[0:4], v+[0:4]).* filter,25,1));
end
end
and this last loop can also be collapsed further but that might make readability an issue.
(edit) The loop can (for instance) be replaced with
IDx = conv2(ID, filter,'same');
I am running the loop in this method for around 1 million times but it is taking a lot of time maybe due O(n^2) , so is there any way to improve these two modules :-
def genIndexList(length,ID):
indexInfoList = []
id = list(str(ID))
for i in range(length):
i3 = (str(decimalToBase3(i)))
while len(i3) != 12:
i3 = '0' + i3
p = (int(str(ID)[0]) + int(i3[0]) + int(i3[2]) + int(i3[4]) + int(i3[6]) + int(i3[8]) + int(i3[10]))%3
indexInfoList.append(str(ID)+i3+str(p))
return indexInfoList
and here is the method for to convert number to base3 :-
def decimalToBase3(num):
i = 0
if num != 0 and num != 1 and num != 2:
number = ""
while num != 0 :
remainder = num % 3
num = num / 3
number = str(remainder) + number
return int(number)
else:
return num
I am using python to make a software and these 2 functions are a part of it.Please suggest why these 2 methods are so slow and how to improve efficiency of these methods.
The first function can be reduced to:
def genIndexList(length, ID):
indexInfoList = []
id0 = str(ID)[0]
for i in xrange(length):
i3 = format(decimalToBase3(i), '012d')
p = sum(map(int, id0 + i3[::2])) % 3
indexInfoList.append('{}{}{}'.format(ID, i3, p))
return indexInfoList
You may want to make it a generator instead:
def genIndexList(length, ID):
id0 = str(ID)[0]
for i in xrange(length):
i3 = format(decimalToBase3(i), '012d')
p = sum(map(int, id0 + i3[::2])) % 3
yield '{}{}{}'.format(ID, i3, p)
The second function could be:
def decimalToBase3(num):
if 0 <= num < 3: return num
result = ""
while num:
num, digit = divmod(num, 3)
result = str(digit) + result
return int(result)
Next step; you are just generating a sequence of base-3 digits. Just generate these directly:
from itertools import product, imap
def base3sequence(l=12, digits='012'):
return imap(''.join, product(digits, repeat=l))
This produces base3 values, 0-padded to 12 digits:
>>> gen = base3sequence()
>>> for i in range(10):
... print next(gen)
...
000000000000
000000000001
000000000002
000000000010
000000000011
000000000012
000000000020
000000000021
000000000022
000000000100
and genIndexList() becomes:
from itertools import islice
def genIndexList(length, ID):
id0 = str(ID)[0]
for i3 in islice(base3sequence(), length):
p = sum(map(int, id0 + i3[::2])) % 3
yield '{}{}{}'.format(ID, i3, p)