I have two functions:
def construct_heirarchy(csv_file):
heirarchy = defaultdict(dict)
for row in read_CSV(csv_file):
row = edit_csv_row_data(
row,
translate_ttypes=translate_ttypes,
include_countries=settings.GEO_COUNTRY.itervalues(),
skip_headers=SKIP_HEADERS,
translate_header=translate_header)
if not row:
continue
pid, _id, ttype = map(row.get, ('pid', 'id', 'type'))
if pid:
heirarchy[pid].setdefault('target', []).append(_id)
heirarchy[_id]['type'] = ttype
return heirarchy
and
def extract_csv_data(csv_file):
csv_data = dict()
for row in read_CSV(csv_file):
row = edit_csv_row_data(
row,
translate_ttypes=translate_ttypes,
include_countries=settings.GEO_COUNTRY.itervalues(),
skip_headers=SKIP_HEADERS,
translate_header=translate_header)
if not row:
continue
# yield row['id'], row
csv_data[row['id']] = row
return csv_data
I track time of these functions using time.time
Heirarchy -2.90870666504e-05
Extracting 1.49716997147
I don't understand why there is such a huge time difference. If I use second function as generator, time is
Extracting 1.90734863281e-06
But then I can't use csv_data.get
Could someone help me understand what is going wrong here and what is the optimized way to do this?
PS: CSV is 6.1 MB with 85263x7 length.
Related
I've tried this
def matrix():
A = [[0,1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
return(A[1:2][0])
I want the function to return the value 5 and 9
If you would like to get the first element of the 2nd and 3rd rows it is:
def matrix():
A = [[0,1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
return (A[1][0], A[2][0])
This returns (5,9) as a tuple.
If you would like to parameterize it as such:
def matrix(rows):
# rows is a tuple or array like (1,2) with the indexes
A = [[0,1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
result = [] # empty list
for i in rows:
result.append(A[i][0])
return result
After calling the method like this: matrix((1,2)), it returns [5,9] as a list.
I'm new to Ruby, so apologies if this is dead easy :-)
I have a .csv file with 5 columns. The first column has a record identifier (in this case a driver number) and the other 4 columns in each row have data relating to that record. For each record there are around 50 rows of data (just under 2,000 rows in total). The .csv file has a header row.
I need to read the .csv file and identify the last entry for each user, so I can move on to the next user. I've tried to get it to compare the first column and the entry in the next row.
I have this so far, it returns incorrect row numbers and they're anywhere between 1 and 5 rows out...?!?!
require 'csv-mapper'
Given(/^I compare the driver numbers from rows "(.*?)" to "(.*?)"$/) do |firstrow, lastrow|
data = CsvMapper.import('C:/auto_test_data/Courts code example csv.csv', headers: true) do
[dln]
end
row = firstrow.to_i
while row <= lastrow.to_i
#licnum1 = data.at(row).dln
#licnum2 = data.at(row+1).dln
if
#licnum2 == #licnum1
$newrecord = "same"
else
$newrecord = #licnum2
end
if
$newrecord != "same"
puts "Last row for #{#licnum1} is #{row}\n"
end
row = row + 1
end
end
This is the layout for the .csv file:
recordidentifier1 dataitem1 dataitem2 code descriptionforcomparison
recordidentifier1 dataitem1 dataitem2 code descriptionforcomparison
recordidentifier2 dataitem1 dataitem2 code descriptionforcomparison
recordidentifier2 dataitem1 dataitem2 code descriptionforcomparison
All help will be greatly appreciated.
Thanks,
Peter
Here's one way to do it
current_identifier = nil
(firstrow.to_i..lastrow.to_i).each do |row|
if current_identifer != data.at(row).dln # current row is new identifier
if current_identifier # this is not the first row
puts "Last row for #{current_identifier} is #{row-1}\n"
end
current_identifier = data.at(row).dln # remember current row
end
# we need to track the last row as the last for the current identifier
puts "Last row for #{current_identifier} is #{lastrow.to_i}\n"
Consider the following html
http://www.carbide-red.com/prog/test_table.html
I have worked out that I can move left to right on the columns using
browser.td(:text => "Equipment").parent.td(:index => "2").flash
to flash the 3rd column over on the line containing "Equipement"
But how can I move down a certain number of rows? I am having terrible luck using .tr & .rows, no matter how I try it just crashes out when using those. Even something as simple as
browser.tr(:text => "Equipment").flash
Am I just misunderstanding how tr/row works?
Specific Row/Column
It sounds like you have already calculated which row/column you want. You can get the cell at a specific row/column index by simply doing:
browser.table[row_index][column_index]
Where row_index and column_index are integers for the row and column you want (note that it is zero-based index).
Specific Row
You can also do the following to select rows based on an index:
browser.table.tr(:index, 1).flash
browser.table.row(:index, 2).flash
Note that .tr includes nested tables while .row ignores nested tables.
Update - Find Rows After Specific Row
To find a row after a specific row containing a certain text, determine the index of the specific row first. Then you can locate the other rows in relation to it. Here are some examples:
#Get the 3rd row down from the row containing the text 'Equipment'
starting_row_index = browser.table.rows.to_a.index{ |row| row.text =~ /Equipment/ }
offset = 3
row = browser.table.row(:index, starting_row_index + offset)
puts row.text
# => CAT03 ...
#Get the 3rd row down from the row containing a cell with yellow background colour
starting_row_index = browser.table.rows.to_a.index{ |row| row.td(:css => "td[bgcolor=yellow]").present? }
offset = 3
row = browser.table.row(:index, starting_row_index + offset)
puts row.text
# => ETS36401 ...
#Output the first column text of each row after the row containing a cell with yellow background colour
starting_row_index = browser.table.rows.to_a.index{ |row| row.td(:css => "td[bgcolor=yellow]").present? }
(starting_row_index + 1).upto(browser.table.rows.length - 1){ |x| puts browser.table[x][0].text }
# => CAT03, CAT08, ..., INTEGRA10, INTEGRA11
Let me know if that helps or if you have a specific example you want.
I have a table called UserManagement that contains information about the user.This table gets updated whenever new user is created. If i create two users then i need check whether two users are actually created or not. Table contains ID,UserName,FirstName,LastName,Bdate..ctc. Here ID will be generated automatically.
I am running Selenium-TestNG script.Using Selenium,how can i get the UserName of the two users which i have created? Should i have to iterate through table? If so how to iterate through the table?
Use ISelenium.GetTable(string) to get the contents of the table cells you want. For example,
selenium.GetTable("UserManagement.0.1");
will return the contents of the table's first row and second column. You could then assert that the correct username or usernames appear in the table.
Get the count of rows using Selenium.getxpathcount(\#id = fjsfj\td\tr") in a variable rowcount
Give the columncount in a variable
Ex:
int colcount = 5;
Give the req i.e New user
String user1 = "ABC"
for(i = 0;i <=rowcount;i++)
{
for(j=0;j<=colcount;j++)
{
if (user1==selenium.gettable("//#[id=dldl/tbody" +i "td"+j))
{
system.out.println(user1 + "Inserted");
break;
}
break;
}
}
Get the number of rows using:
int noOfRowsInTable = selenium.getXpathCount("//table[#id='TableId']//tr");
If the UserName you want to get is at fixed position, let's say at 2nd position, then for each row iterate as given below:
selenium.getText("xpath=//table[#id='TableId']//tr//td[1]");
Note: we can find the number of columns in that table using same procedure
int noOfColumnsInTable = selenium.getXpathCount("//table[#id='TableId']//tr//td");
Generically, something like this?
table = #browser.table(:id,'tableID')
table.rows.each do |row|
# perform row operations here
row.cells.each do |cell|
# do cell operations here
end
end
I have a table that is filled with random content that a user enters. I want my users to be able to rapidly search through this table, and one way of facilitating their search is by sorting the table alphabetically. Originally, the table looked something like this:
myTable = {
Zebra = "black and white",
Apple = "I love them!",
Coin = "25cents"
}
I was able to implement a pairsByKeys() function which allowed me to output the tables contents in alphabetical order, but not to store them that way. Because of the way the searching is setup, the table itself needs to be in alphabetical order.
function pairsByKeys (t, f)
local a = {}
for n in pairs(t) do
table.insert(a, n)
end
table.sort(a, f)
local i = 0 -- iterator variable
local iter = function () -- iterator function
i = i + 1
if a[i] == nil then
return nil
else
return a[i], t[a[i]]
end
end
return iter
end
After a time I came to understand (perhaps incorrectly - you tell me) that non-numerically indexed tables cannot be sorted alphabetically. So then I started thinking of ways around that - one way I thought of is sorting the table and then putting each value into a numerically indexed array, something like below:
myTable = {
[1] = { Apple = "I love them!" },
[2] = { Coin = "25cents" },
[3] = { Zebra = "black and white" },
}
In principle, I feel this should work, but for some reason I am having difficulty with it. My table does not appear to be sorting. Here is the function I use, with the above function, to sort the table:
SortFunc = function ()
local newtbl = {}
local t = {}
for title,value in pairsByKeys(myTable) do
newtbl[title] = value
tinsert(t,newtbl[title])
end
myTable = t
end
myTable still does not end up being sorted. Why?
Lua's table can be hybrid. For numerical keys, starting at 1, it uses a vector and for other keys it uses a hash.
For example, {1="foo", 2="bar", 4="hey", my="name"}
1 & 2, will be placed in a vector, 4 & my will be placed in a hashtable. 4 broke the sequence and that's the reason for including it into the hashtable.
For information on how to sort Lua's table take a look here: 19.3 - Sort
Your new table needs consecutive integer keys and needs values themselves to be tables. So you want something on this order:
SortFunc = function (myTable)
local t = {}
for title,value in pairsByKeys(myTable) do
table.insert(t, { title = title, value = value })
end
myTable = t
return myTable
end
This assumes that pairsByKeys does what I think it does...