Still learning here, so most likely there's a less complicated way to achieve my goals. This is a snippet I pulled from my code that 90% works as intended, but I'm failing on my last step.
inv = {}
for i = 23,mq.TLO.Me.NumBagSlots()+22 do
inv[i] = {}
for j = 1,mq.TLO.Me.Inventory(i).Container() do
inv[i][j]={Item=mq.TLO.Me.Inventory(i).Item(j),bag=i,slot=j}
end
end
local sortthatsucker={}
for _,bags in pairs(inv) do
for _,invcontainer in pairs(bags) do
for _,_ in pairs(invcontainer) do
local compactstring = tostring(invcontainer.Item).."-"..tostring(invcontainer.bag).."_"..tostring(invcontainer.slot)
if tostring(invcontainer.Item) ~= "NULL" then
table.insert(sortthatsucker,compactstring)
end
end
end
end
table.sort(sortthatsucker)
for k,v in pairs(sortthatsucker) do
print(k,"-",v)
end
Will currently extract, filter, and sort all items as I want, creating a compound string showing the item name and current "coords" if you will for simplified k,v table that's sortable by name, but due to I believe the 3 nested for loops to pull the data out, the table.insert will create three identical entries in the final table, albeit with unique key values for each entry.
ie:
163-shovel-23_10
164-shovel-23_10
165-shovel-23_10
166-tonsils-18_5
167-tonsils-18_5
168-tonsils-18_5
169-withers-6_12
170-withers-6_12
171-withers-6_12
etc...
My goal is to have this table with a single entry for each item found, and do a set of new nested for loops to pull the x and y values out of that string to interact with that item at that give coordinate and move it to a new x and y. If there was a way to create the initial table and interact with that, rather make it as a nested table to achieve the same result, that would also be beneficial, as I have not been able to figure that out after two weeks of trying and searching.
Related
I have a data series that contains various names of the same organizations. I want harmonize these names into a given standard using a mapping dictionary. I am currently using a nested for loop to iterate through each series element and if it is within the dictionary's values, I update the series value with the dictionary key.
# For example, corporation_series is:
0 'Corp1'
1 'Corp-1'
2 'Corp 1'
3 'Corp2'
4 'Corp--2'
dtype: object
# Dictionary is:
mapping_dict = {
'Corporation_1': ['Corp1', 'Corp-1', 'Corp 1'],
'Corporation_2': ['Corp2', 'Corp--2'],
}
# I use this logic to replace the values in the series
for index, value in corporation_series.items():
for key, list in mapping_dict.items():
if value in list:
corporation_series = corporation_series.replace(value, key)
So, if the series has a value of 'Corp1', and it exists in the dictionary's values, the logic replaces it with the corresponding key of corporations. However, it is an extremely expensive method. Could someone recommend me a better way of doing this operation? Much appreciated.
I found a solution by using python's .map function. In order to use .map, I had to invert my dictionary:
# Inverted Dict:
mapping_dict = {
'Corp1': ['Corporation_1'],
'Corp-1': ['Corporation_1'],
'Corp 1': ['Corporation_1'],
'Corp2': ['Corporation_2'],
'Corp--2':['Corporation_2'],
}
# use .map
corporation_series.map(newdict)
Instead of 5 minutes of processing, took around 5s. While this is works, I sure there are better solutions out there. Any suggestions would be most welcome.
I have to be honest that I don't quite understand Lua that well yet. I am trying to overwrite a local numeric value assigned to a set table address (is this the right term?).
The addresses are of the type:
project.models.stor1.inputs.T_in.default, project.models.stor2.inputs.T_in.default and so on with the stor number increasing.
I would like to do this in a for loop but cannot find the right expression to make the entire string be accepted by Lua as a table address (again, I hope this is the right term).
So far, I tried the following to concatenate the strings but without success in calling and then overwriting the value:
for k = 1,10,1 do
project.models.["stor"..k].inputs.T_in.default = 25
end
for k = 1,10,1 do
"project.models.stor"..j..".T_in.default" = 25
end
EDIT:
I think I found the solution as per https://www.lua.org/pil/2.5.html:
A common mistake for beginners is to confuse a.x with a[x]. The first form represents a["x"], that is, a table indexed by the string "x". The second form is a table indexed by the value of the variable x. See the difference:
for k = 1,10,1 do
project["models"]["stor"..k]["inputs"]["T_in"]["default"] = 25
end
You were almost close.
Lua supports this representation by providing a.name as syntactic sugar for a["name"].
Read more: https://www.lua.org/pil/2.5.html
You can use only one syntax in time.
Either tbl.key or tbl["key"].
The limitation of . is that you can only use constant strings in it (which are also valid variable names).
In square brackets [] you can evaluate runtime expressions.
Correct way to do it:
project.models["stor"..k].inputs.T_in.default = 25
The . in models.["stor"..k] is unnecessary and causes an error. The correct syntax is just models["stor"..k].
I have the following block of code that iterates through the fields of each table and adds the fields of the current table respectively in order to create a number of tableboxes.
'iterate through every table
For i=1 To arrTCount
'the arrFF array holds the names of the fields of each table
arrFF = Split(arrFields(i), ", ")
arrFFCount = UBound(arrFF)
'create a tablebox
Set TB = ActiveDocument.Sheets("Main").CreateTableBox
'iterate through the fields of the array
For j=0 to (arrFFCount - 1)
'add the field to the tablebox
TB.AddField arrFF(j)
'Msgbox(arrFF(j))
Next
Set tboxprop = TB.GetProperties
tboxprop.Layout.Frame.ObjectId = "TB" + CStr(i)
TB.SetProperties tboxprop
Next
The above code creates the tableboxes, but with one field less every time (the last one is missing). If I change the For loop from For j=0 To (arrFFCount - 1) to For j=0 To (arrFFCount) it creates empty tableboxes and seems to execute forever. Regarding this change, I tested the field names with the Msgbox(arrFF(j)) command and it shows me the correct field names as I want them to be in the tableboxes in the UI of QlikView.
Does anybody have an idea of what seems to be the problem here? Can I do this in a different way?
To clarify the situation here and what I have tested so far, I have 11 tables to make tableboxes of and I have tried with just one of them or some of them. The result I am seeing with the code is on the left and what I am expecting to see is on the right of the following image. Please note that the number of fields vary for each table and the image has just one of them as an example.
I am relatively new to coding and am learning ruby right now. I came across a problem where I have a huge data record (>100k record) consisting of unique ID and another consisting of the date of birth. So it's basically a 2D array. How do I go about creating a method such that every time when I key in method(year), it will give me all the unique ID of those born in the year i choose? And how do I loop this?
The method I tried doing is as follow:
def Id_with_year(year)
emPloyee_ID_for_searching_year = [ ]
employeelist.sort_by!{|a,b|b}
if employeelist.select{|a,b| b == year}.map{|a,b| a}
return emPloyee_ID_for_searching_year
end
end
I should point out that the ID are sorted. That's why I am trying to sort the year in this method so that it will give me all the ID for the year I key in. The output I had was that it returned me [ ] with nothing inside instead of the ID.
Sidenote: methods in ruby are to be named in snake case (this is not mandatory, though.)
The problem you experience is you return what was never changed. The below should work:
def id_with_year(year)
employeelist.sort_by(&:last) # sorting by last element of array
.select{|_,b| b == year} # select
.map(&:first) # map to the first element
end
This is similar to a question I asked before, but is slightly different:
So I have a very large structure array in matlab. Suppose, for argument's sake, to simplify the situation, suppose I have something like:
structure(1).name, structure(2).name, structure(3).name structure(1).returns, structure(2).returns, structure(3).returns (in my real program I have 647 structures)
Suppose further that structure(i).returns is a vector (very large vector, approximately 2,000,000 entries) and that a condition comes along where I want to delete the jth entry from structure(i).returns for all i. How do you do this? or rather, how do you do this reasonably fast? I have tried some things, but they are all insanely slow (I will show them in a second) so I was wondering if the community knew of faster ways to do this.
I have parsed my data two different ways; the first way had everything saved as cell arrays, but because things hadn't been working well for me I parsed the data again and placed everything as vectors.
What I'm actually doing is trying to delete NaN data, as well as all data in the same corresponding row of my data file, and then doing the very same thing after applying the Hampel filter. The relevant part of my code in this attempt is:
for i=numStock+1:-1:1
for j=length(stock(i).return):-1:1
if(isnan(stock(i).return(j)))
for k=numStock+1:-1:1
stock(k).return(j) = [];
end
end
end
stock(i).return = sort(stock(i).return);
stock(i).returnLength = length(stock(i).return);
stock(i).medianReturn = median(stock(i).return);
stock(i).madReturn = mad(stock(i).return,1);
end;
for i=numStock:-1:1
for j = length(stock(i+1).volume):-1:1
if(isnan(stock(i+1).volume(j)))
for k=numStock:-1:1
stock(k+1).volume(j) = [];
end
end
end
stock(i+1).volume = sort(stock(i+1).volume);
stock(i+1).volumeLength = length(stock(i+1).volume);
stock(i+1).medianVolume = median(stock(i+1).volume);
stock(i+1).madVolume = mad(stock(i+1).volume,1);
end;
for i=numStock+1:-1:1
for j=stock(i).returnLength:-1:1
if (abs(stock(i).return(j) - stock(i).medianReturn) > 3*stock(i).madReturn)
for k=numStock+1:-1:1
stock(k).return(j) = [];
end
end;
end;
end;
for i=numStock:-1:1
for j=stock(i+1).volumeLength:-1:1
if (abs(stock(i+1).volume(j) - stock(i+1).medianVolume) > 3*stock(i+1).madVolume)
for k=numStock:-1:1
stock(k+1).volume(j) = [];
end
end;
end;
end;
However, this returns an error:
"Matrix index is out of range for deletion.
Error in Failure (line 110)
stock(k).return(j) = [];"
So instead I tried by parsing everything in as vectors. Then I decided to try and delete the appropriate entries in the vectors prior to building the structure array. This isn't returning an error, but it is very slow:
%% Delete bad data, Hampel Filter
% Delete bad entries
id=strcmp(returns,'');
returns(id)=[];
volume(id)=[];
date(id)=[];
ticker(id)=[];
name(id)=[];
permno(id)=[];
sp500(id) = [];
id=strcmp(returns,'C');
returns(id)=[];
volume(id)=[];
date(id)=[];
ticker(id)=[];
name(id)=[];
permno(id)=[];
sp500(id) = [];
% Convert returns from string to double
returns=cellfun(#str2double,returns);
sp500=cellfun(#str2double,sp500);
% Delete all data for which a return is not a number
nanid=isnan(returns);
returns(nanid)=[];
volume(nanid)=[];
date(nanid)=[];
ticker(nanid)=[];
name(nanid)=[];
permno(nanid)=[];
% Delete all data for which a volume is not a number
nanid=isnan(volume);
returns(nanid)=[];
volume(nanid)=[];
date(nanid)=[];
ticker(nanid)=[];
name(nanid)=[];
permno(nanid)=[];
% Apply the Hampel filter, and delete all data corresponding to
% observations deleted by the filter.
medianReturn = median(returns);
madReturn = mad(returns,1);
for i=length(returns):-1:1
if (abs(returns(i) - medianReturn) > 3*madReturn)
returns(i) = [];
volume(i)=[];
date(i)=[];
ticker(i)=[];
name(i)=[];
permno(i)=[];
end;
end
medianVolume = median(volume);
madVolume = mad(volume,1);
for i=length(volume):-1:1
if (abs(volume(i) - medianVolume) > 3*madVolume)
returns(i) = [];
volume(i)=[];
date(i)=[];
ticker(i)=[];
name(i)=[];
permno(i)=[];
end;
end
As I said, this is very slow, probably because I'm using a for loop on a very large data set; however, I'm not sure how else one would do this. Sorry for the gigantic post, but does anyone have a suggestion as to how I might go about doing what I'm asking in a reasonable way?
EDIT: I should add that getting the vector method to work is probably preferable, since my aim is to put all of the return vectors into a matrix and get all of the volume vectors into a matrix and perform PCA on them, and I'm not sure how I would do that using cell arrays (or even if princomp would work on cell arrays).
EDIT2: I have altered the code to match your suggestion (although I did decide to give up speed and keep with the for-loops to keep with the structure array, since reparsing this data will be way worse time-wise). The new code snipet is:
stock_return = zeros(numStock+1,length(stock(1).return));
for i=1:numStock+1
for j=1:length(stock(i).return)
stock_return(i,j) = stock(i).return(j);
end
end
stock_return = stock_return(~any(isnan(stock_return)), : );
This returns an Index exceeds matrix dimensions error, and I'm not sure why. Any suggestions?
I could not find a convenient way to handle structures, therefore I would restructure the code so that instead of structures it uses just arrays.
For example instead of stock(i).return(j) I would do stock_returns(i,j).
I show you on a part of your code how to get rid of for-loops.
Say we deal with this code:
for j=length(stock(i).return):-1:1
if(isnan(stock(i).return(j)))
for k=numStock+1:-1:1
stock(k).return(j) = [];
end
end
end
Now, the deletion of columns with any NaN data goes like this:
stock_return = stock_return(:, ~any(isnan(stock_return)) );
As for the absolute difference from medianVolume, you can write a similar code:
% stock_return_length is a scalar
% stock_median_return is a column vector (eg. [1;2;3])
% stock_mad_return is also a column vector.
median_return = repmat(stock_median_return, stock_return_length, 1);
is_bad = abs(stock_return - median_return) > 3.* stock_mad_return;
stock_return = stock_return(:, ~any(is_bad));
Using a scalar for stock_return_length means of course that the return lengths are the same, but you implicitly assume it in your original code anyway.
The important point in my answer is using any. Logical indexing is not sufficient in itself, since in your original code you delete all the values if any of them is bad.
Reference to any: http://www.mathworks.co.uk/help/matlab/ref/any.html.
If you want to preserve the original structure, so you stick to stock(i).return, you can speed-up your code using essentially the same scheme but you can only get rid of one less for-loop, meaning that your program will be substantially slower.