Is it possible to index columns in a Rust ndarray matrix using a Vec rather than a Slice object? The only documentation I can find pertains to slicing using contiguous columns
Specifically, I am trying to implement something like the following code in Python:
x = np.array([[1,2,3,4,5,6], [7,8,9,10,11,12]])
idx = [0,1,2,4]
x[:, idx]
The outcome of x[:, idx] would be the subset of the matrix containing all rows and only columns described in idx, i.e., [0,1,2,4].
I am currently using ndarray (as the title suggests) but I cannot find a way to subset on non-contiguous slices. For instance, you can pass ndarray, which can take a Slice with a start, stop and an index, but I cannot find a way to pass a list of columns that cannot be described using a Slice object.
For instance:
#[macro_use]
extern crate ndarray;
fn main() {
let mut x = array![[1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12]];
let idx = vec![0, 1, 2, 4];
// The following works as expected
let y = x.slice(s![.., 0..2]);
println!("{:?}", y);
// This is conceptually what I would like to do but
// It does not work.
let y = x.slice(s![.., idx]);
}
The analogue of "advanced indexing" in Numpy is the ArrayBase::select() method:
use ndarray::{array, Axis};
fn main() {
let x = array![[1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12]];
let idx = vec![0, 1, 2, 4];
println!("{}", x.select(Axis(1), &idx));
}
producing the output
[[1, 2, 3, 5],
[7, 8, 9, 11]]
Note that the resulting array is a copy of the selected elements (just as it is in Numpy). Depending on your use case, you may not need the copy; it's possible you can make do with just iterating over idx and using x.slice(s![.., i]) for all i in idx.
I have a List>
List<List<Byte>> bytes = new List<List<Byte>>()
{
new List<Byte> {1, 1, 2, 3, 4}, // index 0 is original
new List<Byte> {0, 0, 2, 4, 1},
new List<Byte> {1, 2, 2, 1, 1},
new List<Byte> {1, 0, 2, 2, 2}
};
and the first List is original. Then I sort my List, and I have to find my original List index.
bytes:
[0] = {0, 0, 2, 4, 1}
[1] = {1, 0, 2, 2, 2}
[2] = {1, 1, 2, 3, 4} // here is that index
[3] = {1, 2, 2, 1, 1}
One guy suggested me to use reference on first not sorted List then sort List and use ReferenceEqual, but this doesn't work for me. Am I doing something wrong by setting reference or it doesn't work on List? How could I get the index of original in sorted array? P.S I'm sorting with OrderBy and IComparer.
This is how I try to make reference:
List<byte> reference = new List<byte>(bytes[0]);
This is how I try to make reference:
List<byte> reference = new List<byte>(bytes[0]);
That's not the right way to make a reference, because you make a copy by calling a constructor of List<byte>. The copy of bytes[0] is not present in bytes, so you wouldn't be able to find it by checking reference equality.
This is how you should do it:
List<byte> reference = bytes[0];
Now reference references the list that is at position zero before sorting, so you should be able to find its index by using reference equality.
To find all permutations with the length of two one could use the follow simple program:
#include <iostream>
using namespace std;
int main(int argc, const char *argv[])
{
int l[] = {0, 1, 2, 3, 4, 5};
const int length = sizeof(l)/sizeof(l[0]);
for(int i = 0; i + 1 < length; i++)
for(int j = i + 1; j < length; j++)
cout << "(" << l[i] << ", " << l[j] << ")" << endl;
return 0;
}
But in the application where I need this, the single items are BIG, and needs to be constructed before the set can be used. Therefor I am trying to find algorithm, which does the same but with blocking. The blocking should allow me to have a bank which can be used for caching.
The following illustrates one (manually) created sequence with a bank, which can hold 4 items:
SETS, Cache miss, bank
(0,1) * * 0, 1
(0,2) * 0, 1, 2
(0,3) * 0, 1, 2, 3
(1,2) 0, 1, 2, 3
(1,3) 0, 1, 2, 3
(2,3) 0, 1, 2, 3
(0,4) * 0, 1, 2, 4
(1,4) 0, 1, 2, 4
(2,4) 0, 1, 2, 4
(0,5) * 0, 1, 4, 5
(1,5) 0, 1, 4, 5
(4,5) 0, 1, 4, 5
(2,5) 0, 2, 4, 5
(3,4) * 3, 2, 4, 5
(3,5) 3, 2, 4, 5
Does any of you know a solution to this problem? or can you point in the right direction.
--
Allan
Create a class cache with a method get(int ID), which returns the item with the corresponding ID:
(1) If the corresponding item is already cached, return the cached copy (or a pointer to it)
(2) If it is not: create it, remove an item from the cache, add the new item to the cache, and proceed with (1)
The hard part is to decide which item to remove from the cache in (2). The optimal choice is to remove the item which will not be needed for the longest time. If this is not practical (e.g. because you don't know enough about the access pattern), there are many alternatives.
The most common ones are:
LRU: remove the least recently used item
MRU: remove the most recently used item
LFU: remove the least frequently used item
See the Wikipedia article about cache algorithms for more examples.
I reduced a debugging problem in Mathematica 8 to something similar to the following code:
f = Function[x,
list = {1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5};
Count[list, x]
];
f[4]
Maximize{f[x], x, Integers]
Output:
4
{0, {x->0}}
So, while the maximum o function f is obtained when x equals 4 (as confirmed in the first output line), why does Maximize return x->0 (output line 2)?
The reason for this behavior can be easily found using Trace. What happens is that your function is evaluated inside Maximize with still symbolic x, and since your list does not contain symbol x, results in zero. Effectively, you call Maximize[0,x,Integers], hence the result. One thing you can do is to protect the function from immediate evaluation by using pattern-defined function with a restrictive pattern, like this for example:
Clear[ff];
ff[x_?IntegerQ] :=
With[{list = {1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5}}, Count[list, x]]
It appears that Maximize can not easily deal with it however, but NMaximize can:
In[73]:= NMaximize[{ff[x], Element[x, Integers]}, x]
Out[73]= {4., {x -> 4}}
But, generally, either of the Maximize family functions seem not quite appropriate for the job. You may be better off by explicitly computing the maximum, for example like this:
In[78]:= list = {1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5};
Extract[#, Position[#, Max[#], 1, 1] &[#[[All, 2]]]] &[Tally[list]]
Out[79]= {{4, 4}}
HTH
Try this:
list = {1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5};
First#Sort[Tally[list], #1[[2]] > #2[[2]] &]
Output:
{4, 4}
My friend was asked a question in his interview:
The interviewer gave him an array of unsorted numbers and asked him to sort. The restriction is that the number of writes should be minimized while there is no limitation on the number of reads.
Selection sort is not the right algorithm here. Selection sort will swap values, making up to two writes per selection, giving a maximum of 2n writes per sort.
An algorithm that's twice as good as selection sort is "cycle" sort, which does not swap. Cycle sort will give a maximum of n writes per sort. The number of writes is absolutely minimized. It will only write a number once to its final destination, and only then if it's not already there.
It is based on the idea that all permutations are products of cycles and you can simply cycle through each cycle and write each element to its proper place once.
import java.util.Random;
import java.util.Collections;
import java.util.Arrays;
public class CycleSort {
public static final <T extends Comparable<T>> int cycleSort(final T[] array) {
int writes = 0;
// Loop through the array to find cycles to rotate.
for (int cycleStart = 0; cycleStart < array.length - 1; cycleStart++) {
T item = array[cycleStart];
// Find where to put the item.
int pos = cycleStart;
for (int i = cycleStart + 1; i < array.length; i++)
if (array[i].compareTo(item) < 0) pos++;
// If the item is already there, this is not a cycle.
if (pos == cycleStart) continue;
// Otherwise, put the item there or right after any duplicates.
while (item.equals(array[pos])) pos++;
{
final T temp = array[pos];
array[pos] = item;
item = temp;
}
writes++;
// Rotate the rest of the cycle.
while (pos != cycleStart) {
// Find where to put the item.
pos = cycleStart;
for (int i = cycleStart + 1; i < array.length; i++)
if (array[i].compareTo(item) < 0) pos++;
// Put the item there or right after any duplicates.
while (item.equals(array[pos])) pos++;
{
final T temp = array[pos];
array[pos] = item;
item = temp;
}
writes++;
}
}
return writes;
}
public static final void main(String[] args) {
final Random rand = new Random();
final Integer[] array = new Integer[8];
for (int i = 0; i < array.length; i++) { array[i] = rand.nextInt(8); }
for (int iteration = 0; iteration < 10; iteration++) {
System.out.printf("array: %s ", Arrays.toString(array));
final int writes = cycleSort(array);
System.out.printf("sorted: %s writes: %d\n", Arrays.toString(array), writes);
Collections.shuffle(Arrays.asList(array));
}
}
}
A few example runs :
array: [3, 2, 6, 1, 3, 1, 4, 4] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 6
array: [1, 3, 4, 1, 3, 2, 4, 6] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 4
array: [3, 3, 1, 1, 4, 4, 2, 6] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 6
array: [1, 1, 3, 2, 4, 3, 6, 4] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 6
array: [3, 2, 3, 4, 6, 4, 1, 1] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 7
array: [6, 2, 4, 3, 1, 3, 4, 1] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 6
array: [6, 3, 2, 4, 3, 1, 4, 1] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 5
array: [4, 2, 6, 1, 1, 4, 3, 3] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 7
array: [4, 3, 3, 1, 2, 4, 6, 1] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 7
array: [1, 6, 4, 2, 4, 1, 3, 3] sorted: [1, 1, 2, 3, 3, 4, 4, 6] writes: 7
array: [5, 1, 2, 3, 4, 3, 7, 0] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 5
array: [5, 1, 7, 3, 2, 3, 4, 0] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 6
array: [4, 0, 3, 1, 5, 2, 7, 3] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 8
array: [4, 0, 7, 3, 5, 1, 3, 2] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 7
array: [3, 4, 2, 7, 5, 3, 1, 0] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 7
array: [0, 5, 3, 2, 3, 7, 1, 4] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 6
array: [1, 4, 3, 7, 2, 3, 5, 0] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 7
array: [1, 5, 0, 7, 3, 3, 4, 2] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 7
array: [0, 5, 7, 3, 3, 4, 2, 1] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 4
array: [7, 3, 1, 0, 3, 5, 4, 2] sorted: [0, 1, 2, 3, 3, 4, 5, 7] writes: 7
If the array is shorter (ie less than about 100 elements) a Selection sort is often the best choice if you also want to reduce the number of writes.
From wikipedia:
Another key difference is that
selection sort always performs Θ(n)
swaps, while insertion sort performs
Θ(n2) swaps in the average and worst
cases. Because swaps require writing
to the array, selection sort is
preferable if writing to memory is
significantly more expensive than
reading. This is generally the case if
the items are huge but the keys are
small. Another example where writing
times are crucial is an array stored
in EEPROM or Flash. There is no other
algorithm with less data movement.
For larger arrays/lists Quicksort and friends will provide better performance, but may still likely need more writes than a selection sort.
If you're interested this is a fantastic sort visualization site that allows you to watch specific sort algorithms do their job and also "race" different sort algorithms against each other.
You can use a very naive algorithm that satisfies what you need.
The algorithm should look like this:
i = 0
do
search for the minimum in range [i..n)
swap a[i] with a[minPos]
i = i + 1
repeat until i = n.
The search for the minimum can cost you almost nothing, the swap costs you 3 writes, the i++ costs you 1..
This is named selection sort as stated by ash. (Sorry, I didn't knew it was selection sort :( )
One option for large arrays is as follows (assuming n elements):
Initialize an array with n elements numbered 0..n-1
Sort the array using any sorting algorithm. As the comparison function, compare the elements in the input set with the corresponding numbers (eg, to compare 2 and 4, compare the 2nd and 4th elements in the input set). This turns the array from step 1 into a permutation that represents the sorted order of the input set.
Iterate through the elements in the permutation, writing out the blocks in the order specified by the array. This requires exactly n writes, the minimum.
To sort in-place, in step 3 you should instead identify the cycles in the permutation, and 'rotate' them as necessary to result in sorted order.
The ordering I meant in O(n) is like the selection sort(the previous post) useful when you have a small range of keys (or you are ordering numbers between 2 ranges)
If you have a number array where numbers will be between -10 and 100, then you can create an array of 110 and be sure that all numbers will fit in there, if you consider repeated numbers the idea is the same, but you will have lists instead of numbers in the sorted array
the pseudo-idea is like this
N: max value of your array
tosort //array to be sorted
sorted = int[N]
for i = 0 to length(tosort)
do
sorted[tosort[i]]++;
end
finalarray = int[length(tosort)]
k = 0
for i = 0 to N
do
if ( sorted[i] > 0 )
finalarray[k] = i
k++;
endif
end
finalarray will have the final sorted array and you will have o(N) write operations, where N is the range of the array. Once again, this is useful when using keys inside a specific range, but perhaps its your case.
Best regards and good luck!