I have a two dimensional array of integers. I would like to write an optimized and fast code to sum all the columns of the two dimensional array.
Any thoughts how I might be able to do this using LINQ/PLINQ/TASK parallelization ?
Ex:
private int[,] m_indexes = new int[6,4] { {367, 40, 74, 15},
{535, 226, 74, 15},
{368, 313, 74, 15},
{197, 316, 74, 15},
{27, 226, 74, 15},
{194, 41, 74, 15} };
The simplest parallel implementation:
int[,] m_indexes = new int[6, 4] { {367, 40, 74, 15},
{535, 226, 74, 15},
{368, 313, 74, 15},
{197, 316, 74, 15},
{27, 226, 74, 15},
{194, 41, 74, 15} };
var columns = Enumerable.Range(0, 4);
int[] sums = new int[4];
Parallel.ForEach(columns, column => {
int sum = 0;
for (int i = 0; i < 6; i++) {
sum += m_indexes[i, column];
}
sums[column] = sum;
});
This code can obviously be "generalized" (use m_indexes.GetLength(0) and m_indexes.GetLength(1)).
LINQ:
var sums = columns.Select(
column => {
int sum = 0;
for (int i = 0; i < 6; i++) {
sum += m_indexes[i, column];
} return sum;
}
).ToArray();
Be sure to profile on real-world data here if you truly need to optimize for performance here.
Also, if you truly care about optimizing for performance, try to load up your array so that you summing across rows. You'll get better locality for cache performance that way.
Or maybe without for's :
List<List<int>> m_indexes = new List<List<int>>() { new List<int>(){367, 40, 74, 15},
new List<int>(){535, 226, 74, 15},
new List<int>(){368, 313, 74, 15},
new List<int>(){197, 316, 74, 15},
new List<int>(){27, 226, 74, 15},
new List<int>(){194, 41, 74, 15} };
var res = m_indexes.Select(x => x.Sum()).Sum();
Straightforward LINQ way:
var columnSums = m_indexes.OfType<int>().Select((x,i) => new { x, col = i % m_indexes.GetLength(1) } )
.GroupBy(x => x.col)
.Select(x => new { Column = x.Key, Sum = x.Sum(g => g.x) });
It might not be worth it to parallelize. If you need to access the array by index, you spend some cycles on bounds checking, so, as always with performance, do measure it.
Related
i have tow queries
one is
List<int> Scores = new List<int>() { 97, 92, 81, 60 };
IEnumerable<int> queryHighScores =
from a in Scores
where a < 5
select a;
and the other is
int[] numbers = { 12, 43, 44, 323, 44, 45, 34 };
var query1 = from a in numbers
where a < 5
select a;
now anybody tell me what is IEnumerable and why we use this . both query return same result so why we use IEnumerable
I am learning to code in Khan academy. It uses processing as its base language. I learnt that Java doesn't have goto function with it. Then how to implement goto function using processing. Explanatio with snippet is appreciated.
Here's my code and I have comment the places where I want to use the goto functionality:
/* Returns either the index of the location in the array,
or -1 if the array did not contain the targetValue */
var doSearch = function(array, targetValue) {
var min = 0;
var max = array.length - 1;
var guess;
//startover:
if (guess === max || guess === min) {
//goto notFound;
}
guess = round((min + max) / 2);
if (targetValue === array[guess]) {
return guess;
} else {
if (targetValue > array[guess]) {
min = guess + 1;
//goto startover;
} else {
max = guess - 1;
//goto startover;
}
}
}
//notFound:
return -1;
};
var primes = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37,
41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97
];
var result = doSearch(primes, 73);
println("Found prime at index " + result);
//Program.assertEqual(doSearch(primes, 73), 20);
The best you could probably do is to use a for loop and use break or continue for gotos.
for(var unusedVariable = 0;1==1;unusedVariable++){//equivalent of "startover:"
if(condition1){
continue;//goto startover;
}
if(condition2){
continue;//goto startover;
}
if(condition1){
break;//goto notFound;
}
}//notFound:
We perform checksums of some data in sql server as follows:
declare #cs int;
select
#cs = CHECKSUM_AGG(CHECKSUM(someid, position))
from
SomeTable
where
userid = #userId
group by
userid;
This data is then shared with clients. We'd like to be able to repeat the checksum at the client end... however there doesn't seem to be any info about how the checksums in the functions above are calculated. Can anyone enlighten me?
On SQL Server Forum, at this page, it's stated:
The built-in CHECKSUM function in SQL Server is built on a series of 4 bit left rotational xor operations. See this post for more explanation.
The CHECKSUM function doesn't provide a very good quality checksum and IMO is pretty useless for most purposes. As far as I know the algorithm isn't published. If you want a check that you can reproduce yourself then use the HashBytes function and one of the standard, published algorithms such as MD5 or SHA.
//Quick hash sum of SQL and C # mirror Ukraine
private Int64 HASH_ZKCRC64(byte[] Data)
{
Int64 Result = 0x5555555555555555;
if (Data == null || Data.Length <= 0) return 0;
int SizeGlobalBufer = 8000;
int Ost = Data.Length % SizeGlobalBufer;
int LeftLimit = (Data.Length / SizeGlobalBufer) * SizeGlobalBufer;
for (int i = 0; i < LeftLimit; i += 64)
{
Result = Result
^ BitConverter.ToInt64(Data, i)
^ BitConverter.ToInt64(Data, i + 8)
^ BitConverter.ToInt64(Data, i + 16)
^ BitConverter.ToInt64(Data, i + 24)
^ BitConverter.ToInt64(Data, i + 32)
^ BitConverter.ToInt64(Data, i + 40)
^ BitConverter.ToInt64(Data, i + 48)
^ BitConverter.ToInt64(Data, i + 56);
if ((Result & 0x0000000000000080) != 0)
Result = Result ^ BitConverter.ToInt64(Data, i + 28);
}
if (Ost > 0)
{
byte[] Bufer = new byte[SizeGlobalBufer];
Array.Copy(Data, LeftLimit, Bufer, 0, Ost);
for (int i = 0; i < SizeGlobalBufer; i += 64)
{
Result = Result
^ BitConverter.ToInt64(Bufer, i)
^ BitConverter.ToInt64(Bufer, i + 8)
^ BitConverter.ToInt64(Bufer, i + 16)
^ BitConverter.ToInt64(Bufer, i + 24)
^ BitConverter.ToInt64(Bufer, i + 32)
^ BitConverter.ToInt64(Bufer, i + 40)
^ BitConverter.ToInt64(Bufer, i + 48)
^ BitConverter.ToInt64(Bufer, i + 56);
if ((Result & 0x0000000000000080)!=0)
Result = Result ^ BitConverter.ToInt64(Bufer, i + 28);
}
}
byte[] MiniBufer = BitConverter.GetBytes(Result);
Array.Reverse(MiniBufer);
return BitConverter.ToInt64(MiniBufer, 0);
#region SQL_FUNCTION
/* CREATE FUNCTION [dbo].[HASH_ZKCRC64] (#data as varbinary(MAX)) Returns bigint
AS
BEGIN
Declare #I64 as bigint Set #I64=0x5555555555555555
Declare #Bufer as binary(8000)
Declare #i as int Set #i=1
Declare #j as int
Declare #Len as int Set #Len=Len(#data)
if ((#data is null) Or (#Len<=0)) Return 0
While #i<=#Len
Begin
Set #Bufer=Substring(#data,#i,8000)
Set #j=1
While #j<=8000
Begin
Set #I64=#I64
^ CAST(Substring(#Bufer,#j, 8) as bigint)
^ CAST(Substring(#Bufer,#j+8, 8) as bigint)
^ CAST(Substring(#Bufer,#j+16,8) as bigint)
^ CAST(Substring(#Bufer,#j+24,8) as bigint)
^ CAST(Substring(#Bufer,#j+32,8) as bigint)
^ CAST(Substring(#Bufer,#j+40,8) as bigint)
^ CAST(Substring(#Bufer,#j+48,8) as bigint)
^ CAST(Substring(#Bufer,#j+56,8) as bigint)
if #I64<0 Set #I64=#I64 ^ CAST(Substring(#Bufer,#j+28,8) as bigint)
Set #j=#j+64
End;
Set #i=#i+8000
End
Return #I64
END
*/
#endregion
}
I figured out the CHECKSUM algorithm, at least for ASCII characters. I created a proof of it in JavaScript (see https://stackoverflow.com/a/59014293/9642).
In a nutshell: rotate 4 bits left and xor by a code for each character. The trick was figuring out the "XOR codes". Here's the table of those:
var xorcodes = [
0, 1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 11, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23,
24, 25, 26, 27, 28, 29, 30, 31,
0, 33, 34, 35, 36, 37, 38, 39, // !"#$%&'
40, 41, 42, 43, 44, 45, 46, 47, // ()*+,-./
132, 133, 134, 135, 136, 137, 138, 139, // 01234567
140, 141, 48, 49, 50, 51, 52, 53, 54, // 89:;<=>?#
142, 143, 144, 145, 146, 147, 148, 149, // ABCDEFGH
150, 151, 152, 153, 154, 155, 156, 157, // IJKLMNOP
158, 159, 160, 161, 162, 163, 164, 165, // QRSTUVWX
166, 167, 55, 56, 57, 58, 59, 60, // YZ[\]^_`
142, 143, 144, 145, 146, 147, 148, 149, // abcdefgh
150, 151, 152, 153, 154, 155, 156, 157, // ijklmnop
158, 159, 160, 161, 162, 163, 164, 165, // qrstuvwx
166, 167, 61, 62, 63, 64, 65, 66, // yz{|}~
];
The main thing to note is the bias towards alphanumerics (their codes are similar and ascending). English letters use the same code regardless of case.
I haven't tested high codes (128+) nor Unicode.
I have a matrix, IEnumerable<IEnumerable<int>> matrix, for example:
{ {10,23,16,20,2,4}, {22,13,1,33,21,11 }, {7,19,31,12,6,22}, ... }
and another array:
int[] arr={ 10, 23, 16, 20}
I want to filter the matrix on the condition that I group all rows of the matrix which contain the same number of elements from arr.
That is to say the first row in the matrix {10,23,16,20,2,4} has 4 numbers from arr, this array should be grouped with the rest of the rows with 4 numbers from arr.
better to use linq, thank you very much!
This worked for me:
private static void Main(string[] args)
{
int[] searchNums = new int[] {10, 23, 16, 20};
var groupByCount = from o in lists
group o by o.Count(num => searchNums.Contains(num)) into g
orderby g.Key
select g;
foreach(var grouping in groupByCount)
{
int countSearchNums = grouping.Key;
Console.WriteLine("Lists that have " + countSearchNums + " of the numbers:");
foreach(IEnumerable<int> list in grouping)
{
Console.WriteLine("{{ {0} }}", String.Join(", ", list.Select(o => o.ToString()).ToArray()));
}
}
Console.ReadKey();
}
private static List<List<int>> lists = new List<List<int>>
{
new List<int> {10, 23, 16, 20, 2, 4},
new List<int> {22, 13, 1, 33, 21, 11},
new List<int> {7, 19, 31, 12, 6, 22},
new List<int> {10, 13, 31, 12, 6, 22},
new List<int> {10, 19, 20, 16, 6, 22},
new List<int> {10},
new List<int> {10, 13, 16, 20},
};
Output:
Lists that have 0 of the numbers:
{ 22, 13, 1, 33, 21, 11 }
{ 7, 19, 31, 12, 6, 22 }
Lists that have 1 of the numbers:
{ 10, 13, 31, 12, 6, 22 }
{ 10 }
Lists that have 3 of the numbers:
{ 10, 19, 20, 16, 6, 22 }
{ 10, 13, 16, 20 }
Lists that have 4 of the numbers:
{ 10, 23, 16, 20, 2, 4 }
Due to the wonders of branch prediction, a binary search can be slower than a linear search through an array of integers. On a typical desktop processor, how big does that array have to get before it would be better to use a binary search? Assume the structure will be used for many lookups.
I've tried a little C++ benchmarking and I'm surprised - linear search seems to prevail up to several dozen items, and I haven't found a case where binary search is better for those sizes. Maybe gcc's STL is not well tuned? But then -- what would you use to implement either kind of search?-) So here's my code, so everybody can see if I've done something silly that would distort timing grossly...:
#include <vector>
#include <algorithm>
#include <iostream>
#include <stdlib.h>
int data[] = {98, 50, 54, 43, 39, 91, 17, 85, 42, 84, 23, 7, 70, 72, 74, 65, 66, 47, 20, 27, 61, 62, 22, 75, 24, 6, 2, 68, 45, 77, 82, 29, 59, 97, 95, 94, 40, 80, 86, 9, 78, 69, 15, 51, 14, 36, 76, 18, 48, 73, 79, 25, 11, 38, 71, 1, 57, 3, 26, 37, 19, 67, 35, 87, 60, 34, 5, 88, 52, 96, 31, 30, 81, 4, 92, 21, 33, 44, 63, 83, 56, 0, 12, 8, 93, 49, 41, 58, 89, 10, 28, 55, 46, 13, 64, 53, 32, 16, 90
};
int tosearch[] = {53, 5, 40, 71, 37, 14, 52, 28, 25, 11, 23, 13, 70, 81, 77, 10, 17, 26, 56, 15, 94, 42, 18, 39, 50, 78, 93, 19, 87, 43, 63, 67, 79, 4, 64, 6, 38, 45, 91, 86, 20, 30, 58, 68, 33, 12, 97, 95, 9, 89, 32, 72, 74, 1, 2, 34, 62, 57, 29, 21, 49, 69, 0, 31, 3, 27, 60, 59, 24, 41, 80, 7, 51, 8, 47, 54, 90, 36, 76, 22, 44, 84, 48, 73, 65, 96, 83, 66, 61, 16, 88, 92, 98, 85, 75, 82, 55, 35, 46
};
bool binsearch(int i, std::vector<int>::const_iterator begin,
std::vector<int>::const_iterator end) {
return std::binary_search(begin, end, i);
}
bool linsearch(int i, std::vector<int>::const_iterator begin,
std::vector<int>::const_iterator end) {
return std::find(begin, end, i) != end;
}
int main(int argc, char *argv[])
{
int n = 6;
if (argc < 2) {
std::cerr << "need at least 1 arg (l or b!)" << std::endl;
return 1;
}
char algo = argv[1][0];
if (algo != 'b' && algo != 'l') {
std::cerr << "algo must be l or b, not '" << algo << "'" << std::endl;
return 1;
}
if (argc > 2) {
n = atoi(argv[2]);
}
std::vector<int> vv;
for (int i=0; i<n; ++i) {
if(data[i]==-1) break;
vv.push_back(data[i]);
}
if (algo=='b') {
std::sort(vv.begin(), vv.end());
}
bool (*search)(int i, std::vector<int>::const_iterator begin,
std::vector<int>::const_iterator end);
if (algo=='b') search = binsearch;
else search = linsearch;
int nf = 0;
int ns = 0;
for(int k=0; k<10000; ++k) {
for (int j=0; tosearch[j] >= 0; ++j) {
++ns;
if (search(tosearch[j], vv.begin(), vv.end()))
++nf;
}
}
std::cout << nf <<'/'<< ns << std::endl;
return 0;
}
and my a couple of my timings on a core duo:
AmAir:stko aleax$ time ./a.out b 93
1910000/2030000
real 0m0.230s
user 0m0.224s
sys 0m0.005s
AmAir:stko aleax$ time ./a.out l 93
1910000/2030000
real 0m0.169s
user 0m0.164s
sys 0m0.005s
They're pretty repeatable, anyway...
OP says: Alex, I edited your program to just fill the array with 1..n, not run std::sort, and do about 10 million (mod integer division) searches. Binary search starts to pull away from linear search at n=150 on a Pentium 4. Sorry about the chart colors.
binary vs linear search http://spreadsheets.google.com/pub?key=tzWXX9Qmmu3_COpTYkTqsOA&oid=1&output=image
I don't think branch prediction should matter because a linear search also has branches. And to my knowledge there are no SIMD that can do linear search for you.
Having said that, a useful model would be to assume that each step of the binary search has a multiplier cost C.
C log2 n = n
So to reason about this without actually benchmarking, you would make a guess for C, and round n to the next integer. For example if you guess C=3, then it would be faster to use binary search at n=11.
Not many - but hard to say exactly without benchmarking it.
Personally I'd tend to prefer the binary search, because in two years time, when someone else has quadrupled the size of your little array, you haven't lost much performance. Unless I knew very specifically that it's a bottleneck right now and I needed it to be as fast as possible, of course.
Having said that, remember that there are hash tables too; you could ask a similar question about them vs. binary search.