Doing Sum() on a set of anonymous type: is it possible? - linq

If I select an anonymous type (select new) from a query:
var smsPerGroup = from g in db.Groups
select new
{
GroupName = g.Name,
ReceivedSMS = g.Members.SelectMany(p => p.ReceivedSMS).Count()
};
It happends that I can't sum() on it:
int max = smsPerGroup.Max(g => g.ReceivedSMS); // ERROR!
The error thrown is:
There was an error parsing the query.
What is the most correct solution for this?
I don't want to create a class to use ONLY one time, on this query.
Is it possible to have a "not very anonymous" type? this is: to define the type of its properties but not the type of the class itself
Something like this: (which is syntactically incorrect)
...
select new
{
string GroupName = g.Name,
int ReceivedSMS = g.Members.SelectMany(p => p.ReceivedSMS).Count()
};
Edit: the error in detail is the following
There was an error parsing the query. [ Token line number = 6,Token line offset = 5,Token in error = SELECT ]
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Data.SqlServerCe.SqlCeException: There was an error parsing the query. [ Token line number = 6,Token line offset = 5,Token in error = SELECT ]
Source Error:
Line 159: };
Line 160:
Line 161: int max = smsPerGroup.Max(g => g.ReceivedSMS);

A simple solution would be to materialize the query with .ToList() before calling .Sum():
var test = (from g in db.Groups
select new
{
GroupName = g.Name,
ReceivedSMS = g.Members.SelectMany(p => p.ReceivedSMS).Count()
}).ToList();
var sum = test.Sum(t => t.ReceivedSMS);

Related

IGrouping does not contain a definition for Sum

Would anyone have an idea what's up here? I'm loading an XML file, parsing it and using LINQ to get some summary info. Here's the XML document:
<?xml version="1.0" encoding="utf-8"?>
<root>
<Ticker>
<Name>MSFT</Name>
<PurchaseDate>
2009-01-01
</PurchaseDate>
<Shares>44</Shares>
</Ticker>
<Ticker>
<Name>MSFT</Name>
<PurchaseDate>
2009-03-01
</PurchaseDate>
<Shares>33</Shares>
</Ticker>
<Ticker>
<Name>GOOG</Name>
<PurchaseDate>
2009-03-01
</PurchaseDate>
<Shares>122</Shares>
</Ticker>
<Ticker>
<Name>AAPL</Name>
<PurchaseDate>
2019-03-01
</PurchaseDate>
<Shares>89</Shares>
</Ticker>
</root>
My code:
var xmlStr = File.ReadAllText("ClientInfo.xml");
var Summary = XElement.Parse(xmlStr);
var data = from acct in Summary.Elements("Ticker")
select new
{
Ticker = (string)acct.Element("Name"),
Shares = (int)acct.Element("Shares")
};
var groupedDataByTicker = from acct in data
group acct by acct.Ticker into g
select new
{
Ticker = g.Key,
Shares = g.Sum(),
};
Works fine but when I add the g.Sum() bit I get this message:
'IGrouping>' does not contain a definition for 'Sum' and the best extension method overload 'Queryable.Sum(IQueryable)' requires a receiver of type 'IQueryable'
I can't make sense of it, what am I doing wrong?
In your code, you are trying to find the sum of the group itself.
Instead, you should be summing by the Shares property:
var groupedDataByTicker = from acct in data
group acct by acct.Ticker into g
select new
{
Ticker = g.Key,
Shares = g.Sum(row => row.Shares),
};

Oracle decode logic implementation using Slick

I have following problem - there is sql with DECODE oracle function:
SELECT u.URLTYPE, u.URL
FROM KAA.ENTITYURLS u
JOIN KAA.ENTITY e
ON decode(e.isurlconfigured, 0, e.urlparentcode, 1, e.CODE,
NULL)=u.ENTITYCODE
JOIN CASINO.Casinos c ON e.casinocode = c.code
WHERE e.NAME = $entityName
AND C.NAME = $casinoName
I'm trying to realize this sql in my slick code , like:
val queryUrlsEntityName = for {
entityUrl <- entityUrls
entity <- entities.filter(e => e.name.trim===entityName &&
entityUrl.entityCode.asColumnOf[Option[Int]]==(e.isURLConfigured match
{
case Some(0) => e.urlParentCode
case Some(1) => e.code.asColumnOf[Option[Int]]
case _ => None
}
)
)
casino <- casinos.filter(_.name.trim===casinoName) if
entity.casinoCode==casino.code
} yield (entityUrl)
But I don't understand how can I implement of matching of values in line
case Some(0) => e.urlParentCode
because I'm getting error
constructor cannot be instantiated to expected type;
[error] found : Some[A]
[error] required: slick.lifted.Rep[Option[Int]]
[error] case Some(0) => e.urlParentCode
Thanks for any advice
You should rewrite your code in pattern-matching section so you could compare required Rep[Option[Int]] - to left type, in your case it's Option[Int], or transform Rep[Option[Int]] to Option[Int] type. Rep is only the replacement to the column datatype in slick. I would prefer the first variant - this answer shows how to make the transformation from Rep, or you can use map directly:
map(u => (u.someField)).result.map(_.headOption.match {
case Some(0) => .....
})

How to get the fields from a WorkItem using a WIQL in one call to speed up?

I've have the following code to get some specific WorkItems:
string workItemQueryString = "Select Id, State, Type From WorkItems Where [Work Item Type] = 'Code Review Request' And [Area Path] = 'abc' Order By [Changed Date] Desc";
var workItemQuery = new Query(workItemStore, workItemQueryString);
WorkItemCollection queryResults = workItemQuery.RunQuery();
This code runs fast (< 1 sec). However I also want to get some extra fields like "Associated Context Type" and "Associated Context".
So I use this code get get those fields:
var workItemDetails = queryResults.Cast<WorkItem>().Select(workItem => new WorkItemDetail
{
WorkItem = workItem,
AssociatedContextType = workItem.Fields.Contains("Associated Context Type") ? workItem.Fields["Associated Context Type"].Value : null,
AssociatedContext = workItem.Fields.Contains("Associated Context") ? workItem.Fields["Associated Context"].Value : null
}).ToList();
But this code runs very slow (13 to 20 seconds) which looks to me that separate queries (for each workitem?) are fired to the TFS server to get all data.
Note that when I use a Parallel.ForEach statement, the code breaks with an exception.
The total number of WorkItems in the WorkItemCollection is about 2800.
Try to change:
AssociatedContextType = workItem.Fields.Contains("AssociatedContextType") ? workItem.Fields["AssociatedContextType"].Value : null,
AssociatedContext = workItem.Fields.Contains("AssociatedContext") ? workItem.Fields["AssociatedContext"].Value : null
to:
AssociatedContextType = workItem.Fields.Contains("Associated Context Type") ? workItem.Fields["Associated Context Type"].Value : null,
AssociatedContext = workItem.Fields.Contains("Associated Context") ? workItem.Fields["Associated Context"].Value : null

GA API Filter Failure: Invalid value 'ga:userGender == female' for filters parameter."

I tried to add a filter to extract data only from female users, but encounter the following error:
Invalid value 'ga:userGender == female' for filters parameter.
This is my code:
query.list <- Init(start.date = "2015-10-09", end.date = "2016-10-09", dimensions = "ga:userAgeBracket", metrics = "ga:users,ga:pageviews,ga:sessions", filter = "ga:userGender == female", max.results = 10000, table.id = "ga:xxxxxxxx")
ga.query <- QueryBuilder(query.list) cga.data <- GetReportData(ga.query, oauth_token)
I do not understand what I did wrong -- I thought my code was correct.

read json key-values with hive/sql and spark

I am trying to read this json file into a hive table, the top level keys i.e. 1,2.., here are not consistent.
{
"1":"{\"time\":1421169633384,\"reading1\":130.875969,\"reading2\":227.138275}",
"2":"{\"time\":1421169646476,\"reading1\":131.240628,\"reading2\":226.810211}",
"position": 0
}
I only need the time and readings 1,2 in my hive table as columns ignore position.
I can also do a combo of hive query and spark map-reduce code.
Thank you for the help.
Update , here is what I am trying
val hqlContext = new HiveContext(sc)
val rdd = sc.textFile(data_loc)
val json_rdd = hqlContext.jsonRDD(rdd)
json_rdd.registerTempTable("table123")
println(json_rdd.printSchema())
hqlContext.sql("SELECT json_val from table123 lateral view explode_map( json_map(*, 'int,string')) x as json_key, json_val ").foreach(println)
It throws the following error :
Exception in thread "main" org.apache.spark.sql.hive.HiveQl$ParseException: Failed to parse: SELECT json_val from temp_hum_table lateral view explode_map( json_map(*, 'int,string')) x as json_key, json_val
at org.apache.spark.sql.hive.HiveQl$.createPlan(HiveQl.scala:239)
at org.apache.spark.sql.hive.ExtendedHiveQlParser$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:50)
at org.apache.spark.sql.hive.ExtendedHiveQlParser$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:49)
at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
This would work, if you rename "1" and "2" (key names) to "x1" and "x2" (inside the json file or in the rdd):
val resultrdd = sqlContext.sql("SELECT x1.time, x1.reading1, x1.reading1, x2.time, x2.reading1, x2.reading2 from table123 ")
resultrdd.flatMap(row => (Array( (row(0),row(1),row(2)), (row(3),row(4),row(5)) )))
This would give you an RDD of tuples with time, reading1 and reading2. If you need a SchemaRDD, you would map it to a case class inside the flatMap transformation, like this:
case class Record(time: Long, reading1: Double, reading2: Double)
resultrdd.flatMap(row => (Array( Record(row.getLong(0),row.getDouble(1),row.getDouble(2)),
Record(row.getLong(3),row.getDouble(4),row.getDouble(5)) )))
val schrdd = sqlContext.createSchemaRDD(resultrdd)
Update:
In the case of many nested keys, you can parse the row like this:
val allrdd = sqlContext.sql("SELECT * from table123")
allrdd.flatMap(row=>{
var recs = Array[Record]();
for(col <- (0 to row.length-1)) {
row(col) match {
case r:Row => recs = recs :+ Record(r.getLong(2),r.getDouble(0),r.getDouble(1));
case _ => ;
}
};
recs
})

Resources