Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I am building an application using Ruby 2.0 where after parsing a string to JSON, I'm interested in retrieving all of the game_id fields listed below. After parsing has been completed, is there a trivial way in Ruby to map/collect through the parsed JSON response to do that? Here's an example of the JSON I'm working with.
{
"season_type"=>"Regular",
"sport"=>"NBA",
"gamedate"=>"11/29/2013",
"updated_at"=>"2013-11-29 23:35:17 UTC",
"games"=>[
{
"game"=>{
"home_team_name"=>"Charlotte",
"game_id"=>19089,
"visiting_team_name"=>"Milwaukee",
"home_team_id"=>152,
"gametime"=>"07:00 PM",
"visiting_team_id"=>103
}
},
{
"game"=>{
"home_team_name"=>"Orlando",
"game_id"=>19091,
"visiting_team_name"=>"San Antonio",
"home_team_id"=>117,
"gametime"=>"07:00 PM",
"visiting_team_id"=>110
}
},
{
"game"=>{
"home_team_name"=>"Toronto",
"game_id"=>19092,
"visiting_team_name"=>"Miami",
"home_team_id"=>118,
"gametime"=>"07:00 PM",
"visiting_team_id"=>115
}
},
{
"game"=>{
"home_team_name"=>"Atlanta",
"game_id"=>19096,
"visiting_team_name"=>"Dallas",
"home_team_id"=>91,
"gametime"=>"07:30 PM",
"visiting_team_id"=>95
}
},
{
"game"=>{
"home_team_name"=>"Boston",
"game_id"=>19101,
"visiting_team_name"=>"Cleveland",
"home_team_id"=>92,
"gametime"=>"07:30 PM",
"visiting_team_id"=>94
}
},
{
"game"=>{
"home_team_name"=>"Detroit",
"game_id"=>19104,
"visiting_team_name"=>"LA Lakers",
"home_team_id"=>97,
"gametime"=>"07:30 PM",
"visiting_team_id"=>102
}
},
{
"game"=>{
"home_team_name"=>"Houston",
"game_id"=>19106,
"visiting_team_name"=>"Brooklyn",
"home_team_id"=>99,
"gametime"=>"08:00 PM",
"visiting_team_id"=>104
}
},
{
"game"=>{
"home_team_name"=>"Indiana",
"game_id"=>19110,
"visiting_team_name"=>"Washington",
"home_team_id"=>100,
"gametime"=>"08:00 PM",
"visiting_team_id"=>113
}
},
{
"game"=>{
"home_team_name"=>"Oklahoma City",
"game_id"=>19108,
"visiting_team_name"=>"Golden State",
"home_team_id"=>111,
"gametime"=>"08:00 PM",
"visiting_team_id"=>98
}
},
{
"game"=>{
"home_team_name"=>"Philadelphia",
"game_id"=>19112,
"visiting_team_name"=>"New Orleans",
"home_team_id"=>106,
"gametime"=>"08:00 PM",
"visiting_team_id"=>114
}
},
{
"game"=>{
"home_team_name"=>"Denver",
"game_id"=>19116,
"visiting_team_name"=>"New York",
"home_team_id"=>96,
"gametime"=>"09:00 PM",
"visiting_team_id"=>105
}
},
{
"game"=>{
"home_team_name"=>"Utah",
"game_id"=>19119,
"visiting_team_name"=>"Phoenix",
"home_team_id"=>112,
"gametime"=>"09:00 PM",
"visiting_team_id"=>107
}
},
{
"game"=>{
"home_team_name"=>"Sacramento",
"game_id"=>19123,
"visiting_team_name"=>"LA Clippers",
"home_team_id"=>109,
"gametime"=>"10:00 PM",
"visiting_team_id"=>101
}
}
]
}
Simple:
parsed_json['games'].map { |g| g['game']['game_id'] }
Related
Context:
Spring Boot (starter): 2.7.5
Spring Data MongoDB (starter): 2.7
Java version: 17
TLDR
I'd like to generate an aggregation step like this (tested with MongoCompass and works):
{ // "$match" omitted
"value.timestampWithoutTimezoneValue.value" : {
$gt : ISODate('Sat Jan 14 11:29:29 CET 2023')
}
}
but this is generated when I use MongoTemplate instead:
{
"$match":{
"value.timestampWithoutTimezonevalue.value":{
"$gt":{
"$date":"2023-01-14T10:29:29.499Z"
}
}
}
}
I can't really find the reason for this and I get zero results.
Debug details
This is where I generate the Criteria:
case GREATER_THAN -> {
// LocalDateTime: 2023-01-14T11:29:29.499999999
Date out = Date.from(((LocalDateTime) filter.getValue()).atZone(ZoneId.of("UTC")).toInstant()); // out = Sat Jan 14 12:29:29 CET 2023 (I assume it's +0)
yield Criteria.where("value.timestampWithoutTimezonevalue.value").gt(out); // See below
}
EDIT: I noticed that the conversion to Date caused a +1h instead of a +0h, but it's not relevant for this thread's sake. Keep reading and you'll understand.
This is the aggregation pipeline that I submit to MongoTemplate (I omitted the rest because the rest works as intended):
{
...
"pipeline":[
...
{
"$addFields":{
"value.timestampWithoutTimezoneValue.value":{
"$toDate":"$value.timestampWithoutTimezoneValue.value"
}
}
},
{
"$match":{
"value.timestampWithoutTimezonevalue.value":{
"$gt":{
"$date":"2023-01-14T11:29:29.499Z"
}
}
}
}
...
],
...
}
Note that:
Dates are stored in documents as Strings, e.g. "2023-01-07T12:30:30.500", therefore
I need to convert the string to a Date object before $matching
(Note that I'd rather work with Date objects because I might need to run more complex queries in future)
For completion's sake I share the small testing dataset I'm using (only the relevant field):
[{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-07T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-08T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-09T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-10T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-11T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-12T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-13T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-14T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-15T12:30:30.500"
}
}
},{
"value": {
"timestampWithoutTimezoneValue": {
"value": "2023-01-16T12:30:30.500"
}
}
}]
Expectation of the query result:
Return documents with dates:
"2023-01-14T12:30:30.500" (this should be discarded considering my +1h bug)
"2023-01-15T12:30:30.500"
"2023-01-16T12:30:30.500"
Actual Result: Zero results
Conclusions:
This is one of many attempts by the way... I tried to:
Querying by string without converting to Date
Converting string dates to Longs ($toLong) instead of Date objects and querying by timestamp
...
None of the above worked and my impression is that my code is ok. The problem I'm facing is that my aggregation pipeline isn't interpreted the way I expect to (structure-wise) and I can't find a way to make it work out of MongoCompass.
I have a Lambda hooked up to my DynamoDB stream. It is configured to trigger if both criteria are met:
eventName = "MODIFY"
status > 10
My filter looks as follows:
{"eventName": ["MODIFY"], "dynamodb": {"NewImage": {"status": [{"numeric": [">", 10]}]}}}
If the filter is configured to only trigger if the event name is MODIFY it works, however anything more complicated than that does not trigger my Lambda. The event looks as follows:
{
"eventID": "ba1cff0bb53fbd7605b7773fdb4320a8",
"eventName": "MODIFY",
"eventVersion": "1.1",
"eventSource": "aws:dynamodb",
"awsRegion": "us-east-1",
"dynamodb":
{
"ApproximateCreationDateTime": 1643637766,
"Keys":
{
"org":
{
"S": "test"
},
"id":
{
"S": "61f7ebff17afad170f98e046"
}
},
"NewImage":
{
"status":
{
"N": "20"
}
}
}
}
When using the test_event_pattern endpoint it confirms the filter is valid:
filter = {
"eventName": ["MODIFY"],
"dynamodb": {
"NewImage": {
"status": [ { "numeric": [ ">", 10 ] } ]
}
}
}
response = client.test_event_pattern(
EventPattern=json.dumps(filter),
Event="{\"id\": \"e00c66cb-fe7a-4fcc-81ad-58eb60f5d96b\", \"eventName\": \"MODIFY\", \"dynamodb\": {\"NewImage\":{\"status\": 20}}, \"detail-type\": \"myDetailType\", \"source\": \"com.mycompany.myapp\", \"account\": \"123456789012\", \"time\": \"2016-01-10T01:29:23Z\", \"region\": \"us-east-1\"}"
)
print(response) >> {'Result': True, 'ResponseMetadata': {'RequestId':...}
Is there something that I'm overlooking? Do DynamoDB filters not work on the actual new image?
probably already found out yourself but for anyone else
its missing the dynamodb json specific numeric field leaf:
{
"eventName": ["MODIFY"],
"dynamodb": {
"NewImage": {
"status": { "N": [{ "numeric": [">", 10] }] }
}
}
}
I have a somewhat complex JSON coming into a WEBAPI -
{
"TypeInq": "ResidentialStart",
"DataInq": {
"Account_ID": "2491054657",
"Premise_ID": "5394660000",
"DogWarning": "N",
"ServiceStartDate": "2020-01-10",
"PrimaryPerson": {
"PersonId": "0357801000",
"POS_ID_Required": "Y",
"LastName": "HOGAN",
"FirstName": "BARRY L",
"Username": "garmin789#gmail.com",
"EmailList": [
{
"Email": "garmin789#gmail.com",
"EmailType": "PERSONAL EMAIL",
"PrimaryEmailFlag": "Y",
"ActionFlag": "ADD",
"ActiveInactiveFlag": "C1AC"
}
],
"DateOfBirth": "1977-11-04",
"CurrentAddress": {
"AddressLine1": "1923 Chestnut Dr",
"City": "Pleasant hill ",
"State": "MO",
"Zip": "64080"
},
"MailingAddress": {
"PremiseID": "5394660000"
}
},
"AdditionalPersons": [
{
"PersonId": "0357801000",
"POS_ID_Required": "Y",
"LastName": "HOGAN",
"FirstName": "BARRY L",
"Username": "garmin789#gmail.com",
"EmailList": [
{
"Email": "garmin789#gmail.com",
"EmailType": "PERSONAL EMAIL",
"PrimaryEmailFlag": "Y",
"ActionFlag": "ADD",
"ActiveInactiveFlag": "C1AC"
}
],
"DateOfBirth": "1977-11-04",
"CurrentAddress": {
"AddressLine1": "1923 Chestnut Dr",
"City": "Pleasant hill ",
"State": "MO",
"Zip": "64080"
},
"MailingAddress": {
"PremiseID": "5394660000"
}
}
]
}
}
So the part 'TypeInq' seems to be fine
The Part 'DataInq' i need to convert all that into a string before I process it. Not 100% sure how to approach this in C# but was trying this where the 'Base' object just has strings in it for 'TypeInq' and 'DataInq'
[HttpPost]
[Route("CreateInquiry")]
public IHttpActionResult CreateNewInquiry(Base inqBasic)
{
string retval = "";
try
{
//return Ok();
InquiryService svc = new InquiryService();
retval = svc.CreateInquiry(inqBasic);
}
catch (Exception ex)
{
Logger.LogEx(ex);
return WebApiConfig.ExceptionResponse(this);
}
return Ok(retval);
}
You can directly deserialize the incoming json object. Then just reference the appropriate property you want the value of.
object somenewobject = new JavaScriptSerializer().Deserialize(response.Data.ToString());
I was able to come up with this:
{
repository(owner: "hawkular", name: "hawkular.github.io") {
pullRequest(number: 237) {
comments(first: 10) {
nodes {
body
author {
login
}
}
}
}
}
}
which returns all the comments on the given PR:
{
"data": {
"repository": {
"pullRequest": {
"comments": {
"nodes": [
{
"body": "PR was auto-deployed here: http://209.132.178.114:10237 and it will be available for 4 days.\n",
"author": {
"login": "hawkular-website-bot"
}
},
{
"body": "Any ETA for this blog post to be finished/published ?",
"author": {
"login": "theute"
}
},
{
"body": "After PR696 in hawkular-metrics I will continue.. helps me to create something repetable",
"author": {
"login": "burmanm"
}
},
{
"body": "PR was auto-deployed here: http://209.132.178.114:10237 and it will be available for 4 days.",
"author": {
"login": "hawkular-website-bot"
}
},
{
"body": "The article cites `CompressionData` job, but it should be `CompressData`.",
"author": {
"login": "jsanda"
}
},
{
"body": "Is this PR active ? Severla months without activity.",
"author": {
"login": "lucasponce"
}
}
]
}
}
}
}
however I am interested only in those written by hawkular-website-bot. Is there a way to specify the author in Git Hub Graph QL api?
There is no filter criterion on the author in type PullRequestConnection. So you can't filter pull requests based on the author. But you can use search query feature like
{
search(query: "repo: hawkular/hawkular.github.io is:pr is:open author:hawkular-website-bot", last:10, type:ISSUE){
edges{
node{
... on PullRequest{
body
title
}
}
}
}
}
For details see this
I have a very similar task as D3.js nesting and rollup at same time and solution provided by #altocumulus for d3.js v3 works perfectly fine for me (tested). However, I am using d3.js v4 across my website and I am having difficulties replicating the same approach with v4 - I am not getting the same results. Perhaps, because I don't understand the sumChildren function. Please feel free to provide a better or different approach on how to restructure the loaded csv file as json with subtotals at every node level using d3.js v4. In my case, I need to have Population at the Country, State and the City levels.
Disclaimer: I have been using SO for many years and in most of the cases I got my answers from questions posted by other people, but this is my first question. In addition to this I am noob in d3.js
population.csv:
Country,State,City,Population
"USA","California","Los Angeles",18500000
"USA","California","San Diego",1356000
"USA","California","San Francisco",837442
"USA","Texas","Austin",885400
"USA","Texas","Dallas",1258000
"USA","Texas","Houston",2196000
index.html:
<!DOCTYPE html>
<html>
<head>
<title> Test D3.js</title>
</head>
<!-- <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.17/d3.min.js"></script>
-->
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.4.0/d3.min.js"></script>
<body>
<script>
d3.csv("population.csv", function(error, data) {
if (error) throw error;
var nested = d3.nest()
.key(function(d) { return d.Country; })
.key(function(d) { return d.State; })
.rollup(function(cities) {
return cities.map(function(c) {
return {"City": c.City, "Population": +c.Population };
});
})
.entries(data);
// Recursively sum up children's values
function sumChildren(node) {
node.Population = node.values.reduce(function(r, v) {
return r + (v.values ? sumChildren(v) : v.Population);
},0);
return node.Population;
}
// Loop through all top level nodes in nested data,
// i.e. for all countries.
nested.forEach(function(node) {
sumChildren(node);
});
// Output. Nothing of interest below this line.
d3.select("body").append("div")
.style("font-family", "monospace")
.style("white-space", "pre")
.text(JSON.stringify(nested,null,2));
});
</script>
</body>
</html>
Results:
[
{
"key": "USA",
"values": [
{
"key": "California",
"value": [
{
"City": "Los Angeles",
"Population": 18500000
},
{
"City": "San Diego",
"Population": 1356000
},
{
"City": "San Francisco",
"Population": 837442
}
]
},
{
"key": "Texas",
"value": [
{
"City": "Austin",
"Population": 885400
},
{
"City": "Dallas",
"Population": 1258000
},
{
"City": "Houston",
"Population": 2196000
}
]
}
],
"Population": null
}
]
Desired Results:
[
{
"key": "USA",
"values": [
{
"key": "California",
"values": [
{
"City": "Los Angeles",
"Population": 18500000
},
{
"City": "San Diego",
"Population": 1356000
},
{
"City": "San Francisco",
"Population": 837442
}
],
"Population": 20693442
},
{
"key": "Texas",
"values": [
{
"City": "Austin",
"Population": 885400
},
{
"City": "Dallas",
"Population": 1258000
},
{
"City": "Houston",
"Population": 2196000
}
],
"Population": 4339400
}
],
"Population": 25032842
}
]
The v4 changelog tells us that
When used in conjunction with nest.rollup, nest.entries now returns {key, value} objects for the leaf entries, instead of {key, values}.
It is this little renaming from values to value within the leaf nodes which eventually breaks the code. Changing the helper function accordingly should get you back on track:
// Recursively sum up children's values
function sumChildren(node) {
if (node.value) {
node.values = node.value; // Ensure, leaf nodes will also have a values array
delete node.value; // ...instead of a single value
}
node.Population = node.values.reduce(function(r, v) {
return r + (v.value? sumChildren(v) : v.Population);
},0);
return node.Population;
}