from Neo4j to GraphJSON with Ruby - ruby

I'm trying to get visualizations using d3.js or alchemy.js--but alchemy, in particular, requires the datasource to be in GraphJSON.
I've been playing around with the tutorials and examples of Max De Marzi (using neography), Michael Hunger (cy2neo, js), Neo4j, and Neo4j.rb -- but I cannot seem to get all the way there. Mostly because I don't know what I'm doing--but this is how I'm trying to learn.
What I'm trying to achieve would be along the lines of:
https://bl.ocks.org/mbostock/3750558
or the default visualization here: http://graphalchemist.github.io/Alchemy/#/docs
And you can see what GraphJSON formatting should look like by finding it on this page also: http://graphalchemist.github.io/Alchemy/#/docs
If I run the following...
get '/followers' do
Neo4j::Session.open(:server_db, "http://localhost:7474")
query = Neo4j::Session.query('MATCH (a--(b)--(c) RETURN a,b,c LIMIT 30')
puts "--------------"
puts query_to_graph_json(query)
query_to_graph_json(query)
end
# This is supposed to grab nodes and edges, but it never gets edges.
# It's originally from a conversation at the neo4j.rb site
def query_to_graph_json(query)
nodes = {}
edges = {}
add_datum = Proc.new do |datum|
case datum
when Neo4j::ActiveNode, Neo4j::Server::CypherNode
nodes[datum.neo_id] = {
id: datum.neo_id,
properties: datum.props #was attributes, but kept saying that wasn't a method
}
when Neo4j::ActiveRel, Neo4j::Server::CypherRelationship
edges[[datum.start_node.neo_id, datum.end_node.neo_id]] = {
source: datum.start_node.neo_id,
target: datum.end_node.neo_id,
type: datum.rel_type,
properties: datum.props
}
else
raise "Invalid value found: #{datum.inspect}"
end
end
query.each do |row|
row.to_a.each do |datum|
if datum.is_a?(Array)
datum.each {|d| add_datum.call(d) }
else
add_datum.call(datum)
end
end
end
{
nodes: nodes.values,
edges: edges.values
}.to_json
end
I'll get...
{
"nodes": [
{
"id": 597,
"properties": {
"name": "John",
"type": "Person"
}
},
{
"id": 127,
"properties": {
"name": "Chris",
"type": "Person"
}
},
{
"id": 129,
"properties": {
"name": "Suzie",
"type": "Person"
}
},
],
"edges": [
]
}
The problem being that I need the edges.
If I run...
get '/followers' do
content_type :json
neo = Neography::Rest.new("http://localhost:7474")
cypher = "MATCH (a)--(b)--(c) RETURN ID(a),a.name,ID(b),b.name,ID(c),c.name LIMIT 30"
puts neo.execute_query(cypher).to_json
end
I'll get a table of paths. But it's not formatted in the way I need--and I have no idea how it might get from this format to the GraphJSON format.
{
"columns": [
"ID(a)",
"a.name",
"ID(b)",
"b.name",
"ID(c)",
"c.name"
],
"data": [
[
597,
"John",
127,
"Chris",
129,
"Suzie"
],
[
597,
"John",
6,
"Adam",
595,
"Pee-Wee"
]
]
}

I think that one problem that you're having is that, instead of matching two nodes and one relationship, you're matching three nodes and two relationships. Here's your MATCH:
MATCH (a)--(b)--(c)
It should be like:
MATCH (a)-[b]-(c)
In a MATCH clause the [] can be excluded and you can just do a raw -- (or --> or <--) which represents the relationship.
You probably want to be querying for one specific direction though. If you query bidirectionally you'll get the same relationship twice with the start and end nodes switched.
Using neo4j-core (which I biased towards as one of the maintainers ;)
nodes = []
rels = []
session.query('(source)-[rel]->(target)').pluck(:source, :rel, :target).each do |source, rel, target|
nodes << source
nodes << target
rels << rel
end
{
nodes: nodes,
edges: rels
}.to_json
Also note that if you don't specify any labels your query might be slow, depending on the number of nodes). Depends on what you need ;)

This Cypher query should return the edges array as per the example format:
MATCH (a)-[r]-(b)
WITH collect(
{
source: id(a),
target: id(b),
caption: type(r)
}
) AS edges
RETURN edges
Running this against some sample data, the results look like this:
[
{
"source": 9456,
"target": 9454,
"caption": "LIKES"
},
{
"source": 9456,
"target": 9454,
"caption": "LIKES"
},
{
"source": 9456,
"target": 9455,
"caption": "LIKES"
},
{
"source": 9454,
"target": 9456,
"caption": "LIKES"
}
]

Related

Delete existing Records if they are not in sent array Rails 5 API

I need help on how to delete records that exist in the DB but not in array sent in a request;
My Array:
[
{ "id": "509",
"name": "Motions move great",
"body": "",
"subtopics": [
{
"title": "Tywan",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
},
{
"title": "Transportations Gracious",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
},
{
"title": "Transportation part",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
}
]
},
{
"name": "Motions kkk",
"body": "",
"subtopics": [
{
"title": "Transportations",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
}
]
}
]
Below is my implementation: where am going wrong?
#topics = #course.topics.map{|m| m.id()}
#delete= #topics
puts #delete
if Topic.where.not('id IN(?)', #topics).any?
#topics.each do |topic|
topic.destroy
end
end
it's not clear to me where, in your code, you pick the ids sent in the array you showed before... so I'm assuming like this:
objects_sent = [
{ "id": "509",
"name": "Motions move great",
"body": "",
"subtopics": [
{
"title": "Tywan",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
},
{
"title": "Transportations Gracious",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
},
{
"title": "Transportation part",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
}
]
},
{
"name": "Motions kkk",
"body": "",
"subtopics": [
{
"title": "Transportations",
"url_path": "https://ugonline.s3.amazonaws.com/resources/6ca0fd64-8214-4788-8967-b650722ac97f/WhatsApp+Audio+2021-09-24+at+13.57.34.mpeg"
}
]
}
]
since you have your array like this, the only information you need to query on database is the ids (also, assuming the id's in the array are the id's on database, otherwise it wouldn't make sense). You can get them like this:
sent_ids = objects_sent.map{|o| o['id'].to_i}
Also, it seems to me that, for the code you showed, you want to destroy them based on a specific course. There would be 2 ways to do that. First, using the relationship (I prefer like this one):
#course.topics.where.not(id: sent_ids).destroy_all
Or you can do the query directly on the Topic model, but passing the course_id param:
Topic.where(course_id: #course.id).where.not(id: sent_ids).destroy_all
ActiveRecord is smart enough to mount that query correctly in both ways. Give it a test and see which works better for you

Ruby iterate over an array of hashes

I have the below array of hashes.
I want to add a new key,value pair to "hashes" which are in "all" array. Is there any better way of looping through, than what I am doing currently?
stack = {
"all": [
"mango",
"apple",
"banana",
"grapes"
],
"mango": {
"TYPE": "test",
"MAX_SIZE": 50,
"REGION": "us-east-1"
},
"apple": {
"TYPE": "dev",
"MAX_SIZE": 55,
"REGION": "us-east-1"
},
"banana": {
"TYPE": "test",
"MAX_SIZE": 60,
"REGION": "us-east-1"
},
"grapes": {
"TYPE": "dev",
"MAX_SIZE": 80,
"REGION": "us-east-1"
},
"types": [
"dev",
"test"
]
}
My code:
stack['all'].each do |fruit|
stack[fruit].each do |fruit_name|
stack[fruit_name]['COUNT'] = stack[fruit_name]['MAX_SIZE'] * 2
end
end
Expected output:
stack = {
"all": [
"mango",
"apple",
"banana",
"grapes"
],
"mango": {
"TYPE": "test",
"MAX_SIZE": 50,
"REGION": "us-east-1",
"COUNT" : 100
},
"apple": {
"TYPE": "dev",
"MAX_SIZE": 55,
"REGION": "us-east-1",
"COUNT" : 110
},
"banana": {
"TYPE": "test",
"MAX_SIZE": 60,
"REGION": "us-east-1",
"COUNT" : 120
},
"grapes": {
"TYPE": "dev",
"MAX_SIZE": 80,
"REGION": "us-east-1",
"COUNT" : 160
},
"types": [
"dev",
"test"
]
}
There is no need for the second loop. The following does what you want:
keys = stack[:all].map(&:to_sym)
keys.each do |key|
stack[key][:COUNT] = stack[key][:MAX_SIZE] * 2
end
In the above code-block stack[:all] will return an array of keys as strings, .map(&:to_sym) will convert each string in the resulting array into a symbol.
Another way to achieve the same result would be to use either fetch_values or values_at to retrieve an array of values belonging to the provided keys. The difference being that fetch_values raises an exception if a key is missing while values_at returns nil for that key.
fruits = stack.fetch_values(*stack[:all].map(&:to_sym))
fruits.each do |fruit|
fruit[:COUNT] = fruit[:MAX_SIZE] * 2
end
If you are wondering why there is a * before stack[:all].map(&:to_sym), this is to convert the array into individual arguments. In this context * is called the spat operator.
You might write the code as follows.
stack[:all].each do |k|
h = stack[k.to_sym]
h[:COUNT] = 2*h[:MAX_SIZE] unless h.nil?
end
When, for example, `k = "mango",
h #=> h={:TYPE=>"test", :MAX_SIZE=>50, :REGION=>"us-east-1", :COUNT=>100}
I've defined the local variable h for three reasons:
it simplifies the code by avoiding multiple references to stack[k.to_sym]
when debugging it may may be helpful to be able to examine h
it makes the code more readable
Note that h merely holds an existing hash; it does not create a copy of that hash, so it has a neglibile effect on memory requirements.
The technique of defining local variables to hold objects that are parts of other objects is especially useful for more complex objects. Suppose, for example, we had the hash
hash = {
cat: { sound: "purr", lives: 9 },
dog: { sound: "woof", toys: ["ball", "rope"] }
}
Now suppose we wish to add a dog toy
new_toy = "frisbee"
if it is not already present in the array
hash[:dog][:toys]
We could write
hash[:dog][:toys] << new_toy unless hash[:dog][:toys].include?(new_toy)
#=> ["ball", "rope", "frisbee"]
hash
#=> {:cat=>{:sound=>"purr", :lives=>9},
# :dog=>{:sound=>"woof", :toys=>["ball", "rope", "frisbee"]}}
Alternatively, we could write
dog_hash = hash[:dog]
#=> {:sound=>"woof", :toys=>["ball", "rope"]}
dog_toys_arr = dog_hash[:toys]
#=> ["ball", "rope"]
dog_toys_arr << new_toy unless dog_toys_arr.include?(new_toy)
#=> ["ball", "rope", "frisbee"]
hash
#=> {:cat=>{:sound=>"purr", :lives=>9},
# :dog=>{:sound=>"woof", :toys=>["ball", "rope", "frisbee"]}}
Not only does the latter snippet display intermediate results, it probably is a wash with the first snippet in terms of execution speed and storage requirements and arguably is more readable. It also cuts down on careless mistakes such as
hash[:dog][:toys] << new_toy unless hash[:dog][:toy].include?(new_toy)
If one element of stack[:all] were, for example, "pineapple", stack[:pineapple] #=> nil since stack has no key :pineapple. If, however, stack contained the key-value pair
nil=>{ sound: "woof", toys: ["ball", "rope"] }
that would become problem. Far-fetched? Maybe, but it is perhaps good practice--in part for readability--to avoid the assumption that h[k] #=> nil means h has no key k; instead, use if h.key?(k). For example:
stack[:all].each do |k|
key = k.to_sym
if stack.key?(key)
h = stack[key]
h[:COUNT] = 2*h[:MAX_SIZE]
end
end

JSONNET Get the position of an element in an array

I'm using jsonnet to configure my panels in Grafana. I'm using itfor the first time and I like it a lot. However, I'm having a hard time understanding certain aspects.
I have something similar to the following:
.addTemplate(
template.new(
microservices,
datasource='',
query=std.join(',', std.map(function(x) x.text, service['microservices'])),
label= services,
)
What I am trying to do now is to obtain, given a microservice, the position it occupies in order to be able to assign it to the variable service (and then get my custom values whit the query=std.join(',', std.map(function(x) x.text, service['microservices'])),).
local services = std.extVar('services');
local service = services[x?];
The variable service has the following form:
[
{
// I'm trying to get this array position where my value is
"key": "asd",
"microservices": [
{
"key": "I HAVE THIS VALUE",
"text": "ads"
},
{
"key": "asd",
"text": "ads"
},
{
"key": "asd",
"text": "asd"
}
],
"name": "bca",
"services: [
{
"key": "bca",
"name": "bca"
}
]
},
{
"key": "bca",
"microservices": [
{
"key": "bca",
"text": "bca"
},
{
"key": "bca",
"text": "bca"
}
],
"name": "bca",
"services: [
{
"key": "bca",
"name": "bca"
}
]
},
{
"key": "abc",
"microservices": [
{
"key": "bca",
"text": "bca"
}
],
"name": "abc",
"services: [
{
"key": "ccc",
"name": "ccc"
}
]
}
]
In any other language it seems to me a very basic operation.
var srv type
for x, service := range services{
for _, microservice := range service.microservices{
if microservice.key == "my value"{
srv= services[x]
}
}
Any tip?
Thank you so much.
It is also a very simple operation in Jsonnet. The most natural way to do it is with array comprehensions, which also conveniently support filtering:
local service2 = [
s for s in services
if [ms for ms in s.microservices if ms.key == "I HAVE THIS VALUE"] != []
][0];
Please note the indexing [0] - in general there may be more than one or none matching services in general. So if you want to get the first one, you need to take it explicitly.
The code above is written with the assumption that you don't care about the actual index, you only want to retrieve this service. If you need it, it gets a tiny bit more complicated:
local serviceIndex = [
x for x in std.range(0, std.length(services) - 1)
if [ms for ms in services[x].microservices if ms.key == "I HAVE THIS VALUE"] != []
][0];
BTW you could achieve the same result with functions such as std.filter, but that would be a bit more verbose.
BTW2 I would also consider extracting a function hasMicroservice(service, msKey) to enhance readability of filtering.

Iterate through JSON respone Facebook Graph

Im trying to iterate through a returned response from the Facebook Graph api
def get_feed
uri = URI(FACEBOOK_URL)
response = HTTParty.get(uri)
results = JSON.parse(response.body)['data']
puts formatted_data(results)
end
def formatted_data(results)
return unless results
formatted = results['data'].each do |d|
unless d.nil?
{
message: d['message'],
}
end
formatted.delete_if {|x| x.nil?}
end
end
The response is very large so here is a snippet if it helps
{
"data": [
{
"id": "197958940234297_827831980580320",
"from": {
"category": "Amateur sports team",
"category_list": [
{
"id": "189018581118681",
"name": "Sports Club"
},
{
"id": "139721016091877",
"name": "Outdoor Recreation"
},
{
"id": "109615542448700",
"name": "Physical Fitness"
}
],
"name": "Varsity Vandals",
"id": "197958940234297"
},
"to": {
"data": [
{
"id": "668983363",
"name": "Heather Walker"
},
{
"id": "638195502",
"name": "Emma Williams"
},
{
"id": "1286337937",
"name": "Becky Williams"
}
]
},
"with_tags": {
"data": [
{
"id": "668983363",
"name": "Heather Walker"
},
{
"id": "638195502",
"name": "Emma Williams"
},
{
"id": "1286337937",
"name": "Becky Williams"
}
]
},
"message": "Great turnout for the women's intro session today. Cool to have a women's game and a men's game running side by side. Touch is for all.",
"picture": "https://fbcdn-photos-f-a.akamaihd.net/hphotos-ak-prn2/t1.0-0/1507550_827829843913867_410211203232735862_s.jpg",
"link": "https://www.facebook.com/photo.php?fbid=827829843913867&set=pcb.827831980580320&type=1&relevant_count=2",
"icon": "https://fbstatic-a.akamaihd.net/rsrc.php/v2/yz/r/StEh3RhPvjk.gif",
"actions": [
{
"name": "Comment",
"link": "https://www.facebook.com/197958940234297/posts/827831980580320"
},
{
"name": "Like",
"link": "https://www.facebook.com/197958940234297/posts/827831980580320"
}
],
"privacy": {
"value": ""
},
I am getting an error
TypeError: no implicit conversion of String into Integer
At the moment i would just like to pull out all the Messages from the JSON object...Am i handling the extraction correctly
Any help appreciated
Thanks
I tried you code, I change you require is move formatted.delete_if {|x| x.nil?} out of loop, like following, as formatted will be nil inside the loop.
def formatted_data(results)
return unless results
formatted = results['data'].each do |d|
unless d.nil?
{
message: d['message'],
}
end
end
formatted.delete_if {|x| x.nil?}
end
are you sure your not using the data key twice?
results = JSON.parse(response.body)['data'] in main method and formatted = results['data'].each in your formatted_data method?
Thinking maybe?
def def formatted_data(results)
return unless results
results['data'].map {|m| {message: m['message']} }.compact
end
I'd do this:
def get_feed
uri = URI(FACEBOOK_URL)
response = HTTParty.get(uri)
messages = format_data(response)
for message in messages do
puts message
end
end
def format_data(response, new_data = [])
if response.present?
results = JSON.parse(response)
for result in results do
new_data << result[:data][:message] if result[:data][:message].present?
end
return new_data #-> array of messages
end
end

Ruby: Delete a reoccurring hash in a large nested data structure

I am trying to move data between services and need to remove a reoccurring hash from a large record that contains both hashes and arrays.
The hash to remove from every section of the record is
{
"description": "simple identifier",
"name": "id",
"type": "id"
},
Heres example data :
{"stuff": { "defs": [
{
"description": "simple identifiery",
"name": "id",
"type": "id"
},
{
"name": "aDate",
"type": "date"
},
{
"defs": [
{
"description": "simple identifier",
"name": "id",
"type": "id"
},
{
"case-sensitive": true,
"length": null,
"name": "Id",
"type": "string"
},
{
"name": "anotherDate",
"type": "dateTime"
}
],
},
{
"defs": [
{
"description": "simple identifier",
"name": "id",
"type": "id"
},
...lots more....
I created a couple recursive function to remove the element(s) but I'm left with an empty hash '{}'. I also tried to remove the parent but found that I removed the hashes parent and not the hash itself.
I'm pretty sure I could create a new hash and populate it with the data I want but there must be a way to do this.
I am not working in rails and would like to avoid using rails gems.
I figured this out by looking at the data structure closer. The elements that need to be removed are always in an array so before recursing check if the hash key/value exists and delete if so. I'm sure this could be coded better so let me know what you think.
def recursive_delete!(node, key, value)
if node.is_a?(Array)
node.delete_if { |elm| elm[key] == value }
node.each do |elm|
recursive_delete!(elm, key, value)
end
elsif node.is_a?(Hash)
node.each_value do |v|
recursive_delete!(v, key, value)
end
end
end
If you are looking for the way to delete the same hash as you have inside complex Array/Hash data structure, it's easy:
def remove_hash_from(source, hsh)
return unless source.is_a?(Hash) || source.is_a?(Array)
source.each do |*args|
if args.last == hsh
source.delete(args.first)
elsif args.last.is_a?(Hash) || args.last.is_a?(Array)
remove_hash_from(args.last, hsh)
end
end
source
end
data = [
{h: 'v',
j: [{h: 'v'},
{a: 'c'},
8,
'asdf']
},
asdf: {h: 'v', j: 'c'}
]
remove_hash_from(data, {h: 'v'})
# => [{:h=>"v", :j=>[{:a=>"c"}, 8, "asdf"]}, {:asdf=>{:h=>"v", :j=>"c"}}]
Possibly, you will need to adjust method above for your needs. But common idea is clear, I hope.

Resources