How do I save the csv data into the mongodb? - spring

I want to write a service method for saving bulk amount of csv data into the Mongo database keeping things in mind that it will not give some memory overflow error.
How can I do this if anyone already done or know please help.

You can use GridFS to save document larger than 16 MB, here is the link: https://docs.mongodb.com/v3.2/core/gridfs/, come back again when you have further questions.

Try to model a document based on your CSV that you're trying to store. It's hard to comment directly on your CSV format take the following for example:
Id,Name,Student 1 Id,Student 1 Name,Student 2 Id,Student 2 Name,Student 3 Id,Student 3 Name
48,Bill,1001,Kev,1002,Sakis,1005,Lee
78,Fred,1005,Lee,1073,Karen,1021,Jay
We could break this down in to a teacher collection:
{
_id: 48,
name: "Bill",
students: [
{_id: 1001, name: "Kev"},
{_id: 1002, name: "Sakis"},
{_id: 1005, name: "Lee"}
]
}
{
_id: 78,
name: "Fred",
students: [
{_id: 1005, name: "Lee"},
{_id: 1073, name: "Karen"},
{_id: 1021, name: "Jay"},
]
}

Related

Converting Language Detection Score of CLD2 to CLD3 Accuracy

My cld2 language detection model (langID) returns for the input sentence to classify the following values
{ reliable: true,
textBytes: 181,
languages:
[ { name: 'ITALIAN', code: 'it', percent: 61, score: 774 },
{ name: 'ENGLISH', code: 'en', percent: 38, score: 1573 } ],
chunks:
[ { name: 'ITALIAN', code: 'it', offset: 0, bytes: 116 },
{ name: 'ENGLISH', code: 'en', offset: 116, bytes: 71 } ] }
where the textBytes represents the size of the input text, percent the distribution of the code in the sentence, while the score is an indicator of the quality of the detection (the smaller it is the best it is).
That said, in the brand new CLD3 neural network, the result of the classification is just the accuracy (so a probability value between 0 and 1) so like
println(ld.getCode(0))
println(ld.getScore(0))
en
0.99
I would like to figure out how to convert CLD2 score to probabilities values in order to compare the results to the new CLD3 model.

how to create c3js Bar chart from json array

if i have a json array like the below json
[
{id: 27, name: "ACT LIFE BU1", count: 360}
{id: 38, name: "ALUMATREND BU2", count: 802}
{id: 36, name: "ASPATREND BU2", count: 350}
{id: 28, name: "AVOSOYA BU1", count: 569}
]
how to make a chart like the below picture
many thanks for your help,

Sort array of active records by multiple columns

Given an array of active records
jobs = [#<Job _id: 1, created_at: 2014-07-15 19:18:40 UTC, organization: "Acme In.c">,
#<Job _id: 3, created_at: 2014-05-20 09:27:38 UTC, organization: "Baxter">,
#<Job _id: 2, created_at: 2014-11-25 12:21:00 UTC, organization: "Wizard">,
#<Job _id: 3, created_at: 2015-01-15 07:20:10 UTC, organization: "Baxter">]
How can I sort the array first by organization in ascending order A->Z and then by created_at descending to end up with this ...
jobs = [#<Job _id: 1, created_at: 2014-07-15 19:18:40 UTC, organization: "Acme In.c">,
#<Job _id: 3, created_at: 2015-01-15 07:20:10 UTC, organization: "Baxter">,
#<Job _id: 3, created_at: 2014-05-20 09:27:38 UTC, organization: "Baxter">,
#<Job _id: 2, created_at: 2014-11-25 12:21:00 UTC, organization: "Wizard">]
I've tried to to do jobs.sort_by{|j| [j.organization, j.created_at]} but I'm not really sure how to specify multiple asc/desc rules and the closest I've gotten is sorted by organization but then sorting by created gives me oldest to newest instead of newest to oldest. I've also tried to group first and then sort but I couldn't get it to work either. Any help if appreciated.
A good way is to use sort and <=> and nonzero? like this:
jobs.sort{|a,b|
(a.organization <=> b.organization).nonzero? ||
(b.created_at <=> a.created_at)
}
This code says:
Compare A and B by organization.
If they differ, then we have our answer.
If they are the same, then we need to do more.
Compare B and A by time. (Note B & A are in reverse order)
If they differ, then we have our answer.
If they are the same, then the sort order doesn't matter. (Ruby sort is "unstable")
Example code independent of ActiveRecord:
require 'time'
require 'ostruct'
jobs = [
OpenStruct.new(_id: 1, created_at: Time.parse("2014-07-15 19:18:40 UTC"), organization: "Acme Inc"),
OpenStruct.new(_id: 3, created_at: Time.parse("2014-05-20 09:27:38 UTC"), organization: "Baxter"),
OpenStruct.new(_id: 2, created_at: Time.parse("2014-11-25 12:21:00 UTC"), organization: "Wizard"),
OpenStruct.new(_id: 3, created_at: Time.parse("2015-01-15 07:20:10 UTC"), organization: "Baxter")
]
As mentioned in
http://guides.rubyonrails.org/active_record_querying.html
Try jobs.order("organization ASC", "created_at DESC") or the following
jobs.order("organization ASC").order("created_at DESC")

Manipulating Tree Structures for API endpoints

I am working with an RDBMS that contains a list of hierarchical objects stored like this:
Id Name ParentId
====================================
1 Food NULL
2 Drink NULL
3 Vegetables 1
4 Fruit 1
5 Liquor 2
6 Carrots 3
7 Onions 3
8 Strawberries 4
...
999 Celery 3
I do not know the specific reason why this was chosen, but it is fixed in so far as the rest of the system relies on fetching the structure in this form.
I want to expose this data via JSON using a RESTful API, and I wish to output this in the following format (array of arrays):
item:
{
id: 1, Description: "Food",
items: [
{
id: 3, Description: "Vegetables",
items: [ ... ]
},
{
id: 4, Description: "Fruit",
items: [ ... ]
}
]
},
item:
{
id: 2, Description: "Drink",
items: [ ... ]
}
What would be a sensible way of looping through the data and producing the desired output? I'm developing in C# but if there are libraries or examples for other languages I would be happy to re-implement where possible.
Thanks :)

find overlapping times in an array of hashes

I've got an array of classes, and I want to find where there may be a schedule overlap.
My array is something like this
[
{
id:2,
start: "3:30",
length: 40,
break: 30,
num_attendees: 14
},
{
id: 3,
start: "3: 40",
length: 60,
break: 40,
num_attendees: 4
},
{
id: 4,
start: "4: 40",
length: 30,
break: 10,
num_attendees: 40
}
]
pretty simple.
Now, I want to get an array where I add the start and the length, and then get the classes that overlap to notify the user that they have a conflict.
I know I can do a large for loop and compare that way, but I'm thinking there must be a nicer way to do this in Ruby, something like (ignore that we're not working in absolute minutes here, I've got that, I just want to keep the example simple).
overlap = class_list.select{|a,b| if a.start+a.length>b.start return a,b end}
any suggestions?
You can use Array#combination like this:
class_list.combination(2).select{|c1, c2|
# here check if c1 and c2 overlap
}

Resources