Handling JSON+ Stack Traces using logstash+ FileBeat - elasticsearch

Question

I'm trying to parse some JSON log files(that have stackTraces within them) into Logstash, but I think the stack traces are not letting it happen.
here is my logstash conf: (one of the many confs I've tried )
input {
# This configuration works but the input file needs to be set in a format like so [{},{}]
file {
path => "/opt/logstash/confFiles/suite1.json"
start_position => beginning
codec => json
sincedb_path => "/opt/logstash/confFiles/suite1incedb"
}
}
filter {
json{
source => "message"
}
}
output {
stdout {codec => rubydebug}
}
This is the json TYPE I'm trying to work around (its the result of a jsonReporter for jasmine)
{
"suite1": {
"id": "suite1",
"description": "create process - simulate test 2",
"fullName": "create process - simulate test 2",
"failedExpectations": [],
"status": "finished",
"specs": [{
"id": "spec0",
"description": "should redirect to modeler after create a process",
"fullName": "create process - simulate test 2 should redirect to modeler after create a process",
"failedExpectations": [{
"matcherName": "toMatch",
"message": "Expected 'http://localhost:3000/#!/signin' to match /myDashboard/.",
"stack": "Error: Failed expectation\n at Env.<anonymous> (/home/ls/code/ph/app/tests/e2e/create-process.e2e.test.js:18:35)\n at /home/ls/code/ph/node_modules/protractor/node_modules/jasminewd2/index.js:95:14\n at [object Object].webdriver.promise.ControlFlow.runInNewFrame_ (/home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1654:20)\n at [object Object].webdriver.promise.ControlFlow.runEventLoop_ (/home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1518:8)\n at [object Object].wrapper [as _onTimeout] (timers.js:274:14)\n at Timer.listOnTimeout (timers.js:119:15)",
"passed": false,
"expected": {},
"actual": "http://localhost:3000/#!/signin"
}, {
"matcherName": "",
"message": "Failed: No element found using locator: By.id(\"selectedUser\")",
"stack": "Error: Failed: No element found using locator: By.id(\"selectedUser\")\n at stack (/home/ls/code/ph/node_modules/protractor/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1441:17)\n at buildExpectationResult (/home/ls/code/ph/node_modules/protractor/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1411:14)\n at Spec.Env.expectationResultFactory (/home/ls/code/ph/node_modules/protractor/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:533:18)\n at Spec.addExpectationResult (/home/ls/code/ph/node_modules/protractor/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:293:34)\n at Env.fail (/home/ls/code/ph/node_modules/protractor/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:837:25)\n at Function.next.fail (/home/ls/code/ph/node_modules/protractor/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1776:19)\n at /home/ls/code/ph/node_modules/protractor/node_modules/jasminewd2/index.js:104:16\n at /home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:1582:15\n at [object Object].webdriver.promise.ControlFlow.runInNewFrame_ (/home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1654:20)\n at notify (/home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:465:12)",
"passed": false,
"expected": "",
"actual": ""
}],
"passedExpectations": [],
"status": "failed"
}]
},
"suite2": {
"id": "suite2",
"description": "login to PB Modeler",
"fullName": "login to PB Modeler",
"failedExpectations": [],
"status": "finished",
"specs": [{
"id": "spec1",
"description": "should redirect to myDashboard after login",
"fullName": "login to PB Modeler should redirect to myDashboard after login",
"failedExpectations": [{
"matcherName": "toMatch",
"message": "Expected 'http://localhost:3000/#!/signin' to match /myDashboard/.",
"stack": "Error: Failed expectation\n at Env.<anonymous> (/home/ls/code/ph/app/tests/e2e/login.e2e.test.js:18:37)\n at /home/ls/code/ph/node_modules/protractor/node_modules/jasminewd2/index.js:95:14\n at [object Object].webdriver.promise.ControlFlow.runInNewFrame_ (/home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1654:20)\n at [object Object].webdriver.promise.ControlFlow.runEventLoop_ (/home/ls/code/ph/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1518:8)\n at [object Object].wrapper [as _onTimeout] (timers.js:274:14)\n at Timer.listOnTimeout (timers.js:119:15)",
"passed": false,
"expected": {},
"actual": "http://localhost:3000/#!/signin"
}],
"passedExpectations": [],
"status": "failed"
}]
}
I've try many configurations now,
If I try to send it like it is above, Logstash will not map it correctly.
So what I did was remove the spaces and set beautify=false in the reporter and surrounded it with "[ ]" to make it look like an array and Logstash would "randomly" take it.
-So what would it be a good approach to parse both the nested JSON objects,taking into account the stackTraces, so that when its sent to ES and kibana is a workable DATA.
-How can I make a mapping for this structure or Model the data so that it understands it when sending this to ELK
I'm using
ES 2.1
Logstash 2.1
Kibana 4.3.1
Filebeat 1.0.1