I use an apache storm topology on a cluster of 8+1 machines. The date on these machines is not the same and we may have more than 5 minutes of difference.
preprod-storm-nimbus-01:
Thu Feb 25 16:20:30 GMT 2016
preprod-storm-supervisor-01:
Thu Feb 25 16:20:32 GMT 2016
preprod-storm-supervisor-02:
Thu Feb 25 16:20:32 GMT 2016
preprod-storm-supervisor-03:
Thu Feb 25 16:14:54 UTC 2016 <<-- this machine is very late :(
preprod-storm-supervisor-04:
Thu Feb 25 16:20:31 GMT 2016
preprod-storm-supervisor-05:
Thu Feb 25 16:20:17 GMT 2016
preprod-storm-supervisor-06:
Thu Feb 25 16:20:00 GMT 2016
preprod-storm-supervisor-07:
Thu Feb 25 16:20:31 GMT 2016
preprod-storm-supervisor-08:
Thu Feb 25 16:19:55 GMT 2016
preprod-storm-supervisor-09:
Thu Feb 25 16:20:30 GMT 2016
Question:
Is the storm topology affected by this non-synchronization?
Note: I know that synchronizing is better, but the sysadmins won't do it without proving them proofs/reasons that they have to do it. Do they really have to do it, "for the topology's sake" :) ?
Thanks
It depends on the computation you are doing... It might have an effect on your result if you do time based window operations. Otherwise, it doesn't matter.
For Storm as an execution engine it has no effect at all.
Related
I'm using elasticdump and got weird error
Mon, 14 Nov 2022 14:42:21 GMT | starting dump
Mon, 14 Nov 2022 14:42:22 GMT | got 10 objects from source elasticsearch (offset: 0)
Mon, 14 Nov 2022 14:42:22 GMT | sent 10 objects to destination file, wrote 10
Mon, 14 Nov 2022 14:42:22 GMT | Error Emitted => This and all future requests should be directed to the given URI.
Mon, 14 Nov 2022 14:42:22 GMT | Error Emitted => This and all future requests should be directed to the given URI.
Mon, 14 Nov 2022 14:42:22 GMT | Total Writes: 0
Mon, 14 Nov 2022 14:42:22 GMT | dump ended with error (get phase) => MOVED_PERMANENTLY: This and all future requests should be directed to the given URI.
It successfully moved 10 objects and stopped
--input-index is for a different use case.
Try with just --input like this
elasticdump --input=http://localhost/dev_index --output=test2.json
I am trying to import index from json file to elasticsearch server but it is failing.
Specifications:
elasticsearch : 4.10.3
elasticdump : 2.4.2
command I am using
elasticdump --input=/home/ubuntu/Files/stocks.json --output=http://localhost:9200/ --type=data`
My stocks.json file looks like
{"_index":"stocks","_type":"stock","_id":"AVhKm5L8FPDye23IuJqe","_score":1,"_source":{"name":"Sun Pharmaceutical Industries Ltd.","industry":"PHARMA","isin":"INE044A01036","symbol":"SUNPHARMA","tweet":"sun pharma' OR 'SUNPHARMA'"}}
{"_index":"stocks","_type":"stock","_id":"AVhKm5L8FPDye23IuJqV","_score":1,"_source":{"name":"Tata Steel Ltd.","industry":"METALS","isin":"INE081A01012","symbol":"TATASTEEL","tweet":"tata steel' OR 'TATASTEEL'"}}
{"_index":"stocks","_type":"stock","_id":"AVhKm5L7FPDye23IuJp2","_score":1,"_source":{"name":"ICICI Bank Ltd.","industry":"FINANCIAL SERVICES","isin":"INE090A01021","symbol":"ICICIBANK","tweet":"icici bank' OR 'ICICIBANK'"}}
I am getting following message
Sat, 07 Oct 2017 05:46:52 GMT | starting dump
Sat, 07 Oct 2017 05:46:52 GMT | got 100 objects from source file
(offset: 0)
Sat, 07 Oct 2017 05:46:52 GMT | sent 100 objects to destination
elasticsearch, wrote 0
Sat, 07 Oct 2017 05:46:52 GMT | got 0 objects from source file
(offset: 100)
Sat, 07 Oct 2017 05:46:52 GMT | Total Writes: 0
Sat, 07 Oct 2017 05:46:52 GMT | dump complete
I had used same json file before but somehow this is not working in this new server. I have installed elasticsearch, node recently in this server.
Thanks for help
J
Jmeter is taking always the same time to finish remote test. The jmx script is simple and there is no time configured on it, and there are just one request (only 84ms in local test).
It happens only on remote test, in local test is ok.
Starting the test on host x.x.x.x # Thu Aug 14 09:31:43 BRT 2014 (1408019503091)
Finished the test on host x.x.x.x # Thu Aug 14 09:34:43 BRT 2014 (1408019683082)
Starting the test on host x.x.x.x # Thu Aug 14 09:35:53 BRT 2014 (1408019753107)
Finished the test on host x.x.x.x # Thu Aug 14 09:38:53 BRT 2014 (1408019933091)
Starting the test on host x.x.x.x # Thu Aug 14 09:40:33 BRT 2014 (1408020033110)
Finished the test on host x.x.x.x # Thu Aug 14 09:43:33 BRT 2014 (1408020213100)
Starting the test on host x.x.x.x # Thu Aug 14 10:03:23 BRT 2014 (1408021403158)
Finished the test on host x.x.x.x # Thu Aug 14 10:06:23 BRT 2014 (1408021583154)
Starting the test on host x.x.x.x # Thu Aug 14 10:07:53 BRT 2014 (1408021673181)
Finished the test on host x.x.x.x # Thu Aug 14 10:10:53 BRT 2014 (1408021853164)
Starting the test on host x.x.x.x # Thu Aug 14 10:25:23 BRT 2014 (1408022723204)
Finished the test on host x.x.x.x # Thu Aug 14 10:28:23 BRT 2014 (1408022903204)
Starting the test on host x.x.x.x # Thu Aug 14 10:33:13 BRT 2014 (1408023193224)
Finished the test on host x.x.x.x # Thu Aug 14 10:36:53 BRT 2014 (1408023413225)
there are 200 records,for example:
[
{time:"Thu Nov 07 2013 13:09:08",value:"10"},
{time:"Thu Nov 07 2013 11:09:08",value:"30"},
{time:"Thu Nov 07 2013 11:09:08",value:"25"},
....more
{time:"Thu Nov 06 2013 10:09:08",value:"65"},
{time:"Tue Aug 06 2013 16:54:31",value:"25"},
{time:"Tue Aug 06 2013 16:54:31",value:"45"},
]
there are one or two unique recodes that the time is too early.
when i draw a line use time as xAxis, beacuse of the unique recodes ( {time:"Tue Aug 06 2013 16:54:31",value:"25"},{time:"Tue Aug 06 2013 16:54:31",value:"45"}) ,the line has a blank at between Nov to Aug .
how can i deal with the records what is unique and several .
Any help is appreciated.
Just add an unique id field to your data if you want to draw both or remove one (and display an average value or whatever is suitable)
I have a arch 64bit VPS on digitalocean. I installed gwan and run it in deamon mode. It stopped running every midnight.
Here is the log file
[Wed Apr 24 06:10:28 2013 GMT] memory footprint: 3.78 MiB
[Thu, 25 Apr 2013 00:00:19 GMT] * child abort(8) coredump
[Thu, 25 Apr 2013 00:00:19 GMT] * child abort(8) coredump
[Thu, 25 Apr 2013 00:00:19 GMT] * child abort(8) coredump
[Thu, 25 Apr 2013 00:00:19 GMT] * child died 3 times within 3 seconds
[Thu Apr 25 12:39:39 2013 GMT] memory footprint: 3.77 MiB.
[Thu Apr 25 12:39:56 2013 GMT] loaded maintenance script/opt/gwan_linux64-bit/0.0.0.0_8080/#0.0.0.0/csp/crash.c 43.14 KiB MD5:820cf6b4-2152b838-08a13fcb-5f0dc4be
[Fri, 26 Apr 2013 00:00:10 GMT] * child abort(8) coredump
[Fri, 26 Apr 2013 00:00:10 GMT] * child abort(8) coredump
[Fri, 26 Apr 2013 00:00:10 GMT] * child abort(8) coredump
[Fri, 26 Apr 2013 00:00:10 GMT] * child died 3 times within 3 seconds
This problem does not happen on all platforms and so far all the user reports we received used hypervisors which alter the CPU and OS behavior in erratic and undocumented ways (not to cite the additional bugs they inject into the system).
UPDATE
That new problem for 4-years old code that worked fine so far is a platform issue, for which we have found a workaround, to be published with the next release in a few weeks.