I'm doing some optimization work on some database stuff and the system writes down log files. Basically I want to optimize some stuff then restore the original customer db-dump and re-run the whole thing. My log files look something like that:
2016-05-10 15:43:18,135 DEBUG [WorkerThread#0[127.0.0.1:64181]] d.h.h.h.c.d.m... [1517]: doing thing #2081087 took 26831ms
2016-05-10 15:05:18,135 DEBUG [WorkerThread#0[127.0.0.1:64181]] d.h.h.h.c.d.m... [1517]: doing thing #20887 took 4051ms
2016-05-10 15:02:18,135 DEBUG [WorkerThread#0[127.0.0.1:64181]] d.h.h.h.c.d.m... [1517]: doing thing #1087 took 261ms
I want to correlate these events over the different runs now to have some statistical data which change did what. So, what I want is a result like
| run 1 | run 2 | run 3 | ... |
thing #20887 | 261ms | 900ms | 100ms | ... |
thing #1087 | 4051ms | 9000ms | 2000ms | ... |
How can I correlate these events using Elasticsearch and visualize it with Kibana?
Related
We are using SpecFlow for functional tests that suppose to replace manual testing when a human reads generated email and validates that all sections match specification. The problem is that Scenario Outlines become to grow to have too many parameters
Scenario Outline: generate and send confirmation email
Given I have stored itinerary in '<EmbeddedItinerary>'
When Generate confirmation email
Then section1 should have parameters '<Param1_1>', '<Param1_2>', '<Param1_3>',...
Then section2 should have parameters '<Param2_1>', '<Param2_2>', '<Param2_3>',..
Then section3 should have parameters '<Param3_1>', '<Param3_2>', '<Param3_3>',...
....
Examples:
| EmbeddedItinerary | Param1_1| Param1_2| Param1_3| Param2_1| Param2_2| Param2_3| Param3_1| Param3_2| Param3_3|...
| Itinerary_1 | Value1_1 | Value1_2 | Value1_3 | Value2_1 | Value2_2 | Value2_3 |Value3_1 | Value3_2 | Value3_3 |...
| Itinerary_1 | Value1_1 | Value1_2 | Value1_3 | Value2_1 | Value2_2 | Value2_3 |Value3_1 | Value3_2 | Value3_3 |...
But the number of columns in Examples would become unmanageable. I wish to have multi-line examples ( but with different reason then in Multiple Multi-Line Examples in SpecFlow Feature File).
The option that I see is to store all ExpectedResults in embedded xml or json resource file, and have SpecFlow features quite small e.g.
Scenario Outline: generate and send confirmation email with correct email address for flight section
Given I have stored embedded resource '<EmbeddedItinerary>'
When Generate confirmation email
Then sections should be as specified in '<ExpectedResultsFile>'
Examples:
| EmbeddedItinerary | ExpectedResultsFile
| Itinerary_1 | ExpectedResults1 |
| Itinerary_2 | ExpectedResults2 |
...
Is it a good idea?
Can anyone suggest better way ( more SpecFlow style)?
My concern is that moving expected data to separate files I am loosing visibility, that is one of advantages of SpecFlow features.
Update: while writing this question I found commercial product ($AUD 255 per user) Specflow+Excel http://www.specflow.org/plus/excel/getting-started/ , which may satisfy my requirement to maintain many columns.
Is it a mature/reliable product? Should I use it instead of own parsing expected results files in proprietary format?
If I have a lot of parameters in a Scenario Outline, I try to work as much as possible with default parameters or split the Scenario Outline in multiple ones.
I think in your case, it should be possible to split the one 'generate and send confirmation email' Scenario Outline in multiple ones that you have a scenario outline for every section.
This would reduce the amount of needed parameters per scenario and you get faster feedback if an error occurs. You immediately see in which section you have an error.
e.g.:
Scenario Outline: generate and send confirmation email - section 1
Given I have stored itinerary in '<EmbeddedItinerary>'
When Generate confirmation email
Then section1 should have parameters '<Param1_1>', '<Param1_2>', '<Param1_3>',...
Examples:
| EmbeddedItinerary | Param1_1 | Param1_2 | Param1_3 |
| Itinerary_1 | Value1_1 | Value1_2 | Value1_3 |
Scenario Outline: generate and send confirmation email - section 2
Given I have stored itinerary in '<EmbeddedItinerary>'
When Generate confirmation email
Then section2 should have parameters '<Param2_1>', '<Param2_2>', '<Param2_3>',..
Examples:
| EmbeddedItinerary | Param2_1 | Param2_2 | Param2_3 |
| Itinerary_1 | Value2_1 | Value2_2 | Value2_3 |
Scenario Outline: generate and send confirmation email - section 3
Given I have stored itinerary in '<EmbeddedItinerary>'
When Generate confirmation email
Then section3 should have parameters '<Param3_1>', '<Param3_2>', '<Param3_3>',...
Examples:
| EmbeddedItirerary | Param3_1 | Param3_2 | Param3_3 |
| Itinerary_1 | Value3_1 | Value3_2 | Value3_3 |
About SpecFlow+Excel: That is also an option. Maintaining examples in Excel is most of the time easier than in the feature- file. It will at least solve your problem in short term, but you have to be also careful to write scenarios that are still understandable and readable.
You can get a trial license for it from here: http://www.specflow.org/request-your-specflow-trial-license/
Full Disclosure: I am one of the developers of SpecFlow+ (Runner & Excel).
I would like to print an output like the following in a fixed position while the numbers in the block keep updating every couple of seconds. It is similar to what top does.
Jobs monitor:
+-----------------------------------------+
| Waiting | Launched | Running | Finished |
+-----------------------------------------+
| 319 | 364 | 94 | 201 |
+-----------------------------------------+
Elapsed time: 21s
Is there a way to do that?
With only one line, I could do it with STDOUT.flush and "\r", but it does not work for multiple lines since the carriage will put the cursor at the beginning of the new line only.
The curses library is one way to make this work. It allows you to write to locations on a 2-d screen so you're not constrained to the current line. This question has some good resources for learning curses.
Is it possible to use different tables based on previous steps?
So suppose I have something like
When I choose from a <list> of things
And I run a <test> from that thing chosen with special <parameter>
Examples:
|list|
| a |
| b |
| a |
| test | parameter |
| mytest1 | myparameter1 |
| mytest2 | myparameter2 |
| b |
| test | parameter|
| mytest1 | myparameter3 |
| mytest2 | myparameter4 |
Is this possible and what would be the correct structure?
No its not, and it is a very bad idea. Generally when you want to do programming in your scenarios, you should do it by
Giving it a name
Pushing the programming down into the step definitions
In this particular case you seem to want to be doing some sort of exhaustive testing, where you try a single operation under a number of different conditions. Cucumber is not appropriate for this sort of testing. Instead find a way to do these sort of tests at a lower level of abstraction e.g. a unit test.
The main reason for not doing exhaustive testing in Cucumber is the runtime cost. As a rough rule of thumb each integration test (Cucumber) has a run time cost of hundreds of unit tests (maybe thousands of good unit tests).
I am totally green in those kind of things, I tried reading some tutorials and still couldn't do it on my own.
Here lies the problem: I have 2 files (bulid, compile) which are somehow supposed to take other files and create a working warcraft3 map.
There's an instruction I followed:
To build:_
$ ./scripts/compile # most basic way of calling compile
$ ./scripts/build # most basic way of building a map
build
Script which takes an unportected map and applies any build settings passed via argv
upon on it then turns it into a working warcraft3 map file.
Options | Default | Description
----------------|-------------------------|---------------------------------------------------------------------
env | beta | map environment: each environment has default build settings
debug_script | false | debug this build script
do_jasshelper | true | turns vJass & ZINC into JASS
do_compile | true | turns ../src into out.j. When false, looks for {map_script_path}
do_optimizer | false | uses Vexorian's map optimizer to protect and make the map run faser
do_widgetizer | false | uses PitzerMike's map widgetizer to make map load faster
debug | false | whether the --debug flag should be passed to jasshelper
launchwc3 | false | whether the script should launch wc3 with the map loaded on exit
map_unpro_path | base-maps/{highest}.w3x | the base map file to inject script into
map_script_path | ../out.j | map script path to load into map
map_output_path | ITT_{commit}_{time}.w3x | path where to put the compiled map
setting up
This application requires Ruby
$ git clone git#github.com:theQuazz/island-troll-tribes
$ cd island-troll-tribes
$ scripts/build
It might be a easy thing to do, but I am really bad at this kind of stuff, if someone could explain what to do step by step I would greatly appreciate it
I have a problem with SonarQube 3.5.1 / 3.6 / 3.6.1.
I use web API to inject manual metrics values for a project like this :
curl -u user:password -d "resource=<project>&metric=<key of metric>&val=<value>" http://localhost:8081/sonar/api/manual_measures
More over, I use sonar-runner (with a .properties file) to run analysis after injected these values.
One of these metrics is a percentage and this metric is declared as a Percent Unit value in Sonar (in Settings => Manual Metrics menu).
I have a project with components and each project and components have this metric value injected and I can see it in Sonar.
When I want to show this metric as a color metric in a "treemap of components" widget, all the treemap is grey (as if values are not defined).
But if I put mouse on the name of component in treemap, I see the color metric value as a percentage value like this :
myComponent - ncloc: 800 - myMetric: 84,0%
Moreover, scale metric color does not appear in treemap title (after Size ncloc Color <my metric>).
Colors and scale appear if I choose an other metric for color (not a manual metric).
In fact feeding a manual metric is a two steps mechanim:
Define the value of this manual metric as you did by calling for instance the 'manual_measures' web service
Launch a Sonar analysis on this project to 'associate' this manual measure to the latest quality snapshot of this project
Then you should be able to use this manual metric in a treemap.
I finally found a solution for this problem.
In metrics table in database, i realized that my manuals metrics for color treemap don't have worst_value and best_value (value was NULL).
select * from metrics where name like 'sp_violations_density';
Result :
+-----+-----------------------+------------------+-----------+--------+-------------------------------+-------------+----------+--------------+---------+--------+------------------------+--------------------------+----------------------+--------+------------------------+
| id | name | description | direction | domain | short_name | qualitative | val_type | user_managed | enabled | origin | worst_value | best_value | optimized_best_value | hidden | delete_historical_data |
+-----+-----------------------+------------------+-----------+--------+-------------------------------+-------------+----------+--------------+---------+--------+------------------------+--------------------------+----------------------+--------+------------------------+
| 161 | sp_violations_density | Rules compliance | 0 | Issues | SyncPerl Compliance Rules (%) | 1 | PERCENT | 1 | 1 | GUI | NULL | NULL | NULL | NULL | NULL |
+-----+-----------------------+------------------+-----------+--------+-------------------------------+-------------+----------+--------------+---------+--------+------------------------+--------------------------+----------------------+--------+------------------------+
So i set them with 0 and 100.
update metrics set worst_value='0' where name LIKE 'sp_violations_density';
update metrics set best_value='100' where name LIKE 'sp_violations_density';
Need a Tomcat restart, and then it works.
Treemap has colors now.