GetStats duration and interval parameters, clarifying API documentation for Jelastic API - jelastic

https://docs.jelastic.com/api/?class=environment.Control&member=GetStats
At the above link in the Jelastic API documentation for the GetStats method there are two parameters duration and interval.
When querying the api i can't figure out how these two parameters interact with each other.
If i query with the below i would expect 100 records at a resolution of 1 minute
/1.0/environment/control/rest/getstats?domain=[myDomiain]&session=[MySession]&duration=6000&interval=60&nodeid=[MyNode]
What i get back is 4 records for each hour so i'm unsure of how the parameters work.
Should i be using GetSumStats?
My final question would be what format are the cpu and mem stats in? MHz and Bytes?
{
"iops_used": 0,
"duration": 3600,
"cpumhz": 3,
"start": "2016-05-03 08:00:00",
"disk": 2141,
"mem": 194840,
"cpu": 12254,
"capacity": 0,
"net": {
"in_int": 703019,
"out_int": 566947,
"in_ext": 46222,
"out_ext": 367209
}
},
{
"iops_used": 0,
"duration": 3600,
"cpumhz": 3,
"start": "2016-05-03 09:00:00",
"disk": 2141,
"mem": 171992,
"cpu": 10076,
"capacity": 0,
"net": {
"in_int": 156703,
"out_int": 314023,
"in_ext": 12627,
"out_ext": 13535
}
},
{
"iops_used": 0,
"duration": 3580,
"cpumhz": 3,
"start": "2016-05-03 10:00:00",
"disk": 2141,
"mem": 172400,
"cpu": 11198,
"capacity": 0,
"net": {
"in_int": 515521,
"out_int": 551317,
"in_ext": 10329,
"out_ext": 17161
}
},
{
"iops_used": 0,
"duration": 3601,
"cpumhz": 3,
"start": "2016-05-03 11:00:00",
"disk": 2141,
"mem": 172610,
"cpu": 10032,
"capacity": 0,
"net": {
"in_int": 153394,
"out_int": 310694,
"in_ext": 10285,
"out_ext": 11210
}
}

#dlearious, for using interval equal 60 you should set duration value to 3600. This is due to the fact that Jelastic keeps detailed data hourly.
Also, you can start from minimal interval = 20.
Jelastic shows cpu in milliseconds and mem in Bytes.

Related

Need to pass 1 gameID to 9 different users (Signed up and logged in) which is created at the run time of previous request

I have multiple requests in the same thread group and i need to pass 18 users to the 2 game id's (9 users to each gameID) and i am using counter element to create different users and gameID's.
Login Request:
42[
"get_login",
{
"uid": "EiNjkkkxgqq1ClN7gwzJEBK5XPhYKsG5",
"key": "AUEEW891WL",
"socketId":"AvKjxCGPLzgvqGxzAAb-",
"username": "",
"avatar": "",
"language": "en",
"playerMove": "",
"joinGame": "",
"replay": 0,
"gameID": 0,
"gameNo": 0,
"data": "&email=Nov${counter}#gmail.com&password=poker",
"players": 0,
"level": "",
"lastAction": "",
"game": "",
"playMoney": 1,
"role": ""
}
]
Response:
42[
"gotLogin",
{
"response": {
"playerId": 24441,
"totalWin": 0,
"points": 0,
"playerName": "672319400464",
"displayName": "DEC65",
"playerRank": "Amateur",
"playerBank": "11K",
"playerBankAmount": 11000,
"playerAvatar": "avatar1.jpg",
"playerEmail": "Nov65#gmail.com",
"playerLevel": 1,
"playerLevelStr": "Lvl. 1",
"playerLevelPrg": 0,
"playerLevelPoints": 100,
"playerGold": 0,
"playerDealerId": 1,
"playerFbId": "",
"loadVocab": "",
"dailyBonus": 0,
"token": "37333e039503825fded65f7714f04741688af294ff0a32a7c676f5c366e193de"
}
}
]
Create Game
Request:
42[
"create_game",
{
"uid": "Esdv7CHkwo1ATMfvY6NcWBTM5YB4d3nj",
"key": "AUEEW891WL",
"socketId":"AvKjxCGPLzgvqGxzAAb-",
"username": "611839990703",
"avatar": "avatar17.jpg",
"language": "en",
"playerMove": "",
"joinGame": "",
"replay": 0,
"gameID": 0,
"gameNo": 0,
"data": "&gameID=undefined&game=texas&playMoney=1&gameStyle=private-cashgame&tableName=TableOct${counter}&rakeRate=0&speed=60&sb=10&bb=20&tablelow=100&tablelimit=1000&videorequired=false&username=611839990703",
"players": 0,
"level": 347,
"lastAction": "",
"game": "",
"playMoney": 1,
"role": "1",
"token":"84d26de7404377dfa0ade2ff74676fd901852e19100068d43ac8a8d449f49a1c",
"playerId":"23910",
"displayName": "p10"
}
]
Response:
42[
"createGameRsp",
{
"response": {
"title": "Create Successful",
"result": "TableOct2 was successfully created. \n GameID: 630 \n Password: ",
"error": "",
"game": "texas",
"gameID": 630
}
}
]
Join Game:
Request:
42[
"get_joingame",
{
"uid": "Esdv7CHkwo1ATMfvY6NcWBTM5YB4d3nj",
"key": "AUEEW891WL",
"socketId":"AvKjxCGPLzgvqGxzAAb-",
"username":${userName}",
"avatar": "avatar17.jpg",
"language": "en",
"playerMove": "",
"joinGame": "",
"replay": 0,
"gameID":"${gameID}",
"gameNo": 0,
"data": "&seat=${counter2}&buyin=500&privateTable=2&clubId=0",
"players": 0,
"level": 347,
"lastAction": "ep.playNow",
"game": "texas",
"playMoney": 1,
"role": "1",
"playerId":"${playerId}",
"displayName": "p10",
"seat":${counter2}
}
]
Response:
42[
"gotJoinGame",
{
"game": "texas",
"gameID": "630",
"response": {
"error": ""
}
}
]
Right now i am able to pass one game id to only one user. But i need to pass one game id to 9 users. How is it possible?
As per JMeter documentation on variables:
Properties are not the same as variables. Variables are local to a thread; properties are common to all threads
So if you extract the gameID by one thread (virtual user) - it will be available only to this thread. If you want to use the value by other virtual user(s) - you need to convert it into a JMeter Property via __setProperty() function or suitable JSR223 Test Element
If you have 2 groups of users you can assign one gameID to users with even thread number and the other to odd users.
More information: Using JMeter Variables With Multiple Thread Groups

Can't get severity info via API

Java 11
SonarQube 8.9.2 LTS
For my java project the SonarQube show the next issues info:
Severity
Blocker 1.3k
Minor 1.1k
Critical 5.8k
Info 233
Major 1.3k
So I need to get this information via SonarQube WEB API.
I found only this api method:
GET http://some_url_sonar_qube/api/issues/search
And its return all issues on page = 1
And its return all issues on page = 1 with detail info
{
"total": 10049,
"p": 1,
"ps": 100,
"paging": {
"pageIndex": 1,
"pageSize": 100,
"total": 10049
},
"effortTotal": 50995,
"issues": [
{
"key": "dddd",
"rule": "css:S4670",
"severity": "CRITICAL",
...
This:
GET http://some_url_sonar_qube/api/issues/search?p=2
And its return all issues on page = 2
and so on.
Response example:
As you can see has 10049 issues. It's 100 pages.
But I need summary info. Smt like this in json format:
{
"Severity": {
"Blocker": 1300,
"Minor": 1100,
"Critical": 5800,
"Info": 233,
"Major": 1300
}
}
I'm not found api method for this
I found solution (thanks for #gawkface)
Use this method:
GET http://some_url_sonar_qube/api/issues/search?componentKeys=my_project_key&facets=severities
And here result (on section facets)
{
"total": 10049,
"p": 1,
"ps": 100,
"paging": {
"pageIndex": 1,
"pageSize": 100,
"total": 10049
},
"effortTotal": 50995,
"issues": [...],
"components": [...],
"facets": [
{
"property": "severities",
"values": [
{
"val": "CRITICAL",
"count": 5817
},
{
"val": "MAJOR",
"count": 1454
},
{
"val": "BLOCKER",
"count": 1286
},
{
"val": "MINOR",
"count": 1161
},
{
"val": "INFO",
"count": 331
}
]
}
]
}

Determine Hadoop cluster resources used by completed job

How can you determine Hadoop cluster resources used by a completed job?
Our cluster resource manager is yarn. Access to certain yarn API endpoints is available through HTTP; for example:
curl -L http://my.hadoop.instance:8088/ws/v1/cluster/apps/application_1547448533998_502644
would return:
{
"app": {
"allocatedMB": -1,
"allocatedVCores": -1,
"amContainerLogs": "http://someNode.hadoop.instance:8042/node/containerlogs/container_e149_1547448533998_502644_01_000001/someUser",
"amHostHttpAddress": "someNode.hadoop.instance:8042",
"amNodeLabelExpression": "",
"applicationTags": "",
"applicationType": "SPARK",
"clusterId": 1547448533998,
"clusterUsagePercentage": 0.0,
"diagnostics": "",
"elapsedTime": 583889,
"finalStatus": "SUCCEEDED",
"finishedTime": 1550621490747,
"id": "application_1547448533998_502644",
"logAggregationStatus": "TIME_OUT",
"memorySeconds": 15821179,
"name": "ProjectCantor",
"numAMContainerPreempted": 0,
"numNonAMContainerPreempted": 0,
"preemptedResourceMB": 0,
"preemptedResourceVCores": 0,
"priority": 0,
"progress": 100.0,
"queue": "dsg",
"queueUsagePercentage": 0.0,
"runningContainers": -1,
"startedTime": 1550620906858,
"state": "FINISHED",
"trackingUI": "History",
"trackingUrl": "http://my.hadoop.instance:8088/proxy/application_1547448533998_502644/",
"unmanagedApplication": false,
"user": "someUser",
"vcoreSeconds": 14713
}
}
However, there are several attributes such as allocatedMB and allocatedVCores set to -1 rather than any meaningful value.
Thanks in advance.

How can I resolve the increase in index size when using nested objects in elasticsearch?

The total number of data is 1 billion.
When I configure an index by setting some fields of data as nested objects, the number of data increases and the index size increases.
There are about 20 nested objects in a document.
When I index 1 billion documents, the number of indexes is 20 billion, and the index size is about 20TB.
However, when I remove nested objects, the number of indexes is 1 billion, and the index size is about 5TB.
It's simply removed nested object and can not provide services with this index structure.
I know why nested objects have a higher index count than a simple object configuration.
But I ask why the index is four times larger and how to fix it.
version of elasticsearch : 5.1.1
The Sample Data is as follows:
nested object Mapping : idds, ishs, resources, versions
{
"fileType": {
"asdFormat": 1
},
"dac": {
"pe": {
"cal": {
"d1": -4634692645508395000,
"d2": -5805223225419042000,
"d3": -1705264433
},
"bytes": "6a7068e0",
"entry": 0,
"count": 7,
"css": {
"idh": 0,
"ish": 0,
"ifh": 0,
"ioh": 0,
"ish": 0,
"ied": 0,
"exp": 0,
"imp": 0,
"sec": 0
},
"ff": {
"field1": 23117,
"field2": 144,
"field3": 3,
"field4": 0,
"field5": 4,
"field6": 0,
"field7": 65535,
"field8": 0,
"field9": 184,
"field10": 0,
"field11": 0,
"field12": 0,
"field13": 64,
"field14": 0,
"field15": 40104,
"field16": 64563,
"field17": 0,
"field18": 0,
"field19": 0,
"field20": 0,
"field21": 0,
"field22": 0,
"field23": 0,
"field24": 0,
"field25": 0,
"field26": 0,
"field27": 0,
"field28": 0,
"field29": 0,
"field30": 0,
"field31": 224
},
"ifh": {
"mc": 332,
"nos": 3,
"time": 1091599505,
"ps": 0,
"ns": 0,
"soh": 224,
"chart": 271
},
"ioh": {
"magic": 267,
"mlv": 7,
"nlv": 10,
"soc": 80384,
"soid": 137216,
"soud": 0,
"aep": 70290,
"boc": 4096,
"bod": 86016,
"aib": "16777216",
"si": 4096,
"fa": 512,
"mosv": 5,
"nosv": 1,
"miv": 5,
"niv": 1,
"msv": 4,
"nsv": 0,
"wv": 0,
"si": 262144,
"sh": 1024,
"cs": 0,
"ss": 2,
"dllchart": 32768,
"ssr": "262144",
"ssc": "262144",
"ssh": "1048576",
"shc": "4096",
"lf": 0,
"nor": 16
},
"idds": [
{
"id": 1,
"address": 77504,
"size": 300
},
{
"id": 2,
"address": 106496,
"size": 134960
},
{
"id": 6,
"address": 5264,
"size": 28
},
{
"id": 11,
"address": 592,
"size": 300
},
{
"id": 12,
"address": 4096,
"size": 1156
}
],
"ishs": [
{
"id": 0,
"name": ".text",
"size": 79920,
"address": 4096,
"srd": 80384,
"ptr": 1024,
"ptrl": 0,
"ptl": 0,
"nor": 0,
"nol": 0,
"chart": 3758096480,
"ex1": 60404022,
"ex2": 61903965,
"ex": 61153993.5
},
{
"id": 1,
"name": ".data",
"size": 17884,
"address": 86016,
"srd": 2048,
"ptr": 81408,
"ptrl": 0,
"ptl": 0,
"nor": 0,
"nol": 0,
"chart": 3221225536,
"ex1": 27817394,
"ex2": -1,
"ex": 27817394
},
{
"id": 2,
"name": ".rsrc",
"size": 155648,
"address": 106496,
"srd": 135680,
"ptr": 83456,
"ptrl": 0,
"ptl": 0,
"nor": 0,
"nol": 0,
"chart": 3758096448,
"ex1": 38215005,
"ex2": 46960547,
"ex": 42587776
}
],
"resources": [
{
"id": 2,
"count": 3,
"hash": 658696779440676200
},
{
"id": 3,
"count": 14,
"hash": 4671329014159995000
},
{
"id": 5,
"count": 30,
"hash": -6413921454731808000
},
{
"id": 6,
"count": 17,
"hash": 8148183923057157000
},
{
"id": 14,
"count": 4,
"hash": 8004262029246967000
},
{
"id": 16,
"count": 1,
"hash": 7310592488525726000
},
{
"id": 2147487240,
"count": 2,
"hash": -7466967570237519000
}
],
"upx": {
"path": "xps",
"d64": 3570326159822345700
},
"versions": [
{
"language": 1042,
"codePage": 1200,
"companyName": "Microsoft Corporation",
"fileDescription": "Files and Settings Transfer Wizard",
"fileVersion": "5.1.2600.2180 (xpsp_sp2_rtm.040803-2158)",
"internalName": "Microsoft",
"legalCopyright": "Copyright (C) Microsoft Corp. 1999-2000",
"originalFileName": "calc.exe",
"productName": "Microsoft(R) Windows (R) 2000 Operating System",
"productVersion": "5.1.2600.2180"
}
],
"import": {
"dll": [
"GDI32.dll",
"KERNEL32.dll",
"USER32.dll",
"ole32.dll",
"ADVAPI32.dll",
"COMCTL32.dll",
"SHELL32.dll",
"msvcrt.dll",
"comdlg32.dll",
"SHLWAPI.dll",
"SETUPAPI.dll",
"Cabinet.dll",
"LOG.dll",
"MIGISM.dll"
],
"count": 14,
"d1": -149422985349905340,
"d2": -5344971616648705000,
"d3": 947564411044974800
},
"ddSec0": {
"d1": -3007779250746558000,
"d4": -2515772085422514700
},
"ddSec2": {
"d2": -4422408392580008000,
"d4": -8199520081862749000
},
"ddSec3": {
"d1": -8199520081862749000
},
"cdp": {
"d1": 787971,
"d2": 39,
"d3": 101980696,
"d4": 3,
"d5": 285349133
},
"cde": {
"d1": 67242500,
"d2": 33687042,
"d3": 218303490,
"d4": 1663632132,
"d5": 0
},
"cdm": {
"d1": 319293444,
"d2": 2819,
"d3": 168364553,
"d4": 50467081,
"d5": 198664
},
"cdb": {
"d1": 0,
"d2": 0,
"d3": 0,
"d4": 0,
"d5": 0
},
"mm": {
"d0": -3545367393134139000,
"d1": 1008464166428372900,
"d2": -6313842304565328000,
"d3": -5015640502060250000
},
"ser": 17744,
"ideal": 0,
"map": 130,
"ol": 0
}
},
"fileSize": 219136
}

finding active framework current resource usage in mesos

Which HTTP endpoint will help me to find all the active frameworks current resource utilization?
We want this information because we want to dynamically scale Mesos cluster and our algorithm needs information regarding what resources each active framework is using.
I think to focus on the frameworks is not really what you would want to to. What you're after is probably the Mesos Slave utilization, which can be requested via calling
http://{mesos-master}:5050/master/state-summary
In the JSON answer, you'll find a slaves property which contains an array of slave objects:
{
"hostname": "192.168.0.3",
"cluster": "mesos-hw-cluster",
"slaves": [{
"id": "bd9c29d7-8530-4c5b-8c50-5d2f60dffbf6-S2",
"pid": "slave(1)#192.168.0.1:5051",
"hostname": "192.168.0.1",
"registered_time": 1456826950.99075,
"resources": {
"cpus": 12.0,
"disk": 1840852.0,
"mem": 63304.0,
"ports": "[31000-32000]"
},
"used_resources": {
"cpus": 5.75,
"disk": 0.0,
"mem": 14376.0,
"ports": "[31000-31000, 31109-31109, 31267-31267, 31699-31699, 31717-31717, 31907-31907, 31979-31980]"
},
"offered_resources": {
"cpus": 0.0,
"disk": 0.0,
"mem": 0.0
},
"reserved_resources": {},
"unreserved_resources": {
"cpus": 12.0,
"disk": 1840852.0,
"mem": 63304.0,
"ports": "[31000-32000]"
},
"attributes": {},
"active": true,
"version": "0.27.1",
"TASK_STAGING": 0,
"TASK_STARTING": 0,
"TASK_RUNNING": 7,
"TASK_FINISHED": 18,
"TASK_KILLED": 27,
"TASK_FAILED": 3,
"TASK_LOST": 0,
"TASK_ERROR": 0,
"framework_ids": ["bd9c29d7-8530-4c5b-8c50-5d2f60dffbf6-0000", "bd9c29d7-8530-4c5b-8c50-5d2f60dffbf6-0002"]
},
...
}
You could iterate over all the slave objects and calculate the overall ressource usage by summarizing the resources and then subtract the summary of the used_resources.
See
http://mesos.apache.org/documentation/latest/endpoints/master/state-summary/
http://mesos.apache.org/documentation/latest/endpoints/

Resources