API call:
https://www.googleapis.com/youtube/analytics/v1/reports?ids=channel==UCs4uXj0TcstDHqhHDUWlINg&start-date=2016-10-02&end-date=2016-10-02&metrics=views%2ccomments%2clikes%2cdislikes%2cshares%2cestimatedMinutesWatched%2caverageViewDuration%2caverageViewPercentage%2cannotationClickThroughRate%2cannotationCloseRate%2csubscribersGained%2csubscribersLost
token:
ya29
.OAJc4bDrDoA6XVEmCI9KZK6rfIz68aXjibhZFQowWZxHJx7tt0qyvpxUryxtPZtN8IrN
Observe the parameters I used in reports.query Try-it:
The correct format in the ids textfield is 'channel=={YOUR_CHANNEL_ID}'
The response will correspond to the order you enumerated your metrics. For example, the response to the parameters above was:
"rows": [
[
201,
0,
9
]
]
meaning:
201 - views
0 - likes
9 - comments
because my metrics textfield was in this order -> views,likes,comments.
Related
Below is a python script that subscribes order book information via Biance's Websocket API (Documentation Here).
In both requests(btcusdt#depth and btcusdt#depth#100ms), each json payload is streamed with a varying depth.
Please shed light on what might be the cause of this? Am I doing something wrong? Or might they have certain criteria as to how many depths of an order book to fetch?
import json
import websocket
socket='wss://stream.binance.com:9443/ws'
def on_open(self):
print("opened")
subscribe_message = {
"method": "SUBSCRIBE",
"params":
[
"btcusdt#depth#100ms"
],
"id": 1
}
ws.send(json.dumps(subscribe_message))
def on_message(self, message):
print("received a message")
###### depths of bid/ask ######
d = json.loads(message)
for k, v in d.items():
if k == "b":
print(f"bid depth : {len(v)}")
if k == "a":
print(f"ask depth : {len(v)}")
def on_close(self):
print("closed connection")
ws = websocket.WebSocketApp(socket,
on_open=on_open,
on_message=on_message,
on_close=on_close)
ws.run_forever()
btcusdt#depth#100ms
opened
received a message
received a message
bid depth : 3
ask depth : 12
received a message
bid depth : 14
ask depth : 12
received a message
bid depth : 17
ask depth : 24
received a message
bid depth : 14
ask depth : 16
received a message
bid depth : 3
ask depth : 5
received a message
bid depth : 16
ask depth : 6
.
.
.
btcusdt#depth
opened
received a message
received a message
bid depth : 135
ask depth : 127
received a message
bid depth : 125
ask depth : 135
received a message
bid depth : 95
ask depth : 85
received a message
bid depth : 68
ask depth : 88
received a message
bid depth : 119
ask depth : 145
received a message
bid depth : 127
ask depth : 145
.
.
.
Your code reads the length of the diff for the last 100 ms or 1000 ms (the default value when you don't specify the timeframe). I.e. the remote API sends just the diff, not the full list.
The varying length of the diff is expected.
Example:
An order book has 2 bids and 2 asks:
ask price 1.02, amount 10
ask price 1.01, amount 10
bid price 0.99, amount 10
bid price 0.98, amount 10
During the timeframe, one more bid is added and one ask is updated. So the message returns:
"b": [
[ // added new bid
0.97,
10
]
],
"a": [
[ // updated existing ask
1.01,
20
]
]
And your code reads this message as
bid depth: 1
ask depth: 1
During another timeframe, two bids are updated
"b": [
[ // updated existing bid
0.98,
20
],
[ // updated existing bid
0.99,
20
]
],
"a": [] // no changes
So your code reads this as
bid depth: 2
ask depth: 0
"btcusdt#depth#100ms" only provides the change in the order book, not the order book itself (as mentioned by the other answer)
Use: "btcusdt#depth10#100ms" if you want to stream the book 10 best bids and 10 best asks.
can someone explain what does the MAX statistic refers to in the below response. I don't see it documented anywhere.
localhost:8081/actuator/metrics/http.server.requests?tag=uri:/myControllerMethod
Response:
{
"name":"http.server.requests",
"description":null,
"baseUnit":"milliseconds",
"measurements":[
{
"statistic":"COUNT",
"value":13
},
{
"statistic":"TOTAL_TIME",
"value":57.430899
},
{
"statistic":"MAX",
"value":0
}
],
"availableTags":[
{
"tag":"exception",
"values":[
"None"
]
},
{
"tag":"method",
"values":[
"GET"
]
},
{
"tag":"outcome",
"values":[
"SUCCESS"
]
},
{
"tag":"status",
"values":[
"200"
]
},
{
"tag":"commonTag",
"values":[
"somePrefix"
]
}
]
}
You can see the individual metrics by using ?tag=url:{endpoint_tag} as defined in the response of the root /actuator/metrics/http.server.requests call. The details of the measurements values are;
COUNT: Rate per second for calls.
TOTAL_TIME: The sum of the times recorded. Reported in the monitoring system's base unit of time
MAX: The maximum amount recorded. When this represents a time, it is reported in the monitoring system's base unit of time.
As given here, also here.
The discrepancies you are seeing is due to the presence of a timer. Meaning after some time currently defined MAX value for any tagged metric can be reset back to 0. Can you add some new calls to /myControllerMethod then immediately do a call to /actuator/metrics/http.server.requests to see a non-zero MAX value for given tag?
This is due to the idea behind getting MAX metric for each smaller period. When you are seeing these metrics, you will be able to get an array of MAX values rather than a single value for a long period of time.
You can get to see this in action within Micrometer source code. There is a rotate() method focused on resetting the MAX value to create above described behaviour.
You can see this is called for every poll() call, which is triggered every some period for metric gathering.
What does MAX represent
MAX represents the maximum time taken to execute endpoint.
Analysis for /user/asset/getAllAssets
COUNT TOTAL_TIME MAX
5 115 17
6 122 17 (Execution Time = 122 - 115 = 17)
7 131 17 (Execution Time = 131 - 122 = 17)
8 187 56 (Execution Time = 187 - 131 = 56)
9 204 56 From Now MAX will be 56 (Execution Time = 204 - 187 = 17)
Will MAX be 0 if we have less number of request (or 1 request) to the particular endpoint?
No number of request for particular endPoint does not affect the MAX (see Image from Spring Boot Admin)
When MAX will be 0
There is Timer which set the value 0. When the endpoint is not being called or executed for sometime Timer sets MAX to 0. Here approximate timer value is 2 minutes (120 seconds)
DistributionStatisticConfig has .expiry(Duration.ofMinutes(2)).
which sets some measurements to 0 if there is no request has been made in between expiry time or rotate time.
How I have determined the timer value?
For that, I have taken 6 samples (executed the same endpoint for 6 times). For that, I have determined the time difference between the time of calling the endpoint - time for when MAX set back to zero
More Details
UPDATE
Document has been updated.
NOTE:
Max for basic DistributionSummary implementations such as CumulativeDistributionSummary, StepDistributionSummary is a time
window max (TimeWindowMax).
It means that its value is the maximum value during a time window.
If the time window ends, it'll be reset to 0 and a new time window starts again.
Time window size will be the step size of the meter registry unless expiry in DistributionStatisticConfig is set to other value
explicitly.
I have a Spring Boot application and I am using Spring Boot Actuator and Micrometer in order to track metrics about my application. I am specifically concerned about the 'http.server.requests' metric and the MAX statistic:
{
"name": "http.server.requests",
"measurements": [
{
"statistic": "COUNT",
"value": 2
},
{
"statistic": "TOTAL_TIME",
"value": 0.079653001
},
{
"statistic": "MAX",
"value": 0.032696019
}
],
"availableTags": [
{
"tag": "exception",
"values": [
"None"
]
},
{
"tag": "method",
"values": [
"GET"
]
},
{
"tag": "status",
"values": [
"200",
"400"
]
}
]
}
I suppose the MAX statistic is the maximum time of execution of a request (since I have made two requests, it's the the time of the longer processing of one of them).
Whenever I filter the metric by any tag, like localhost:9090/actuator/metrics?tag=status:200
{
"name": "http.server.requests",
"measurements": [
{
"statistic": "COUNT",
"value": 1
},
{
"statistic": "TOTAL_TIME",
"value": 0.029653001
},
{
"statistic": "MAX",
"value": 0.0
}
],
"availableTags": [
{
"tag": "exception",
"values": [
"None"
]
},
{
"tag": "method",
"values": [
"GET"
]
}
]
}
I am always getting 0.0 as a max time. What is the reason of this?
What does MAX represent (MAX Discussion)
MAX represents the maximum time taken to execute endpoint.
Analysis for /user/asset/getAllAssets
COUNT TOTAL_TIME MAX
5 115 17
6 122 17 (Execution Time = 122 - 115 = 17)
7 131 17 (Execution Time = 131 - 122 = 17)
8 187 56 (Execution Time = 187 - 131 = 56)
9 204 56 From Now MAX will be 56 (Execution Time = 204 - 187 = 17)
Will MAX be 0 if we have less number of request (or 1 request) to the particular endpoint?
No number of request for particular endPoint does not affect the MAX (see an image from Spring Boot Admin)
When MAX will be 0
There is Timer which set the value 0. When the endpoint is not being called or executed for sometime Timer sets MAX to 0. Here approximate timer value is 2 to 2.30 minutes (120 to 150 seconds)
DistributionStatisticConfig has .expiry(Duration.ofMinutes(2)) which sets the some measutement to 0 if there is no request has been made for last 2 minutes (120 seconds)
Methods such as public TimeWindowMax(Clock clock,...), private void rotate() Clock interface has been written for the same. You may see the implementation here
How I have determined the timer value?
For that, I have taken 6 samples (executed the same endpoint for 6 times). For that, I have determined the time difference between the time of calling the endpoint - time for when MAX set back to zero
MAX property belongs to enum Statistic which is used by Measurement
(In Measurement we get COUNT, TOTAL_TIME, MAX)
public static final Statistic MAX
The maximum amount recorded. When this represents a time, it is
reported in the monitoring system's base unit of time.
Notes:
This is the cases from metric for a particular endpoint (here /actuator/metrics/http.server.requests?tag=uri:/user/asset/getAllAssets).
For generalize metric of actuator/metrics/http.server.requests
MAX for some endPoint will be set backed to 0 due to a timer. In my view for MAX for /http.server.requests will be same as a particular endpoint.
UPDATE
The document has been updated for the MAX.
NOTE: Max for basic DistributionSummary implementations such as
CumulativeDistributionSummary, StepDistributionSummary is a time
window max (TimeWindowMax). It means that its value is the maximum
value during a time window. If the time window ends, it'll be reset to
0 and a new time window starts again. Time window size will be the
step size of the meter registry unless expiry in
DistributionStatisticConfig is set to other value explicitly.
You can see the individual metrics by using ?tag=url:{endpoint_tag} as defined in the response of the root /actuator/metrics/http.server.requests call. The details of the measurements values are;
COUNT: Rate per second for calls.
TOTAL_TIME: The sum of the times recorded. Reported in the monitoring system's base unit of time
MAX: The maximum amount recorded. When this represents a time, it is reported in the monitoring system's base unit of time.
As given here, also here.
The discrepancies you are seeing is due to the presence of a timer. Meaning after some time currently defined MAX value for any tagged metric can be reset back to 0. Can you add some new calls to your endpoint then immediately do a call to /actuator/metrics/http.server.requests to see a non-zero MAX value for given tag?
This is due to the idea behind getting MAX metric for each smaller period. When you are seeing these metrics, you will be able to get an array of MAX values rather than a single value for a long period of time.
You can get to see this in action within Micrometer source code. There is a rotate() method focused on resetting the MAX value to create above described behaviour.
You can see this is called for every poll() call, which is triggered every some period for metric gathering.
I intend to use a Prometheus Histogram vector to monitor the execution time of request handlers in Go.
I register it so:
var RequestTimeHistogramVec = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "request_duration_seconds",
Help: "Request duration distribution",
Buckets: []float64{0.125, 0.25, 0.5, 1, 1.5, 2, 3, 4, 5, 7.5, 10, 20},
},
[]string{"endpoint"},
)
func init() {
prometheus.MustRegister(RequestTimeHistogramVec)
}
I use it so:
startTime := time.Now()
// handle request here
metrics.RequestTimeHistogramVec.WithLabelValues("get:" + endpointName).Observe(time.Since(startTime).Seconds())
When I do a HTTP GET to the /metrics endpoint after using my endpoint a couple of times, I get - amongst other things - the following:
# HELP request_duration_seconds Request duration distribution
# TYPE request_duration_seconds histogram
request_duration_seconds_bucket{endpoint="get:/position",le="0.125"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="0.25"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="0.5"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="1"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="1.5"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="2"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="3"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="4"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="5"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="7.5"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="10"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="20"} 6
request_duration_seconds_bucket{endpoint="get:/position",le="+Inf"} 6
request_duration_seconds_sum{endpoint="get:/position"} 0.022002387
request_duration_seconds_count{endpoint="get:/position"} 6
From the looks of it, all buckets are filled by the same amount, equal to the total amount of times I used my endpoint (6 times).
Why does this happen and how may I fix it?
Prometheus histogram buckets are cumulative, so in this case all the requests took less than or equal to 125ms.
In this case your choice of buckets may not be the best, you might want to make some of the buckets smaller.
This is not an error. Notice the rule for filling the bucket is le=..., meaning less or equal. Since all 6 requests succeeded quickly, all buckets were filled.
I am working with an RDBMS that contains a list of hierarchical objects stored like this:
Id Name ParentId
====================================
1 Food NULL
2 Drink NULL
3 Vegetables 1
4 Fruit 1
5 Liquor 2
6 Carrots 3
7 Onions 3
8 Strawberries 4
...
999 Celery 3
I do not know the specific reason why this was chosen, but it is fixed in so far as the rest of the system relies on fetching the structure in this form.
I want to expose this data via JSON using a RESTful API, and I wish to output this in the following format (array of arrays):
item:
{
id: 1, Description: "Food",
items: [
{
id: 3, Description: "Vegetables",
items: [ ... ]
},
{
id: 4, Description: "Fruit",
items: [ ... ]
}
]
},
item:
{
id: 2, Description: "Drink",
items: [ ... ]
}
What would be a sensible way of looping through the data and producing the desired output? I'm developing in C# but if there are libraries or examples for other languages I would be happy to re-implement where possible.
Thanks :)