I would like to bind a NSProgressIndicator to HTTP activity in STTwitter library. Just wondering if there is a way to determine if a HTTP connection is currently being active?
In order to display a network activity indicator, I would keep track of the number of active connections.
Just increment this counter when you start a request, and decrease it in successBlock / errorBlock.
Start the animation when the counter changes from 0 to 1, and stop it when it changes from 1 to 0.
Related
I deployed an apache beam pipeline to GCP dataflow in a DEV environment and everything worked well. Then I deployed it to production in Europe environment (to be specific - job region:europe-west1, worker location:europe-west1-d) where we get high data velocity and things started to get complicated.
I am using a session window to group events into sessions. The session key is the tenantId/visitorId and its gap is 30 minutes. I am also using a trigger to emit events every 30 seconds to release events sooner than the end of session (writing them to BigQuery).
The problem appears to happen in the EventToSession/GroupPairsByKey. In this step there are thousands of events under the droppedDueToLateness counter and the dataFreshness keeps increasing (increasing since when I deployed it). All steps before this one operates good and all steps after are affected by it, but doesn't seem to have any other problems.
I looked into some metrics and see that the EventToSession/GroupPairsByKey step is processing between 100K keys to 200K keys per second (depends on time of day), which seems quite a lot to me. The cpu utilization doesn't go over the 70% and I am using streaming engine. Number of workers most of the time is 2. Max worker memory capacity is 32GB while the max worker memory usage currently stands on 23GB. I am using e2-standard-8 machine type.
I don't have any hot keys since each session contains at most a few dozen events.
My biggest suspicious is the huge amount of keys being processed in the EventToSession/GroupPairsByKey step. But on the other, session is usually related to a single customer so google should expect handle this amount of keys to handle per second, no?
Would like to get suggestions how to solve the dataFreshness and events droppedDueToLateness issues.
Adding the piece of code that generates the sessions:
input = input.apply("SetEventTimestamp", WithTimestamps.of(event -> Instant.parse(getEventTimestamp(event))
.withAllowedTimestampSkew(new Duration(Long.MAX_VALUE)))
.apply("SetKeyForRow", WithKeys.of(event -> getSessionKey(event))).setCoder(KvCoder.of(StringUtf8Coder.of(), input.getCoder()))
.apply("CreatingWindow", Window.<KV<String, TableRow>>into(Sessions.withGapDuration(Duration.standardMinutes(30)))
.triggering(Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane().plusDelayOf(Duration.standardSeconds(30))))
.discardingFiredPanes()
.withAllowedLateness(Duration.standardDays(30)))
.apply("GroupPairsByKey", GroupByKey.create())
.apply("CreateCollectionOfValuesOnly", Values.create())
.apply("FlattenTheValues", Flatten.iterables());
After doing some research I found the following:
regarding constantly increasing data freshness: as long as allowing late data to arrive a session window, that specific window will persist in memory. This means that allowing 30 days late data will keep every session for at least 30 days in memory, which obviously can over load the system. Moreover, I found we had some ever-lasting sessions by bots visiting and taking actions in websites we are monitoring. These bots can hold sessions forever which also can over load the system. The solution was decreasing allowed lateness to 2 days and use bounded sessions (look for "bounded sessions").
regarding events dropped due to lateness: these are events that on time of arrival they belong to an expired window, such window that the watermark has passed it's end (See documentation for the droppedDueToLateness here). These events are being dropped in the first GroupByKey after the session window function and can't be processed later. We didn't want to drop any late data so the solution was to check each event's timestamp before it is going to the sessions part and stream to the session part only events that won't be dropped - events that meet this condition: event_timestamp >= event_arrival_time - (gap_duration + allowed_lateness). The rest will be written to BigQuery without the session data (Apparently apache beam drops an event if the event's timestamp is before event_arrival_time - (gap_duration + allowed_lateness) even if there is a live session this event belongs to...)
p.s - in the bounded sessions part where he demonstrates how to implement a time bounded session I believe he has a bug allowing a session to grow beyond the provided max size. Once a session exceeded the max size, one can send late data that intersects this session and is prior to the session, to make the start time of the session earlier and by that expanding the session. Furthermore, once a session exceeded max size it can't be added events that belong to it but don't extend it.
In order to fix that I switched the order of the current window span and if-statement and edited the if-statement (the one checking for session max size) in the mergeWindows function in the window spanning part, so a session can't pass the max size and can only be added data that doesn't extend it beyond the max size. This is my implementation:
public void mergeWindows(MergeContext c) throws Exception {
List<IntervalWindow> sortedWindows = new ArrayList<>();
for (IntervalWindow window : c.windows()) {
sortedWindows.add(window);
}
Collections.sort(sortedWindows);
List<MergeCandidate> merges = new ArrayList<>();
MergeCandidate current = new MergeCandidate();
for (IntervalWindow window : sortedWindows) {
MergeCandidate next = new MergeCandidate(window);
if (current.intersects(window)) {
if ((current.union == null || new Duration(current.union.start(), window.end()).getMillis() <= maxSize.plus(gapDuration).getMillis())) {
current.add(window);
continue;
}
}
merges.add(current);
current = next;
}
merges.add(current);
for (MergeCandidate merge : merges) {
merge.apply(c);
}
}
I'm trying to trigger a notification which has no expiry but must be closed by pressing the top-right X close button. Is this possible?
I've been able to trigger a timed notification which also closes when anywhere else is clicked. With this answer.
[reflection.assembly]::loadwithpartialname("System.Windows.Forms")
[reflection.assembly]::loadwithpartialname("System.Drawing")
$notify = new-object system.windows.forms.notifyicon
$notify.icon = [System.Drawing.SystemIcons]::Information
$notify.visible = $true
$notify.showballoontip(10,"New Chat!","You have received New Chat!",[system.windows.forms.tooltipicon]::None)
Per Microsofts NotifyIcon.ShowBalloonTip Method documentation, the actual timeout property is set by the current system settings.
Minimum and maximum timeout values are enforced by the operating system and are typically 10 and 30 seconds, respectively, however this can vary depending on the operating system. Timeout values that are too large or too small are adjusted to the appropriate minimum or maximum value. In addition, if the user does not appear to be using the computer (no keyboard or mouse events are occurring) then the system does not count this time towards the timeout.
According to a couple of more google searches, you can set the time for your profile through the Registry ( Regedit - HKEY_CURRENT_USER\Control Panel\Accessibility: MessageDuration - didn't work for me).
Through group policy, or using theSystemParametersInfo API which is out of my league to explain any further. Only reference I can find was configuring the Accessibility/System Parameter: SPI_SETMESSAGEDURATION.
Its C++ though and only other article I could find was this one:SystemParametersInfoA function.
Seems possible but, it will definitely be a hassle to get it working.
Trying to understand (new to kafka)how the poll event loop in kafka works.
Use Case : 25 records on the topic, max poll size is set to 5.
max.poll.interval.ms = 5000 //5 seconds by default max.poll.records = 5
Sequence of tasks
Poll the records from the topic.
Process the records in a for loop.
Some processing login where the logic would either pass or fail.
If logic passes (with offset) will be added to a map.
Then it will be committed using commitSync call.
If fails then the loop will break and whatever was success before this would be committed.The problem starts after this.
The next poll would just keep moving in batches of 5 even after error, is it expected?
What we basically expect is that the loop breaks and the offsets till success process message logic should get committed, then the next poll should continue from the failed message.
Example, 1st batch of poll 5 messages polled and 1,2 offsets successful and committed then 3rd failed.So the poll call keep moving to next batch like 5-10,10-15 if there are any errors in between we expect it to stop at that point and poll should start from 3 in first case or if it fails in 2nd batch at 8 then the next poll should start from 8th offset not from next max poll batch settings which would be like 5 in this case.IF IT MATTERS USING SPRING BOOT PROJECT and enable autocommit is false.
I have tried finding this in documentation but no help.
tried tweaking this but no help max.poll.interval.ms
EDIT: Not accepted answer because there is no direct solution for a customer consumer.Keeping this for informational purpose
max.poll.interval.ms is milliseconds, not seconds so it should be 5000.
Once the records have been returned by the poll (and offsets not committed), they won't be returned again unless you restart the consumer or perform seek() operations on the consumer to reset the offset to the unprocessed ones.
The Spring for Apache Kafka project provides a SeekToCurrentErrorHandler to perform this task for you.
If you are using the consumer yourself (which it sounds like), you must do the seeks.
You can manually seek to the beginning offset of the poll for all the assigned partitions on failure. I am not sure using spring consumer.
Sample code for seeking offset to beginning for normal consumer.
In the code below I am getting the records list per partition and then getting the offset of the first record to seek to.
def seekBack(records: ConsumerRecords[String, String]) = {
records.partitions().map(partition => {
val partitionedRecords = records.records(partition)
val offset = partitionedRecords.get(0).offset()
consumer.seek(partition, offset)
})
}
One problem doing this in production is bad since you don't want seekback all the time only in cases where you have a transient error otherwise you will end up retrying infinitely.
We're running an enterprise scale SAP application with front-end springboot clients connecting via Jco adapter 3.0 on Oracle VM using the connection pool (size 100). We're experiencing unsystematic long-running requests > 10s that are not visible in the SAP application server log, i.e. the bottleneck does not appear to be on SAP side.
Looking at the trace files (level 4) for an example request we can see that the time seems lost when the adapter thread tries to get the client from the pool (other threads continue execution, removed the irrelevant threads for clarity):
[20:05:50:259]: [JCoAPI] JCoContext.isStateful(P-foo-CPIC0) in session ID Client-53-1 returns false
[20:05:50:259]: [JCoAPI] JCoContext.begin(P-foo-CPIC0) in session ID Client-53-1
[20:05:50:259]: [JCoAPI] Started context for session Client-53-1
[20:05:50:259]: [JCoAPI] JCoContext.begin() for destination PFOO_200 (P-foo-CPIC0) on context with id Client-53-1; current state counter is 1
[20:05:50:259]: [JCoAPI] destination PFOO_200 destinationID=P-foo-CPIC0 executes Z_foo sessionID=Client-53-1, threadID=0x35
[20:05:50:259]: [JCoAPI] Context.getConnection on destination PFOO_200 (state: destination = STATEFUL, default = STATELESS)
[20:05:50:259]: [JCoAPI] PoolingFactory.getClient() on pool P-foo-CPIC0
--> time lost here
[20:06:20:840]: [JCoAPI] PoolingFactory.getClient() returns handle [3/84977415]
[20:06:20:840]: [JCoAPI] Context.getConnection on destination PFOO_200 nothing found in the context - got client from ConnectionManager [3/84977415]
[20:06:20:840]: [JCoAPI] JCoClient before execute(Z_foo) on handle [3/84977415]
[20:06:20:840]: [JCoRFC] Executing function Z_foo on handle [3/84977415]
[20:06:20:866]: [JCoAPI] JCoClient after execute(Z_foo) on handle [3/84977415] returns after 26 ms
[20:06:20:866]: [JCoAPI] Context.releaseConnection on destination PFOO_200 [3/84977415]
[20:06:20:867]: [JCoAPI] JCoContext.end(P-foo-CPIC0) in session ID Client-53-1
[20:06:20:867]: [JCoAPI] PoolingFactory.releaseClient() handle [3/84977415] into pool P-foo-CPIC0 [pool size: 3, peak limit: 100, waiting threads: 0, currently used: 1]
[20:06:20:879]: [JCoAPI] Finished context for session Client-53-1
[20:06:20:879]: [JCoAPI] JCoContext.end() for destination PFOO_200 (P-foo-CPIC0) on context with id Client-53-1; current state counter is 0
For a typical request the step is handled in milliseconds.
Are there any known limitations or configurations regarding pool handling for the Jco adapter, either on adapter or on SAP side?
Update we've on Jco adapter 3.0.16 and will double-check 3.0.17 now. DNS seems unlikely since we're monitoring dig/nslookup and they're running without delays.
Which JCo patch level do you use?
Did you try to update to the latest JCo patch level 3.0.17 first?
In your time gap the RFC connection will be opened and the RFC logon will be done, if the pool is empty at that time. Did you have a closer look with a higher trace level, or did you have a look into the RFC trace?
This can be anything from not having a free dialog work process at ABAP side, to SAP system database issues (required for the RFC logon authentication checks), slow response times from the SAP message server (if using load balanced logons), SNC handshake issues (if using SNC) or general network issues with the DNS (try using the IP address instead of a hostname).
Another point worth checking: you say your connection pool has size 100. Is it possible, that your program has more than 100 threads? Then it may happen from time to time, that all connections are currently busy in other threads and the current thread has to wait until a function call in another thread completes and a connection is returned to the pool.
(How long a thread waits on an empty pool can be customized via the "pool wait time" parameter.)
I am using veins4.6 with sumo 0.30 and omnet++5.1.1 in ubuntu 14.04. I have created a custom network with a cross(one intersection with 4 roads) and ran the simulation with 200 vehicles. I did not observe this behaviour for 4vehicles. I have seen it with 50 vehicles too. I need to get the count of total lost packets for my masters project. So I was looking at statistics and found that RXTXLostPackets is not zero. As far as I understood from documentation it should be zero if allowTxDuringRx=false. Default is false(PhyLayer80211p.ned). As I did not change any code yet, I was confused if that is expected behaviour.
What I have done so far.
from Mac1609_4::handleLowerControl, statsTXRXLostPackets is updated when Decider80211p responds with RECWHILESEND.
In Decider80211p::processSignalEnd, if value of whileSending is true RECWHILESEND is sent to mac layer as control message.
In Decider80211p::processSignalEnd, if(frame->getWasTransmitting() || phy11p->getRadioState() == Radio::TX) , this frame was considered as received while sending and sets the value for whileSending as true.
The wasTransmitting varilable is set to true in Decider80211p::switchToTx and Decider80211p::processNewSignal functions.
currentFrame->setWasTransmitting(true);
currentFrame->setBitError(true);
in Decider80211p::processNewSignal:
if (phy11p->getRadioState() == Radio::TX ) {
frame->setBitError(true); --> tried disabling both these values and the RXTXLostPackets was zero.
frame->setWasTransmitting(true);
DBG_D11P << "AirFrame: " << frame->getId() << " (" << recvPower << ") received, while already sending. Setting BitErrors to true" << std::endl;
}
There is one thread with similar issue with the fix of adding this line in processSignalEnd function. But looks like veins4.6 does not use curSyncFrame anymore.
Veins - Unexpected behavior with lost packets in certain vehicles
if (!frame->getWasTransmitting()){
curSyncFrame = 0;
}
I could not clearly understand the issue. The code and configuration files I have used are here. https://github.com/Rajeswar59/veins_learning.
Can anyone please take a look and help me with this. Thanks in advance.
edit: I went through the logs. This is what I could understand as of now.
https://drive.google.com/open?id=0BzjDW8PQhkSmSEUtZ2lpcld4ZXc --> some portion of logs are here.
---> order of sending
#13332 0.247987176594 node[30] --> node[48] id=22266
#13375 0.247987796864 node[18] --> node[20] id=22447
#13384 0.247987864534 node[20] --> node[30] id=22573
From logs I have concentrated on node 18. Two nodes that transmitted before 30 are 32 and 4. These 2 messages are received successfully by all 3 nodes. When a message arrives decider tries to set channel state as busy in processnewsignal and set idle after processing packet. This calls mac1609_4.cc channelBusy and channelIdle functions respectively. So the channelIdle variable is set accordingly. Also if channel is to be set busy it will stop contention and calculate currentBackoff if any packet is waiting to be transmitted. If channel is being set idle at the end of reception, startContent is called. Based on this only the lastIdle variable is set which is used to calculate nextMacEvent. So when the last successful message was received all the nodes which have a packet to send decide nextMacEvent and it is sent as self message in Mac1609_4.cc. on receiving the nextMacEvent self message we will start transmitting without checking if any other node has started transmission. We can not identify that probably because we are setting channel busy when we receive messages after some propagation delay. So between last successful transmission and nextMacEvent other nodes also take decision to transmit without checking current channel state. That's why the node has some receive events while sending. As far as my understanding goes before transmission we should sense current state of channel and retry backoff accordingly. We do not check this at the nextMacEvent. It looks like a collision behaviour but should we not check the current state of channel when backoff counter reaches zero and retry. Please correct me if I am wrong anywhere.
Thanks for your patience.
Any help or advice??
My Learnings(probably last update):
After Some digging, these are my learnings if it helps some one. The basic CSMA mechanism says before attempting for transmission, the node has to sense the channel, initiate transmission if the channel is sensed idle for AIFS time, or go in to back off if channel is busy. In veins the channel busy status is stored in idleChannel variable whose status is checked in Mac1609_4:channelBusySelf() function before initiating transmission (nextMacEvent in Mac1609_4::handleSelfMsg). The idleChannel is updated in Mac1609_4::channelBusy and Mac1609_4::channelIdle functions when a message reception starts and when message reception ends respectively. So when a previously transmitting node sends a packet, all the recieving nodes will receive the packet with varying delay i.e., starts receiving at different times and update their channelIdle variable. After that they calculate best time to transmit and starts transmission. It does check if channel is idle or not but as the channelIdle status is updated at next reception and because of transmission delay it takes some time between transmission start at sender and reception start at receiver side, both the transmitting nodes cant see other transmission. As far as I understand this is called a collision when more than two nodes start transmission at the same time. So the BitError statistic is set and statsTXRXLostPackets is also set. So while calculating totalLostPackets we can take only one of these two values.