Let's assume i have the following implementation
For each incoming event, one of the following combination of services will be called
combo 1: service 1 -> service 2 -> service 3
combo 2: service 1 -> service 3 [service 3 still need to read output from service 2]
Each service depends on the output of the upstream service.
I will be persisting "event id" in each of the service output. So in combo #1, i can use "event id" to fetch the data from upstream.
But in combo #2, service 2 will not be called so i cannot use "event id" to source the data in service 3. in my understanding, i have the following options
option #1: call all the microservices for each event whether it's needed or not. Persist the latest output in case if the service didn't produce any output for the event
option #2: Use the latest event's timestamp and get the output prior to this timestamp from service #2.
Question is "even if my event was generated by different systems, can i still rely on the timestamp to fetch the latest output".
Related
My goal is to filter notifications coming from different applications (mainly from different browser window).
I found that with the help of the dbus-monitor I can write a small script that could filter the notification messages that I am interested in.
The filter script is working well, but I have a small problem:
I am starting with the
dbus-monitor "interface='org.freedesktop.Notifications', destination=':1.40'"
command. I have to added the "destination=':1.40'" because on Ubuntu 20.04 I always got twice the same notification.
The following output of
dbus-monitor --profile "interface='org.freedesktop.Notifications'"
demonstrate the reason:
type timestamp serial sender destination path interface member
# in_reply_to
mc 1612194356.476927 7 :1.227 :1.56 /org/freedesktop/Notifications org.freedesktop.Notifications Notify
mc 1612194356.483161 188 :1.56 :1.40 /org/freedesktop/Notifications org.freedesktop.Notifications Notify
As you can see the sender :1.277 sends to :1.56 first than this will be the sender to :1.40 destination. (Simply notify-send hello test message was sent)
My script is working on that way, but every time system boot up, I have to check the destination number and modify my script accordingly to get worked.
I have two questions:
how to discover the destination string automatically? (:1.40 in the above example)
how to prevent system sending the same message twice? (If this question would be answered, than the question under point 1. became pointless.)
I'm using a spring-kafka to run Kafka Stream in a Spring Boot application using StreamsBuilderFactoryBean. I changed the number of partitions in some of the topics from 100 to 20 by deleting and recreating them, but now on running the application, I get the following error:
Existing internal topic MyAppId-KSTREAM-AGGREGATE-STATE-STORE-0000000092-changelog has invalid partitions: expected: 20; actual: 100. Use 'kafka.tools.StreamsResetter' tool to clean up invalid topics before processing.
I couldn't access the class kafka.tools.StreamsResetter and tried calling StreamsBuilderFactoryBean.getKafkaStreams.cleanup() but it gave NullPointerException. How do I do the said cleanup?
The relevant documentation is at here.
Step 1: Local Cleanup
For Spring Boot with StreamsBuilderFactoryBean, the first step can be done by simply adding CleanerConfig to the constructor:
// Before
new StreamsBuilderFactoryBean(new KafkaStreamsConfiguration(config));
// After
new StreamsBuilderFactoryBean(new KafkaStreamsConfiguration(config), new CleanupConfig(true, true));
This enables calling the KafkaStreams.cleanUp() method on both before start() & after stop().
Step 2: Global Cleanup
For step two, with all instances of the application stopped, simply use the tool as explained in the documentation:
# In kafka directory
bin/kafka-streams-application-reset.sh --application-id "MyAppId" --bootstrap-servers 1.2.3.4:9092 --input-topics x --intermediate-topics first_x,second_x,third_x --zookeeper 1.2.3.4:2181
What this does:
For any specified input topics: Reset the application’s committed consumer offsets to "beginning of the topic" for all partitions (for consumer group application.id).
For any specified intermediate topics: Skip to the end of the topic, i.e. set the application’s committed consumer offsets for all partitions to each partition’s logSize (for consumer group application.id).
For any internal topics: Delete the internal topic (this will also delete committed the corresponding committed offsets).
I'm getting error for "SNMPv3" of invalid engine ID? what causes this error usually?
it perfectly works when i select no auth,no priv. but for other two options of auth,no priv and auth,priv it gives error of invalid engine id.
ManageEngine MibBrowser supports auto-discovery of SnmpEngineID, EngineBoots and EngineTime from an Authoritative SNMP Engine.
If user has enabled the "Set EngineID For Adding V3 entry" and entered SNMPV3 user details with EngineID parameter, MibBrowser displays the message received from Agent.
As per RFC3414, If the value of the msgAuthoritativeEngineID
field in the securityParameters is unknown then:
a) a non-authoritative SNMP engine that performs discovery may
optionally create a new entry in its Local Configuration Datastore
(LCD) and continue processing;
or
b) the usmStatsUnknownEngineIDs counter is incremented, and an error
indication (unknownEngineID) together with the OID and value of the
incremented counter are returned to the calling module.
Note in the event that a zero-length or other illegally sized
msgAuthoritativeEngineID is received,b) should be chosen to facilitate
engineID discovery. Otherwise, the choice between a) and b) is an
implementation issue.
If Authoritative SNMP Engine returns "unKnownEngineID OID"(.1.3.6.1.6.3.15.1.1.4.0) with incremental counter value to manager. Then, Mibbrowser displays the "Invalid EngineID" error.
Regards,
ManageEngine
We have been implementing an instant message service and want to use HBEASE to store message history (and using redis to caching ongoing conversation). The incoming message for a user looks like
Userid (to whom)
time
message body (combined with from, message body ....)
Regarding Schema design:
Option A: one message per row
Row key: md5(userid) + timesample
column/valye: null / message
Option B: one user per row
Row key: md5(userid)
column/valye: time / message
could you help me to figure out pro and cont? thanks
chatting type include: peer-2-peer, group chating
As far as I know Facebook has done a great job on message system use hbase; Maybe these links will help you:http://www.slideshare.net/brizzzdotcom/facebook-messages-hbase
http://sites.computer.org/debull/A12june/facebook.pdf
I'm trying to read the data off a contactless Visa Paywave card.
For the Paywave, I have to submit a SELECT using PPSE (2PAY.SYS.DDF01) instead of PSE (1PAY.SYS.DDF01).
The EMV book 1, section 11.3.4, table 43 only describes how to interpret the response for a successful SELECT command using PSE. Does anyone know or can refer me to a source that shows how to process the data returned from a successful SELECT command using PPSE?
Here's my request APDU:
00A404000e325041592e5359532e444446303100
Here's the response:
6F2F840E325041592E5359532E4444463031A51DBF0C1A61184F07A0000000031010500A564953412044454249548701019000
I understand tag 84, tag 85, tag BF0C from the response. According to the examples for reading PSE, I should be able to just send GET PROCESSION OPTIONS (to get the AIP and AFL) with PDOL = null after this successful response as follows: 80A80000830000.
But request 80A80000830000 returns error code 6985 - Command not allowed; conditions of use not satisfied.
I also tried reading all the files after successfully selecting the PPSE by traversing through every single SFI (0-30) and every single record (0-16) of each SFI. Yes, I also did the 3 bit shift and bitwise-OR the SFI with 0x4. But I got no data.
I'm stuck, any help that would point me into getting some info from my Paywave card would be appreciated!
Have you tried this tool from EMVLAB http://www.emvlab.org/emvtags/
Using that tool,
http://www.emvlab.org/tlvutils/?data=6F2F840E325041592E5359532E4444463031A51DBF0C1A61184F07A0000000031010500A564953412044454249548701019000
2PAY.SYS.DDF01 is for contactless (e.g. NFC ) cards, while 1PAY.SYS.DDF01 is for contact cards.
After successfully (SW1 SW2 = 90 00) reading a PSE, you should only search for the SFI (tag 88) which is a mandatory field in the FCI template returned.
With the SFI as your start index, your would have to read the records starting from the start index until you get a 6A83 (RECORD_NOT_FOUND). E.g. if your SFI is 1, you would do a readRecord with record_number=1. That would probably be successful. Then you increament record_number to 2 and do readRecord again. The increament to 3 .... Repeat it until you get 6A83 as your status.
The records read would be ADFs (at least 1). Then your would have to compare the read ADF Names with what your terminal support and also based on the ASI (Application Selection Indicator). At the end you would have a list of possible ADFs (Candidate list)
All the above steps (1-3) are documented in chapter 12.3.2 Book1 v4.3 of the EMV spec.
You would have to make a final selection (Chapter 12.4 Book1)
Read the spec book 1 chapter 12.3 - 12.4 for all the detailed steps.
You seem to have the flow mixed up a bit, you want to:
Send 1PAY or 2PAY, it doesn't actually matter for all of the cards I've tested. This will return a list of the AIDs available on the card. Alternately you can just select an AID straight away if you know it's there but good practice would be to check first.
Get the list of AIDs returned in response to 1PAY/2PAY, in PayWave's case this will probably be A0000000031010 if you sent 2PAY but you may get more if you send 1PAY.
Select one of the AIDs sent back (or one you already know is on there).
Then loop through the SFIs and records sending the Read Records command to get the data.
You don't have to send Get Processing Options before sending the Read Records command even though that's now a normal transaction flow goes.
I think the information you're looking for is available from this VISA website. But only if you're a registered and/or licensed partner of VISA.
EDIT: Looking at the resulting TLV struct under BF0C:
tag=0xBF0C, length=0x1A
tag=0x61, length=0x18
tag=0x4F, length=0x07, value=0xA0000000031010 // looks like an AID to me
tag=0x50, length=0x0A, value="VISA DEBIT"
tag=0x87, length=0x01, value=0x01
I would guess that you need to first select A0000000031010 before getting the processing options.
I was selecting application 2PAY.SYS.DDF01. when I should have been selecting AID = 0xA0000000031010. It looks like there's no records under application 2PAY.SYS.DDF01.
But there was 1 record under application 0xA0000000031010. After I got this application, I performed a READ RECORD, and the first record gave me the PAN and all the credit card info I wanted.
Thanks everyone for chiming in.