#sys.any is not working reliably when it's used outside of the main flow.
I created an intent with 9 different words, and every word has been highlighted with the #sys.any entity type.
The idea of #sys.any is to grab ANY input the user does, however, it's only grabbing some words. If I put the same intent in the main flow at the start, it is able to grab anything I type correctly.
However, if I use the same exact intent in a second flow on the same project, it only grabs some words. If I type "pyramid," the intent is triggered. If I type "train", it's not a match. "Cat" is not a match. "Sandwich" is not a match. "Power" is not a match. "Soda" IS a match.
If I create an entirely new agent with the same exact intent, and have it go to a second flow, it's able to capture everything correctly on the second flow. It almost looks like my secondary flow is bugged. On my "bugged" secondary flow I can put the same intent on it on the start page, and it still only grabs some of the words, not all of them.
Any suggestions would be appreciated.
I have pages later on in the flow that also use sys.any. I actually have four different intents that use sys.any. If I change the intents on the pages later on to all use the same sys.any intent, then the sys.any at the start works properly. It shouldn't interact like this because the pages later on are not connected to the first page that utilizes a sys.any intent. Either way, it solves my problem, well, kind of. It's more of a work-around than a solution.
Related
I'm in the process of building a bot and the experience has been challenging for me so far. This is most likely since I'm coming from v1 and I'm trying to rebuild my bot in v4 style, which is pretty much a completely different framework it seems.
I find there's quite a lot of documentation out there, but it's been split up into theory and practice, probably due to the different development frameworks you can use (i.e. Node, C#). But having to go back and forth between these articles is not helping,
After quite a bit of messing around, I got to a point where things are beginning to get a bit more decent, but I still feel there's lots of room for improvement. I can't share the whole project at the moment, but I've created a gist of the most important code here: https://gist.github.com/jsiegmund/831d5337b1a438133991070daba8a27e
So my issues/questions with this code are the following:
The way to add dialogs and mainly the need to add prompts for retrieving the answers is confusing. I get the idea, but not the inner workings. For instance: I now have the prompts named after the same method names of the corresponding dialog step, is that the way it's supposed to work? There seems to be some magic code linking everything together, by convention? It would make much more sense to me when the waterfall steps would also include the prompts.
What's the right way of feeding the dialog with information so it can skip steps? I've got LUIS intents set-up in the main dialog which then open up this dialog for hour booking. Suppose my user says "I'd like to book 8 hours on customer X", I'd like the dialog to pre-populate the amount to 8 and the customer to X.
The customer/project resolving is maybe a not-so-standard requirement here. These come from a third party application, retrieved through API/SDK. So based on the logged-in user I need to go out to that application and retrieve the data for this user. This comes back in key/value pairs, where the key is a GUID. I don't want the user to type in GUIDs, so I have created these action buttons with the names of the customers, but to get the ID value into the next step it now 'writes' the GUID in the chat instead of the customer name. Using the name is tricky as I can't fully rely on those being unique. Also, for selecting the project I need the customer GUID and saving the final entry I also need the ID's. But I don't want the user to see those.
The way I now have the cards built is also weird to me. I first need to add a dialog for the card, and later when calling stepContext.PromptAsync I need to supply the card as an attachment as well. Feels duplicate to me, but removing either one of the steps fails. The normal style prompt doesn't work for me as that doesn't handle key/value but just strings (see number 3).
Okay, so those are some of the things I'm struggling with. I'm getting there and it works for now, but as said I can't escape the feeling that I'm not doing it right. If anyone could shine a light on this that would be highly appreciated.
Yeah, there is a lot of changes from version to version. Do you really mean v1?! 😲 Or v3?
The way to add dialogs and mainly the need to add prompts for retrieving the answers is confusing. I get the idea, but not really
the inner workings. For instance: I now have the prompts named after
the same method names of the corresponding dialog step, is that the
way it's supposed to work? There seems to be some magic code linking
everything together, by convention? It would make much more sense to
me when the waterfall steps would also include the prompts.
Essentially. The steps listed in the waterfall array are the names of the method names you've created. Basically, this is where you are giving the order of the steps that should be done by the bot. Prompts are classes used to retrieve data and are populated into the ("main") dialog using AddDialog() and are stored in dialog state with unique names so that they can be retrieved correctly. I see your point on how it might be simple to have everything setup in one "call" or declaration, and there probably could have been other approaches to how this was implemented; but this is what we got.
What's the right way of feeding the dialog with information so it can skip steps? I've got LUIS intents set-up in a main dialog which
then open up this dialog for hour booking. Suppose my user says "I'd
like to book 8 hours on customer X", I'd like the dialog to
pre-populate the amount to 8 and the customer to X.
Typically, steps use the previous steps value to reply, act or continue. In simple scenarios, skipping steps can be done by checking the state for those values. In the multiturn sample, if the user does not want to supply their age, it goes on to the next step and then it checks for the value and skips the step (it really doesn't skip it, it reports no age given, but you could just continue without any reply). Assuming the LUIS side of things is correct and getting the right intent+entities (let's say 'booking' intent and entities ['time' and 'customer']), then that should be doable. You would populate state info for both of those entities and then the later step (say prompting for the customer step) would just skip/bypass.
But, what you really want to do is look at adaptive dialogs. They are new and make this type of scenarios much more dynamic and flexible. Look here:
https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-adaptive-dialog-introduction?view=azure-bot-service-4.0
https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-dialogs-adaptive?view=azure-bot-service-4.0&tabs=csharp
https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/csharp_dotnetcore/adaptive-dialog
The customer / project resolving is maybe a not-so-standard requirement here. These come from a third party application, retrieved
through API/SDK. So based on the logged in user I need to go out to
that application and retrieve the data for this user. This comes back
in key/value pairs, where the key is a GUID. Obviously I don't want
the user to type in GUIDs, so I have created these action buttons with
the names of the customers, but to get the ID value into the next step
it now 'writes' the GUID in the chat instead of the customer name.
Using the name is tricky as I can't fully rely on those being unique.
I'm not 100% sure on this part. Let me look into it and get back to you.
Also, for selecting the project I need the customer GUID and saving
the final entry I also need the ID's. But I don't want the user to see
those.
State (conversation|user|etc) would be a good place to manage this.
The way I now have the cards built is also weird to me. I first need to add a dialog for the card, and later when calling
stepContext.PromptAsync I need to supply the card as an attachment as
well. Feels duplicate to me, but removing either one of the steps ends
in failure. The normal style prompt doesn't work for me as that
doesn't handle key/value but just strings (see number 3).
Nope, that's correct. I know it feels weird, but that is the way to do it. Basically, anything but simple text will be an attachment. Cards are JSON, therefore an attachment and you need to send that to the user/client.
You're doing it all correct. Again; I would suggest on looking at adaptive dialogs as that's the newer tech and the move forward. But otherwise; you're on the right path!
I want to go down recursively and automatically at the host and clusters tab (the blue one in the picture), and then take the text like guest os, compability, etc.
I already know how to get the text from summary, but I got the problem to loop (go down) at the host and cluster tabs.
Any idea how to do it?
Thank you very much.
It's probably not the best idea to have one robot do everything - i.e. setting filters, parsing results, clicking on each result, parsing data, then going back, and so on. Instead of that approach, divide and conquer. Create multiple sequences/workflows, each one with one specific task in mind.
Here's how I would tackle it:
Create a sequence that opens the target page, setting the filters (e.g. Germany, as mentioned in your link: https://www.autoscout24.de/ergebnisse?cy=D&powertype=kw&atype=C&ustate=N%2CU&sort=standard&desc=0&page=1&size=20)
Have the same robot extract the link for each result. Don't go into the details, this could potentially be the task of another robot.
Have the links stored in a queue. Read more about Queues and Transactions here.
Create another sequence that takes the next item in the queue and open the stored link. This will take you directly to the results page (e.g. https://www.autoscout24.de/angebote/opel-corsa-12v-city-servo-ahk-benzin-silber-84a66193-b6d6-4fc9-b222-ec5a7319f221?cldtidx=1)
Parse any kind of data you need.
This comes with the benefit that you can use multiple robots at the same time extracting data, potentially increasing scraping significantly. The Queues and Transactions feature will make sure each result is visited only once, and that multiple robots don't process the same item multiple times.
Edit: you might want to start with ReFramework, which I'd recommend.
I'm trying to understand how Event Sourcing changes the data architecture of a service. I've been doing a lot of research, but I can't seem to understand how data is supposed to be properly stored with event sourcing.
Let's say I have a service that keeps track of vehicles transporting packages. The current non relational structure for the data model is that each document represents a vehicle, and has many fields representing origin location, destination location, types of packages, amount of packages, status of the vehicle, etc. Normally this gets queried for information to be read to the front end. When changes are made by the user, the appropriate changes are made to this document in order to update this.
With event sourcing, it seems that a snapshot of every event is stored, but there seem to be a few ways to interpret that:
The first is that the multiple versions of the document I described exist, each a new snapshot every time a change is made. Each event would create a new version of this document and alter it. This is the easiest way for me to wrap my head around it, but I believe this to be incorrect.
Another interpretation I have is that each event stores SPECIFIC information about what's been altered in the document. When the vehicle status changes from On Road to Available, for example, an event specifically for vehicle status changes is triggered. Let's say it's called VehicleStatusUpdatedEvent, and contains the Vehicle ID number, the new status, and the timestamp for this event. So this event is stored and is published to a messaging queue. When picked up from the queue, the appropriate changes are made to the current version of the document. I can understand this, but I think I still have some misconceptions here. My understanding is that event sourcing allows us to have a snapshot of data upon each change, so we can know what it looks like at any point. What I just described would keep a log of changes, but still only have one version of the file, as the events only contain specific pieces of the whole file.
Can someone describe how the data flow and architecture works with event sourcing? Using the vehicle data example I provided might help me frame it better. I feel that I am close to understanding this, but I am missing some fundamental pieces that I can't seem to understand by searching online.
The current non relational structure for the data model is that each document represents a vehicle
OK, let's start from there.
In the data model you've described, storage of a document destroys the earlier copy.
Now imagine that instead we were storing the the document in a git repository. Then then saving the document would also save metadata, and that metadata would include a pointer to the previous document.
Of course, we've probably got a lot of duplication in that case. So instead of storing the complete document every time, we'll store a patch document (think JSON Patch), and metadata pointing to the original patch.
Take that same idea again, but instead of storing generic patch documents, we use domain specific messages that describe what is going on in terms of the model.
That's what the data model of an event sourced entity looks like: a list of domain specific descriptions of document transformations.
When you need to reconstitute the current state, you start with a state you know (which could be the "null" state of the document before anything happened to it, and replay onto that document all of the patches (events) that have occurred since.
If you want to do a temporal query, the game is the same, you replay the events up to the point in time that you are interested in.
So essentially when referring to an older build, you reconstruct the document using the events, correct?
Yes, that's exactly right.
So is there still a "current status" document or is that considered bad practice?
"It depends". In the general case, there is no current status document; only the write-ordered list of events is "real", and everything else is derived from that.
Conversations about event sourcing often lead to consideration of dedicated message stores for managing persistence of those ordered lists, and it is common that the message stores do not also support document storage. So trying to keep a "current version" around would require commits to two different stores.
At this point, designers typically either decide that "recent version" is good enough, in which case they build eventually consistent representations of documents outside of the transaction boundary... OR they decide current version is important, and look into storage solutions that support storing the current version in the same transaction as the events (ex: using an RDBMS).
what is the procedure used to generate the snapshot you want using the events?
IF you want to generate a snapshot, then you'll normally end up using a pattern called a projection, to iterate over the events and either fold or reduce them to create the document.
Roughly, you have a function somewhere that looks like
document-with-meta-data = projection(event-history-with-metadata)
At the moment I'm developing a piece of code which first gathers sentences from a set of documents, then tokenises these, then uses the results to analyse recurring frequencies of token sequences, including case variations (upper case/lower case/leading cap/other), then prints out the results.
Now I want to introduce two more stages before printing out the results:
1. firstly, removing "stop words" (i.e. words or short sequences the frequency of which can never be of interest, such as, in English, "the", "of the", "of which", etc.) - these stop words/"stop sequences" to be taken from a database table
2. secondly, bringing up a dialog enabling the user to identify sequences of new stop words, which would then remove the token sequences involved and also add the sequence in question to the database table.
The thing is, this is a multi-stage process, and I'm just wondering what TDD experts do faced with a situation like this: do I create a new test method for each individual stage...? The problem being that each individual stage requires the use of "live memory data" from the previous stage: another possibility could be to somehow serialise this data and then deserialise it when testing for the next stage... but then this would involve the app code doing things which were of benefit only for the testing code, i.e. it would mean tweaking ("distorting"?) the app code for the benefit of the testing code, which seems wrong in principle...
Also, if anyone can point me in the direction of a book or site which helps TDD newbs like myself go to "the next level" I would be very grateful.
later
To the person who marked this as "favorite": I've now got hold of a book called "Growing Object-Oriented Software, Guided by Tests", which is well-reviewed and appears to be for someone wanting to move from beginner to intermediate. First impressions good.
Any views on this book by experts also welcome, of course...
On the face of it, you seem to be building a pipeline. From what I can tell, you're currently implementing all of it within a single class, which stores both the data that's being worked on and implements the methods that do the processing. One approach that you could take would be to break down the problem into smaller chunks. Rather than having a single class, you have a class for each stage of the pipeline and another class for orchestrating the process which is responsible for plugging the stages together in the correct order.
So, scanning through what you've described, you appear to have the following processors:
DocumentReader (reads documents from somewhere into in memory document)
SentenceExtractor (document/list of documents in, list of sentences out)
1 or more SentenceAnalysers (sentences in, statistics out), you might want to break this down depending on the type of analysis and how complex it is.
StopWordExtractor (StopWordProvider and sentences in, sentences out)
There are additional supporting classes that would be needed, to support writing of new stopwords to the database and depending on how the stopwordprovider was implemented keeping it in sync as the user selects new ones.
Essentially, what I'm saying is that you appear to be doing too much in a single location. If you're really happy that the code as you've described it is a single unit, then there is nothing wrong with you testing it all in one place, but then your inputs will be your starting documents/sentences and your outputs will be the end of the process. If you agree with me that really, there are several distinct components involved in the process that could change independently, then I would suggest breaking the process down into smaller classes and testing that those perform as expected for given sets of inputs/outputs...
For context: this is an HTML app, with little or no browser side JavaScript. I can't easily change that so need to do this on the server.
CouchDB is built to not have side effects. This is fair enough. But there seems to be no method that i can conceive of with shows, views, lists to change what is shown to a user with subsequent requests, or based on user objects, without writing data.
And can a get request for document result in the creation of a new record? Im guessing not as that would be a side effect.
But if you can, you could just create a log and then have a view that picks an advert firm a set of documents describing adverts which is affected by the change in the log when a previous ad was shown.
I'm not actually going to show adverts on my site, I'm going to have tips, and article summaries and minor features that vary from page load to page load.
Any suggestions appreciated.
I've wrapped my head around how to work with the grain for the rest of the functionality I need, but this bit seems contrary to the way couchdb works.
I think you're going to need a list function that receives a set of documents from the view and then chooses only one to return, either at random or some other method. However, because you're inside a list function you gain access to the user's request details, including cookies (which you can also set, btw.) That sounds more like what you want.
In addition, you could specify different Views for the list function to use at query-time. This means you could, say, have only random articles show up on the homepage, but any type of content show up on all others.
Note: You can't get access to the request in a map/reduce function and you'll run into problems if you do something like Math.random() inside a map function.
So a list function is the way to go.
http://guide.couchdb.org/draft/transforming.html
Look into the various methods of selecting a random document from a view. That should enable you to choose a random document (presumably representing an ad, tip, etc.) to display.