I'm trying to find usage statistics for the "statuses" in our Redmine data. I'm using two filters: The tracker and the status. I realized that these filters only show tasks which are currently in that particular status. To find out the total usage statistics I need to find out the historical data, too, which means if a task was marked with a particular status in the past, it should be counted.
There isn't any current filter or report in Redmine which can give me those numbers. Do you know of any other methods to query historical data in Redmine?
(We're changing our tracker, status and workflows and we want to remove those unused statuses from the workflows. To make that decision, we need to find out which statuses were useful and functional in the past.)
Related
I am trying to split traffic from a given flow into different versions to measure statistical performance over time using the Experiment feature. However, it always shows the state "Collecting Data".
Here are the steps to reproduce the issue --
Create an Experiment on a flow and select different versions
Select Auto rollout and Select Steps option
Add steps for gradual progress of traffic increase and minimum duration
Save and Start the Experiment
Send queries to chatbot triggering the configured flow for the Experiment
The experiment should show some results in the Status tab and compare the performance of multiple flow versions. However, It does not produce any results. Always show the status as "Collecting Data" and Auto Rollout as "Not Started".
The only prerequisite for the Experiments feature to work is to enable the Interaction logs which are already enabled on my virtual agent.
About 2.5K sessions (~4K interactions ) were created in the last 48 hours. Are there any minimum requirements for it to generate results like the minimum number of sessions etc.?
I've already read official documentation and find no way.
My datas to es are from kafka which sometimes can be out of order. In the past, message from kafka is parsed and directly insert or update ES doc with specific ID. To avoid the older data override the newer data, I have to check whether the doc with specific ID is already exists and some properties of this doc are meet the conditions. Then I do the UPDATE action(or INSERT).
What I'm doing now is 'search before update'.
Before updating a doc, I search from ES with specific ID(included in kafka msg). Then check if this doc meets the conditions(for example, whether update_time is older?). Lastly I update the doc. And I set refresh to true to update index instantly.
What I'm worried about?
It seems Transactional.
If there is only one Thread executing synchronously, is it possible that When I process next message the doc updated in last message process is not refresh at ES?
If I have several Threads consuming kafka message, how to check before update? Can I use script to solve this problem?
If there is only one Thread executing synchronously, is it possible that When I process next message the doc updated in last message process is not refresh at ES?
That is a possibility since indexes are refreshed once in every second (by default), reducing this value is neither recommended nor guaranteed to give you the desired result since Elasticsearch is NOT designed for this.
If I have several Threads consuming kafka message, how to check before update? Can I use script to solve this problem?
You can use script if the number of fields being updated are very limited. Personally I have found script to be best suited for single field update and that too for corner use cases, it should not be used as a general practice. Any more than that and you are running into the same risk as that with stored procedures in the RDBMS world. It makes data management volatile overall and a system which is harder to maintain/extend in the longer run.
Your use case is best suited for optimistic locking support available from Elasticsearch out of the box. Take a look at Elasticsearch Versioning Support for full details.
You can very well use the inbuilt doc version if concurrency is the only problem that you need to solve. If, however, you need more than concurrency (out of order message delivery and respective ES updates) then you should use your application/domain specific field as the inbuilt version wouldn't work as-is.
You can very well use any of the app specific (numeric) field as a version field and use it for optimistic locking during document updates. If you use this approach, please pay special attention to all insert, update, delete operations for that index. Quoting AS-IS from versioning support - when using external versioning, make sure you always add the current version (and version_type) to any index, update or delete calls. If you forget, Elasticsearch will use it's internal system to process that request, which will cause the version to be incremented erroneously
I'll recommend you evaluate the inbuilt version first and use it if it fulfills your needs. It'll make the overall design much simpler. Consider the app specific version as the second option if the inbuilt version does not meet your requirements.
If there is only one Thread executing synchronously, is it possible that When I process next message the doc updated in last message process is not refresh at ES?
Ad 1. It is possible to save data in ElasticSearch and in a short while after receive stale result (before the index is updated)
If I have several Threads consuming kafka message, how to check before update? Can I use script to solve this problem?
Ad 2. If you process Kafka messages in several threads, it would be the best to use business data (eg. some business ids) as partition keys in Kafka to ensure data is processed in order. Remember to use Kafka to consume messages in many threads and don't consume messages by single consumer to fan out later to multiple threads.
It seems it would be best to ensure data is processed in order and then drop checking in Elasticsearch since it is not guaranteed to give valid results.
We are planning to use the "history" interaction to support viewing of historical snapshots of a resource (for example, viewing care event details for an encounter as historical snapshots of the encounter)
For example,
GET encounter/{id}/_history/{vid}
We wanted to use the same structure to perform retrospective updates to a particular history entry using a PUT interaction
PUT encounter/{id}/_history/{vid}
However, there seems to be a restriction in doing so as mentioned here
Accordingly, there is no way to update or delete past versions of the
record, except that the metadata can be modified (mainly for access
control purposes)
Is there any other mechanism for performing retrospective updates?
There is no mechanism to adjust history. History does not represent "the history of what occurred". It represents "the set of versions that existed on this server at a particular period of time". As such, short of time travel, there's no meaningful need to change history records. If you wanted to assert multiple separate things about a resource at different times, you could create multiple instances and link them together using Linkage - e.g. A Condition that had one severity for a year, then escalated for 2 years, then went into remission for a year, then came back, then got resolved could be represented using multiple Condition records each with different effective periods. Linkage could be used to indicate that they were all talking about the same Condition. And all could be created "now" as the time when the server first became aware of that historical information.
I have just started playing around with ELK to develop our log analytics solution.
I had a few questions regarding the best practices so that I don't make any bad choice to begin with.
This tool will analyze various types of logs to find out and correlate any issue. It will run on multiple 'devices' and each device will be uniquely identifiable with a serial number.
Question 1) Is it possible to create a dashboard where the serial number is taken as an user input?
Details: I would like to have 1 dashboard created to analyze various fields and I should be able to specify the serial number of the device as an input. From what I see, I could use filter but then this would need the visualization to be 'edited'. So it appears to be me that right now, if I need to analyze multiple devices then I need to create a dashboard for each of the device. This will be a problem that if I need to modify the dashboard then I will have to make changes to all. The problem can be minimized by importing additional dashboards as a JSON file, still it is inconvenient.
Is there a better way that I am not aware of?
Question 2) On the main dashboard, I want to show a heatmap of various 'services' and their status as a time series. For e.g. say I am monitoring, CPU, memory, network and our service then I want to see something like below:
Now the heatmap visualization doesn't provide a way to uniquely specify the condition. I generated above image by populating dummy data where values were one of 0,1,2,3. Which means that I need to create such data periodically which the visualization can then use. Is there any built-in mechanism (scheduled jobs for e.g.) provided by ELK to do such processing. One option could be to run an external problem which queries Elasticsearch, fetches all the relevant information, analyzes it and puts it back into Elasticssearch. Is that the only way?
If there are any other suggestions, please feel free to share. Thanks.
For a summer internship, I am asked to collect some specific data relative to the pages a user visits on the startup's website.
In order to simplify things, we can consider the website as a dating site, where each user has its profile page and is tagged under certain categories (hair color, city, etc).
I would like to know the best way, in the Rails framework, to keep traces of each visits a user makes to a profile or to a tag page.
Should it be logged in a file or added in a database, where exactly in the code should the functions be called ?
Maybe a gem already exists for this specific purpose?
The question is both about where functions should be called in Rails and how data should be stored because the goal is to build a recommendation system, ultimately.
There are a wide range of options available to you. I'd recommend one of the following:
Instrument detailed logging of the relevant controller actions. Periodically run a rake task that aggregates data from the log files and makes it available to your relevance engine.
Use a key/value store such as Redis to increment user/action specific counters during requests. Your relevance engine can query this store for the required metrics. Again, periodic aggregation of metrics is advised.
Both approaches lend themselves well to before_filter statements. You can interrogate the input params before the controller action executes to transparently implement the collection of statistics.
I wouldn't recommend using a relational database to store the raw data.