We have an app and we want to log how the user is interacting with it. For example are they using the pages we expect them to. I dont want to log this via the app as it will be very hard for me to then get this information from the device. Each page interacts with webservices so I was planning to log that interaction.
I have had some thoughts on this
* as the webservice is being called add a logging table to the database - problem here could be performance impact
* use log4j async mode to log these details.
Does anyone have any other suggestion on how to do this? Im reading the Lean Startup at the moment (very good so far) and this sort of thing seems fundamental to it so Im wondering if there are any other tips to this.
Thanks
Since no one answered this for a couple months, I thought a couple pointers might help you...
Use mobile analytics tools
Fabric.io
Google Analytics for Mobile Apps
Flurry
Amazon Mobile Analytics
appsee
Have the server record what users access (that's the approach you're considering). To offload the overhead, there are a couple tactics you could employ (mix 'n match as you will):
Use async mechanisms (async operations in the server, such as Futures; log4j async mode; async databases; etc).
Use a separate database.
Use a NoSQL database only to write accesses. Later on you process that information in a separate analytics application.
Have the client (mobile app) record the actions and send them in bulk to the server once in a while (as frequently as you need / want / can afford).
Cheers
Related
This is for a project on Bot Framework Composer (not SDK, so i'm using built in telemetry export settings).
I am looking for the best way to store event logs from bot conversations for analysis. From what I've researched, the method recommended is going through Application Insights, which I activated and tested. The data I require seems to be all captured in table customEvents.
The issue is I need to be able manipulate the data for analysis. But in Application Insights it's read only (and possibly purge via API). I need to be able to add tables, edit text, etc. I have a lot of experice with postgreSQL so that's my first choice for bot log storage.
So my question is, what is the efficient way to get the customEvents data table that is in application insights to a postgres database? From what I see, application insights only exports to azure storage? But that does not have a database option. And if I understand some of the pipelines suggested, they copy data to storage, and then copy to a database. Isn't that a lot of storage cost, as same data will be in application insights, storageBlobs AND postgres?
What is the best pipeline? The goal is to have non-redundant pipeline that transfers event data that is in 'customEvents' to a postgres table with same columns.
(If there is a way to redirect data that goes to customEvents in application isights directly to postgres table that would be perfect too. )
There is no such to redirect data from application insights directly into postgres table.
The first solution is continuous export to azure storage as you know. Storage blob does not cost very much and you can clear the old data periodically to reduce the cost.
Another way is to use the application insights query api. To do that, you need to write your own logic to query the custom events from application insights, then insert them into your DB by your code.
I am looking at few Performance testing features Azure Web Application has. I a following a tutorial given below. I tested a small application. My question is on how I can test an application where I have to login. My application users have to login to get to the home page. How will I do it? Can somone please help me on this
https://learn.microsoft.com/en-us/vsts/test/load-test/app-service-web-app-performance-test?view=vsts
The load test you refer to is very limited. It supports only GET requests, so presumably it is meant for public pages. For more realistic scenarios you need to record (or build) test cases with navigation between multiple pages, where the login page is one and where state such as the login cookie/token is propagated between them.
Personally I work mostly with JMeter, see http://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.html on how to record a test. The recorded steps need to be edited before they can be used (hard-coded data replaced with variables). There is probably a similar tool from Microsoft, but the function in Azure portal seems overly simplistic.
In summary use another tool for more complex load testing, as the specified one handles only GET requests and can't do the job.
I have an Azure App Service running an MVC Web API. It connects to a DB and Redis cache. The calls to the API are taking a massive amount of time or timing out. I have stripped back the methods to be doing pretty much nothing.
public HttpResponseMessage GetData()
{
Request.CreateResponse(HttpStatusCode.OK, "abc");
}
I still have the same issue. The webserver isn't under any pressure, nor is the DB, both below 30%. I'm at a loss to know where to even start.
This method is called quite a lot so there may be a lot of concurrent requests. I'm running an S3 App Service plan which should more than suffice.
Any suggestions on how to trouble shoot greatly welcome? I can only think it is down to the number of simultaneous requests.
As far as I know, this issue is often caused by:
requests taking a long time
application using high memory/CPU
application crashing due to an exception.
I suggest you could firstly enable diagnostics logging for your web app.
By using this way, you could find the details information about Detailed Error, Failed Request Tracing,Web Server Logging in your web application.
More details, you could refer to bellow link:
Enable diagnostics logging for web apps in Azure App Service
You could also use the Azure App Service Support Portal.
By using this way, you could troubleshoot issues related to your web app by looking at HTTP logs, event logs, process dumps, and more.
You can access all this information using our Support portal at http://.scm.azurewebsites.net/Support
More details, you could refer to below link:
New Updates to Support Site Extension for Azure Websites
Besides, you could also refer to this article:Troubleshoot slow web app performance issues in Azure App Service
So say I've got an MVC app hosted in the cloud somewhere, meaning I don't have access to IIS or any infrastructure.
All I have control over is the App code itself, and what comes down to the client.
My goal
Is to collect data over time of how well the MVC app is performing in terms of response times.
Current Problems
I can get a lot of data from Google Analyics, and other client-side tricks, but that won't tell if say, the App Pool is recycling too often.
Similarly if I put stop watches in the actions, that won't tell me about any delays in the App Startup (if it has to start up again).
Also, if I do put a stop watch in the Action, it doesn't take into account any delays in redering the View. For example, even though it's bad practice, there might be a DB call being made from the View, and my action metrics won't take that into account.
My Question
So, if I want to get true metrics of how long requests are taking overal from mulitple clients and users, where are the best places to but Stopwatches in the App. Or is it impossible to get true metrics from the app itself, and I have to place counters outside of the App (like in IIS).
Add New Relic, it's available for free as part of the AppHarbor service - https://appharbor.com/addons/newrelic
Since you mention "in the cloud somewhere" are you using Microsoft Azure for hosting? If so, there's some great diagnostics you can log to your Azure storage with DiagnosticsMonitorConfiguration.
Here's a tutorial on how to add diagnostics to your web and worker roles. You can find a full list of performance counters on MSDN
You can get everything from application requests/second, memory and CPU utilization, network adapter statistics, output cache hits/misses, request execution time, etc.
I am looking for some recommendations on a good data store for activity feeds. The goal is to have a Twitter/Facebook type feed log consisting of various activities users can do throughout our website. The "wall" or "feed" would updated via AJAX showing what the users of the website are currently doing. It will be written to often and then the most recent will be displayed on the site.
(e.g. John Smith recommended Jane Smith's article 2 seconds ago)
We currently are storing the feeds in MySQL but performance has been poor and I'm concerned with hindering performance throughout the rest of the website if we are constantly hitting the database to grab the most recent user activity as well as writing the feeds.
Any recommendations would be greatly appreciated!
Make use of the best caching solutions like memcache to increase performance. Other than scaling, there are no performance-increasing possibilities for an activity feed.
I would vote for using http://redis.io/ or http://www.mongodb.org/ as an alternative to MySQL for short-term, almost live activity feeds across a site. And a cron job to dump history of activities into MySQL for record keeping.
A look at tumblr's or twitters architectures can push you to the right direction as well.
You should take the microservices approach to separate between the datastore that stores the users' actions to the one that store the actual data.
Pub/Sub is the right approach to handle the big stream of users' actions.
Use Kafka or Google Pub/Sub cloud service for a scalable data pipeline. They can take the load with its scalable architecture.
Independently consume the messages from Kafka to some database such as MySQL or Google BigQuery for analytics purposes you must have.