I am considering using Azure Blob Storage's build-in lifecycle management feature for deleting blobs of a certain age.
However, due to a business requirement, it must be possible to generate a report or log statement after each daily execution of the defined ruleset. The report or log must state the number of blob blocks that were affected, e.g. deleted during the run.
I have read through the documentation and Googled to see if others have had similar inquiries, but so far without any luck.
So my question: Does any of you know if and how I can get a build-in Lifecycle management system to do one of the following after each daily run:
Add a log statement to the storage account containing the Blob storage.
Generate and send a report to an endpoint I define.
If the above can't be done I will have to code the daily deletion job and report generation myself, which surely I can do, but I would like to use the built-in feature if possible.
I summarize the solution as below.
If you want to know which blobs are deleted every day, we can configure Diagnostics settings in the storqge account. After doing that, we will get the logs for read, write, and delete requests for the blob. For more detail, please refer to here and here
Regarding how to enable it, we can use PowerShell command Set-AzStorageServiceLoggingProperty.
Related
I am stuck in a complicated situation and appreciate that if somebody can help.
So I was testing indexing blob storage( pdf files) and indexed a copy of my storage in qa environment that cost me some money.
My question is that:
Is there any solution to use this index in production without indexing again?
I found a solution to copy the index and that works fine but when I add an indexer that is connect to production blob storage it start indexing from scratch again( as I expected). Is there any solution to avid this? Is there any solution to ask indexer to index from now on?
I tried to use the index and the indexer that I already have by changing the subscription to prod. But I have to change the data source for indexer to point at production blob storage and in this case I get an error :
Indexer 'filesIndexer' currently references data source 'qafilesds' and cannot be updated to reference a different datasource 'prodfilesds' because it has a non-empty change tracking state, or it is currently in progress. You can use Reset API to reset the indexer's change tracking state when it is no longer in progress, and retry this call.
A simple answer to your first question is to simply use the qa index you built.
A more complicated answer is to switch from the push model you are using now to a pull model. From your explanation above I assume all of your content comes from blob storage. And you have configured an indexer to do the indexing for you. This is known as the pull model.
The alternative is to use the Azure Cognitive Search SDK to write your own application that submits content to the index instead. In this case you do not use the built-in indexer, only the index itself. Then you are free to use whatever logic you want to determine what to index and what to skip. You can even enable your storage accounts to notify your application with events when content is updated.
I've taken over support of a CRM 2016 On-Premise system. I don't know the history of the particular instance, but I suspect it's been copied and/or imported many times.
The BulkDeleteFailureBase tables has just short of 2 million rows, almost all of which contain an error description like:
Not enough privilege to access the Microsoft Dynamics CRM object or
perform the requested operation. The current Organizationid '<GUID1>'
does not match with userOrTeam's organization id '<GUID2>'.
OrganisationBase has only one record with <GUID2> in it.
Has this happened because the instance has been copied/moved around incorrectly? If so, is this likely an indication more problems are heading my way in the future?
How can I recover from this?
BulkDeleteFailureBase is one of the system async jobs logging table where platform captures the run/success/failure logs.
Probably someone might have tried to clean the data like Plugin Trace log which were copied over from different DB backup/restore or CRM Org restoration. They used Bulk delete & all that fails, ended up here.
MS Support recommendation gives the script to clean those tables safely. Leaving it only gives you performance head-ache.
Hey to every alfresco pro out there!
Is there any way to create a report (graphical or textually, i don't care) to see the following information:
download count per file
how many times did user X download a specific file
which permissions do the users have
Are my goals easy to realize? Is there any plugin out there that i can use for this? (Already searched for some but couldn't find one) Hope that you can help me :)
mtzE
There is nothing out-of-the-box that is counting downloads. Maybe the audit service can be used to count reads, but you'll have to turn it on and configure it. Once turned on, the audit service writes records to a set of audit tables in your Alfresco database. You can then use any reporting tool to query those tables.
If you want to check the permissions a user has you can use something like OpenCMIS to connect to the repository, traverse a folder path, and then, for each object, you can inspect the ACL of that object to use as data in your report.
As Lista said, one way to create such reports is to use AAAR, but that is not required.
Hi Fellow SSRS Developers,
I have a scenario that I'm trying to tend to but need to know if what I want to do is even possible.
I have 4 reports that I would like to have run and then store the actual report in a file on a server. The reason for this need is because the response time on the reports is a bit long and I've done everything in SQL to speed it up.
What I want to have happen, is when a user clicks on the report name, instead of rendering the report on their screen I simply want to call the report that is already in a file so that it will load in lightning quick time.
Has anyone ever done this with SSRS and is it even possible?
Thanks,
Other than running reports on demand, there are two specific options: Running from a Cached report and running from a Snapshot.
You can see details on all of this in Setting Report Processing Properties.
Caching
From Books Online:
To enhance performance, you can specify a report (and data) to be
cached temporarily when a user runs the report. The cached copy is
subsequently available to other users who access the same report. With
this approach, if ten users open the report, only the first request
results in report processing. The report is subsequently cached, and
the remaining nine users view the cached report.
So here you can see that it is a specific user action that causes a stored report to be created.
See Report Caching in Reporting Services.
Snapshots
From Books Online:
A report snapshot is a report that contains layout information and
data that is retrieved at a specific point in time. You can run a
report as a report snapshot to prevent the report from being run at
arbitrary times (for example, during a scheduled backup). A report
snapshot is usually created and subsequently refreshed on a schedule,
allowing you to time exactly when report and data processing will
occur. If a report is based on queries that take a long time to run,
or on queries that use data from a data source that you prefer no one
access during certain hours, you should run the report as a snapshot.
Here you can see that these are these are generally set up on a regular schedule, i.e. independent of user activity.
See Creating, Modifying, and Deleting Snapshots in Report History.
In this case it seems like Snapshots might be your best option so you have more control of when the stored report is created. The main issue with Snapshots is that they need either stored credentials or an unattended execution account so might not be possible in all cases.
I have 10 applications they have same logic to write the log on a text file located on the application root folder.
I have an application which reads the log files of all the applicaiton and shows details in a web page.
Can the same be achieved on Windows Azure? I don't want to use the 'DiagnosticMonitor' API's. As I cannot change logging logic of application.
Thanks,
Aman
Even if technically this is possible, this is not advisable as the Fabric Controller can re-create any role at a whim (well - with good reasons, but unpredictable none-the-less) and so whenever this happens you will lose any files stored locally on a role.
So - primarily you should be looking for a different place to store those logs, and there are many options, but all require that you change the logging logic of the application.
You could do this, but aside from the issue Yossi pointed out (the log would be ephemeral; it could get deleted at any time), you'd have a different log file on each role instance (VM). That means when you hit your web page to view the log, you'd see whatever happened to be on the log on that particular VM, instead of what you presumably want (a roll-up of the log files across all VMs).
Windows Azure Diagnostics could help, since you can configure it to copy log files off to blob storage (so no need to change the logging). But honestly I find Diagnostics a bit cumbersome for this. It will end up creating a lot of different blobs, and you'll have to change the log viewer to read all those blobs and combine them.
I personally would suggest writing a separate piece of code that monitors the log file and, for each new line, stores the line as an entity (row) in table storage. This bit of code could be launched as a startup task and just run continuously as a separate process (leaving everything else unchanged). Then modify the log viewer to read the last n entities from table storage and display them.
(I'm assuming you can modify the log viewer even if you can't modify the apps that log to the file.)
What about writing logs to something like azure storage table? Just need to define unique ParitionKey/RowKey, then you can easily retrieve the log for the web page.