I want to be able to store an arbitrary C# struct with some variables on the Azure SQL server and retrieve it later into a similar struct. How can I do this without knowing the structure of the database?
SQL Azure is very similar to SQL Server, as you would build your schema, tables, rows, etc. the same way. If you wanted a schemaless approach to data types, you'd need to serialize your objects to some generic column, along with a Type column. Or use a Property table approach.
Alternatively, Windows Azure has a schema-free storage construct, the Windows Azure Table. Each row may contain different data. You'd just need some mechanism for determining the type of data you wrote (maybe one of the row properties, perhaps). Azure Tables are lightweight compared to SQL Azure, in that it's not a relational database. Each row is referenced by a Partition Key and Row Key (the pair being essentially a composite key).
So... assuming you don't have complex search / index requirements, you should be able to use Azure Tables to accomplish what you're trying to do.
This blog post goes over the basics of both SQL Azure and Azure Tables.
There are also examples of using Azure Tables in the Platform Training Kit.
Related
I am a business analyst. I use Tableau a lot but have limited knowledge about the back-end of Salesforce. The majority of our company's data is stored in Salesforce and our data team does not support business users for understanding such topics.
In many of my projects, I use the Salesforce connector inside Tableau to extract Salesforce tables, but it requires knowledge about joins relationships among tables. Most of the time, I can guess correctly about the primary key among tables, but I still want to learn systematically about the data structure and have my data independence.
So, how do I learn the data structure by myself? Or how do I ask specific structure questions to data team so I don't trouble them as much?
Do you have Salesforce account with "Customize Application" permission? If you don't have in production - maybe they'll be willing to promote you to sysadmin in one of sandboxes.
If you do - Setup -> Schema Builder might be easiest tool to visualise relations. It's bit old, flash-based but pretty neat way to model relationships. https://trailhead.salesforce.com/en/content/learn/modules/data_modeling/schema_builder
Another one might be workbench, http://workbench.developerforce.com/ It's not as neat but lets you experiment with metadata & queries, learn which object has what child relationships...
For standard objects if you have a primary key / foreign key you can use some lookup tables to learn more about target table. All Account Ids in all SF instances start with 001. Contacts with 003, Users with 005... Combine some blogs like http://www.fishofprey.com/2011/09/obscure-salesforce-object-key-prefixes.html with https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_objects_account.htm and it's a good start. Won't help much with custom objects and fields (specific to your company) but well.
It's bit "meta" but you can query info about tables and columns too. After all - you might be more comfortable in Tableau ;) Querying Salesforce Object Column Names w/SOQL might give you some hints.
If your job is to build advanced reports off these data sources, I would imagine you need to understand the data structure to some extent. This would mean you need to have authorization to view and access the database table list to get familiar with it and possibly run raw queries to verify data integrity.
If they are not comfortable with you touching the production system, ask for access to a development system which is a copy of production or even just realistic test data.
I happen to find myself in a situation where i am using Oracle SQL Developer Version 1.5.5 and there's this huge database for which the documentation is very poor. I'd like to create a star or a snowflake schema for better understanding of the data. Is there a simple way to do it?
You can reverse engineer the physical data model using SQL Developer Data Modeler. This is actually a separate tool from SQL Developer but shares some branding. It is also free.
The quality of the resultant diagram will depend heavily on how well the physical data structures have been implemented. You will only get relationships if the database has defined foreign key constraints (disabled is good enough). Likewise UIDs require defined primary key constraints. If your database lacks constraints you'll have to rely on column naming conventions, data analysis and your business knowledge.
Star or Snowflake schemas are for data warehouses. Is that the sort of database you're dealing with?
I want to store some user data in memory, like some in-memory noSQL database.
But later on I want to query that data with a dynamic query constructed from the user. That query is stored in a classic DB like a string, so when I need to query the data stored in memory I would like to parse that string and construct the desired query (by some known rules).
I looked at Redis and I figured out it isn't maintained for Windows anymore, I have also looked at RavenDB but it's main query language is LINQ, even though it can be created dynamic Lucene Query.
Can you suggest me another in memory DB that work with ASP.NET and can be queried with a dynamically created query? Maybe I haven't seen all the options.
I prefer name-value or JSON based noSQL so it's schema can be easyly modified without the constraints of the relation type of DBs
I would suggest to simply use sqlite. It can be easily used as an in-memory database (just open the database using ":memory:" instead of a file name).
You can use a simple 2 columns table with a primary key to emulate a key/value store.
Here are a few links you might find helpful:
http://www.sqlite.org/inmemorydb.html
How to create asp.net web application using sqlite
Is there a data structure which gives you functions of a database (like insert, update, delete etc)? For example:
create a struct like the database table
store data on it and query on it
selectively delete it
I know that with a hashtable you can do this (ex: uthash library). But as far as I know updating one column element only is not easy in a hash table.
Look at sqlite. Rather than a relational database system, it is essentially a connectionless, file-based database library that supports SQL. You link your program against it and it provides functions to perform SQL queries over data files.
Look At NoSQL itis The RMDBS used By FaceBook
Use C structs to represent rows of data and then trees (or maybe hashes) for indexes. There are a lot of little problems you will need to solve, specially in order to make all the operations efficient, but this forms the basis for an in-memory table.
For simple things, a tree structure may be enough.
I am aware of the generation of the Performance Counters and Diagnosis in webrole and worker-role in Azure.
My question is can I get the Performance Counter on a remote place or remote app, given the subscription ID and other certificates (3rd Party app to give performance Counter).
Question in other words, Can I get the Performance Counter Data, the way I use Service Management API for any hosted service...?
What are the pre-configurations required to be done in Server...? to get CPU data...???
Following is the description of the attributes for Performance counters table:
EventTickCount: Stores the tick count (in UTC) when the log entry was recorded.
DeploymentId: Id of your deployment.
Role: Role name
RoleInstance: Role instance name
CounterName: Name of the counter
CounterValue: Value of the performance counter
One of the key thing here is to understand how to effectively query this table (and other diagnostics table). One of the things we would want from the diagnostics table is to fetch the data for a certain period of time. Our natural instinct would be to query this table on Timestamp attribute. However that's a BAD DESIGN choice because you know in an Azure table the data is indexed on PartitionKey and RowKey. Querying on any other attribute will result in full table scan which will create a problem when your table contains a lot of data.
The good thing about these logs table is that PartitionKey value in a way represents the date/time when the data point was collected. Basically PartitionKey is created by using higher order bits of DateTime.Ticks (in UTC). So if you were to fetch the data for a certain date/time range, first you would need to calculate the Ticks for your range (in UTC) and then prepend a "0" in front of it and use those values in your query.
If you're querying using REST API, you would use syntax like:
PartitionKey ge '0<from date/time ticks in UTC>' and PartitionKey le '0<to date/time in UTC>'.
You could use this syntax if you're querying table storage in our tool Cloud Storage Studio, Visual Studio or Azure Storage Explorer.
Unfortunately I don't have much experience with the Storage Client library but let me work something out. May be I will write a blog post about it. Once I do that, I will post the link to my blog post here.
Gaurav
Since the performance counters data gets persisted in Windows Azure Table Storage (WADPerformanceCountersTable), you can query that table through a remote app (either by using Microsoft's Storage Client library or writing your own custom wrapper around Azure Table Service REST API to retrieve the data. All you will need is the storage account name and key.