I'm begin to developing a scial sharing website so I'm curious about database design Schema... So in Data-Mining Star-Schema is the best one but how about a social sharing website... And as a nature of the SS websites there will be (i hope :)) many users in same time... Which better for performance for overdose using...
What do you want to do? Star Schema and Snow Flake are reporting schemas. Social sharing would not need that except mayby then for reporting?
You need something representing the social relations, that is usually done with a graph database http://en.wikipedia.org/wiki/Graph_database or in a RDBMS there are graph techniques such as this More details in the book by Celko
Star and Snowflake are not actually design methods. They are common patterns that arise as part or whole of a schema. As far as I'm aware the term "Snowflake" was invented by Ralph Kimball and is only relevant if you are using his "Dimensional" design methodology (which I certainly wouldn't recommend for a social networking site!).
The best default design for your database should generally be a Normal Form one. Aim to be in at least Boyce-Codd / 5th Normal Form unless you find compelling reasons to modify that.
Related
What is the best considered way to save and use Views, SPs and UDFs for SSRS reporting services that will be used by many users and some reports subscribed being sent out?
Do I:
Write it to a table overnight via scheduled jobs to do a direct read to the pre-saved query results?
Use a SP with temp tables with indexes based on each Views SQL to have it all in one place for the SSRS?
If the answer is that 'it depends on what I want', I would be grateful if you point me to any resources that can give me an idea of ideal setup to get query data to SSRS with minimal performance issues.
Thanking you kindly
Background/Explanation
SQL Server is not foreign to me but I don’t consider myself experienced (1 year) enough in developing 'etiquette' when it comes to crafting the parts of SQL Server. I feel I'm developing a lot of bad habits formed from using basic SQL knowledge, online searches and the odd MS SQL server course. The amount of searching done has been endless and I’m not saying there isn’t an answer out there for each part of SQL Server (UDFs, SPs and Views) out there.
The company I work for has many servers, many databases, for many outsourced front end systems being used. The issue is performance and the more I search the more I realize the setup of our databases could now maybe completely negligent and amateur. When I joined the setup used a lot of views each 'end' view had a dependency tree of over 4+ views including use of functions, each view ranging from aggregate calculations for Statistics to rearranging via pivots and unpivots. The reason given to me was so that we can pick out the parts that have gone wrong in which view. To no surprise the server has now suddenly had enough of this and peaks at 100% every time a report or view ran affecting the front end systems performance for the users.
My PP stresses my frustration and my position with the company (code monkey) in finding an answer myself which has resulted in pushing the keys back in the keyboard with opposable thumbs and appeal to the experts here.
This question is really too broad for stackoverflow. I'll try to give you a quick overview of what I think you're asking but really you're asking for way too much for a single answer here. This site is mainly focused around solving a specific problem and not the general process of development. I expect someone will probably come along and close your question.
Nightly table loads
Depending on the complexity of the task this is exactly what SSIS (SQL Server Integration Services) is for. You can build automated processes that do data transformations and data loads. It is used to build maintainable data integration solutions. Learning to use SSIS (especially properly) is a whole task though. In fact the 3rd exam for the SQL Server 2012 MCSA is exclusively about SSIS. Though if your table loads are not that complicated running them as SQL Tasks could be just as effective.
Database structure and use of views/SPs/functions/etc
This is an incredibly deep subject and it is totally dependent on what you're trying to do, how your data is structured, what kind of hardware you've got running, etc. Certainly using views, functions, and stored procedures can be good. They enable code re-use and allow you to encapsulate the logic for SSRS reports away from the actual report writers.
However, the SQL needs to be well written or it will suffer from performance problems. But, of course, that is just how it is no matter where you put the code. Even if the SQL is just a dataset in an SSRS report it will run slowly and hammer the server if it isn't written well. If the database isn't configured correctly it can have terrible performance. Indexes and other techniques for speeding up databases will always be important.
Above all everything needs to be documented so that someone else (or your later self) can make sense of it in ten months when something breaks.
Training
I would highly recommend trying to convince your employer to send you on some courses to learn SQL Server if they expect you to be developing complex database solutions. Certainly taking the courses to get your MCSA in SQL Server 2012 would be very useful. Getting the certification certainly opened my eyes to many possibilities for achieving things that I didn't know about before or just hadn't thought of.
The first exam will cover writing SQL queries and the different things that can help performance and the many cool features that you can leverage when retrieving or writing data. The second exam will cover database server administration, troubleshooting, and some performance tuning. The third exam is all about SSIS and how to warehouse your data to enable better analysis and reporting.
Even if you just read the Microsoft Learning books for these exams and never take the tests you will gain a lot of knowledge. There are other books that are good too such as T-SQL Fundamentals by Itzik Ben-Gan but ultimately it sounds like you need to get a lot deeper knowledge of the SQL Server platform before you can really make good design decisions about how to implement your solutions.
Conclusion
In the end, programming is programming. Trying to make a maintainable solution that works is your first goal. Tuning the performance of the system comes after that. The specifics of the languages and platforms don't take away from any of that. But in order to get the best performance out of a system you need knowledge about that system. An answer on here isn't going to be able to give you everything you need to know.
I wonder about that can I write native SQL to add or delete operations instead of using Query Expression or FetchXML etc. I know Query Expression is very useful but my real concern is performance and I've thought writing SQL can be faster than the others.
To put it simply, using direct SQL (especially for create/update actions) is not supported. DO NOT DO IT!
The database model for CRM is complex and updates to data can have effects that extend beyond a simple update to a single table or two.
my real concern is performance
Have you validated this concern? Take a look at this link which documents performance tests on CRM. This is an enterprise-level, scalable platform. If you have proven performance issues then perhaps your code needs optimising or your kit needs beefing up...? :)
I totally agree with Greg's answer, this is just as a side note regarding performance. If you really are seeing "performance issues", maybe you should spend your time focusing on seeing if adding an index would be helpful. Although database indexes aren't included within CRM solutions, and will require manual propagation between dev, qa, staging, and prod environments, and are only supported for on-site installations, they can make some queries, 10 or 100 times faster... (of course if they are abused, they can slow everything down as well. Know what you are doing before you use them)
On top of what #Greg & #Daryl have said, when you say performance do you mean its quicker for you to write sql?
Regardless, CRM has some unique ways of doing things.
For example activate/deactivate a record, invoice related actions or the way CRM converts an Opportunity.
It's not that hard to do. You should spend some time in the sdk...
I'm experimenting with Dancer some time, and looking for the right blocks to build my application. Frameworks tend to have flat example applications, dealing with one table at time. So I have no good idea which tools should be used to build a little bit more complex CRUD forms.
Let's say I create a Booklovers app. It should have a form to add/edit books with authors. To cover this I need 3 tables in our database: books, authors and books_to_authors. Which is best way to build a form to add a book with authors?
Note:
It is not known how many authors a book may have, we need dynamic adding of rows.
The authors table may have tens of thousands of records, so a select form element is not suitable.
An author may be missing from our database, we need to add them dynamically, too.
All these dynamic parts needs some AJAX. Is there a good solution to integrate it with form creating tools in Perl? I looked at CGI::FormBuilder and am still looking, but I did not find something that could build forms for 3 joined tables as described. The dynamic client-side part also still needs to be covered.
Are there some best practices for such a pretty simple case?
AJAX is for rendering data in a Web browser, it doesn't affect the back-end data storage.
The books and authors tables have a many-to-many relationship, using ORMs such as DBIx-Class can help.
In the Web form, there is a new book and a collection of authors. At the server side, create a book, then add authors to book.
Browse the DBIx::Class cookbook to get some ideas.
Sorry for not providing a simple answer to solve your problem.
Seems it is somehow too complex question to have straightforward solution.
I stick using templates forms and generete dynamic part with different jQuery plugins.
I don't think it is best way, but have not seen better for now.
I'm looking for either a web-based or Windows-based way to point to a relational data source using automated schema exploration (or, even better, a reflection-based approach that would work on any IQueryable in-memory data source) and allow easy exploration of data, traversing between records in related tables, etc. Basically a dynamic UI that doesn't have to look perfect. Any recommended approaches? Looking less for a rapid prototyping tool and more for a generic data explorer that can work out of the box and can work in multiple contexts against multiple data sources.
There is an application called LinqPad that I use for a similar idea mentioned above.
linqpad.net
I have a legacy VB6 app which I am rewriting in .Net. I have not used an ORM package before (being the old fashioned type who likes to know what SQL is being used), but I have seen good reports of NNibernate and I am tempted to use it for this project. I just want to check I won't be shooting myself in the foot.
Because my new app will initially run alongside the existing one, any ORM I use must work with the existing database schema. Also, I need to use SQL server text searching. From what I gather, LINQ to SQL does not support Text searching, so this will rule it out.
The app uses it's own method of allocating IDs for new objects - will NHibernate allow this or does it expect to use it's own mechanisms?
Also I have read that NHibernate does caching. I need to make sure that rows inserted outside of NHibernate are immediately accessible when accessing the database from NHibernate and vice versa.
There are 4 or 5 main tables and 10 or so subsidiary tables. although a couple of the main tables have up to a million rows, the app itself will normally be only returning a few. The user load is low so I don't anticipate performance being a problem.
At the moment I'm not sure whether it will be ASP.NET or win forms but either way I will be expecting to use data binding.
In terms of functionality, The app is not particulatly complicated - the budget to re-implement it is about 20 man days, so if I am going to use ORM it has to be something that will start paying for itself pretty quickly. Similarly I want the app to be simple to deploy and not require some monster enterprise framework.
Any thoughts on whether this is a suitable project for NHibernate would be much appreciated.
While ORMs are good, I personally wouldn't take on the risk of trying to use any ORM on a 20 day project if I had to absorb the ORM learning curve as I went.
If you have ADO.NET infrastructure you are comfortable with and you can live without ORM features, that is the much less risky approach to take.
You should learn ORMs and Linq (not necessarily Linq To Sql) eventually, but it's much more enjoyable when there is no immediate time pressure.
This sounds more like a risk management issue and that requires you to make a personal decision about how willing you are to see the project fail due to embracing new (to you) technologies.
You might also check out LLBL Gen Pro. It is a very mature ORM that handles a lot of different scenarios.
I have successfully fitted an NHibernate domain model to a few legacy database schemas - it's not yet proved impossible, but it is sometimes not without its difficulties. The easiest schemas to map are those where all primary keys and foreign keys are single column ones, but with so few tables you should be able to do the mapping relatively quickly even if this is not true of yours.
I strongly recommend, particularly given your timescale, that you use Fluent NHibernate to do your mappings - the time to learn the XML mapping file syntax may be too big an ask. However, you will need to use an XML mapping file for your full-text indexing stuff (assuming that's what you meant), writing these as named SQL queries. (See nhibernate.info documentation for details.)
Suggest you spend a day or two trying to create a model for a couple of your tables, and writing code to interact with them. There'll always be people on SO ready to answer any questions you have.
You may also want to take a look at Linq to NHibernate - we've found it helpful in terms of abstracting even more of our database access stuff away behind a simple interface. But it's Fluent NHibernate that will give you the biggest and quickest win in terms of "cheating" on the NHibernate learning curve.