I've read the following post regarding the iText 7 performance issue:
How to improve iText performance when creating tables
Since I can't add comments on that same thread, I've to create a new thread. My question is: is there any conclusion on how to further improve the performance with the .NET library? I'm in the same situation and with similar performance. My application will need to produce PDF report with even more complex table structures.
Thanks for any helps!
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I was thinking on some social media applications like facebook or linkedin. I read lots of articles on websites like http://highscalability.com/ and didn't find the correct answer.
Because, the biggest apps of now, use their custom systems. They use custom file systems or customized db-engines or customized web servers. They don't use the original iis, apache, mssql, mysql, windows or linux. They use lots of programming language for different problems. It's OK for them because of their load. They have to calculate every bit or something. They started on some small enviroments and they encountered problems and saw bottlenecks. So they founded new solutions.
Now, we can find some articles about their current systems. But we have no answer about what is the best start.
I need to learn the answer of "What kind of architecture is a correct start?"
I have some ideas on it but we need to be sure about it.
We think,
Use mysql for relational database. And a caching mechanism like memcached over mysql. And a rest api for business layer. We think using python for codding of rest api. And all systems run on a suitable linux distro. After all of these enviroments is ok, we can use any language or system for UIs. It can be a PHP site for web or a native application for IOS or Android.
We need your advice. Thank you so much.
(I am a good reader but it's my first question. I hope there's no problem.)
Following a similar question last year I compiled the techniques and technologies used by some of the larger social networking sites.
The following architecture concepts are prevalent among such sites:
Scalability
Caching (heavily, across multiple tiers and layers)
Data Sharding (preferrably by data-locality criteria)
In-Memory DBs for often referenced data
Efficient wire-level protocols (as opposed to what an enterprise typically considers state of the art)
asynchronous processing
Flexibility
service oriented architecture as a baseline principle
decoupled and layered components
asynchronous processing
Reliability
asynchronous processing
replication
cell architecture (independently operated subsets, e.g. by geographical criteria)
NB: If you start a new site, you are unlikely to have the kind of scaling or reliability requirements that these extremely large sites face. Hence the best advice is to start small but keep it flexible. One approach is to use an application framework that starts out simple but has flexibility to scale later, e.g. Django.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I have some experience in working by Structuremap. And now I'm creating a big and large scale web app (really a social networking site) using ASP.NET MVC 3, Entity Framework 4.1 code-first, SqlServer 2008, via C#.NET 4. Which one of Structuremap, AutoFac, and Ninject would you suggest me to use in this case? Performance and easy-of-use is important. Thanks to any help, idea and suggestion.
This article has a good summary of most popular IoC frameworks and discusses their performance as well as ease of use. All in all a great read!
http://philipm.at/2011/di_speed.html
& a follow-up at http://philipm.at/2011/di_speed_redux.html
I think there is no differences between them! Its associated on your-self. Every body has some favorites, and you too. I use StructureMap and its good enough for large and/or small scale applications.
UPDATE:
Well, a few months ago I met with Simple Injector which is really really simple, and actually fast. I've used it in several projects with different scales, and it really works perfectly.
If you are interested in HOW they are used in large scale web apps, one good one to look at is OrchardCMS; it uses Autofac.
I've only really looked at / used Ninject and simply haven't bothered looking at any others. The simplicity in usage & it's low learning curve fit my last couple of projects very well. IMO, it is a great IoC container for projects large and small.
I've used Unity (some time ago), AutoFac and StructureMap and prefer StructureMap, specifically for its AssertConfigurationIsValid method which I use in unit tests to, well, assert that my StructureMap configuration is valid (relevant if you're using constructor-based DI).
As far as I'm aware only StructureMap offers this facility.
When using ASP.NET MVC, I also like how the StructureMap NuGet package wires things up for you nicely.
I have a BIRT report with performance problems: it takes approximately 5 minutes to run.
At the beginning I though the problem was the database: this report uses a quite complex SQL Server stored procedure to retrieve data. After a lot of SQL optimizations this procedure now takes ~20 seconds to run (in the management console).
However, the report itself still takes too much time (several minutes). How do I identify the other bottlenecks in BIRT report generation? Is there a way to profile the entire process? I'm running it using the www viewer (running inside Tomcat 5.5), and I don't have any Java event handlers, everything is done using standard SQL and JavaScript.
I watched the webinar "Designing High Performance BIRT Reports" 1, it has some interesting considerations but it didn't help much...
As I write this answer the question is getting close to 2 years old, so presumably you found a way around the problem. No one has offered a profiler for the entire process, so here are some ways of identifying bottle necks.
Start up time - About a minute can be spent here
running a couple reports one after the other or starting a second after the first is running can help diagnosis issues.
SQL Query run time - Good solutions are mentioned in the question
any SQL trace and performance testing will identify issues.
Building the report - This is where I notice the lions share of time being taken. Run a SQL trace while the report is being created. Even a relatively simple tables with lots of data can take around a minute to configure and display (HTML via apache tomcat) after the SQL trace indicates the query is done.
simplify the report or create a clone with fewer graphs or tables run with and without pieces to see if any create a notable difference
modify the query to bring back less records, less records are easier to display,
Delivery method PDF, Excel, HTML each can have different issues
try the report to different formats
if one is significantly greater, try different emitters.
For anyone else having problems with BIRT performance, here are a few more hints.
Profiling a BIRT report can be done using any Java profiler - write a simple Java test that runs your report and then profile that.
As an example I use the unit tests from the SpudSoft BIRT Excel Emitters and run JProfiler from within Eclipse.
The problem isn't with the difficulty in profiling it, it's in understanding the data produced :)
Scripts associated with DataSources can absolutely kill performance. Even a script that looks as though it should only have an impact up-front can really stop this thing. This is the biggest performance killer I've found (so big I rewrote quite a chunk of the Excel Emitters to make it unnecessary).
The emitter you use has an impact.
If you are trying to narrow down performance problems always do separate Run and Render tasks so you can easily see where to concentrate your efforts.
Different emitter options can impact performance, particularly with the third party emitters (the SpudSoft emitters now have a few options for making large reports faster).
The difference between Fixed-Layout and Auto-Layout is significant, try both.
Have you checked how much memory you are using in Tomcat? You may not be assigning enough memory. A quick test is to launch the BIRT Designer and assign it additional memory. Then, within the BIRT Designer software, run the report.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
We are starting off on a new from scratch implementation of an eCommerce solution and have decided on the framework to use too. There will be people joining the project who never really had any experience working on this framework. How should we go about doing the knowledge management/transfer? What could be the other challenges we need to be prepared for and how?
I can think of starting a WIKI with the most often needed content and addressing the most common roadblocks...Is that a good idea?
A glossary will go a long way.
Make sure that you and your team use the glossary when talking. There's no point having specific words for things if you don't use them correctly.
Stand up meetings each day should help with whatever roadblocks you have.
Make sure that your team has as high bandwidth communication as possible but also allow for quiet time where people can focus. Maybe have the first 15min. of each hour as a time when people can walk up to each other and ask questions etc. Then the other 45min is silent time. Reassess this to get a balance that everyone fits with.
I second the wiki recommendation - we have had a great deal of success using it to form a knowledge base and glossary for our projects.
Another technique that has worked well for us is creating a scratch repository (e.g. SVN, Git, etc.) for the purpose of technology ramp up spikes. We're currently working on an enterprise-scale project leveraging Spring's OSGi support, and we created several spike projects to explore different facets of the technology. This helped us grasp the technology before getting too encumbered by the business needs.
As far as challenges for which to be prepared? Expect the unexpected. Any time you embark on using a new framework/technology, you will run into roadblocks and your initial velocity will suffer. My best advice - simple determination. Don't give up on your framework at the first sign of stormy weather. Work through the issues. Eventually you'll clear most of the hurdles and your velocity will increase exponentially as the entire team gets more comfortable with the technology.
I think this is the ideal situation for a wiki. Let the developers choose which wiki to use, because they're the ones that will be using it!
There is this thread:
http://discuss.joelonsoftware.com/default.asp?biz.5.738060.3
You might consider Alfresco.com's open source solution for content management.
I think the Wiki is a good idea, but there is also no substitute for real code. To that end a good quality (reference) implementation of a single function, which shows all the layers in the code stack, ie from browser/form down to the DB and back again.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
What advice would you have for someone who wants to start in the BI (Business Intelligence) domain?
I where and what I should start with: Books, Blogs, WebCasts...
What I should pay attention to and what I should stay away from.
Are the Microsoft technologies worth while ?
The MS technology stack is quite good and is by far the most accessible (try to get hold of a copy of Cognos Reportnet for self-learning). Where you will run into trouble (and this is the main barrier to entry for gaining a B.I. skillset) is to actually get experience working with real data. It's quite hard to come up with a realistic toy scenario for this sort of thing.
This means that you have to overcome the chicken-and-egg problem that this poses. One option would be to try to get a job as a B.I. developer somewhere like a government department or other place that has trouble recruiting due to salary constraints. Clear evidence of technical skills and a demonstrated interest in the business might get your foot in the door.
This will be a bit harder in a recession. However there is still an ongoing skill shortage of good B.I. people. The reason is (IMO) not the lack of technical skills (the technology isn't rocket science). Instead, I think it is the aforementioned chicken-and-egg problem and the fact that the B.I. domain involves customer intimacy to do it well. It lends itself to working in an analyst/programmer mode with direct customer contact (one of the reasons I do this type of work). If you like working in this mode it might be a good line for you to get into.
Edit: Someone who's just had a job offer in this space asked whether he should take the job.
I found the "project real" from microsoft really helpful while getting into the bi-world. Its a real world bi project, supported by microsoft, to develop and show best practices regarding to all the areas of bi like etl, data warehouse design, cube design, etc.
Business Objects http://www.businessobjects.com/ are quite a big player in this area and familiarity with their products will also help you break into B.I. roles.
For practise data, I would recommend something like the anonomised search records from aol that came out a couple of years back - http://www.techcrunch.com/2006/08/06/aol-proudly-releases-massive-amounts-of-user-search-data/ This is real world size and is an interesting database with some published search sets.
I would stress you to read this book; might seem kind of outdated but the same theory still applies today. It is probably the best starter for general BI.
The Data Warehouse Toolkit - Ralph Kimball
Regarding Microsoft's BI it is a medium-sized tool that can do the job in your first steps (I have more experience with Cognos though). Haven't used MS tools since 2005 so I can't tell much about it.
In case you happen to be interested in Cognos, I have a few videos which can be of help: Cognos Tutorials
Good luck with your project.
Get the Kimball Books (specially this one http://www.amazon.com/The-Data-Warehouse-Toolkit-Dimensional/dp/0471200247) and for starters you may want to start with the MS BI Framework The Microsoft Data Warehouse Toolkit and the SQL Server Enterprise (MS BI Bundle with the database, ETL and reporting), it's easy a readily available, specially if you are a student with the MSDNAA, you can get the enterprise version for free!!!
For general business intelligence, I found the Kimball Group as a great source for best practices.
If you would like to start building your own project, check out GoodData Platform. We have full BI stack Platform as a Service and you can start for free (evaluation period) with access to all resources and learn from Tutorials on our Developer Portal.
I would say try to find a few classes. Microsoft technologies are worth the time. There are many large companies running on the .Net framework.
We use this to get a feel of Microstrategy: http://www.teradatauniversitynetwork.com/apply-and-do