Related
Non-optimization question about CPLEX Studio....
So you make your awesome OPL model in CPLEX Studio and it brilliantly solves your amazeballs problem.
Suppose you wanted to allow other users to access this model in a nice user friendly way: Basically, specify some simple parameters in a simple user interface (without having to edit code etc), then, output the solution in some arbitrary way you coded up like an Excel file, HTML report, or whatever.
1) What are the options for a user interface, without adding in too much other technology?
(eg. I currently have a Java program doing exactly this, but I'd rather not rely on Java code / programmers / compiling / hosting source code etc)
2) What are the options for triggering some user friendly output, eg. in a standard format like Excel, some HTML report you coded up, or maybe just triggering a Python script, etc?
(eg. I currently render them in a Java FX application on grids, charts and HTML windows, I would prefer something more lightweight and accessible, like Python etc, HTML5 output)
3) In industry, what is the typical role of CPLEX in a production environment: Is it just called by an external application (Java/.NET etc), or is CPLEX Studio used more actively?
Embed the optimisation model in wider business applications using Java, C#, Python, C++, whatever. Make it just part of the normal business systems that people use. It is just software. Make it so that the users really appreciate that the new software actually benefits them each time they use it. Make it easier to use the model than to not use it. Hide the model inside other software. Probably never even mention optimisation to your end users.
The best model in the world that could deliver amazing benefits will actually achieve nothing of practical value if it doesn't actually get used.
If your target audience or users have to do extra stuff or perform extra steps to use your model, then it will likely not get used very much and may wither and die. If they have to learn new applications etc to use it, it probably won't get used by most people.
By making your model part of their normal day-to-day processes, it will get used, and the practical benefits will come.
I have implemented and support a number of live optimisation applications in several large companies, making decisions that directly affect billions of pounds/dollars of products/revenues per year. Almost all of them have the real optimisation models totally hidden from the users, most of whom have no idea of optimisation or CPLEX; the software in their business systems just works.
There are many options. You may write the model with an algebraic modeling language (AML) like OPL or a general purpose language. (GPL)
If you use OPL then you may call your model from many GPL like C++, Java, Python ...
Or you could plug that model in an existing application.
You could call OPL from Excel or DSX Python Notebook as can be read at https://www.ibm.com/developerworks/community/forums/html/topic?id=306f3ded-33b8-4d9a-8568-b4288aa64265&ps=25
See the survey I mentioned in 1.
Some users use CPLEX OPL IDE in order to make decisions and simulations
Other use Decision Optimization Centre : https://www.ibm.com/us-en/marketplace/ibm-decision-optimization-center
Finally, some write new applications from zero or plug the model into an existing application.
What 3 things would you tell developers new to XPages to do to help maximize the performance of their XPages apps?
Tim Tripcony had given a bunch of suggestion here.
http://www-10.lotus.com/ldd/xpagesforum.nsf/topicThread.xsp?action=openDocument&documentId=365493C31B0352E3852578FD001435D2#AEFBCF8B111E149B852578FD001E617B
Not sure if this tipp is for beginners, but use any of the LifeCyclePhaseListeners from the OpenNTF Snippets to see what is going on in your datasources during a complete or partial refresh (http://openntf.org/XSnippets.nsf/snippet.xsp?id=a-simple-lifecyclelistener-)
Use the extension Library. Report Bugs ( or what you consider a bug ) at OpenNTF.
Use the SampleDb from the extLib. ou can easily modify the samples to your own need. Even good for testing if the issue you encounter is reproducable in this DB.
Use Firebug ( or a similar tool that comes with the browser of your choice ) If you see an error in the error tab, go and fix it.
Since you're asking for only 3, here are the tips I feel make the biggest difference:
Determine what your users / customers mean by "performance", and set the page persistence option accordingly. If they mean scalability (max concurrent users), keep pages on disk. If they mean speed, keep pages in memory. If they want an ideal mixture of speed and scalability, keep the current page in memory. This latter option really should be the server default (set in the server's xsp.properties file), overridden only as needed per application.
Set value bindings to compute on page load (denoted by a $ in the source XML) wherever possible instead of compute dynamically (denoted by a #). $ bindings are only evaluated once, # bindings are recalculated over and over again, so changing computations that only need to be loaded once per page to $ bindings speed up both initial page load and any events fired against the page once loaded.
Minimize the use of SSJS. Wherever possible, use standard EL instead (e.g. ${database.title} instead of ${javascript:return database.getTitle();}). Every SSJS expression must be parsed into an abstract syntax tree to be evaluated, which is incrementally slower than the standard EL resolver.
There are many other ways to maximize performance, of course, but in my opinion these are the easiest ways to gain noticeable improvement.
1. Use the Script Library instead writing a bulk of code into the Xpage.
2. Use the Theme or separate CSS class for each elements [Relational]
3. Moreover try to control your SSJS code. Because server side request only reduce our system performance.
4. Final point consider this as sub point of 3, Try to get the direct functions from our SSJS, Don't use the while llop and for loop for like document collection, count and other things.
The basics like
Use the immediate flags ( or one of the other flags) on serverside events if possible
Check the Flag which (forgot its name..) generates the css and js as
one big file at runtime therefore minimizing the ammount of
requests.
Choose your scope wisely. Dont put everything in your sessionscope but define when, where and how your are using the data and based on that use the correct scope. This can lead to better memory usage..
And of course the most important one read the mastering xpages book.
Other tips I would add:
When retrieving data use viewentrycollections or the viewnavigstor
Upgrade to 8.5.3
Use default html tags if possible. If you dont need the functionality of a xp:div or xp:panel use a <div> instead so you dont generate an extra uicomponent on the tree.
Define what page persistance mode you need
Depends a lot what you mean by performance. For performance of the app:
Use compute on page load wherever feasible. It significantly improves performance.
In larger XPages particularly, combine code into single controls where possible. E.g. Use a single Computed Field control combining literal strings, EL and SSJS rather than one control for each language. On that point, EL performs better than SSJS, and SSJS on the XPage performs better than SSJS in a Script Library.
Use dataContexts for properties that are calculated more than once on an XPage.
Partial Execution mode is a very strong recommendation, but probably beyond new XPages developers at this point. Java will also perform better than SSJS in a Script Library, but again beyond new developers. XPages controls you've created with the Extensibility Framework should perform better, because they should run fewer lines of Java than multiple controls, but I haven't tested that.
If you mean performance of the developer:
Get the Extension Library.
Use themes to set default properties, e.g. A standard style for all your pagers.
Use Firebug. If you're developing for Notes Client or IE, still use Firebug. You'll spend longer suffering through Client/IE thank you will fixing the few quirks that will remain.
Does anyone know of a solution or strategy for parsing and translating recurring events in any given language?
I've got some code that sort of works, but I'm having trouble even wrapping my brain around a solid approach.
By "parse" you mean free-form text in any language?
That is nearly impossible to do. The problem is, you can describe that in many ways (even in English, right?). Put inflection in the picture and...
You would need some kind of cognitive system, that is you would need to employ machine learning. Theoretically speaking you'll be interested in only few ontologies but getting translated versions of them might not be too easy (I have seen a presentation of such web service but I believe it was more academic work and the service is not publicly available).
Taking these facts into account, I can suggest (unless you really must parse free form text, i.e. sent via email or something) that you create a form where people can actually choose recurring frequency and options (i.e. ignore holidays, etc.). It is still uneasy task but...
There are various libraries which can parse RRULE's in several languages.
RRULE Libraries
C - libical
Python - python-dateutil
Ruby - ice_cube
Javascript - google-caja or rrule
Java - iCal4j
PHP - when
As for strategies, take a look at the standard here which provides architectural pseudo-code for abstracting rrule data.
From their website http://www.playframework.org/documentation/1.0/faq
"
The biggest CPU consumer in the Play stack at this time is the template engine based on Groovy. But as Play applications are easily scalable, it is not really a problem if you need to serve a very high traffic: you can balance the load between several servers. And we hope for a performance gain at this level with the new JDK7 and its better support of dynamic languages.
"
So there are no better choices? What about JSP?
JSP is not feasible as every JSP compiles to a Servlet and the servlet API provides things like the server side session which are not compatible with the RESTful paradigm. We don't want to go back to the dark ages of badly scalable server side sessions, back buttoning problems in the browser, reposts etc.
Japid templates are interesting, but they are not backed by a great community and perhaps didn't even exist at the time play was created (I don't know for sure though). I tried Japid as a replacement for the Groovy templates in my own application and found out in a JMeter test that the benefit would be only marginal, 10% to max. 25% improvement.
I guess in the end it all depends on your scalability requirements and the structure of your pages. I picked the 90% use case of my application and did the test. To me, the little improvement did not justify for the additional costs of the extra dependency (I like to keep dependencies to a minimum for maintainability).
Groovy templates are not bad or slow in general. Use typed variables wherever possible (instead of "def"), even in closures! Keep values of accessed properties in local variables. Do reasonable results paging. Then keep your fingers crossed that GSP might be able to run on groovy++ in the future and you're done ;)
To me, the question would not be why they used groovy in the views. That is, because I rather do miss it so much in the controller layer. Groovy would make coding the controller behaviour a lot easier IMHO.
First off, JSP was not a valid option for Play since it chose not to go down the Java EE route (which JSP is part of). Instead, Play chose to use Groovy as an intuitive, simple but powerful templating engine.
However, one of Play's greatest features is that it is a pluggable system, meaning that many parts of the core system can simply be replaced. This includes the template engine, and there are a couple that are already available.
The most popular is Japid. It claims to be 2-20x faster than the standard templating engine, and has been around for a while. For more info, see here http://www.playframework.org/modules/japid.
A second option is Cambridge, although this has only been out for a little while, but is reasonably active in the message boards (see https://groups.google.com/forum/?hl=en#!searchin/play-framework/cambridge/play-framework/IxSei-9BGq8/X-3pF5tWAKAJ).
I tend to stick to Groovy, as I like the way it works, and have not found it to be too bad in terms of performance, but every application is individual, so your own performance tests should lead you down your own particular path.
Yes there is Japid. Which is much, much faster.
http://www.playframework.org/modules/japid
I totally agree with the choice of ease over speed the Play Framework designers made here. My guess is that if the templating starts getting in the way in terms of performance, you can (and should!) measure the slow bits, and refactor them into fast tags where possible. With that, you're likely to save 80% of CPU by moving 20% into fast tags, leaving you with flexibility and adequate speed.
Having said that, I'm looking forward to an experiment I'm planning to see how well the new Scala templates (loosely "borrowed" from Razor.NET - awesome clean syntax) work with Java controllers/models. The Scala backend isn't there yet in terms of comparative ease, but the templating language certainly rocks.
I may be late to the party in 2016. Play 2 is out, and the JDK (not to mention the hardware) drastically improved. I am not using Play or Spring Boot, since my platform doesn't need them - just pure runtime text/HTML generation from templates.
First, when talking about Groovy templates, there is more than one. I use the original Groovy SimpleTemplateEngine for anything from emails to rich Web pages, whether most people nowadays favor the "advanced" MarkupTemplateEngine with its non-HTML builder syntax. I did not go that route because of the IDE (e.g. UntelliJ) support for JSPish HTML files with JavaScript - catching unclosed tags, braces, etc. Besides, how would you include JavaScript into the curly brace based "builder" style template?
Whichever you chose, both SimpleTemplateEngine and MarkupTemplateEngine statically compile their templates, even though the Groovy doc only mentions it for the latter. Why wouldn't it generates a class for the former? I didn't compare them against each other, but I'd expect SimpleTemplateEngine to be faster, since it is... well, simpler - doesn't translate any syntax into String concatenations with ifs and loops in between.
And it is indeed very fast. I was concerned about invoking it in a loop. Doesn't make any difference. There is no overhead, as the template is compiled once.
I use multiple small templates responsible for generating individual form control markup (HTML + JS) to generate a composite form, included in a higher-level container, included in another container, and so on, until the entire page is formed. Decomposing your view like that makes it, as you already guessed, modular, encapsulated, and "object-oriented" - composed of many individual MVC components building upon each other. Sort of like good old custom JSP tags, only evaluated at runtime and compatible with technologies like Spring Boot, if you cannot resist trendy resume-boosting stuff like that.
A test form with a 100 fields (all complex "smart" controls with encapsulated state management and behavior) renders in 150ms the first time, and then 10-14ms thereafter. In an IDE debugger on my memory-starved 4y.o. notebook. I also verified it is thread-safe, since Groovy never mentioned it explicitly. Why wouldn't it be, if it is compiled into a stateless Groovy class like any other? Call createTemplate() once, store the Template somewhere, and use it (call Template.make()) in your servlet or another concurrent context. Obviously I'll never have a 100-field form in a real application. Anyone who does, needs to reconsider his/her UX.
The performance is quite adequate. I'd even accept one second to render a 100-field page. Think of it, you don't need ultimate nanotrading or nuclear missile tracking performance in a Web app. If you do, pick Jamon: http://www.jamon.org/Overview.html, which generates a Java class, you'd normally write to concatenate Strings. I didn't test it, as I don't like extra compilation steps (automatically executed by Maven, but still). Compiled Groovy bytecode is good enough for me - compared to the compiled, yes, strongly-typed Java. The difference would be marginal unless you are doing something complex, which you shouldn't inside a template (see below). Playing with typed Groovy variables vs. def as suggested in this thread, only saved me a couple of milliseconds on that 100-template run.
Templates should not have much procedural logic (internal variables, ifs and loops) anyway - that's the controller's, not view's responsibility. That said, ifs and loops are a must for a template engine. Why would one use Handlebars/Mustache, if he/she can simply call String.replace()?
The rest of the template engines is also irrelevant. No String concatenation (e.g. Velocity, or Freemarker) or interpreted JS-based technology (e.g. Jade) would ever beat the most direct Jamon's approach performance-wise. And being a Java programmer, you want to use your favorite language/syntax: either directly (Jamon) or 90% close to Java, Groovy is (being a scripting-centric concise interpreted Java). I wouldn't comment on Scala - the matter of preference. Other than its allegedly "concise" syntax (less and less relevant with Java 8+) comes at a price. And only matters for complex loops. You do not want to write your entire app inside one template, like I already said. A couple of loops and up to ten if statements max.
And, like everyone mentioned, the intuitive syntax, and ease of use is the key. They drastically reduce the number of bugs. A good (additional) server costs a $1000, while developer salaries - to fix all of the bugs stemming form the complexity of marginal performance optimization, are 100 times higher.
Together with another developer, I have embarked on a journey to create a hosted 'CRM Style' application that will cater to enterprise level businesses. These businesses will be accessing our application remotely and so the hosted nature of the application will require certain features. For example, to guarantee a level of professional service the following things must be true:
internationalization requires multiple languages and presentation of date/time for various timezones and locales
transactional capability for batch processing of tasks and rollback capabilities
security concerns for keeping data safe and remote invocations secure from attack
etcetera, the list goes on and on
Due to these concerns and my role as the developer most responsible for the server side development, I am very interested in the choices I make early on. Regarding timezones and languages for example, are there issues related to my choice of database or data fields? Do I choose to use a UTC timestamp or date field throughout the application and if so is there a standard format for that? Also, regarding different languages, am I supposed to ensure the data is stored in the database as UTF-8 or unicode?
I really want to avoid laying down the infustructure of the system only to discover later that a fundamental decision was incorrect or not big enough, wide enough, smart enough, etc. Can someone point me in the right direction regarding these basic 'early' decisions?
EDIT _ Ok I appreciate the broad responses and now I see my question was a little too non-specific. I'd like to focus on the more specific elements that WERE present in the question, such as how to choose the proper format for storing a UTC Date/Time or how to save my text data (do I specify a UTF format?)
If you are targeting enterprise CRM, then you will need a very high level of customizability and integrations with all kinds of systems. You will make mistakes in the design. Your only hope is to isolate each little piece of the code so that you can have a chance of fixing it later.
In short, basic software engineering principles are your best bet.
What you are discussing is called a multi-tenant application wherein you have the same code base used by multiple customers (tenants) with logical or physical separation of data. Remember the fundamental rule of development: flexibility is relational to complexity. The more flexible you make the system, the more complicated it will be.
RE: UTC
For a CRM application that stores things like when calls were made and when meetings took place, I would definitely store all those in UTC and let the user set their local timezone. However, you might run into dates which are timezone agnostic and for those, I would store whatever date was entered.
RE: Unicode
Yes, I would use Unicode for all user-entered data. However, that will not get you localization. If for a single company for example, you have a user in Hong Kong entering text in Chinese and user in Amsterdam entering text in Dutch, you are not going to get automatic translation. Things like dates and number formats can be localized, but raw text like names used in drop lists and such can be a chore to localize.
As you have not mentioned what you think about the issue, you may find my answer or parts of it rather basic.
If you don't need to, don't use a low-level language. I'd use python usually for the first version of a CRM application (with the hope that it would be good enough for the next versions), but this decision depends also on the domain community.
Try to write the minimal code on your own, instead relying on the third-party libraries. People may disagree on this, but I would write the code myself as the last option. But the next point is important.
When selecting a library/framework to use, make sure the party behind it is going to last, the library is stable and the software license suits you needs.
Other general rules apply: focus on the customer, use continuous integration/testing, etc., use good software practices like logging etc.
Nothing is ever stored as "unicode" because this is an abstract concept. Unicode is always stored in some kind of unicode transformation format (UTF) (well or UCS but I never saw that used somewhere). The most commonly used UTF is UTF-8 but I suggest to use what is native/default to your platform. wikipedia