I am developing a firefox add-on using XUL, and I want to measure and profile my extension memory usage.
How can I do this? and check which function is taking the most memory usage and how much memory usage my extension is adding to firefox?
You cannot measure the impact of a single function, the memory management in Firefox doesn't work at this level - it works with compartments. If your extension has its own window then you will be able to see the compartment of this window under about:memory?verbose (click "Minimize memory usage", otherwise you might see objects there that will be garbage collected anyway). If your extension's code runs in the context of the browser window then you are usually out of luck - you will not be able to distinguish it from the other scripts running there. It's the same with XPCOM components and JavaScript modules - all of them get loaded into the "[System Principal]" compartment.
What you can do to get your scripts separated from a large compartment however: use sandboxes, a sandbox always gets its own compartment. For example, in a browser window you would do something like this:
Components.utils.import("resource://gre/modules/Services.jsm");
var mySandbox = Components.utils.Sandbox(window,
{sandboxName: "myExtension/browserScript.js"});
mySandbox.window = window; // Expose window variable to scripts in the sandbox
Services.scriptloader.loadSubScript("chrome://myextension/content/browserScript.js",
mySandbox);
mySandbox.init(); // Call function init() of the script in the sandbox
As a result, a compartment called myExtension/browserScript.js will be displayed under about:memory?verbose and you will be able to see how much memory this script (along with objects it creates etc.) takes exactly. Things to keep in mind:
The script in the sandbox won't have access to the variables from "outside". You have to explicitly set these variables as properties of the sandbox (like I've done with the window variable in the example).
Compartments aren't cheap, and passing objects between compartments isn't cheap either. So creating one compartment for each function would be a bad idea because of the overheads involved.
Documentation: Sandbox, Services.jsm
Update: As of Firefox 13 things changed. There is this extension for example that will show you all the objects currently in memory. Still far from being comfortable, also getting the whole picture is non-trivial - but it gives you granularity on a level below compartments.
Related
Ok, it may be a bit difficult to explain:
Suppose someone creates a Windows application (using C# or any other language) that uses the GetDesktopWindow() function on the user32.dll to capture a Screenshot and then sends this image to any online service.
Since it's custom made application, no anti-virus software will be able to determine that it's a virus because it's still an unknown application for it. Also, there are legitimate uses for such API, so it's not necessarily a virus, it can be a harmless window capture tool or some kind of espionage tool.
What I want to know is: Is there any way to see what a specific EXE file does regarding the Windows functions? Can I know if "myapp.exe" uses GetDesktopWindow() of user32.dll?
This is only one example. There are plenty other Windows endpoints that I would like to know when they're used by any application.
Is there a way to do that?
It depends to what lengths you want to go doing that. It's essentially a game of cat and mouse - bad actors will attempt to find new ways to circumvent your detection by jumping through some obscure hoops, you will add more sophisticated detection methods for those tricks, they will think of new tricks, and so on.
Also, it depends on whether you want to statically or dynamically determine that, and whether you actually want to know if GetDesktopWindow is called or if "the program gets a handle to the desktop window" (which can be achieved in other ways as well).
Here is a non-exhaustive list of ideas:
You could statically determine whether the function is imported by looking at the import directory. Research the PE file structure to find out more. This article may help.
This method of detection can be easily circumvented by dynamically importing the function using LoadLibrary and GetProcAddress.
You could scan the file for the string GetDesktopWindow to detect possible usage for dynamic import.
This method of detection can be easily circumvented by packing, encrypting or otherwise obfuscating the name of the dynamically imported function.
You could dynamically observe whether the GetDesktopWindow function gets called by registering an AppInit_DLL or a global hook which is injected into every new process and hook the GetDesktopWindow function from inside the process by overwriting its first bytes with a jump to your own code, notifying your detection component somehow, executing the original bytes and jumping back. (Microsoft Detours can help there.)
This method of detection can be circumvented if the target notices the hook and removes it before calling, since its in its own process space. (You could also do some tricks with acting like a debugger and setting a hardware breakpoint on the first instruction of GetDesktopWindow, but yet again there would be ways to detect or circumvent that since the target could also modify the debug registers.)
You could build a driver that does this from kernel-mode instead, but now we are getting really deep.
Note that until now we focused on the actual GetDesktopWindow function from user32.dll. But what if the target will just use a different way to achieve its goal of getting a desktop window handle?
The desktop window handle for the current thread is stored in the TIB (thread information block) which is accessible via fs:[18] from user mode. You can see this in the GetDesktopWindow source code of ReactOS which is pretty accurate compared to Microsoft's actual implementation (which you can verify by looking at it in a debugger). The target could therefore just access the TIB and extract this value, without even calling GetDesktopWindow at all.
The target could just take a known top-level window such as the shell's hidden compatibility window which you'll get via GetShellWindow() or - to avoid detection of GetShellWindow too - for example FindWindow(NULL, "Program Manager") (or even a newly created window!) and call GetAncestor(hWnd, GA_PARENT) on it to get the desktop window handle.
I'm sure, with some creativity, your adversaries will come up with more clever ideas than these.
Also, if we take this one step further and take a look at the ultimate goal of taking a screenshot, there as well exist other ways to achieve that. First example coming to mind: They could use keybd_event to emulate pressing the PrnSc key and then read the screenshot out of the clipboard data.
So it's all a matter of how far you want to take this.
By the way, you may find the drltrace project interesting - it is a library call tracer.
I have a MacOS app that creates many NSViews of differing types, some of which contain WKWebViews. The Number of WKWebViews can become fairly large (dozens or more).
The problem is that each of these WKWebViews creates several processes, as can be easily seen in Activity Monitor. When the number of processes becomes too large, the entire system essentially grinds to a halt: running apps still function slowly, but new apps can't be launched, the Finder stops working, and the system must be rebooted.
So I am looking for a way to control the number of processes started by WKWebViews.
According to Apple docs, this is managed by the processPool property of WKWebView, as they state: "The process pool associated with a web view is specified by its web view configuration. Each web view is given its own Web Content process until an implementation-defined process limit is reached; after that, web views with the same process pool end up sharing Web Content processes." (from the page at: https://developer.apple.com/documentation/webkit/wkprocesspool?language=objc).
I cannot figure out how to define that "implementation-defined process limit". When I NSLog the description of the WKWebView's configuration, I get this:
WKWebView configuration description:
>; preferences = >
I can see there's a processPool and a configuration for it (called _WKProcessPoolConfiguration), and a value called maximumProcessCount, which is set to 0. I am assuming that 0 means that the process count is unlimited, and I would like to set it to a reasonable number. But I cannot figure out how to set this value! The processPool itself has no properties, so I have no idea how to access this maximumProcessCount value.
Any help is greatly appreciated.
you can use singleton WKProcessPool and all wkwebview.configuration.processpool use that WKProcessPool singleton；
it will resolve cpu too high.
What 3 things would you tell developers new to XPages to do to help maximize the performance of their XPages apps?
Tim Tripcony had given a bunch of suggestion here.
http://www-10.lotus.com/ldd/xpagesforum.nsf/topicThread.xsp?action=openDocument&documentId=365493C31B0352E3852578FD001435D2#AEFBCF8B111E149B852578FD001E617B
Not sure if this tipp is for beginners, but use any of the LifeCyclePhaseListeners from the OpenNTF Snippets to see what is going on in your datasources during a complete or partial refresh (http://openntf.org/XSnippets.nsf/snippet.xsp?id=a-simple-lifecyclelistener-)
Use the extension Library. Report Bugs ( or what you consider a bug ) at OpenNTF.
Use the SampleDb from the extLib. ou can easily modify the samples to your own need. Even good for testing if the issue you encounter is reproducable in this DB.
Use Firebug ( or a similar tool that comes with the browser of your choice ) If you see an error in the error tab, go and fix it.
Since you're asking for only 3, here are the tips I feel make the biggest difference:
Determine what your users / customers mean by "performance", and set the page persistence option accordingly. If they mean scalability (max concurrent users), keep pages on disk. If they mean speed, keep pages in memory. If they want an ideal mixture of speed and scalability, keep the current page in memory. This latter option really should be the server default (set in the server's xsp.properties file), overridden only as needed per application.
Set value bindings to compute on page load (denoted by a $ in the source XML) wherever possible instead of compute dynamically (denoted by a #). $ bindings are only evaluated once, # bindings are recalculated over and over again, so changing computations that only need to be loaded once per page to $ bindings speed up both initial page load and any events fired against the page once loaded.
Minimize the use of SSJS. Wherever possible, use standard EL instead (e.g. ${database.title} instead of ${javascript:return database.getTitle();}). Every SSJS expression must be parsed into an abstract syntax tree to be evaluated, which is incrementally slower than the standard EL resolver.
There are many other ways to maximize performance, of course, but in my opinion these are the easiest ways to gain noticeable improvement.
1. Use the Script Library instead writing a bulk of code into the Xpage.
2. Use the Theme or separate CSS class for each elements [Relational]
3. Moreover try to control your SSJS code. Because server side request only reduce our system performance.
4. Final point consider this as sub point of 3, Try to get the direct functions from our SSJS, Don't use the while llop and for loop for like document collection, count and other things.
The basics like
Use the immediate flags ( or one of the other flags) on serverside events if possible
Check the Flag which (forgot its name..) generates the css and js as
one big file at runtime therefore minimizing the ammount of
requests.
Choose your scope wisely. Dont put everything in your sessionscope but define when, where and how your are using the data and based on that use the correct scope. This can lead to better memory usage..
And of course the most important one read the mastering xpages book.
Other tips I would add:
When retrieving data use viewentrycollections or the viewnavigstor
Upgrade to 8.5.3
Use default html tags if possible. If you dont need the functionality of a xp:div or xp:panel use a <div> instead so you dont generate an extra uicomponent on the tree.
Define what page persistance mode you need
Depends a lot what you mean by performance. For performance of the app:
Use compute on page load wherever feasible. It significantly improves performance.
In larger XPages particularly, combine code into single controls where possible. E.g. Use a single Computed Field control combining literal strings, EL and SSJS rather than one control for each language. On that point, EL performs better than SSJS, and SSJS on the XPage performs better than SSJS in a Script Library.
Use dataContexts for properties that are calculated more than once on an XPage.
Partial Execution mode is a very strong recommendation, but probably beyond new XPages developers at this point. Java will also perform better than SSJS in a Script Library, but again beyond new developers. XPages controls you've created with the Extensibility Framework should perform better, because they should run fewer lines of Java than multiple controls, but I haven't tested that.
If you mean performance of the developer:
Get the Extension Library.
Use themes to set default properties, e.g. A standard style for all your pagers.
Use Firebug. If you're developing for Notes Client or IE, still use Firebug. You'll spend longer suffering through Client/IE thank you will fixing the few quirks that will remain.
I'm trying to track down some peculiar memory behavior in my Cocoa desktop app. My app does a lot of image processing using NSImage and uploads those images to a website over HTTP using NSURLConnection.
After uploading several hundred images (some very large), when I run Instrument I get no leaks. I've also run through MallocDebug and get no leaks. When I dig into object allocations using Instrument I get output like this:
GeneralBlock-9437184, Net Bytes 9437184, # Net 1
GeneralBlock-192512, Net Bytes 2695168, # Net 14
and etc., for smaller sizes. When I look at these in detail, they're marked as being owned by 'Foundation' and created via NSConcreteMutableData initWithCapacity. During HTTP upload I'm creating a post body using NSMutableData, so I'm guessing these are buffers Cocoa is caching for me when I create the NSMutableData objects.
Is there a way to force Cocoa to free these? I'm 90% positive I'm releasing correctly (and Instruments and MallocDebug seem to confirm this), but I think Cocoa is keeping these around for perf reasons since I'm allocating so many MSMutableData buffers.
If you're certain you're releasing the objects you own correctly, then there's really nothing you can (or should) do. Those blocks are, as Instruments says, owned by Foundation because NSConcreteMutableData, a Foundation object, created them. It's possible that these are some sort of cache that NSData is keeping around on purpose, but there's no way to know what they are.
If you believe this is a bug, you should report it at http://bugreport.apple.com. The rules of memory ownership apply to classes that don't manage memory well, too.
Also, this might be a silly question, but which option are you using for the Object Alloc tool? All objects created or Created and still living? You might be looking at allocations that don't matter anymore.
I'm looking for advanced strategies for dealing with User Object Handle limits when building heavy-weight windows interfaces. Please explain how you overcame or bypassed this issue using SWT or direct Windows GUI APIs. The only thing I am not interested in is strategies to optimize widget usage as I have done this extensively and it does not solve the problem, only makes it less likely.
My Situation:
I have an SWT based GUI that allows for multiple sessions within the same parent shell and within each session their are 3 separate places where a list of user generated comments are displayed. As a user opens multiple sessions and pulls data that populates those lists, the number of user object handles can increase dramatically depending on the number of comments.
My current solutions:
1. I page the comments by default thereby limiting the number of comment rows in each session, but due to management demands, i also have what is effectively a "View All" button which bypasses this completely.
2. I custom draw all non-editable information in each row. This means each row utilizes only 2 object handles.
3. I created JNI calls which query the OS for the current usage and the Max usage. With this i can give indications to users that a crash is imminent. Needless to say, they ignore this warning.
First off, are you sure the problem isn't desktop heap vs. handle count? Each handle can consume a certain amount of Windows desktop heap. One USER handle may eat a lot of space, some very little. I'm suggesting this to make sure you're not chasing user handle counts when it's really something else. (google for Microsoft's dheapmon tool, it may help)
I've read that you can alter the maxes on handles by changing keys in the registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\
CurrentVersion\Windows\ USERProcessHandleQuota and GDIProcessHandleQuota
This could be a short term fix for users.
I'd approach this by first figuring out what 2 user handles need to be maintained for each item (like 2 for each item in a listbox?). This seems suspect. User handles are for only a few top-level Windows UI objects (Windows, menus, cursors, Window positions, icons, etc...). I don't see why your widget needs to keep 2 objects around for each item (is it an icon handle??).
If you're looking to rip the whole thing apart - this sounds like a job for a virtual-mode List-View (LVS_OWNERDATA).
You should think about using windowless controls. They are designed for precisely this situation. See "Windowless controls are not magic", by Raymond Chen
Not only top-level windows, but most native controls use one user object each. See Give Me a Handle, and I'll Show You an Object for an in-depth explanation of user- and other handle types. This also means that SWT uses at least one user handle per widget, even for a Composite.
If you truly are hitting the limit of 10000 user objects per process, and you don't have a leak, then your only option is to reduce the number of widget instances in your application. I wrote a blog article about how we did this for our application.