$ORDER vs counting to scan global range - performance

I have a choice between two ways of scanning through a key level in a large global array and am trying to figure out if one method is more efficient than the other.
This is a vendor supplied application and database on the Intersystems Caché database platform. It is written in the old MUMPS style and does not use any of Caché's object persistence functions: all data is stored in globals directly and any indexes are application maintained.
There is a common convention for repeating data elements attached to entities where the first record will contain a count of child records and then each child record is numbered sequentially at the next key level. For example:
^GBDATA(12345,100)="3"
^GBDATA(12345,100,1)="A^Record"
^GBDATA(12345,100,2)="B^Record"
^GBDATA(12345,100,3)="C^Record"
Where "12345" is the entity key, and "100" is one of the attached detail types. Note that the first "100" record with no other keys has the count of subrecords. There could be anywhere between 0 and hundreds of subrecords attached. The entities are often very wide and there is a lot of other data besides this subrecord type (not shown in example).
Given an entity key, I want to scan through all the subrecords of one type. Would it be faster to use $ORDER to go through the subkeys or to use a FOR loop to anticipate the key values? Does it matter?
$ORDER method:
SET EKEY=12345
SET SEQ=""
FOR
{
SET SEQ=$ORDER(^GBDATA(EKEY,100,SEQ), 1, ROWDATA)
QUIT:SEQ=""
WRITE ROWDATA,!
}
FOR count method:
SET EKEY=12345
SET LIM=^GBDATA(EKEY,100)
FOR SEQ=1:1:LIM
{
WRITE ^GBDATA(EKEY,100,SEQ),!
}
Does anyone know how $ORDER vs $GET is implemented internally in Caché?
I'm having trouble testing this empirically since we only have one production instance with appropriate data and I can't take it offline to clear the cache. I'm most interested in from-disk performance as opposed to from-cache performance.

You could use %SYS.MONLBL to figure out definitively. My guess is that $ORDER is slightly better.
http://docs.intersystems.com/cache20122/csp/docbook/DocBook.UI.Page.cls?KEY=GCM_monlbl

In regards to your question, "Does anyone know how $ORDER vs $GET is implemented internally in Caché?" The two are completely different functions.
$Order is used for the direction that you're going in when reviewing your ^Global.
$Get is used to pull the data within the ^Global. Below is an example of it's use. I use Cache ObjectScript; however, this should give you a general idea
Global Structure
^People(LastName,FirstName)="Phone"
Global Data
^People(Doe,John)="1035001234"
^People(Smith,Jane)="7405241305"
^People(Wood,Edgar)="7555127598"
Code Sample
SET LASTNAME=0
FOR QUIT:LASTNAME?." " DO
.SET LASTNAME=$ORDER(^People(LASTNAME)) QUIT:LASTNAME?." "
.SET FIRSTNAME=0
.FOR QUIT:FIRSTNAME?." " DO
..SET FIRSTNAME=$ORDER(^People(LASTNAME,FIRSTNAME)) QUIT:FIRSTNAME?." "
..SET PHONE=$GET(^People(LASTNAME,FIRSTNAME))
In the sample provided above, it will start with the first record within the ^People global and then start with the first record within the last name by utilizing $Order. It will then $Get the data for the ^People(LASTNAME,FIRSTNAME) node, which is the phone number.
For some samples and reference areas, check out the following links:
$Get Information
$Order Information

Related

Store a sheet object in cache of my google app script [duplicate]

I am trying to develop a webapp using Google Apps Script to be embedded into a Google Site which simply displays the contents of a Google Sheet and filters it using some simple parameters. For the time being, at least. I may add more features later.
I got a functional app, but found that filtering could often take a while as the client sometimes had to wait up to 5 seconds for a response from the server. I decided that this was most likely due to the fact that I was loading the spreadsheet by ID using the SpreadsheetApp class every time it was called.
I decided to cache the spreadsheet values in my doGet function using the CacheService and retrieve the data from the cache each time instead.
However, for some reason this has meant that what was a 2-dimensional array is now treated as a 1-dimensional array. And, so, when displaying the data in an HTML table, I end up with a single column, with each cell being occupied by a single character.
This is how I have implemented the caching; as far as I can tell from the API reference I am not doing anything wrong:
function doGet() {
CacheService.getScriptCache().put('data', SpreadsheetApp
.openById('####')
.getActiveSheet()
.getDataRange()
.getValues());
return HtmlService
.createTemplateFromFile('index')
.evaluate()
.setSandboxMode(HtmlService.SandboxMode.IFRAME);
}
function getData() {
return CacheService.getScriptCache().get('data');
}
This is my first time developing a proper application using GAS (I have used it in Sheets before). Is there something very obvious I am missing? I didn't see any type restrictions on the CacheService reference page...
CacheService stores Strings, so objects such as your two-dimensional array will be coerced to Strings, which may not meet your needs.
Use the JSON utility to take control of the results.
myCache.put( 'tag', JSON.stringify( myObj ) );
...
var cachedObj = JSON.parse( myCache.get( 'tag' ) );
Cache expires. The put method, without an expirationInSeconds parameter expires in 10 minutes. If you need your data to stay alive for more than 10 minutes, you need to specify an expirationInSeconds, and the maximum is 6 hours. So, if you specifically do NOT need the data to expire, Cache might not be the best use.
You can use Cache for something like controlling how long a user can be logged in.
You could also try using a global variable, which some people would tell you to never use. To declare a global variable, define the variable outside of any function.

How does one read the data value from a Function outside FunctionCallbackInfo?

When I create a function like this:
v8::Function::New(<Isolate>, <C_Function>, <Data_Value>);
The Data_Value that I supply is useful for many things and I can access that when the function is called, with something like FunctionCallbackInfo->GetData().
But I have found no way to get back this data in a different scenario. Let's say I store that Function in a Persistent object, and then I would like to read which data is currently bound to it. Any ideas?
I don't think it's exposed via the API.
But there's an alternative:
manually construct a v8::FunctionTemplate
set its ->InstanceTemplate()->SetInternalFieldCount(num_fields)
get the v8::Function from the template with template->GetFunction(context),
now you should have function->InternalFieldCount() == num_fields
you can use function->SetInternalField(index, value) and function->GetInternalField(index) to store any data you want.
For complete examples, search for "SetInternalFieldCount" in V8's test-api.cc.

Improve Script performance by caching Spreadsheet values

I am trying to develop a webapp using Google Apps Script to be embedded into a Google Site which simply displays the contents of a Google Sheet and filters it using some simple parameters. For the time being, at least. I may add more features later.
I got a functional app, but found that filtering could often take a while as the client sometimes had to wait up to 5 seconds for a response from the server. I decided that this was most likely due to the fact that I was loading the spreadsheet by ID using the SpreadsheetApp class every time it was called.
I decided to cache the spreadsheet values in my doGet function using the CacheService and retrieve the data from the cache each time instead.
However, for some reason this has meant that what was a 2-dimensional array is now treated as a 1-dimensional array. And, so, when displaying the data in an HTML table, I end up with a single column, with each cell being occupied by a single character.
This is how I have implemented the caching; as far as I can tell from the API reference I am not doing anything wrong:
function doGet() {
CacheService.getScriptCache().put('data', SpreadsheetApp
.openById('####')
.getActiveSheet()
.getDataRange()
.getValues());
return HtmlService
.createTemplateFromFile('index')
.evaluate()
.setSandboxMode(HtmlService.SandboxMode.IFRAME);
}
function getData() {
return CacheService.getScriptCache().get('data');
}
This is my first time developing a proper application using GAS (I have used it in Sheets before). Is there something very obvious I am missing? I didn't see any type restrictions on the CacheService reference page...
CacheService stores Strings, so objects such as your two-dimensional array will be coerced to Strings, which may not meet your needs.
Use the JSON utility to take control of the results.
myCache.put( 'tag', JSON.stringify( myObj ) );
...
var cachedObj = JSON.parse( myCache.get( 'tag' ) );
Cache expires. The put method, without an expirationInSeconds parameter expires in 10 minutes. If you need your data to stay alive for more than 10 minutes, you need to specify an expirationInSeconds, and the maximum is 6 hours. So, if you specifically do NOT need the data to expire, Cache might not be the best use.
You can use Cache for something like controlling how long a user can be logged in.
You could also try using a global variable, which some people would tell you to never use. To declare a global variable, define the variable outside of any function.

In Local Datastore chapter, fromPin and fromlocaldatastore

Just like as title, I want to ask what difference between
fromPin()
and
fromLocalDatastore()
By the way, Pin and datastore two terminologies. What difference between two of them ?
Thanks.
There is a slight difference and you can see it from the docs and from the decompiled code of the Parse library (okay, this last one is more complicated...).
The docs says:
fromLocalDatastore(): Change the source of this query to all pinned objects.
fromPin(): Change the source of this query to the default group of pinned objects.
Here you can see that, interally on Parse, there is a way to get all the objects from the entire set of pinned data, without filters, but also from a so-called "default group". This group is defined in the Parse code with the following string: _default (o'rly?).
When you pin something using pinInBackground, you can do it in different ways:
pinInBackground() [without arguments]: Stores the object and every object it points to in the local datastore.
This is what the docs say, but if you look at the code you'll discover that the pin will be actually performed to the... _default group!
public Task<Void> pinInBackground() {
return pinAllInBackground("_default", Arrays.asList(new ParseObject[] { this }));
}
On the other hand, you can always call pinInBackground(String group) to specify a precise group.
Conclusion: every time you pin an object, it's guaranteed to be pinned to a certain group. The group is "_default" if you don't specify anything in the parameters. If you pin an object to your custom group "G", then a query using fromPin() will not find it! Because you didn't put it on "_default", but "G".
Instead, using fromLocalDatastore(), the query is guaranteed to find your object because it will search into "_default", "G" and so on.

How to cache a collection in Magento?

I have a collection that takes significant time to load. What I would like is to cache it (APC, Memcache). It is not possible to cache the entire object (as it cannot be unserialized and it is over 1 MB). I'm thinking that caching the collection data ($col->getData() ) is the way to go, but I found no way to rebuild the object based on this array. Any clues?
Collections already have some caching built in but they need a little prompting so put this in the constructor of a collection:
$cache = Mage::app()->getCacheInstance();
$prefix = "SomeUniqueValue";
$this->initCache($cache, $prefix, array(Mage_Catalog_Model_Product::CACHE_TAG));
Choose tags appropriate to the content of the collection so that it will be flushed automatically. This way builds an ID based on the query being executed, it is most useful when the collection is filtered, ordered or paged - it avoids a version conflict.
Generally this hardly gets used because when you retrieve data you almost always end up displaying it, probably as HTML, so it makes sense to cache the output instead. Block caching is widely used and better documented.
I really don't know, but I searched for all the files that have the word "cache" in them with file names of "Collection.php" and got a few results. The most promising example to look at might be Mage_Sales_Model_Entity_Quote_Item_Collection (_getProductCollection() method). Looks like Varien_Data_Collection (which is a parent class of any magento collection) has a few cache-related methods: initCache() and _getCacheInstance().
Can't say I have used them before but might be useful someday.
Good luck.
You can get more information here: Can I use Magento's Caching layer as a Key/Value Store?
I'll be posting more info there as I find it.

Resources