I have a simple Oracle table with 3 fields: ID (int), Name (nvarchar), and configJSON (CLOB).
The configJSON field is over 10,000 characters.
I am trying to use this table in an oData web service using entity framework. I do not get an error message, but the response is simply blank. When I remember the CLOB column I see the other 2 data fields so I think the issue relates to the large CLOB field. I am not tied to that particular data type but I thought that was the best to use since my field is so large.
What is the best way to return a large data field within an oData web service?
I was able to resolve the issue by changing the field type to a LONG.
Related
I have a data JPA entity where it contains a "price" type Double. Now, the users need to able to filter the records based on that field (Between min and max). Now the problem is, the value in the DB can be null for some records. My data JPA repository uses a native query like "price BETWEEN :priceFrom AND :priceTo". Now, if the user does not specify anything in the filter conditions, all record including the ones where prices is null should be returned. However, this query does not return those record. I know, I can create a new method with query "price IS NULL" and check the filter values in my service layer and call the null version if nothing is specified. But, I have multiple fields with the same requirement then it results in a lot of duplicate methods to maintain. Is there a better approach to handle that situation?
It seems to me, that you can specify
(:priceFrom is null and :priceTo is null and price is null)
OR price between :priceFrom and :priceTo
if priceFrom and priceTo are entered, second part of OR will be used, otherwise it selects records where price is null
Since you are using Spring Data JPA - Specifications should solve this for you. This is a JPA Criteria based solution.
For any complex-enough API – searching/filtering your resources by very simple fields is simply not enough. A query language is more flexible and allows you to filter down to exactly the resources you need. Hence you should easily be able to program for NULL (in the scenario that you currently need) and anything else that you might need.
This is scalable for multiple fields and easy to code/configure. There are a few links which will give you more insight into it
Spring Blog
Tutorial 1
Tutorial 2
Hopefully, this is helpful.
Using Kibana, is it possible to display one row of data which is a summary of other rows?
This is our requirement:
Given an entry in an index with the following structure:
string requestId
boolean raisedException
boolean requiredExternalLookup
We want to create a tabular output with the following structure
requestId numberRaisedException numberNoException numberRequiredLookup
So, if there were three rows (or entries) in the index for the same request id, two where an exception was raised, the output may look like this:
requestId numberRaisedException numberNoException numberRequiredLookup
REQUEST_123 2 1 3
Presumably the correct Kibana visualization widget to represent this would be a Data Table. But how in Kibana would one create a row like the above which is a summary of several rows, somewhat akin to a sql GROUP BY clause. Is it at all possible?
You can probably do this with 'scripted_fields', but the status of the 'scripted_fields' feature in kibana isn't clear. I think it was recently blocked in kibana due to security issues - Leaving this open is dangerous since you can do anything.
If you have access to your elasticsearch cluster then you might be able to create the field on your elasticsearch index.
You can read about it here : http://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html
It used to be, and the documentation still says: Each PFObject class may only have one key with a PFGeoPoint object.
But in my tests today, I created an object with 2 GeoPoint columns, was able to query on either GeoPoint, and was able to modify and save either GeoPoint. Previously, this would lead to an error like: only 1 ParseGeoPoint object can be stored in a class.
Is this really supported now?
Some additional info: I first have to create the 2 geoPoint columns in the data browser. If they don't exist and my iPhone code tries to save an object with 2 geoPoints, then I get the "only one GeoPoint field may exist in an object". But as long as the 2 columns exist, my client code appears to be able to use both.
As of July 2015, Parse still does not support more than one GeoPoint column on a class. They have, however, fixed the Data Browser to prevent users from creating two GeoPoint columns.
Got this response from Parse (in the Google Group forum):
Hmm, that sounds like a problem with the data browser's mechanism of altering the schema. Could you report a bug? I would not recommend using objects created in this way - the underlying data store can only index one geopoint field per object, so whichever field gets indexed second just will have the index fail and you won't be able to run queries against it.
The solution is to put the second GeoPoint (which you will not be able to search on) into a singleton array.
I am using elasticsearch as a document database and each record I create has a guid id that the system uses for the record id. Business people want to offer a feature to let the user have their own auto file name convention based on date and how many records were created so far this day/month.
What I need is to prevent duplicate user file names. Is there a way to setup an indexed field to be unique? Like a sql unique constraint?
You'd need to use the field that is supposed to be unique as id for your documents. By default a new document with existing id would override the existing document with same id, but you can switch to op_type=create in order to get back an error if a document with same id already exists.
There's no way to have the same behaviour with arbitrary fields though, only the _id field works that way. I would probably consider handling this logic in the application layer instead of within elasticsearch.
One solution will be to use uniqueId field value for specifying document ID and use op_type=create while storing the documents in ES. With this you can make sure your uniqueId field will have unique value and will not be overridden by another same valued document.
For this, the elasticsearch document says:
The index operation also accepts an op_type that can be used to force a create operation, allowing for "put-if-absent" behavior. When create is used, the index operation will fail if a document by that id already exists in the index.
Here is an example of using the op_type parameter:
$ curl -XPUT 'http://localhost:9200/es_index/es_type/unique_a?op_type=create' -d '{
"user" : "kimchy",
"uniqueId" : "unique_a"
}'
If you run the above request it is ok, but running it the next time will give you an error.
You can use the _id in the column you want to have unique contraint on.
Here is the sample river that uses postgresql. Yo can change the Database Driver/DB-URL according to your usage.
curl -XPUT localhost:9200/_river/simple_jdbc_river/_meta -d "{\"type\":\"jdbc\",\"jdbc\":{\"strategy\":\"simple\",\"poll\":\"1s\",\"driver\":\"org.postgresql.Driver\",\"url\":\"jdbc:postgresql://DB-URL/DB-INSTANCE\",\"user\":\"USERNAME\",\"password\":\"PASSWORD\",\"sql\":\"select t.id as _id,t.name from topic as t \",\"digesting\" : true},\"index\":{\"index\":\"jdbc\",\"type\":\"topic_jdbc_river1\"}}"
So far as to ES 7.5, there is no such extra "constraint" to ensure uniqueness using a custom field in the mapping.
But you still can walk around it via your own application UUID, which could be used directly explicitly as the _id (which is unique implictly) to achieve your goals.
PUT <your_index_name>/_doc/<your_app_uuid>
{
"a_field": "a_value"
}
Another approach might be to generate the string you store in a field that should be unique by integrating an auto-incrementing integer. This way you ensure from the start that your field values are unique.
You would put your file name together like this:
<current day/month>_<auto-incremented integer>
Auto-incrementing integers are not supported by Elasticsearch per se but you could mimic them using this approach. If you happen to use node.js you can use the es-sequence module.
I have a problem that only happens rarely with FT search. but once it happens it stays. I use the following search term in the FT search box in Lotus Notes
[Tags] = "foo"
in most application this search term work fine. but for some applications this search term gives the error "query is not understandable".
It does not matter if I replace the value, e.g [Tags] = "boo" produce the same result. and also FIELD Tags = "boo". for the record [Tag] = "foo" works fine so it seem be issues with the field or field name.
The problem only happens in some applications. Once this problem start happening no views can be searched using that search query and I get the error message everytime I search.
It does not help to remove, compact and re-create the FT index.
I get the same error in xpages when using the same search query in a view data source.
I have seen this problem using other fieldnames as well in other application.
If I remove the FT index the search query works
Creating a new copy of the "broken" database does not resolve the problem
I tried to have only one document in database, create a new FT index. the document in view does not have the field "Tags" still not working. (there are other forms in db with the fieldname "Tags")
This is a real show stopper for me as I have built some of my XPages based on search values from specific fields
In my own invstigation of this problem I think it has to do with some sort of bug in the FT index. There seem to be some data contained in documents or forms that causes the FT index to not work correctly.
I am looking for a solution to this problem as I have not found a way to repair it once it has become broken.
Update:
It does not help to follow this procedure
https://www-304.ibm.com/support/docview.wss?uid=swg21261002
Here is my debug info
[1078:0002-2250] IN FTGSearch
[1078:0002-2250] option = 0x400219
[1078:0002-2250] Query: ( FIELD Tags = "foo")
[1078:0002-2250] OUT FTGSearch error = F09
[1078:0002-2250] FTGSearch: found=0, returned=0, start=0, count=0, limit=0
It sounds like you need to fix the UNK table with a compact. Here is the listing of compact options, use a copy style not in place.
http://www-01.ibm.com/support/docview.wss?uid=swg21084388
If Tags field is sometimes numeric, I would advise looking at the database design. The UNK table is a table of all fields in the NSF. The first time a field name is used, it is stored in the UNK table as that data type. Full text searching uses that data type and only that data type. If you have a field Tags on more than one form in a database, once numeric and once text, you're in for big trouble with full text searches. The datatype in searches will depend on which datatype the field was on the first document saved which had that field. Even if you delete all documents that have it as numeric, you won't change the UNK table without the compact. Sounds like that's what you have here. Ensure the database never stores Tags as numeric. Delete or change all docs with it stored numeric. Then compact.
Thank you all for answering. I learned a whole lot about UNK tables and FT index today.
The problem was that I had a numeric field called "Tags" in a form that I hadn't looked at and really didn't think that it would contain a field by that name.
after using the DDE search I found all instances of the tags field and could eaily locate the problem form. I renamed the field in the form, removed the FT indx , used compact -c and recreated the ft index. now everythig is working fine.
One other thing to notice is that I have several databases with the same design but only a few of them had the ft index problem, the reason for this is probably because some of these databases was created after the form with the faulty Tags field was created.
I am so happy to have solved this.
lessons learned
If you plan to use fulltext index in your application. make sure you do not have the same field name in different forms and use different field types.
from now on I will probably use shared fields more :-)
One more thing we discovered
You actually do not need notes peek to find out which field tpe is stored in the UNK table. you can use the "Fields" button in the searchbar. if you select the field and the right hand box displays "contains" you know the unk table has a text field type set.