Full-text searth JSON-string - elasticsearch

I have a question: in my DB i have a table, who has a field with JSON-string, like:
field "description"
{
solve_what: "Add project problem",
solve_where: "In project CRUD",
shortname: "Add error"
}
How can i full-text search for this string? For example, I need to find all records, who has "project" in description.solve_what. In my sphinx.conf i have
sql_attr_json = description
P.S.Mb i can do this with elasticSearch?

I've just answered a very similar questio here:
http://sphinxsearch.com/forum/view.html?id=13861
Note there is no support for extracting them as FIELDs at this time -
so cant 'full-text' search the text within the json elements.
(To do that would have to use mysql string manipulation functions to
create a new column to index as a normal field. Something like:
SELECT id, SUBSTR(json_column,
LOCATE('"tag":"', json_column)+7,
LOCATE('"', json_column, LOCATE('"tag":"', json_column)+7)-LOCATE('"tag":"',
json_column)-7 ) AS tag, ...
is messy but should work... )
The code is untested.

Related

Filtering a list of values by a field value in GraphQL

So I'm doing some tests with GraphQL, and I'm failing in doing something that I believe is fairly simple.
When going to the GraphQL demo site (https://graphql.org/swapi-graphql) I'm presented with a default query which goes like this:
{
allFilms {
films {
title,
director,
releaseDate
}
}
}
This works as expected and returns a list of films.
Now - I would like to modify this query to return only the films where the director is George Lucas, and for the life of me - I can't figure out how to do that.
I've tried using the where and filter expressions, and also change the second line to films: (director: "George Lucas") but keep getting error messages.
What's the correct syntax for doing that?
Thanks!
If you check the docs of the provided GraphQL schema, you'll see that this is not possible. Following is the definition of the allFilms field:
allFilms(
after: String
first: Int
before: String
last: Int
): FilmsConnection
As per the doc, it has 4 input arguments, which are after, first, before, and last. There is no way to filter this out using the director's name.
GraphQL is not SQL. You cannot use expressions like WHERE or FILTER in GraphQL. The schema is already defined and the filters are pre-defined too. If the schema does not allow you to filter values using a certain field, you just can't do it.
You can to see the graphql schema here https://github.com/graphql/swapi-graphql/blob/master/schema.graphql
The allFilms query does not contain a filter for the field director. Also i can't find other query with this filter.
Most likely you need to write a filter on the result of the query.

Is there a way to compare each item to a aggreated value?

I'm new to graphQL and Hasura. I'm trying(in Hasura) to let me users provide custom aggregation (ideally in the form of a normal graphQL query) and have then each item the results compared against the aggreation.
Here's a example. Assume I have this schema:
USERTABLE:
userID
Name
Age
City
Country
Gender
HairColor
INCOMETABLE:
userID
Income
I created a relationship in hasura and I can query the data but my users want to do custom scoring of users' income level. For example, one user may want to query the data broken down by country and gender.
For the first example the result maybe:
{Country : Canada
{ gender : female
{ userID: 1,
Name: Nancy Smith,..
#data below is on aggregated results
rank: 1
%fromAverage: 35%
}...
Where I'm struggling is the data showing the users info relative to the aggregated data.
for Rank, I get the order by sorting but I'm not sure how to display the relative ranking and for the %fromAverage, I'm not sure how to do it at all.
Is there a way to do this in Hasura? I suspected that actions might be able to do this but I'm not sure.
You can use track a Postgres view. Your view would have as many fields as you'd like calculated in SQL and tracked as a separate "table" on your graphql api.
I am giving examples below based on a simplification where you have just table called contacts with just a single field called: id which is an auto-integer. I am just adding the id of the current contact to the avg(id) (a useless endeavor to be sure; just to illustrate...). Obviously you can customize the logic to your liking.
A simple implementation of a view would look like this (make sure to hit 'track this' in hasura:
CREATE OR REPLACE VIEW contact_with_custom AS
SELECT id, (SELECT AVG(ID) FROM contacts) + id as custom FROM contacts;
See Extend with views
Another option is to use a computed field. This is just a postgres function that takes a row as an argument and returns some data and it just adds a new field to your existing 'table' in the Graphql API that is the return value of said function. (you don't 'track this' function; once created in the SQL section of Hasura, you add it as a 'computed field' under 'Modify' for the relevant table) Important to note that this option does not allow you to filter by this computed function, whereas in a view, all fields are filterable.
In the same schema mentioned above, a function for a computed field would look like this:
CREATE OR REPLACE FUNCTION custom(contact contacts)
RETURNS Numeric AS $$
SELECT (SELECT AVG(ID) from contacts ) + contact.id
$$ LANGUAGE sql STABLE;
Then you select this function for your computed field, naming it whatever you'd like...
See Computed fields

Index JSON Array in Postgres DB

I have a table where each row has a JSON structure as follows that I'm trying to index in a postgresql database and was wondering what the best way to do it is:
{
"name" : "Mr. Jones",
"wish_list": [
{"present_name": "Counting Crows",
"present_link": "www.amazon.com"},
{ "present_name": "Justin Bieber",
"present_link": "www.amazon.com"},
]
}
I'd like to put an index on each present_name within the wish_list array. The goal here is that I'd like to be able to find each row where the person wants a particular gift through an index.
I've been reading on how to create an index on a JSON which makes sense. The problem I'm having is creating an index on each element of an array within a JSON object.
The best guess I have is using something like the json_array_elements function and creating an index on each item returned through that.
Thanks for a push in the right direction!
Please check JSONB Indexing section in Postgres documentation.
For your case index config may be the following:
CREATE INDEX idx_gin_wishlist ON your_table USING gin ((jsonb_column -> 'wish_list'));
It will store copies of every key and value inside wish_list, but you should be careful with a query which hits the index. You should use #> operator:
SELECT jsonb_column->'wish_list'
FROM your_table WHERE jsonb_column->'wish_list' #> '[{"present_link": "www.amazon.com", "present_name": "Counting Crows"}]';
Strongly suggested to check existing nswers:
How to query for array elements inside JSON type
Index for finding an element in a JSON array

How to create text fields dynamically

Say I have a record like this:
type Library = { books : [Book] }
type Book = { title : String, year : Int }
And I want to dynamically create a UI to fill the Library record.
I would want a button "Add" which would create two text fields per book.
And editing inside any text field would immediately be synced in the record structure.
How could I achieve this with elm 0.12?
Learn about interactive UI elements and understand the TODO list example, which is pretty much isomorphic with what you want to acquire.

Substring with spacebar search in RavenDB

I'm using such a query:
var query = "*" + QueryParser.Escape(input) + "*";
session.Query<User, UsersByEmailAndName>().Where(x => x.Email.In(query) || x.DisplayName.In(query));
With the support of a simple index:
public UsersByEmailAndName()
{
Map = users => from user in users
select new
{
user.Email,
user.DisplayName,
};
}
Here I've read that:
"By default, RavenDB uses a custom analyzer called
LowerCaseKeywordAnalyzer for all content. (...) The default values for
each field are FieldStorage.No in Stores and FieldIndexing.Default in
Indexes."
The index contains fields:
DisplayName - "jarek waliszko" and Email - "my_email#domain.com"
And finally the thing is:
If the query is something like *_email#* or *ali* the result is fine. But while I use spacebar inside e.g. *ek wa*, nothing is returned. Why and how to fix it ?
Btw: I'm using RavenDB - Build #960
Change the Index option for the fields you want to search on to be Analyzed, instead of Default
Also, take a look here:
http://ayende.com/blog/152833/orders-search-in-ravendb
Lucene’s query parser interprets the space in the search term as a break in the actual query, and doesn’t include it in the search.
Any part of the search term that appears after the space is also disregarded.
So you should escape space character by prepending the backslash character before whitespace character.
Try to query *jarek\ waliszko*.
So.., I've came up with an idea how to do it. I don't know if this is the "right way" but it works for me.
query changes to:
var query = string.Format("*{0}*", Regex.Replace(QueryParser.Escape(input), #"\s+", "-"));
index changes to:
public UsersByEmailAndName()
{
Map = users => from user in users
select new
{
user.Email,
DisplayName = user.DisplayName.Replace(" ", "-"),
};
}
I've just changed whitespaces into dashes for the user input text and spacebars to dashes in the indexed display name. The query gives expected results right now. Nothing else really changed, I'm still using LowerCaseKeywordAnalyzer as before.

Resources