Elastic search "InnerHits" Vs Doing an extra call - elasticsearch

Currently i am having product(child) and supplier(parent) types which have parent-child relationship.In scenario where i need to show the natched products with there respective suppliers i am using "innerhits" to fetch the supplier(parent) along with product(child), which is working fine.
But now i have gone through an article which says that "innerhits" slows up the query.so now i am having an alternate approach which is to make a single call to supplier(parent) type and fetch all the suppliers for the needed products.
BRIEFLY PUTTING : (InnerHits) Vs (making separate call), which of the two approaches is advisable.

Related

How to make an API endpoint in Laravel, that can be used to search any table we want?

Background
Let us consider a hypothetical scenario. Suppose we have five different tables in our database,
Customers
Categories
Products
Orders
OderDetails
Our client wants us to add a search bar to the frontend, where a user can search for a specific product and upon tapping the search button, matching products has to be displayed on the frontend.
My Approach for Tackling This Problem
In order to add the aforementioned functionality, I came across the following strategy.
👉 I would add an input box to ender the product name and a submit button.
👉 Upon pressing the submit button, a GET request will be sent to the backend. On the query parameters, the product name that has been entered by the user will be included
👉 Once the GET request is received to the backend, I will pass it down to the ProductsController and from within a method defined inside the ProductController, I will use the Product model to query the products table to see if there are any matching results.
👉 If there are any matching results, I will send them to the frontend inside a JSON object and if there aren't any matching results, I will set a success flag to false inside the JSON object and send it to the frontend
👉 In the frontend, if there are any matching results, I will display them on the screen. Else, I will display "No Results Found!"
Problem with My Approach
Everything works fine if we only want to search the products table. But what if, later our client tells us something like "Remember that search functionality you added for the products? I thought that functionality should be added to orders as well. I think as the list of orders by a user grows and grows, they should be able to search for their previous orders too."
👉 Now, since in our previous approach to search products was implemented in the ProductController using the Product model, when we are adding the same functionality to Orders, WE WOULD HAVE TO DO THE SAME THINGS WE DID IN THE ProductsController AGAIN INSIDE THE OrdersController USING THE Order model. It does not take seconds to understand that this leads to duplication of code.
Problem Summary
❓ How do we add an API endpoint in laravel for a search functionality that can be used to search any table in our database and get results, instead of the search functionality being scoped to a specific controller and a corresponding model?
A good start would be to create a trait called something like searchable and add it to all the models you want to search and put any shared logic between the different models in there. Possibly you'd want to manually allow different columns so in each model you have an array of searchable columns which you'd refer to in your trait.
Your controller would have to point to the related model and call a method in the trait that searches.
As others have pointed out this is a high level question so I won't go too much in detail. Of course there are a million ways to implement this, just try and keep shared logic in place.

The generally accepted way to do bulk product importing in a Shopify App

I'm currently working on my first Shopify app and it requires bulk importing products. I'm looking around the web, and it appears there's no query to do bulk importing. It also looks like if I want to add the price to the items I import, I'll have to make a separate query from the one that creates the products in the first place.
I'm thinking the easier way would be to create a .csv but there's no query to upload a .csv either.
Has anyone tackled something like this before, and what's the usual way to go about it?
If not supported then standard graphql possibilities [possibly (not tested)] can (should be possible) be applied:
Aliases
You can make multiple queries/mutations within one request.
Guaranteed order of mutations on root level
Multiple (aliased) chains of mutations:
m1a: insert #1,
m1b: define #1 price,
m2a: insert #2,
m2b: define #2 price
...
Probably you can use arguments used for element creation (one mutation) to search item for following mutation (if id not required).
You can try/check this scenario in the playground.
Of course in this case you need to construct query dynamically (usually not advised) in the app. You will need dynamic aliases and input variable names.

Distributed GraphQL in microservices

I'm trying to write microservices in Java. I've implemented GraphQL endpoints using graphql-spring-boot-starter.
Now I have a problem how to make it efficient.
Datamodel is like a tree and I need to query for data from multiple services at once. The problem is how to filter for a member of collection, something like CONTAINS in database, but data is not in separate table, but separate microservice. Maybe the problem is that domain is not correctly splitted between services?
Let's make an example: I have 3 microservices: users, libriaries, books. Every library have collection of users and books (just list of identifiers, like foreign keys). Every book has a name and genre. Every library have lists of books borrowed by user (identifiers too).
Question 1 - should library hosts list of books and users (just identifiers, like foreign keys)? Is it correct approach?
Question 2 - I want to find libraries in which specified users (by surname) have borrowed books of specified genre. Going from top I need to first find libraries containing users. Not easy, as we have names in different service. We need to query first for users, gathers their identifiers, and now we are able to query for libraries. But it isn't all. Now we need to find books for every user and check genres - in different service. And it's not all. I want to have everything presented in nice way, so whole output should be sorted and paged. It force me to collect all data from all services, then page and sort it, which of course will not be efficient.
Please don't concentrate on this example, I'm looking how to solve general approach, not this one example. I've tried to use Datafetchers but it's troublesome and there are not good examples of calling Graphql-to-GraphQL. Most examples covers calling REST endpoints etc.

Is there a way to sort a content query by the value of a field programmatically?

I'm working on a portal based on Orchard CMS. We're using Orchard to manage the "normal" content of the site, as well as to model what's essentially data for a small application embedded in it.
We figured that doing it that way is "recommended" for working in Orchard, and that it would save us duplicating a bunch of effort in features that Orchard already provides, mainly generating a good enough admin UI. This is also why we're using fields wherever possible.
However, for said application, the client wants to be able to display the data in the regular UI in a garden-variety datagrid that can be filtered, sorted, and paged.
I first tried to implement this by cobbling together a page with a bunch of form elements for the filtering, above a projection with filters bound to query string parameters. However, I ran into the following issues with this approach:
Filters for numeric fields crash when the value is missing - as would be pretty common to indicate that the given field shouldn't be considered when filtering. (This I could achieve by changing the implementation in the Orchard source, which would however make upgrading trickier later. I'd prefer to keep anything I haven't written untouched.)
It seems the sort order can only be defined in the administration UI, it doesn't seem to support tokens to allow for the field to sort by to be changed when querying.
So I decided to dump that approach and switched to trying to do this with just MVC controllers that access data using IContentQuery. However, there I found out that:
I have no clue how, if at all, it's possible to sort the query based on field values.
Or, for that matter, how / if I can filter.
I did take a look at the code of Orchard.Projections, however, how it handles sorting is pretty inscrutable to me, and there doesn't seem to be a straightforward way to change the sort order for just one query either.
So, is there any way to achieve what I need here with the rest of the setup (which isn't little) unchanged, or am I in a trap here, and I'll have to move every single property I wish to use for sorting / filtering into a content part and code the admin UI myself? (Or do something ludicrous, like create one query for every sortable property and direction.)
EDIT: Another thought I had was having my custom content part duplicate the fields that are displayed in the datagrids into Hibernate-backed properties accessible to query code, and whenever the content item is updated, copy values from these fields into the properties before saving. However, again, I'm not sure if this is feasible, and how I would be able to modify a content item just before it's saved on update.
Right so I have actually done a similar thing here to you. I ended up going down both approaches, creating some custom filters for projections so I could manage filters on the frontend. It turned out pretty cool but in the end projections lacked the raw querying power I needed (I needed to filter and sort based on joins to aggregated tables which I think I decided I didn't know how I could do that in projections, or if its nature of query building would allow it). I then decided to move all my data into a record so I could query and filter it. This felt like the right way to go about it, since if I was building a UI to filter records it made sense those records should be defined in code. However, I was sorting on users where each site had different registration data associated to users and (I think the following is a terrible affliction many Orchard devs suffer from) I wanted to build a reusable, modular system so I wouldn't have to change anything, ever!
Didn't really work out quite like I hoped, but to eventually answer the question in your title: yes, you can query fields. Orchard projections builds an index that it uses for querying fields. You can access these in HQL, get the ids of the content items, then call getmany to get them all. I did this several years ago, and I cant remember much but I do remember having a distinctly unenjoyable time with it haha. So after you have an nhibernate session you can write your hql
select distinct civr.Id
from Orchard.ContentManagement.Records.ContentItemVersionRecord civr
join civ.ContentItemRecord cir
join ci.FieldIndexPartRecord fipr
join fipr.StringFieldIndexRecord sfir
This just shows you how to join to the field indexes. There are a few, for each different data type. This is the string one I'm joining here. They are all basically the same, with a PropertyName and value field. Hql allows you to add conditions to your join so we can use that to join with the relevant field index records. If you have a part called Group attached directly to your content type then it would be like this:
join fipr.StringFieldIndexRecord sfir
with sfir.PropertyName = 'MyContentType.Group.'
where sfir.Value = 'HR'
If your field is attached to a part, replace MyContentType with the name of your part. Hql is pretty awesome, can learn more here: https://docs.jboss.org/hibernate/orm/3.3/reference/en/html/queryhql.html But I dunno, it gave me a headache haha. At least HQL has documentation though, unlike Orchard's query layer. Also can always fall back to pure SQL when HQL wont do what you want, there is an option to write SQL queries from the NHibernate session.
Your other option is to index your content types with lucene (easy if you are using fields) then filter and search by that. I quite liked using that, although sometimes indexes are corrupted, or need to be rebuilt etc. So I've found it dangerous to rely on it for something that populates pages regularly.
And pretty much whatever you do, one query to filter and sort, then another query to getmany on the contentmanager to get the content items is what you should accept is the way to go. Good luck!
You can use indexing and the Orchard Search API for this. Sebastien demoed something similar to what you're trying to achieve at Orchard Harvest recently: https://www.youtube.com/watch?v=7v5qSR4g7E0

Store Umbraco Member Properties in Separate Table

I want to create a membership based site in Umbraco 7, following the umbraco.tv videos and reading through the docs have got me quite far.
My members will have custom properties, firstname, lastname, favourite colours, hats owned etc. I have been adding each of these as custom properties and then assigning them to the tab I want. This works fine and I can then access them from code using:
Members.GetCurrentMember().GetProperty("lastname").Value.ToString();
When I looked in my database I noticed that each of these custom properties is a row in the cmsPropertyData table, linked to the cmsMember table by the nodeId column. Is there a way I can set all of this information to store in it's own table?
Ideally, I want each Member to have a one to many relationship with favourite colours, as well as one to many relationships with other tables; each member might have 100 hats for example. What is the best way for me to set this up? Shall I create custom tables in my Umbraco database for HatsOwned and FavouriteColours, then assign each Member a unique ID so I can set my foreign keys up correctly? That way I would only need to store the Members Unique Id in the cmsPropertyTable. Is there a better way to let Umbraco deal with it? Would I have difficulty retrieving Members using either the Umbraco orm, or EF?
Any help or pointers greatly appreciated!
I would store all data in the PROFILE of the member, in the umbraco membership. E.g. timezone, hair color, ... This makes sense for other developers to find back the data.
For all other data, you have a few options:
Relationships
If you want to link nodes to members, or nodes to nodes, or... Relations link 2 umbraco entities and can be one way or two way. If you have a color node, you can link all members to this node. Just create a "favoriteColor" relationship on the developer section, linking up nodes to members. Do some programming and you are done. Don't forget that a relation is a database record linking 2 umbraco entities. So think of some caching if you use this in your front end to take off some database load. Read more on the Relationship Api in the umbraco documentation.
Content
It's pretty easy to create new nodes using code to store e.g. comments on an article. Because you are republishing the xml cache every time you create (and publish) a node, don't use content nodes for stroring your data if you have a lot of updates.
External data
It is perfectly legit to store data outside of umbraco. Just create your own tables (or content to any service you created). You could use every ORM you want to, but I would recommend PetaPoco. The reason is obvious. Umbraco uses it also. And it will make you a better Umbraco developer. There is a detailed post on stackoverflow on how to work with external data in umbraco.

Resources