I am new to solr.I am trying to add data to solr .I have class Education with properties and getter/setter methods. I am trying to add field to solr List. What should be field type in schema.xml ?
<field name="education" type="string" indexed="true" stored="true" multiValued="true" />
Related
I indexed 2M+ documents and when I request the last result order by name, the response is very slow (18s).
Is there a wy to optimize the sorting ?
My request :
http://localhost:8983/solr/items/select?&q=*:*&start=2274000&rows=100&sort=name_sort asc&fl=id,name
Below is the field definition in schema.xml file
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<!-- docValues are enabled by default for long type so we don't need to index the version field -->
<field name="_version_" type="plong" indexed="false" stored="false"/>
<field name="name" type="text_general" indexed="true" stored="true" multiValued="false" />
<field name="name_sort" type="string" indexed="true" stored="false" multiValued="false" />
I am trying to implement a solr search for a project. Everything was fine so far, a first simple version worked. Now I try to import from a postgres data base where multiple columns should end up in the same field. My config:
<entity name="address" query="SELECT objectid, ags2, ags3, ags5, ags8, ags11, ags20, ags22, pt, stn, hnr_min, hnr_max, plz, ort, ortz, ot1, ot2 FROM variablen2018.ags22_tmp_solr LIMIT 10000;">
<field column="objectid" name="id" />
<field column="plz" name="plz" />
<field column="ort" name="ort" />
<field column="ortz" name="ort" />
<field column="ot1" name="ort" />
<field column="ot2" name="ort" />
<field column="ort" name="ort_res" />
<field column="stn" name="stn" />
<field column="stn" name="stn_res" />
<field column="ags2" name="ags2" />
<field column="ags3" name="ags3" />
<field column="ags5" name="ags5" />
<field column="ags8" name="ags8" />
<field column="ags11" name="ags11" />
<field column="ags20" name="ags20" />
<field column="ags22" name="ags22" />
<field column="pt" name="coord" />
<field column="hnr_min" name="hnr_min" />
<field column="hnr_max" name="hnr_max" />
</entity>
As you can see there are 4 columns from the DB (ort, ortz, ot1, ot2) going into one field (ort). Most of the times only one of the columns is populated at all, in which case the document is indexed normally. But when there are actually multiple entries the indexing of the document fails. The field is defined this way:
<field name="ort" type="text_de" uninvertible="true" indexed="true" required="true" stored="true"/>
DataImporthandler maps the result view of the query to a schema view and hence I don't think that you will be able to map multiple columns to one field. Instead you can assign each column to a new Solr field and then do a copy of them in your schema.
eg
<field name="ort" type="string" />
<field name="ortz" type="string" />
<field name="ot1" type="string" />
<field name="ot2" type="string" />
<field name="ortCombined" type="string" multiValued="true"/>
<copyField source="ort" dest="ortCombined" />
<copyField source="ortz" dest="ortCombined" />
<copyField source="ot1" dest="ortCombined" />
<copyField source="ot2" dest="ortCombined" />
Hope this helps !
you do it this way:
you concatenate all values into a single value in the Select:
select ...,ort||','||ortz||','||or1||','||ort2 AS ort_all FROM variablen2018.ags22_tmp_solr
and then split it into individual values when indexing into solr (this is done with RegexTransformer/splitBy)
< entity name="address" transformer="RegexTransformer"
...
< field column="ort_all" name="ort" splitBy=","/>
Note: inserted a space after < or the text does not show up here...
To watch out:
handle possible nulls, check concat_ws etc
handle possible , inside ort values (use another separator or replace , etc)
I'm trying to get Solr to sort by Title, but I'm having no luck.
In my Schema I have the "title" field as text_general for searching, and then a "title_sort" field as a string for sorting. I've created a copyField that should be taking the "title" text_general field and putting it into the "title_sort" field as a string.
<fields>
<field name="title" type="text_general" indexed="true" stored="true"/>
<field name="title_sort" type="string" indexed="true" stored="false" />
</fields>
<copyField source="title" dest="title_sort" />
When I run the sort query "title_sort desc" this is what I get back
title: Don’t Mind If I Do
title: Men Don't Run Marathons
title: Danny
Can a copyField not convert a text_general field into a string?
Solved with the help of #MatsLindh.
I'd been reloading the schema in the CoreAdmin panel, thinking that would make the changes. I had to reindex after adding the copyField instruction, as Solr will not go through all the documents and update secondary fields.
I am stuck with ClobTransformer in solr from the past 3 days. I want to convert an oracle clob field to text field in solr. I am using multiple cores and I started my config and schema files from scratch.
This is my config file:
<lib dir="../../../dist/" regex="apache-solr-dataimporthandler-.*\.jar" />
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</requestHandler>
These are the columns in my schema file for a core:
<field name="id" type="string" indexed="true" stored="true" multiValued="false" required="true"/>
<field name="mandp" type="text_en_splitting" indexed="true" stored="true" multiValued="false" />
This is my data-config.xml for the core:
<dataConfig>
<dataSource type="JdbcDataSource"
driver="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:thin:#***"
user="***"
password="****"/>
<document>
<entity name="wiki" transformer="ClobTransformer"
query="Select t.id as id, t.mandp From table1 t">
<field column="mandp" name="mandp" clob="true" />
</entity>
</document>
</dataConfig>
When I start solr, I can see that dataimporthandler*.jar files have loaded successfully in the console. When I run my dataimport from http://localhost:8983/solr/wiki/dataimport?command=full-import&clean=false, I don't see any errors in the console neither do I see anything related to transformer or clob. So, If I type anything in my transformer parameter (transformer="bla bla bla"), it doesn't throw any errors in the console, that could mean my transformer argument is completely ignored or the full logging is turned off.
When I query solr, I see oracle.sql.CLOB#375c929a in the mandp field. Nothing happens of course if I use HTMLStripTransformer class too. I want to use both on this field.
Any ideas are appreciated!!!
It looks like the ClobTransformer is not fired. I would personally change the mandp column name inside the query like this:
Select t.id as id, t.mandp as mandp From table1 t
please add transformer="ClobTransformer, RegexTransformer" to the entity in your data-config.xml file
I'm trying to implement faceting on product catelogue app with Solr, SolrNet and its built with asp.net MCV 3. So far I managed to list all the product results but not the faceting. I could print the facets as shown below.
<ul>
#foreach (var facet in Model.Products.FacetFields["brand"])
{
<li>#facet.Key (#facet.Value)</li>
}
</ul>
I have two issues with above code,
1) If the Search results doesn't contain facets for brand its throwing this error
The given key was not present in the dictionary.
System.Collections.Generic.KeyNotFoundException:
The given key was not present in the
dictionary. at
System.Collections.Generic.Dictionary
2) I need to show facets keys and values as links. So on click of that facet I should be able to list the products of the facet.
Here is the schema.xml, please help me if you know the answers for the above questions.
<field name="product_id" type="string" indexed="true" stored="true" required="true" />
<field name="name" type="string" indexed="true" stored="true"/>
<field name="merchant" type="string" indexed="true" stored="true"/>
<field name="merchant_id" type="string" indexed="true" stored="true"/>
<field name="brand" type="string" indexed="true" stored="true"/>
<field name="brand_id" type="string" indexed="true" stored="true"/>
<field name="categories" type="string" multiValued="true" indexed="true" stored="true"/>
1) If the Search results doesn't contain facets for brand its throwing this error The given key was not present in the dictionary.
If you're not doing a facet field query on that field, just don't ask for it in the results.
2) I need to show facets keys and values as links. So on click of that facet I should be able to list the products of the facet.
Basically you need to convert the clicked facet value into a filter query.
There are many ways to implement this depending on your specific application needs. See the SolrNet sample app for one way to do it, use its source code as guidance.