Solr dataimport jdbc multiple columns into one field - jdbc

Question

I am trying to implement a solr search for a project. Everything was fine so far, a first simple version worked. Now I try to import from a postgres data base where multiple columns should end up in the same field. My config:
<entity name="address" query="SELECT objectid, ags2, ags3, ags5, ags8, ags11, ags20, ags22, pt, stn, hnr_min, hnr_max, plz, ort, ortz, ot1, ot2 FROM variablen2018.ags22_tmp_solr LIMIT 10000;">
<field column="objectid" name="id" />
<field column="plz" name="plz" />
<field column="ort" name="ort" />
<field column="ortz" name="ort" />
<field column="ot1" name="ort" />
<field column="ot2" name="ort" />
<field column="ort" name="ort_res" />
<field column="stn" name="stn" />
<field column="stn" name="stn_res" />
<field column="ags2" name="ags2" />
<field column="ags3" name="ags3" />
<field column="ags5" name="ags5" />
<field column="ags8" name="ags8" />
<field column="ags11" name="ags11" />
<field column="ags20" name="ags20" />
<field column="ags22" name="ags22" />
<field column="pt" name="coord" />
<field column="hnr_min" name="hnr_min" />
<field column="hnr_max" name="hnr_max" />
</entity>
As you can see there are 4 columns from the DB (ort, ortz, ot1, ot2) going into one field (ort). Most of the times only one of the columns is populated at all, in which case the document is indexed normally. But when there are actually multiple entries the indexing of the document fails. The field is defined this way:
<field name="ort" type="text_de" uninvertible="true" indexed="true" required="true" stored="true"/>

Answer 1

DataImporthandler maps the result view of the query to a schema view and hence I don't think that you will be able to map multiple columns to one field. Instead you can assign each column to a new Solr field and then do a copy of them in your schema.
eg
<field name="ort" type="string" />
<field name="ortz" type="string" />
<field name="ot1" type="string" />
<field name="ot2" type="string" />
<field name="ortCombined" type="string" multiValued="true"/>
<copyField source="ort" dest="ortCombined" />
<copyField source="ortz" dest="ortCombined" />
<copyField source="ot1" dest="ortCombined" />
<copyField source="ot2" dest="ortCombined" />
Hope this helps !

Answer 2

you do it this way:
you concatenate all values into a single value in the Select:
select ...,ort||','||ortz||','||or1||','||ort2 AS ort_all FROM variablen2018.ags22_tmp_solr
and then split it into individual values when indexing into solr (this is done with RegexTransformer/splitBy)
< entity name="address" transformer="RegexTransformer"
...
< field column="ort_all" name="ort" splitBy=","/>
Note: inserted a space after < or the text does not show up here...
To watch out:
handle possible nulls, check concat_ws etc
handle possible , inside ort values (use another separator or replace , etc)