NIFI - How to insert distinct data from the flow and refer to that data ID in other places - apache-nifi

I'm trying to learn NIFI so this is all new to me, I used to work with Talend and I have hard time translating to NIFI. So the main idea: For example is I have two tables in Postgresql
Table CITY :
ID (auto generated), city_name
Table PERSON :
ID (auto generated), first_name, last_name, city_id
and I have a CSV file :
first_name, last_name, city_name
Can you please explain how I can insert in tow tables from one flowfile and refer in the table PERSON to the ID of the city not the name from the table CITY.
Thank you

you could use LookupRecord to enrich each record with city id and split input in two files: matched/unmatched.
for matched you have to execute simple insert into PERSON table - because city id was found.
for unmatched you have to generate insert/upsert into CITY table and then route all those records to lookup record again.
or you could insert everything as is into temp table with structure that matches your CSV.
and then execute 2 simple sql statements:
populate missing cities from temp table
insert into person from temp table with lookup city id on the level of SQL server

Related

Hibernate - How to get single value from another table without primary/Foreign key relationship?

I have 2 tables(Employee & Department).
Employee(ID, employee name, dept id, employee address) --> ID primary key
Department(ID, department name, address) --> ID primary key
Note: Consider no primary key / foreign key relationship between these 2 tables
Now based on dept id in Employee, i need to fetch its corresponding dept name from Department & display it in Employee.
Employee(ID, employee name, dept id(Display dept name instead of dept id, query department table to get dept name through passing dept id), employee address)
What are all possible ways to achieve this?
Regards
Raj
I see at least three possible solutions:
Create a "normal" SQL query for this, joins can be perfectly done without a PK/FK-relationship.
Perform two SQL queries: One query for fetching the employee, then extract the ID and then a second query to fetch the department by ID.
Change your JPA entities to include the relationship as you would normally do having a #OneToMany relationship so you can load the employee with its department as usual. Same as 1 applies, joins (whether done by you or by the JPA provider) don't need a PK/FK-relationship.
Whilst 1 can be done with plain SQL, 2 can be done using the entity manager without the need to write "plain" SQL. 3 would be the easiest solution, obviously.
You can #Query in repository and write your JPA query using JOIN.
Sample Query:
#Query( value = "select new map(employee.name as empName, employee.id as id, department.id as depId, department.name as depName) from Employee employee INNER JOIN Department department ON employee.depId = department.id where employee.id = :id" )
public Map<String,Object> getEmployeeById(long id);

How to get distinct single column value while fetching data by joining two tables

$data['establishments2'] = Establishments::Join("establishment_categories",'establishment_categories.establishment_id','=','establishments.id')->where('establishments.city','LIKE',$location)->where('establishments.status',0)->whereIn('establishment_id',array($est_data))->get(array('establishments.*'));
this is controller condition.
I have two tables, in table1 i am matching id with table2 and then fetching data from table1, and in table2 i have multiple values of same id in table 1. i want to get data of table1 values only one time, but as i am hvg multiple data of same id in table2 , data is repeating multiple times, can anyone please tell me how to get data only one time wheater table2 having single value or multiple value of same id... thank you
you can do it by selecting the field name you desired
instead get all field from table establishments
$data['establishments2'] = Establishments::Join("establishment_categories",'establishment_categories.establishment_id','=','establishments.id')->where('establishments.city','LIKE',$location)->where('establishments.status',0)->whereIn('establishment_id',array($est_data))->get(array('establishments.*'));
you can select specific field from table establishments like
$data['establishments2'] = Establishments::Join("establishment_categories",'establishment_categories.establishment_id','=','establishments.id')->where('establishments.city','LIKE',$location)->where('establishments.status',0)->whereIn('establishment_id',array($est_data))->get('establishments.fieldName');
or you can also do
$data['establishments2'] = `Establishments::Join("establishment_categories",'establishment_categories.establishment_id','=','establishments.id')->where('establishments.city','LIKE',$location)->where('establishments.status',0)->whereIn('establishment_id',array($est_data))->select('establishments.fieldName')->get();`

how to join two tables using non primary key and remove duplicates in the result set

I am using MS Access 2013 DB
I have two tables
Table1:
StartDate,EndDate, ID1, ID2,ProgramName, LanguageID,Language, Gender,CenterName,ZoneName
Table2
StartDate,EndDate, ID3,ProgramName, LanguageID,Language, Gender,CenterName,ZoneName
I want to join these two tables and remove duplicates by comparing the following columns from both tables
StartDate,EndDate,ProgramName, LanguageID,Language, Gender,CenterName,ZoneName
some data in the columns StartDate, EndDate have null values also. The resultant table should contain the following columns with no duplicate data
StartDate,EndDate, ID1, ID2,ID3,ProgramName, LanguageID,Language, Gender,CenterName,ZoneName
First you want create new table (other structer)
Second insert every record to new table all record where value from table1 and table2 are diffrent (without duplicate). Remember about check

sql statement to select and insert one value from one table rest from outside the tables

I have two db2 tables, table one contains rows of constant data with id as one of the columns. second table contains id column which is foreign key to id column in first table and has 3 more varchar columns.
I am trying to insert rows into second table, where entry into id col is based on a where clause and the remaining columns get values from outside.
table1 has columns id, t1c1, t1c2, t1c3
table2 has columns id, t2c1, t2c2, t2c3
to insert into table2, I am trying this query:
insert into table2 values
(select id from table1 where t1c2 like 'xxx', 'abc1','abc2');
I know it is something basic I am missing here.
Please help correcting this query.

SSAS - Creating named calculation from different tables

I am having troubles when i want to create a named calculation from two different tables.
I have the table "CallesDim" with an id(PK) and a description and the table "UbicacionesDim" with an id (PK), another id (FK to "CallesDim") and a description:
--
CallesDim
id PK
Descripcion VARCHAR
--
UbicacionesDim
id PK
CalleId FK to id from CallesDIM
Altura INT
--
I want to concatenate "Descripcion" from "CallesDim" with Altura from "UbicacionesDim".
I try doing this:
CallesDim.Descripcion + ' ' + CONVERT(VARCHAR,UbicacionesDim.Altura)
but i am having the following error:
the multi-part identifier "CallesDim.Descripcion" could not be bound
Any ideas?
Thanks!
In a named calculation you can only access columns from the table that it is defined on.
Which record of the other table should it take in case it would accept columns from other tables? How should it join? All this cannot be configured.
If you need to join two (or more) tables, you can define a named query that can contain joins and access as many tables as you like. A named query can contain everything that you can state in a single select statement.

Resources