fetching JSON data from a table without model using JPA native query - spring-boot

I have a table something like this
TableA :
pid | data
ac1e |{ "cId":"ac-ga-55" ,"name":"omar","dob":"20/09/1999","age":"23","occupation":"SDE"}
ns7f |{ "cId":"as-op-00","name":"raj","dob":"20/06/1999","age":"23","occupation":"SDE II"}
I am trying to fetch name and age from this table using a JPA native query like this
#Query(value = "select data->>'name' as Name,data->>'age' as Age from TableA where data->>'cId'=:processId", nativeQuery = true)
String findData(#Param("processId") String processId);
what i ultimately want is a query in JPA which will return to me a string or a jsonObject which looks like this {"Name" : "","Age": } which i will add into a JsonArray.
But i am not getting any result from this query and my code is breaking and going into the catch block.
NOTE : This table does not have a model class it just exist in database
MyActualQuery :
#Query(value = "select json_build_object (
'processInstanceId' , data ->> 'processInstanceId' ,
'lead_id' , data ->> 'leadId_lap' ,
'loanApplicationNumber' , data ->> 'loanApplicationNumber',
'constitution' , data ->> 'applicant1EntityType_ch',
'coborrower_pan_name' , data ->> 'applicant2FullNamePan',
'lead_generation_date_and_time_timestamp' , data ->> 'leadGenerationDate_lap' ,
'loan_moved_to_sent_for_disbursal_stage_date_and_time_timestamp' , data ->> 'loanOnBoardingTimestamp' ,
'pan_number' , CASE when data ->> 'applicant1EntityPan_lap' IS null THEN data ->> 'applicant1Pan_lap' ELSE data ->> 'applicant1EntityPan_lap' END,
'co_app_pan' , data ->> 'applicant2EntityPan_lap',
'date_of_incorporation' , data ->> 'applicant1DateofIncorpProp_lap' ,
'dob' , data ->> 'applicant1DateOfBirth_lap',
'coborrower_date_of_incorporation' , data ->> 'applicant2DateofIncorpProp_lap' ,
'co_app_dob' , data ->> 'applicant2DateOfBirth_lap' ,
'mobile_number' , data ->> 'applicant1MobileNumber_lap',
'co_app_mobileno' , data ->> 'applicant2MobileNumber_lap',
'loan_type' ,data ->> 'applicantloantype_ch' ,
'requested_amount_of_loan' , data ->> 'requestedLoanAmount_lap',
'sanctioned_loan_amount' , data ->> 'sanctionedLoanAmount' ,
'tenure' , data ->> 'requestedTenure_lap' ,
'sanctioned_roi' , data ->> 'sanctionedRoi' ,
'calculated_emi' , data ->> 'calculatedEmi' ,
'cibil_score' , data ->> 'applicant1CibilScore_lap' ,
'coborrower_bureau_score' , data ->> 'applicant2bureauScore_lap' ,
'case_stage' , data ->> 'leadStatus_lap'
) from kul_flat_vars where data->>'processInstanceId'=:processInstanceId"
,nativeQuery = true)
JSONObject findData(#Param("processInstanceId") String processInstanceId);
StackTrace :
org.springframework.orm.jpa.JpaSystemException: No Dialect mapping for JDBC type: 1111; nested exception is org.hibernate.MappingException: No Dialect mapping for JDBC type: 1111
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:331)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:233)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:551)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:152)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke

I have resolved this error by simply changing the return type of native query from JsonObject to String.

Related

Complex queries on JPA

Can i use complex queries on jpa #Query Annotations like these
#Query(value = "select json_build_object (
" 'sanctioned_roi' , data ->> 'sanctionedRoi' ," +
" 'calculated_emi' , data ->> 'calculatedEmi' ," +
" 'cibil_score' , data ->> 'CibilScore' ," +
" 'coborrower_bureau_score' , data ->> 'bureauScore' ," +
" 'case_stage' , data ->> 'leadStatus' " +
") from table where data->>'processInstanceId'=:processInstanceId"
,nativeQuery = true)
JSONObject findData(#Param("processInstanceId") String processInstanceId);

Join relation query for an Excel File

Hello i try to display some values from my Eloquent model "Licencies" who are in others tables in relation with an id but i get
Integrity constraint violation: 1052 Column 'id' in on clause is
ambiguous
I'm not sure to make the join method correctly , someone could help me ?
Here my query :
$licencies = Licencies::where('lb_assurance' , '=' , 'Lafont')
->leftJoin('activite_licencie' , 'id' , '=' , 'licencie.activite_licencie_id')
->leftJoin('saisons' , 'id' , '=' , 'licencie.saison_id')
->leftJoin('pays' , 'id' , '=' , 'licencie.pays_naissance_id')
->leftJoin('type_licence' , 'id' , '=' , 'licencie.type_licence_id')
->leftJoin('structures' , 'id' , '=' , 'licencie.structure.id')
->select('num_licence' , 'lb_nom' , 'lb_prenom' , 'dt_naissance' , 'lb_surclassement' , 'lb_assurance' , 'cd_dept_naissance' , 'activite_licencie.lb_activite' , 'saisons.lb_saison' , 'pays.fr' , 'type_licence.lb_type' , 'structures.nom_structure' )->get();
There are column name id in all the related tables so mysql does not know which id you are referring to. To resolve this issue prepend the ids with their respective table names. eg. 'structures.id'
->leftJoin('structures' , 'structures.id' , '=' , 'licencie.structure.id')

I have Oracle Join query that picks data very slow

I have an Oracle join query that picks data very slow. It is like 1000 rows for 7 mins. Please could you help in writing the code in a different way so the data is pulled faster. The next steps for it is using the Select values and dumping the data into MySQL table. I am using Pentaho tool here. Thanks
select
null id,
ss.ILOAN_CODE ,
ss.INST_NUM ,
ss.INST_AMT ,
ss.INST_PRINCIPAL ,
ss.INST_INTEREST ,
ss.BALANCE_PRINCIPAL ,
ss.INST_DUE_DATE ,
ss.PAID_FLAG ,
ss.LATE_FEE ,
ss.PAYMENT_DATE ,
ss.INST_AMT_PAID ,
ss.INST_AMT_DUE ,
ss.REV_CHECK_NUM ,
ss.REV_CHECK_AMT ,
ss.CREATED_BY ,
ss.DATE_CREATED ,
ss.UPDATED_BY ,
ss.DATE_UPDATED ,
ss.INST_DAYS ,
ss.MATURED_INTEREST ,
ss.UNPAID_INTEREST ,
ss.ADJ_INST_PRINCIPAL ,
ss.ADJ_INST_AMT ,
ss.ADJ_INST_INTEREST ,
ss.ADJ_BALANCE_PRINCIPAL ,
ss.ADJ_MATURED_INTEREST ,
ss.ADJ_UNPAID_INTEREST ,
ss.IS_PRINTED ,
ss.RTN_FEE_AMT ,
ss.WAIVE_FEE_AMT ,
ss.LATE_FEE_AMT ,
ss.APR_BALANCE_PRINCIPAL ,
ss.ACHDEPOSIT_DATE ,
ss.ACHRETURN_DATE ,
ss.ACHCLEAR_DATE ,
ss.APR_INST_INTEREST ,
ss.APR_UNPAID_INTEREST ,
ss.CSO_FEE ,
ss.MATURED_CSO_FEE ,
ss.UNPAID_CSO_FEE ,
ss.CSO_FEE_BALANCE
from ST_IL_SCHEDULE ss,
ST_IL_MASTER sm,
BO_MASTER bm
where sm.iloan_code = ss.iloan_code
and sm.bo_code = bm.bo_code
and ss.ILOAN_CODE in (select distinct loan_Number from SVP_LOAN_MASTER_INVENTORY)
and ss.ILOAN_CODE in (select distinct loan_Number from SVP_LOAN_MASTER_INVENTORY)
This is candidate for being slow. You don't need distinct here and also please use explicit join for readability.
Try:
Select
null id,
ss.ILOAN_CODE ,
ss.INST_NUM ,
ss.INST_AMT ,
ss.INST_PRINCIPAL ,
ss.INST_INTEREST ,
ss.BALANCE_PRINCIPAL ,
ss.INST_DUE_DATE ,
ss.PAID_FLAG ,
ss.LATE_FEE ,
ss.PAYMENT_DATE ,
ss.INST_AMT_PAID ,
ss.INST_AMT_DUE ,
ss.REV_CHECK_NUM ,
ss.REV_CHECK_AMT ,
ss.CREATED_BY ,
ss.DATE_CREATED ,
ss.UPDATED_BY ,
ss.DATE_UPDATED ,
ss.INST_DAYS ,
ss.MATURED_INTEREST ,
ss.UNPAID_INTEREST ,
ss.ADJ_INST_PRINCIPAL ,
ss.ADJ_INST_AMT ,
ss.ADJ_INST_INTEREST ,
ss.ADJ_BALANCE_PRINCIPAL ,
ss.ADJ_MATURED_INTEREST ,
ss.ADJ_UNPAID_INTEREST ,
ss.IS_PRINTED ,
ss.RTN_FEE_AMT ,
ss.WAIVE_FEE_AMT ,
ss.LATE_FEE_AMT ,
ss.APR_BALANCE_PRINCIPAL ,
ss.ACHDEPOSIT_DATE ,
ss.ACHRETURN_DATE ,
ss.ACHCLEAR_DATE ,
ss.APR_INST_INTEREST ,
ss.APR_UNPAID_INTEREST ,
ss.CSO_FEE ,
ss.MATURED_CSO_FEE ,
ss.UNPAID_CSO_FEE ,
ss.CSO_FEE_BALANCE
from ST_IL_SCHEDULE ss,
inner join ST_IL_MASTER sm on (sm.iloan_code = ss.iloan_code)
inner join BO_MASTER bm on (sm.bo_code = bm.bo_code)
inner join SVP_LOAN_MASTER_INVENTORY slm on (ss.loan_code = slm.loan number)
If that not helps please consider creating indexes on columns used in join.

Postgresql query is performing really bad

I have a huge issue with query performance at my postgresql database.
The version of postgresql is: "PostgreSQL 8.4.3 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20070115 (prerelease) (SUSE Linux), 64-bit"
I have config file set as follows:
shared_buffers = 8GB
effective_cache_size = 24GB
work_mem = 419430kB
maintenance_work_mem = 2GB
checkpoint_segments = 128
checkpoint_completion_target = 0.9
wal_buffers = 16MB
default_statistics_target = 500
constraint_exclusion = on
The query which I need to execute is:
SELECT events.evt_id, events.evt_time, events.device_evt_time , to_ip_char(events.sip) , evt_agent.port , events.rv40 , events.evt , events.msg , events.sun , events.rv35 , events.dun , events.rv45 , events.fn , events.dp , events.trgt_trust_name , events.trgt_trust_domain , events.rv36 , events.rv43 , events.cv21 , events.cv40 , events.cv41 , events.cv42 , events.cv43 , events.cv44 , events.cv50 , events.cv51 , events.cv52 , events.cv53 , events.cv54 , events.cv55 , events.cv56 , events.cv35 , events.cv60 , events.cv61 , events.cv62
FROM events, evt_agent
WHERE
events.agent_id = evt_agent.agent_id AND (evt_agent.port::text = ANY (ARRAY['x'::character varying, 'y'::character varying, 'z'::character varying]::text[]))
AND events.evt::text <> 'Internal Message'::text
AND event_time > '2015-12-31 13:23:55.767+00'::timestamptz limit 10;
ORDER BY events.evt_time;
Table events is a partitioned table, one partition for each day.
Each partition has two constrains:
Name events_p_YYYYMMDDHHMISS_events_p_max_pk
Columns evt_time, evt_id
Name events_p_YYYYMMDDHHMISS_dc
Definition evt_time > '2015-12-04 13:24:25.267973+00'::timestamp with time zone AND evt_time <= '2015-12-05 13:24:25.267973+00'::timestamp with time zone
and seven indexes:
Name events_p_YYYYMMDDHHMISS_events_p_max_identity_ix1
Columns evt_time, init_usr_identity_guid, rid02
Operator classes timestamptz_ops, uuid_ops, int8_ops
Name events_p_YYYYMMDDHHMISS_events_p_max_identity_ix2
Columns evt_time, trgt_usr_identity_guid, rid02
Operator classes timestamptz_ops, uuid_ops, int8_ops
Name events_p_YYYYMMDDHHMISS_events_p_max_ix1
Columns evt_time, sev, agent_id
Operator classes timestamptz_ops, int4_ops, int8_ops
Name events_p_YYYYMMDDHHMISS_events_p_max_ix2
Columns evt_time, dip, sev
Operator classes timestamptz_ops, int4_ops, int4_ops
Name events_p_YYYYMMDDHHMISS_events_p_max_ix3
Columns evt_time, res, sev
Operator classes timestamptz_ops, text_ops, int4_ops
Name events_p_YYYYMMDDHHMISS_events_p_max_ix4
Columns evt_time, sip, sev
Operator classes timestamptz_ops, int4_ops, int4_ops
Name events_p_YYYYMMDDHHMISS_events_p_max_ix5
Columns evt_time, txnmy_id, agent_id
Operator classes timestamptz_ops, int8_ops, int8_ops
Table evt_agent is a dictionary table with about 20-30 rows only.
constrains:
Name evt_agent_pk
Columns agent_id
indexes:
Name evt_agent_ak1
Columns agent, port, rn, pn, sn, st, device_ctgry, src_id, cust_id
Operator classes text_ops, text_ops, text_ops, text_ops, text_ops, text_ops, text_ops, uuid_ops, int8_ops
Name evt_agent_ix1
Columns device_ctgry, agent_id
Operator classes text_ops, int8_ops
When I execute the query I got this explain plan:
http://explain.depesz.com/s/3xcu
Name evt_agent_ix2
Columns st, agent_id
Operator classes text_ops, int8_ops
Name test_ev_ag_indx1
Columns agent_id
Operator classes int8_ops
I thought there is a problem with statistics so I "vaccuum analyze" all related tables, but no improvement in the query performance, explain plan.
I tried with "inner query" trick, explain plan is here:
http://explain.depesz.com/s/FAkH
From what I can understand it got kind of worse.
Do you have nay idea how to get better execution plan for this query?
Now it takes about 42 minutes to get any results from the query.
Thanks in advance!
We will have to try and eliminate the hash join. A lateral join might just be the answer:
SELECT events.evt_id, events.evt_time, events.device_evt_time , to_ip_char(events.sip) , evt_agent.port , events.rv40 , events.evt , events.msg , events.sun , events.rv35 , events.dun , events.rv45 , events.fn , events.dp , events.trgt_trust_name , events.trgt_trust_domain , events.rv36 , events.rv43 , events.cv21 , events.cv40 , events.cv41 , events.cv42 , events.cv43 , events.cv44 , events.cv50 , events.cv51 , events.cv52 , events.cv53 , events.cv54 , events.cv55 , events.cv56 , events.cv35 , events.cv60 , events.cv61 , events.cv62
FROM evt_agent,
LATERAL (SELECT *
FROM events AS e
WHERE
e.agent_id = evt_agent.agent_id
AND e.evt::text <> 'Internal Message'::text
AND e.event_time > '2015-12-31 13:23:55.767+00'::timestamptz
LIMIT 10000) AS events
WHERE (evt_agent.port::text = ANY (ARRAY['x'::character varying, 'y'::character varying, 'z'::character varying]::text[]))
ORDER BY events.evt_time
LIMIT 10000;
Please try it with and without the inner limit and post the explain plans for both.

If statement in Oracle

I need to create a query, that if a certain field is blank or null. I need to do a select statement to another table and retrieve the blank field . Could you please advise on a way to accomplish this. Below is the query. The field in question is BEAT.
SELECT COALESCE(ADDRESSES.BEAT,Incident_addresses.beat)
, COALESCE (ADDRESSES.SUB_BEAT,Incident_addresses.sub_beat)
, ADDRESSES.STREET_NAME
, ADDRESSES.STREET_NUMBER
, ADDRESSES.SUB_NUMBER
, WARRANT_PEOPLE_VW.LNAME
, WARRANT_PEOPLE_VW.FNAME
, WARRANT_PEOPLE_VW.DOB
, WARRANT_PEOPLE_VW.RACE_RACE_CODE
, WARRANT_PEOPLE_VW.SEX_SEX_CODE
, WARRANT_PEOPLE_VW.CASE_NUMBER
, E_WARRANTS.DATE_ISSUED
, E_WARRANTS.TELETYPE_NUMBER
, E_WARRANTS.ORDINANCE_VIOLATION
FROM EJSDBA.ADDRESSES
, POL_LEEAL.E_WARRANTS
, POL_LEEAL.WARRANT_PEOPLE_VW,incident_people,Incident_addresses
WHERE ADDRESSES.ADDRESS_ID =E_WARRANTS.ADDR_ADDRESS_ID
AND E_WARRANTS.WARRANT_ID = WARRANT_PEOPLE_VW.WARRANT_ID
AND WARRANT_PEOPLE_VW.NME_TYP_NAME_TYPE_CODE = 'P'
AND WARRANT_PEOPLE_VW.AGNCY_CD_AGENCY_CODE = 'MCPD'
AND WARRANT_PEOPLE_VW.WSC_CODE='A'
AND EJSDBA.ADDRESSES.ADDRESS_ID= Incident_addresses.ADDRESS_ID
and incident_people.inc_incident_id=Incident_addresses.incident_id
ORDER BY ADDRESSES.BEAT
, ADDRESSES.SUB_BEAT
, ADDRESSES.STREET_NAME
, ADDRESSES.STREET_NUMBER
;
You can embed a correlated subquery into a case expression, but you MUST reference the table inside the subquery to some values of the outer query so that the correct value can be located.
SELECT
COALESCE(ADDRESSES.BEAT, Incident_addresses.beat)
, CASE
WHEN Addresses.BEAT IS NULL THEN (
SELECT
Beat
FROM incidents inner_ref
WHERE ???outer??.incident_id = inner_ref.id
AND rownum = 1)
END AS x
, COALESCE(ADDRESSES.SUB_BEAT, Incident_addresses.sub_beat)
...
and that subquery MUST only return a single value (hence I have used "and rownum = 1".
(In Oracle 12 you could use FETCH FIRST 1 ROW ONLY)

Resources