Long running Informatica Job - oracle

I am running a job where in source, i have oracle sql query to read the rows, followed by sorter transformation. Below are session logs where we can see after the start of sorter transformation [srt] it takes more than 3 hours to process 410516 rows. I am failing to understand if sorter is taking time or source query?
Appreciate your response.
READER_1_1_1> RR_4049 [2022-06-26 23:40:13.888] SQL Query issued to database : (Sun Jun 26 23:40:13 2022)
READER_1_1_1> RR_4050 [2022-06-26 23:49:01.147] First row returned from database to reader : (Sun Jun 26 23:49:01 2022)
TRANSF_1_1_1> SORT_40420 [2022-06-26 23:49:01.148] Start of input for Transformation [srt]. : (Sun Jun 26 23:49:01 2022)
READER_1_1_1> BLKR_16019 [2022-06-27 03:04:51.901] Read [410516] rows, read [0] error rows for source table [DUMMY_src] instance name [DUMMY_src]
READER_1_1_1> BLKR_16008 [2022-06-27 03:04:51.902] Reader run completed.
TRANSF_1_1_1> SORT_40421 [2022-06-27 03:04:51.909] End of input for Transformation [srt]. : (Mon Jun 27 03:04:51 2022)
TRANSF_1_1_1> SORT_40422 [2022-06-27 03:04:52.180] End of output from Sorter Transformation [srt]. Processed 410516 rows (6568256 input bytes; 0 temp I/O bytes). : (Mon Jun 27 03:04:52 2022)
TRANSF_1_1_1> SORT_40423 [2022-06-27 03:04:52.181] End of sort for Sorter Transformation [srt]. : (Mon Jun 27 03:04:52 2022)
WRITER_1_*_1> WRT_8167 [2022-06-27 03:04:52.201] Start loading table
WRT_8035 [2022-06-27 03:09:47.457] Load complete time: Mon Jun 27 03:09:47 2022

Related

Repository creation fails while upgrading OWB11gR1(11.1.0.7) to OWB11gR2(11.2.0.4)

I need to a new workspace in OWB11gR2(11.2.0.4) to upgrade OWB11gR1(11.1.0.7). Repository Assistant fails after processing 64%. The following is the error log.
main.TaskScheduler timer[5]20200714#08:45:58.058: 00> oracle.wh.service.impl.assistant.ProcessEngine.display(ProcessEngine.java:2122): % = 0.8051529790660225
main.TaskScheduler timer[5]20200714#08:45:58.058: 00> oracle.wh.service.impl.assistant.ProcessEngine.display(ProcessEngine.java:2122): -token name = LOADJAVA; -token type = 13
main.TaskScheduler timer[5]20200714#08:45:58.058: 00> oracle.wh.service.impl.assistant.ProcessEngine.display(ProcessEngine.java:2122): ProcessEngine.token_db_min_ver =
main.TaskScheduler timer[5]20200714#08:45:58.058: 00> oracle.wh.service.impl.assistant.ProcessEngine.display(ProcessEngine.java:2122): Before processing LOADJAVA Token
main.TaskScheduler timer[5]20200714#08:45:58.058: 00> oracle.wh.service.impl.assistant.ProcessEngine.display(ProcessEngine.java:2122): ... I am in processLoadJavaToken ...
main.AWT-EventQueue-0[6]20200714#08:48:36.036: 00> oracle.wh.ui.jcommon.WhButton#424c414: WhButton setLabel rtsString = Yes
main.AWT-EventQueue-0[6]20200714#08:48:36.036: 00> oracle.wh.ui.jcommon.WhButton#424c414: WhButton setLabel rtsString = No
The following is the list of database patches.
Patch 17906774: applied on Wed Aug 04 11:21:52 BDT 2021
Unique Patch ID: 17692968
Created on 14 May 2014, 22:56:54 hrs PST8PDT
Bugs fixed:
17607032, 17974168, 17669786, 17561509, 16885825, 18274560, 17613052
17461930, 16829998, 17251918, 17435868, 17279666, 17328020, 17006987
18260620, 16833468, 18180599, 17292119, 17340242, 17296559, 15990966
17438322, 17939651, 17359696, 18385759, 17820353, 17939225, 17715818
18192446, 16960088, 17191248, 17422695
Patch 31668908 : applied on Mon Jul 12 16:13:02 BDT 2021
Unique Patch ID: 23822194
Patch description: "OJVM PATCH SET UPDATE 11.2.0.4.201020"
Created on 18 Sep 2020, 03:30:45 hrs PST8PDT
Bugs fixed:
23727132, 19554117, 19006757, 14774730, 18933818, 18458318, 18166577
19231857, 19153980, 19058059, 19007266, 17285560, 17201047, 17056813
19223010, 19852360, 19909862, 19895326, 19374518, 20408829, 21047766
21566944, 19176885, 17804361, 17528315, 21811517, 22253904, 19187988
21911849, 22118835, 22670385, 23265914, 22675136, 24448240, 25067795
24534298, 25076732, 25494379, 26023002, 19699946, 26637592, 27000663
25649873, 27461842, 27952577, 27642235, 28502128, 28915933, 29254615
29774367, 29992392, 29448234, 30160639, 30534664, 30855121, 31306274
30772207, 31476032, 30561292, 28394726, 26716835, 24817447, 23082876
31668867
Patch 31537677 : applied on Thu Jul 08 11:53:10 BDT 2021
Unique Patch ID: 23852314
Patch description: "Database Patch Set Update : 11.2.0.4.201020 (31537677)"
The following is the workaround to fix the issue.
Step 1: Rollback the patches 31668908 and 31537677. OWB does not support with OJVM patch newer than December 2018
Step 2:Re-run the repository assistant.

FORTRAN : maxloc

my data looks like
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
22.60 24.60 30.60 34.60 36.20 35.70 32.10 30.20 31.40 31.60 28.00 24.80
25.40 27.60 32.40 34.60 36.50 38.10 31.70 31.40 30.30 30.20 27.00 23.90
and there are like hundreds of rows! I want to find a maximum value in each row and write it in different column next to data along with month
so my out put will be
36.20 MAY
38.10 JUN
.
.
I want to use maxloc function, but i have no idea how to use it!
Try
index = maxloc(myTable(3,:))
print *, myTable((/1,3/), index)
It should select the highest value from the third row and display the first and third value at this index.

How do I create a cron expression running in Kibana on weekday?

I would like my watcher to run from Monday to Friday only. So I'm trying to use this schedule:
"trigger": {
"schedule" : { "cron" : "0 0 0/4 * * MON-FRI" }
},
"input": {
...
However, I'm getting
Error
Watcher: [parse_exception] could not parse [cron] schedule
when I'm trying to save the watcher. Removing MON-FRI does helps but I need it.
This expression works:
0 0 0/4 ? * MON-FRI
But I'm not sure I understand why ? is required for either the day_of_week or day_of_month
Thank you!
I believe this is what you are looking for:
"0 0 0/4 ? * MON-FRI"
You can use croneval to check your cron expressions 1:
$ /usr/share/elasticsearch/bin/x-pack/croneval "0 0 0/4 ? * MON-FRI"
Valid!
Now is [Mon, 20 Aug 2018 13:32:26]
Here are the next 10 times this cron expression will trigger:
1. Mon, 20 Aug 2018 09:00:00
2. Mon, 20 Aug 2018 13:00:00
3. Mon, 20 Aug 2018 17:00:00
4. Mon, 20 Aug 2018 21:00:00
5. Tue, 21 Aug 2018 01:00:00
6. Tue, 21 Aug 2018 05:00:00
7. Tue, 21 Aug 2018 09:00:00
8. Tue, 21 Aug 2018 13:00:00
9. Tue, 21 Aug 2018 17:00:00
10. Tue, 21 Aug 2018 21:00:00
For the first expression you'll get following java exception:
java.lang.IllegalArgumentException: support for specifying both a day-of-week AND a day-of-month parameter is not implemented.
You can also use Crontab guru to get human readable descriptions like:
At every minute past every 4th hour from 0 through 23 on every day-of-week from Monday through Friday.
The question mark means 'No Specific value'. From the documentation on Quartz's website:
? (“no specific value”) - useful when you need to specify something in one of the two fields in which the character is allowed, but not the other. For example, if I want my trigger to fire on a particular day of the month (say, the 10th), but don’t care what day of the week that happens to be, I would put “10” in the day-of-month field, and “?” in the day-of-week field. See the examples below for clarification.
http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html
I suppose since you want your schedule to run every 4 hours, mon-fri, the actual day of the month is irrelevant, so the ? specifies that. * on teh other hand would be 'all values' which would not make sense since you are specifying only mon-fri for day of the week.
Hope that helps!

Sort a hash with key as month date with many values for the same month

I create a hash with months as keys and timelaps as values
biens_delai[bien_date.mon] = b.delai
I get this result without month parsing
{Wed, 18 Jan 2017=>3.0, Sat, 25 Feb 2017=>2.0, Fri, 17 Mar 2017=>3.0, Sat, 25 Mar 2017=>5.0, Tue, 18 Apr 2017=>2.0, Thu, 29 Jun 2017=>2.0}
In March i have 2 values but when i parse by month i get the most high value and i want a addition of 2 values for March not the most high
{1=>3.0, 2=>2.0, 3=>5.0, 4=>2.0, 6=>2.0}
That's not the high value which you are getting, the values are getting overwritten, try the following
biens_delai[bien_date.mon] = biens_delai[bien_date.mon].to_f + b.delai

GLPK MathProg sets and groups

I hope this is obvious to someone. I have only had a vanilla use of GLPK/MathProg.
I am having trouble figuring out the syntax in GNU MathProg (within GLPK) to do the following, for example:
set PartsOfWeek;
set WeekDays;
data;
set PartsOfWeek := WorkWeek WeekEnd;
set WorkWeek := Mon Tue Wed Thu Fri;
set WeekEnd := Sat Sun;
set WeekDays := setof{d in (WorkWeek union WeekEnd)}(d);
The problem is that this is rejected by MathProg.
In general, I just want to be able to:
- declare a Partition (here PartsOfWeek) and a set (here Weekdays)
- build the partition from data
- populate the set with the elements of the of the sets from the partition.
A better example might be with seasons and months.
with #ALi's literature reference help:
set seasons;
set months;
set monthsOfseason {seasons} within months;
data;
set seasons := winter spring summer fall;
set months := jan feb mar apr may jun jul aug sep oct nov dec;
set monthsOfseason[winter] := dec jan feb;
set monthsOfseason[spring] := mar apr may;
set monthsOfseason[summer] := jun jul aug;
set monthsOfseason[fall] := sep oct nov;

Resources