Invalid memory alloc request size 1610613056 in Greenplum 4.3.14.0 - greenplum

I have table "test" and I create it with this query:
CREATE TABLE test
(
id bigint,
str1 timestamp without time zone,
str2 text,
str3 text
);
After create table I added data:
INSERT INTO test VALUES (1, '2017-08-29 10:51:40.190913', 'gfsdfg1', 'sfgsdhgy1');
INSERT INTO test VALUES (2, '2016-08-29 10:51:40.190913', 'gfsdfg2', 'sfgsdhgy2');
INSERT INTO test VALUES (3, '2015-08-29 10:51:40.190913', 'gfsdfg3', 'sfgsdhgy3');
INSERT INTO test VALUES (4, '2014-08-29 10:51:40.190913', 'gfsdfg4', 'sfgsdhgy4');
INSERT INTO test VALUES (5, '2013-08-29 10:51:40.190913', 'gfsdfg5', 'sfgsdhgy5');
INSERT INTO test VALUES (6, '2012-08-29 10:51:40.190913', 'gfsdfg6', 'sfgsdhgy6');
INSERT INTO test VALUES (7, '2011-08-29 10:51:40.190913', 'gfsdfg7', 'sfgsdhgy7');
INSERT INTO test VALUES (8, '2010-08-29 10:51:40.190913', 'gfsdfg8', 'sfgsdhgy8');
INSERT INTO test VALUES (9, '2009-08-29 10:51:40.190913', 'gfsdfg9', 'sfgsdhgy9');
INSERT INTO test VALUES (10, '2008-08-29 10:51:40.190913', 'gfsdfg10', 'sfgsdhgy10');
INSERT INTO test VALUES (11, '2009-08-29 10:51:40.190913', 'gfsdfg11', 'sfgsdhgy11');
INSERT INTO test VALUES (12, '2015-08-29 10:51:40.190913', 'gfsdfg12', 'sfgsdhgy12');
INSERT INTO test VALUES (13, '2020-08-29 10:51:40.190913', 'gfsdfg13', 'sfgsdhgy13');
And then we I trying UPDATE this table with such query:
UPDATE test SET
str1 = c.str1,
str2 = c.str2,
str3 = c.str3
FROM (
VALUES
(10, '2017-08-29 11:11:37'::timestamp without time zone, 'str2-10', 'str3-10'),
(11, '2017-08-29 11:11:37'::timestamp without time zone, 'str2-11', 'str3-11'),
(12, '2017-08-29 11:11:37'::timestamp without time zone, 'str2-12', 'str3-12'),
(13, '2017-08-29 11:11:37'::timestamp without time zone, 'str2-13', 'str3-13')
) AS c (id, str1, str2, str3)
WHERE c.id = test.id;
And I got error:
ERROR: invalid memory alloc request size 1610613056 (context 'ExecutorState') (mcxt.c:1069) (mcxt.c:477) (seg0 node03:40000 pid=113577) (cdbdisp.c:1322)
How can I fix this error?

What version of GPDB are you using?
There are a few known planner errors -- this looks like an older legacy planner issue.
Can you try the same with set optimizer=on; or off?
With the large memory allocation size it's more likely this is an issue with statistics for the tables causing the planner to blow up.
As a best practice, follow up every CREATE with an ANALYZE.
ANALYZE the tables then running the query again.

Related

Why aren't rows returned in the order of insertion?

Oracle 18c:
I have 1000 rows of test data:
create table lines (id number, shape sdo_geometry);
begin
insert into lines (id, shape) values (1, sdo_geometry(2002, 26917, null, sdo_elem_info_array(1, 2, 1), sdo_ordinate_array(574360, 4767080, 574200, 4766980)));
insert into lines (id, shape) values (2, sdo_geometry(2002, 26917, null, sdo_elem_info_array(1, 2, 1), sdo_ordinate_array(573650, 4769050, 573580, 4768870)));
insert into lines (id, shape) values (3, sdo_geometry(2002, 26917, null, sdo_elem_info_array(1, 2, 1), sdo_ordinate_array(574290, 4767090, 574200, 4767070)));
insert into lines (id, shape) values (4, sdo_geometry(2002, 26917, null, sdo_elem_info_array(1, 2, 1), sdo_ordinate_array(571430, 4768160, 571260, 4768040)));
insert into lines (id, shape) values (5, sdo_geometry(2002, 26917, null, sdo_elem_info_array(1, 2, 1), sdo_ordinate_array(571500, 4769030, 571350, 4768930)));
...
end;
/
Full data here: db<>fiddle
When I select the data:
select
id,
sdo_util.to_wktgeometry(shape)
from
lines
...the data doesn't get returned in the order that I inserted it.
SQL Developer / on-prem db:
db<>fiddle:
I would have expected ID #1 to be the first row, and so on.
I know in reality, we would never rely on the order of the rows in the table. We would sort the data using order by if the ordering was important.
But I'm still curious, why wouldn't the data be returned in the order that it was inserted in? What's going on there?
The short answer is: tables are basically “heaps" and the rows are inserted where ever they want.

"insert into" is overwriting data

internet is not helping and my knowledge on subject is limited.
I have a table with below schema :
CREATE EXTERNAL TABLE `db.temp_entries`(
`id` bigint,
`random_id` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION
'hdfs://xxxx/xxxxx/xxx/temp_entries'
TBLPROPERTIES (
'COLUMN_STATS_ACCURATE'='{\"BASIC_STATS\":\"true\"}',
'numFiles'='1',
'numRows'='1',
'orc.compress'='ZLIB',
'rawDataSize'='115',
'totalSize'='381',
'transient_lastDdlTime'='1532514067')
Here are the insert queries m using :
Query 1
insert into `db.temp_entries`
values (1, 'P1804010001249002159939')
Query 2
insert into `db.temp_entries`
values (2, 'P1804010001495232931398'),
(3, 'P1804010002374640308088'),
(4, 'P1804010009196709498065')
I am generating this through a python script and doing the insert through python pyhive package -> from pyhive import hive
Although m not using insert overwrite, data of Query#1 is being overwritten by Query#2. Is there something wrong in my aproach?
Remove the backquotes `` around the table name.
Query 1
insert into db.temp_entries
values (1, 'P1804010001249002159939')
Query 2
insert into db.temp_entries
values (2, 'P1804010001495232931398'),
(3, 'P1804010002374640308088'),
(4, 'P1804010009196709498065')

Oracle - Text Classification (adding a secondary column)

I'm following the demo on training on classification from Oracle.
I've attached the script.
In what way can this be used to create multiple sets of rules based on a secondary column? The column wouldn't be an extra predictor, but really a way to completely isolate the rules.
That column might have its own list of data, and I'd hate to create separate sets of tables to handle the scenario.
So for example, if I added a doc_type field to ml_docs (and to the categories). When its 1 I might want to use one set of rules and if its a 2, a different set of rules. I want them to be completely isolated. What implications would that have on the script? Does Oracle support this?
create table ml_docs (
doc_id number primary key,
doc_text clob);
insert into ml_docs values
(1, 'MacTavishes is a fast-food chain specializing in burgers, fries and -
shakes. Burgers are clearly their most important line.');
insert into ml_docs values
(2, 'Burger Prince are an up-market chain of burger shops, who sell burgers -
and fries in competition with the likes of MacTavishes.');
insert into ml_docs values
(3, 'Shakes 2 Go are a new venture in the low-cost restaurant arena,
specializing in semi-liquid frozen fruit-flavored vegetable oil products.');
insert into ml_docs values
(4, 'TCP/IP network engineers generally need to know about routers,
firewalls, hosts, patch cables networking etc');
insert into ml_docs values
(5, 'Firewalls are used to protect a network from attack by remote hosts,
generally across TCP/IP');
create table ml_category_descriptions (
cd_category number,
cd_description varchar2(80));
create table ml_doc_categories (
dc_category number,
dc_doc_id number,
primary key (dc_category, dc_doc_id))
organization index;
-- descriptons for categories
insert into ml_category_descriptions values (1, 'fast food');
insert into ml_category_descriptions values (2, 'computer networking');
insert into ml_doc_categories values (1, 1);
insert into ml_doc_categories values (1, 2);
insert into ml_doc_categories values (1, 3);
insert into ml_doc_categories values (2, 4);
insert into ml_doc_categories values (2, 5);
exec ctx_ddl.create_preference('bid_lex', 'basic_lexer');
exec ctx_ddl.set_attribute ('bid_lex', 'index_themes', 'no');
exec ctx_ddl.set_attribute ('bid_lex', 'index_text', 'yes');
create index ml_docsindex on ml_docs(doc_text) indextype is ctxsys.context
parameters ('lexer bid_lex');
create table ml_rules(
rule_cat_id number,
rule_text varchar2(4000),
rule_confidence number
);
begin
ctx_cls.train(
index_name => 'ml_docsindex',
docid => 'doc_id',
cattab => 'ml_doc_categories',
catdocid => 'dc_doc_id',
catid => 'dc_category',
restab => 'ml_rules',
rescatid => 'rule_cat_id',
resquery => 'rule_text',
resconfid => 'rule_confidence'
);
end;
create index rules_idx on ml_rules (rule_text) indextype is ctxsys.ctxrule;
set serveroutput on;
declare
incoming_doc clob;
begin
incoming_doc
:= 'I have spent my entire life managing restaurants selling burgers';
for c in
( select distinct cd_description from ml_rules, ml_category_descriptions
where cd_category = rule_cat_id
and matches (rule_text, incoming_doc) > 0) loop
dbms_output.put_line('CATEGORY: '||c.cd_description);
end loop;
end;

SQL Server : bulk Insert and ignore duplicates

I'm trying to do a bulk insert (SQL Server 2008) into a table but the insert must ignore any duplicate already in the table.
The simplified table will look like this with existing values.
TBL_STOCK
id | Stock
---------------
1 | S1
2 | S2
3 | S3
Now I want to do a bulk insert that looks like
INSERT INTO TBL_STOCK (Id, Stock)
VALUES
(3, S3),
(4, S4),
(5, S5)
This works but will cause duplicate entries
How do I go about ignoring duplicate entries in the Stock column?
By "ignoring duplicate entries", you mean avoiding them in TBL_STOCK, right ?
I might be a bit late, but have you tried the following:
INSERT INTO #TempStock (Id, Stock) -- temporary table
VALUES
(3, S3),
(4, S4),
(5, S5)
INSERT INTO TBL_STOCK
SELECT * FROM #TempStock
WHERE NOT EXISTS (SELECT Stock FROM #TempStock WHERE #TempStock.Stock = TBL_STOCK.Stock)
DROP TABLE #TempStock

Automating pl\sql functions

I want to automate the registrations system so that it automatically registers students. Then I want a procedure to automate the grading system. I have included the code that I am written to make most of this assignment work which it does but unsure how to incorporate PL\SQL automated functions for the registrations system, and the grading system. So any help or ideas I would greatly appreciated please.
set Linesize 250
set pagesize 150
drop table student;
drop table faculty;
drop table Course;
drop table Section;
drop table location;
DROP TABLE courseInstructor;
DROP TABLE Registration;
DROP TABLE grade;
create table student(
studentid number(10),
Lastname varchar2(20),
Firstname Varchar2(20),
MI Char(1),
address Varchar2(20),
city Varchar2(20),
state Char(2),
zip Varchar2(10),
HomePhone Varchar2(10),
Workphone Varchar2(10),
DOB Date,
Pin VARCHAR2(10),
Status Char(1));
ALTER TABLE Student
Add Constraint Student_StudentID_pk Primary Key (studentID);
Insert into student values (1,'xxxxxxxx','xxxxxxxxxx','x','xxxxxxxxxxxxxxx','Columbus','oh','44159','xxx-xxx-xxxx','xxx-xxx-xxxx','06-Mar-1957','1211','c');
create table faculty(
FacultyID Number(10),
FirstName Varchar2(20),
Lastname Varchar2(20),
MI Char(1),
workphone Varchar2(10),
CellPhone Varchar2(10),
Rank Varchar2(20),
Experience Varchar2(10),
Status Char(1));
ALTER TABLE Faculty
ADD Constraint Faculty_facultyId_PK PRIMARY KEY (FacultyID);
insert into faculty values (1,'xxx','xxxxxxxxxxxx','xxx-xxx-xxxx','xxx-xxx-xxxx','professor','20','f');
create table Course(
CourseId number(10),
CourseNumber Varchar2(20),
CourseName Varchar(20),
Description Varchar(20),
CreditHours Number(4),
Status Char(1));
ALTER TABLE Course
ADD Constraint Course_CourseID_pk PRIMARY KEY(CourseID);
insert into course values (1,'cit 100','computer concepts','introduction to PCs','3.0','o');
insert into course values (2,'cit 101','Database Program','Database Programming','4.0','o');
insert into course values (3,'Math 101','Algebra I','Algebra I Concepts','5.0','o');
insert into course values (4,'cit 102a','Pc applications','Aplications 1','3.0','o');
insert into course values (5,'cit 102b','pc applications','applications 2','3.0','o');
insert into course values (6,'cit 102c','pc applications','applications 3','3.0','o');
insert into course values (7,'cit 103','computer concepts','introduction systems','3.0','c');
insert into course values (8,'cit 110','Unified language','UML design','3.0','o');
insert into course values (9,'cit 165','cobol','cobol programming','3.0','o');
insert into course values (10,'cit 167','C++ Programming 1','c++ programming','4.0','o');
insert into course values (11,'cit 231','Expert Excel','spreadsheet apps','3.0','o');
insert into course values (12,'cit 233','expert Access','database devel.','3.0','o');
insert into course values (13,'cit 169','Java Programming I','Java Programming I','3.0','o');
insert into course values (14,'cit 263','Visual Basic','Visual Basic Prog','3.0','o');
insert into course values (15,'cit 275','system analysis 2','System Analysis 2','3.0','o');
create table Section(
SectionID Number(10),
CourseId Number(10),
SectionNumber VarChar2(10),
Days Varchar2(10),
StartTime Date,
EndTime Date,
LocationID Number(10),
SeatAvailable Number(3),
Status Char(1));
ALTER TABLE Section
ADD Constraint Section_SectionID_PK PRIMARY KEY(SectionID);
insert into section values (1,1,'18977','r','21-Sep-2011','10-Dec-2011','1','89','o');
create table Location(
LocationId Number(10),
Building Varchar2(20),
Room Varchar2(5),
Capacity Number(5),
Satus Char(1));
ALTER TABLE Location
ADD Constraint Location_LocationID_pk PRIMARY KEY (LocationID);
insert into Location values (1,'Clevleand Hall','cl209','35','o');
insert into Location values (2,'Toledo Circle','tc211','45','o');
insert into Location values (3,'Akron Square','as154','65','o');
insert into Location values (4,'Cincy Hall','ch100','45','o');
insert into Location values (5,'Springfield Dome','SD','35','o');
insert into Location values (6,'Dayton Dorm','dd225','25','o');
insert into Location values (7,'Columbus Hall','CB354','15','o');
insert into Location values (8,'Cleveland Hall','cl204','85','o');
insert into Location values (9,'Toledo Circle','tc103','75','o');
insert into Location values (10,'Akron Square','as201','46','o');
insert into Location values (11,'Cincy Hall','ch301','73','o');
insert into Location values (12,'Dayton Dorm','dd245','57','o');
insert into Location values (13,'Springfield Dome','SD','65','o');
insert into Location values (14,'Cleveland Hall','cl241','10','o');
insert into Location values (15,'Toledo Circle','tc211','27','o');
insert into Location values (16,'Akron Square','as311','28','o');
insert into Location values (17,'Cincy Hall','ch415','73','o');
insert into Location values (18,'Toledo Circle','tc111','67','o');
insert into Location values (19,'Springfield Dome','SD','69','o');
insert into Location values (20,'Dayton Dorm','dd211','45','o');
Alter Table Student
Add Constraint student_Zip_CK Check(Rtrim (Zip,'1234567890-') is null);
Alter Table Student
ADD Constraint Student_Status_CK Check(Status In('c','t'));
Alter Table Student
ADD Constraint Student_MI_CK2 Check(RTRIM(MI,'abcdefghijklmnopqrstuvwxyz')is Null);
Alter Table Student
Modify pin not Null;
Alter table Faculty
Add Constraint Faculty_Status_CK Check(Status In('f','a','i'));
Alter table Faculty
ADD Constraint Faculty_Rank_CK Check(Rank In ('professor','doctor','instructor','assistant','tenure'));
Alter table Faculty
ADD Constraint Faculty_MI_CK2 Check(RTRIM(MI,'abcdefghijklmnopqrstuvwxyz')is Null);
Update Section Set Starttime = To_date('09-21-2011 6:00 PM', 'mm-dd-yyyy hh:mi pm');
Update Section Set Endtime = To_date('12-10-2011 9:50 PM', 'mm-dd-yyyy hh:mi pm');
alter table Section
Add Constraint StartTime_Status_CK Check (starttime < Endtime);
Alter Table Section
Add Constraint Section_StartTime_ck check (StartTime < EndTime);
Alter Table Section
ADD Constraint Section_CourseId_FK FOREIGN KEY (CourseID) References Course(CourseId);
Alter Table Section
ADD Constraint Section_LocationID_FK FOREIGN KEY (LocationID) References Location (LocationId);
Alter Table Section
ADD Constraint Section_Days_CK Check(RTRIM(Days,'mtwrfsu')IS Null);
update section set seatavailable = '99';
Alter Table Section
ADD Constraint Section_SeatsAvailable_CK Check (SeatAvailable < 100);
Alter Table Course
Add Constraint Course_CreditHours_ck check(CreditHours < = 6.0);
update location set capacity = '99';
Alter Table Location
Add Constraint Location_Capacity_CK Check(Capacity < 100);
Create Table Registration (
StudentID Number(10),
SectionID Number(10),
Constraint Registration_pk Primary key (studentId, Sectionid));
Insert into registration values (1, 2);
Insert into Registration values (2, 3);
Insert into registration values (3, 4);
Insert into registration values (4, 5);
Insert into registration values (5, 6);
Insert into registration values (6, 7);
Insert into registration values (7, 8);
Insert into registration values (8, 9);
insert into registration values (9, 10);
insert into registration values (10, 11);
insert into registration values (9, 12);
insert into registration values (8, 13);
insert into registration values (7, 14);
insert into registration values (6, 15);
insert into registration values (5, 17);
insert into registration values (4, 18);
insert into registration values (3, 19);
insert into registration values (2, 20);
insert into registration values (1, 21);
insert into registration values (2, 22);
insert into registration values (3, 23);
insert into registration values (4, 24);
insert into registration values (5, 25);
Insert into registration values (6, 24);
insert into registration values (7, 23);
insert into registration values (8, 22);
insert into registration values (9, 21);
insert into registration values (10, 20);
insert into registration values (9, 19);
insert into registration values (8, 17);
Create Table courseInstructor(
FacultyID Number(10),
SectionID Number(10),
Constraint CourseInstructor_pk Primary key (FacultyId, SectionID));
insert into courseInstructor values (1, 1);
insert into courseInstructor values (2, 2);
insert into courseInstructor values (3, 3);
insert into courseInstructor values (4, 4);
insert into courseInstructor values (5, 5);
insert into courseInstructor values (5, 6);
insert into courseInstructor values (4, 7);
insert into courseInstructor values (3, 8);
insert into courseInstructor values (2, 9);
insert into courseInstructor values (1, 10);
insert into courseInstructor values (5, 11);
insert into courseInstructor values (4, 12);
insert into courseInstructor values (3, 13);
insert into courseInstructor values (2, 14);
insert into courseInstructor values (1, 15);
Create table grade(
StudentID Number(10),
SectionID Number(10),
Grade Varchar2(1),
Constraint grade_pk Primary key (StudentID, SectionID));
CREATE OR REPLACE TRIGGER TR_CreateGrade
AFTER INSERT ON Registration
FOR EACH ROW
BEGIN
INSERT INTO grade (SectionID,StudentID,Grade)
VALUES(:New.SectionID,:New.StudentID,NULL);
END TR_createGrade;
/
CREATE OR REPLACE FORCE VIEW V_reg_student_course AS
SELECT
Registration.StudentID,
student.LastName,
student.FirstName,
course.CourseName,
Registration.SectionID,
course.CreditHours,
section.Days,
TO_CHAR(StartTime, 'MM/DD/YYYY') AS StartDate,
TO_CHAR(StartTime, 'HH:MI PM') AS StartTime,
TO_CHAR(EndTime, 'MM/DD/YYYY') AS EndDate,
TO_CHAR(EndTime, 'HH:MI PM') AS EndTime,
location.Building,
location.Room
FROM registration, student, section, course, location
WHERE registration.StudentID = student.StudentID
AND registration.SectionID = section.SectionID
AND section.LocationID = location.LocationID
AND section.CourseID = course.CourseID;
CREATE OR REPLACE FORCE VIEW V_teacher_to_course AS
SELECT
courseInstructor.FacultyID,
faculty.FirstName,
faculty.LastName,
courseInstructor.SectionID,
section.Days,
TO_CHAR(StartTime, 'MM/DD/YYYY') AS StartDate,
TO_CHAR(StartTime, 'HH:MI PM') AS StartTime,
TO_CHAR(EndTime, 'MM/DD/YYYY') AS EndDate,
TO_CHAR(EndTime, 'HH:MI PM') AS EndTime,
location.Building,
location.Room
FROM courseInstructor, faculty, section, course, location
WHERE courseInstructor.FacultyID = faculty.FacultyID
AND courseInstructor.SectionID = section.SectionID
AND section.LocationID = location.LocationID
AND section.CourseID = course.CourseID;
SELECT * FROM V_reg_student_course;
SELECT * FROM V_teacher_to_course;
I do not know how much detail is required for this exercise but these are the things I see:
You are mixing CHAR and VARCHAR2 types, there is no reason to do this. Make your life easier and stick with VARCHAR2 if the entries are expected to be less than 4000 characters
Many people's names, cities and addresses are much longer than 20 characters. Think big, 250 characters.
Middle Initial is problematic. Not everyone has one and if they do they may want their full middle name spelled out.
Address information should be broken out into a series of separate tables but I grant this may be beyond the scope of your assignment
Not everyone lives in the United States, think about a numeric key to a secondary table for cities and states/provinces.
Many of these fields should be not null such as course number, course name
Now, onto your question about automating functions. I am not sure what you really want. A student registering for a course is initiated outside of the database. Staff would enter the students registration choices based on a form. Or the student would submit them electronically, either way some front-end application would initiate the transaction.
For example, on your front end, someone presses "Submit". Then the old fashioned way is to open a connection to the database, add some parameters, make a procedure call like
Add_student_to_course(student_id_in IN NUMBER, section_id_in IN NUMBER, registration_out OUT sys_refcursor)
Inside the procedure you check to see whether the student exists, course exists, course has seats available.
There are many different ways of dealing with error handling depending on the complexity of the validation process. If it is possible for one registration to violate several business rules then you want to return complex information. (if the course must have a seat available, if the course must have a teacher, if the student is paid up).
So, do your processing as if statements. When everything is fine, in a simple application, the sys_refcursor could look like
select registration_no, error_number from dual;
If everything is fine error_number is 0, if not return an error code. Otherwise the registration code is 0 and the error code refers to a table of errors with a human readable text.
(I am aware that this is a simplistic approach but this is a simplified exercise)

Resources