There are 2 tables (Say Station and Terminal). Terminal table has 2 columns Origin and Destination and both accept Station name (which is primary key in STATION table).
How to get all records from TERMINAL table with STATION object in JPA?
Can we associate ManyToOne relation to Origin and Destination columns (From Terminal Table) to StationName in STATION table?
There are a 100 records in TERMINAL table, but when trying to retrieve all 100 records I get back only 20 records.
Related
Purpose: Remove duplicate records from large table.
My Process:
Create table 2 with 9 fields. No indexes. Same data_types per field as Table 1.
insert 9 fields, all records into table 2 from existing table 1
Table 1 contains 71+ Million rows and 232 columns and many duplicate records.
No joins. No Where Clause.
Table 1 contains several indexes.
8 fields are required to get unique records.
I'm trying to set up a process to de-dup large tables, using dense_rank partitioning to identify the most recently entered duplicate. Thus, those 8 required fields from Table 1 plus the auto-increment from Table 1 are loaded into Table 2.
Version: 10.5.17 MariaDB
The next steps would be:
Create new Table 3 identical to table 1 but with no indexes.
Load all data from Table 1 into Table 3, joining Table 1 to Table 2 on the auto-increment fields, where table 2.Dense_Rank field value = 1. This inserts ~17 Million unique records
Drop any existing Foreign_Keys related to Table 1
Truncate Table 1
Insert all records from Table 3 into Table 1
Nullify columns in related tables where the foreign key values in Table 1 no longer exist
re-create Foreign Keys that had been dropped.
Creating a test instance of an existing system I accomplish everything I need to once - only the first time. But If I then drop table 2 before refreshing Table 1 as outlined immediately above, re-create and try to reload, workbench shows query running until 7200 second timeout.
While the insert into Table 2 query is running, opening a second instance of Workbench and selecting count of records in table 2 after 15 minutes gives me the 71+ Million records I'm looking for, but Workbench continues running until timeout.
The query shows up in Show Processlist for those 15 minutes, but disappears around the 15 minute mark - presumably once all records are loaded.
I have tried running with timeouts set to 0 as well as 86,400 seconds, indicating no read timeout and 24 hours timeout, respectively, but query still times out at 7200.0xx seconds, or 2 hours, every time.
The exact error message I get is: Error Code: 2013. Lost connection to MySQL server during query 7200.125 sec
I have tried running the insert statement with COMMIT and without.
This is being done in a Test Instance set up for this development where I am the only user, and only a single table is in use during the insert process.
Finding one idea on line I ran the following suggested query to identify locked tables but got the error message that the table does not exist:
SELECT TRX_ID, TRX_REQUESTED_LOCK_ID, TRX_MYSQL_THREAD_ID, TRX_QUERY
FROM INNODB_TRX
and, of course, with only a single table being called by a single user in the system nothing should be locked.
As noted above I can fulfill the entire process a single time but, when trying to run up to the point of stopping just before truncating Table 1 so I can start over, I am consistently unable to succeed. Table 2 never gets released after being loaded again.
The reason it is important for me test a second iteration is that once this process is successful it will be applied to several database instances that were not just set up for the purpose of testing this process, and if this only works on a newly created database instance that has had no other processing performed it may not be dependable.
I have three tables:
Table1(id, name, no)
Table2(id, table1id, no)
Table3(id, table2id, no).
I have an object suitable for the structure of these tables and I want to save them in the database. While recording, I want to use the sequence of the first table in my second table. I will pull the sequence of my first table from the database. I will save tables 2 and 3 in the same way. I want to save these transactions in my oracle database using sqlkata, dapper in .net environment.
I tried pulling the sequence value and setting it each time.
I'm trying to auto-generate ID's for my entity, but it's not generating. Instead, it's starting from 1 when there already exists an entry with id "1" in my DB. Why is it not generating id "9" for my new entity?
Typically when creating a table with GenerationType.IDENTITY on postgres, Hibernate will setup the id column plus a database sequence to manage this id.
By convention the sequence name will be "tablename_id_seq". E.g., for the table ad_group_action there will be a corresponding sequence ad_group_action_id_seq. You can connect to the database to double-check the actual sequence name created.
The sequence just starts from 1 and increments each time a row is inserted by Hibernate.
But if there are pre-existing rows -- or if rows with existing IDs are inserted "manually" into the table -- those rows can conflict with the sequence.
One solution is to simply reset the sequence (from pgAdmin or another database client) to start at a higher number (say 100), using something like:
ALTER SEQUENCE ad_group_action_id_seq RESTART WITH 100;
Now Hibernate will not conflict with the existing rows (assuming their max id is < 100).
Alternatively, when inserting rows manually, omit the id column and let postgres automatically set them. This way the table and the sequence will always be in sync.
I have a scenario as below.
a) Table A with 10 columns of which 3 columns form a composite primary key.
b) A MV is created on top of table A.(Direct select query from table A)
c) MV logs are created on top of table A and MV is configured for fast refresh
Now i need to remove 1 column from the primary key combination so that i will still have 10 columns in my table A but only 2 columns form the composite primary key.
How should i handle my MV on top of the table? Should i recreate the MV and MVLogs?
I have one table values that have 80 million records. Another table values_history that has 250 million records.
I want to filter the values_history table and want to keep the only data for which id is preset in values table.
delete from values_history where id not in (select id from values);
This query takes such a long time that I have to abort the process.
Please some idea to speed up the process.
Can I delete the records in bunch like 1000000 at a time?
I have extracted out the required record and inserted into temp table .This took 2 hrs after that i dropped the table then again inserted extracted data back to the main table whole process took 4 hrs around that is fine for me.I have dropped foreign key and all other constraint before that..