I've a small program in Linq that inserts records in my database based on a number of conditions (depending on existing data in the same db). The SQL output are lots of SELECT and INSERT statements. In my production environment this should be executing from a SQL script and reviewed first. Is it possible to generate the SQL output without actually running the query?
Related
I have a requirement in which i have to execute multiple sql queries in Nifi using executesql processor.
Those queries are dependent on each other as I am storing data in session based temporary table.
So I need to know if I do so , whether all the queries will get executed in single database session.
Example:
Query a data is inserted into Temp_A, Now I need that data in next query so will it be possible.
Note: I am talking about session based Temporary table only.
you can use ExecuteSQL processor where in SQL Pre-Query parameter that could contain multiple commands separated with semicolon ;
all those commands with main sql query will be executed using the same connection for one flow file.
note that the same sql connection could be used for the next flow file.
Using Oracle SQL Developer, I develop queries that are copy/pasted as a subquery into a number of other queries. If I make a change, I have to go through all of the files and manually update those queries as well. It's very inefficient.
Is there a way of using sql files like programing modules? That way I only have to make a change in one place?
If you create a stored procedure you can reference that in your queries and make the change in the procedure itself.
I am not an expert of oracle, I just write queries and scripts through TOAD but I don't know how oracle works.
In order to make a data analysis I have created several tables in order to build a DB from them.
In order to build them I have used the following command
CREATE TABLE SCHEMA.NAME
TABLESPACE SCHEMASPACE NOLOGGING PARALLEL AS
select [...]
with the parallel option active in order to make the server work on more processors.
Once that the DB was created I have tried to do some queries on it but the same query done on the same DB 2 times gave me two different answers. In detail the first time I started the query it returned to me a table with about 2500 rows and the second time it returned to me a table with about 2650 rows.
I have tried to redo the entire process without the parallel option and now it seems to work, but it take too much time and I need to run the process several times, does anyone have a solution?
Hey EXPERIENCED SSIS DEVELOPERS, I need your help.
High-Level Requirements
Query SQL Server table (on a different server than my SSIS server) resulting in about 200-300k records results set.
Use three output colums for each row to lookup date in Oracle database.
Insert or Update SQL Server table with results.
Use SSIS.
SQL Server 2008
Sounds easy, right?
Here is what I have done:
Created on Control Flow Execute SQL Task that gets a recordset from SQL Server. Very fast, easy query, like select field1, field2, field 3 from table where condition > 0. That's it. Takes less than a second.
Created a variable (evaluated as expression) for the Oracle query that uses the results set from the above in the WHERE clause.
Created a ForEachLoop Container that takes the results (from #1 above) for each row in the recordset and runs it through a Data Flow that uses the Oracle query (from #2 above) with Data access mode: SQL command from variable against an Oracle data source. Fast, simple query with only about 6 columns returned.
Data Conversion - obvious reasons - changing 3 columns from Oracle data types to SQL Server data types.
OLE DB Destination to insert to SQL Server using Fast Load to staging table.
It works perfectly! Hooray! Bad news - it is very, very slow. When I say slow, I mean it process 3000 records per hour. Holy moly - so freaking slow.
Question: am I missing a way to speed it up? It seems like the ForEachLoop Container is the bottleneck. Growl.
Important Points:
- I have NO write access in Oracle environment, so don't even suggest a potential solution that requires it. Not a possibility. At all.
Oracle sources do not allow for direct parameter definition. So no SELECT FIELD FROM TABLE WHERE ?. Don't suggest it - doesn't work.
Ideas
- Should I find a way to break down the results of the Execute SQL task and send them through several ForEachLoop Containers for faster processing?
Is there another design that is more appropriate?
Is there a script I can use that is faster?
Would it be faster to create a temporary table in memory and populate it - then use the results to bulk insert to SQL Server? Does this work when using an Oracle data source?
ANY OTHER IDEAS?
I have a single instance called sp_admin with several tables. One of them is called 'tenant_api_user'. From sqlplus, I can select * from any of the tables, including tenant_api_user (or select count(*) etc) and the results are fine. However, when I run queries against this one specific table in SQL Developer, it always returns zero records.
When I run the select * from tenant_api_user; from SQL Developer, it returns 0 records instantly - no waiting. That same query from sqlplus returns the only record in the table. When I try to update a record that is visible from sqlplus, SQL Developer says 0 rows updated.
Has anyone experienced this kind of behavior before? It seems to be isolated to just this table. SQL Developer doesn't behave this way with any other table.
If you have entered the data in one session and you have not run a commit statement then you will not be able to see the data in another session until it has been committed.