I'm new using Azure Data Factory and i would like to know if there is a mode to run two queries in one activity copy, for example:
I have a Data Set Oracle for my origin and other for my stage, both use Oracle DB.
i need copy all registries in a specific query, but before to run it, i need to alter session
ALTER session SET nls_language = 'AMERICAN';
and immediately run my query
Select * from ... where ....
This is posible? or it's not the way? thks for your answers
Important: We don't have the posibility to create objects in DB Origin
Of course you can, in the copy activity, click on the tab Source, and then select Query. Input your 2 step query into the textbox and you are good to go.
Hope this helped!
Related
I created a view in Redshift that unions two queries, and it works great. We've thought of a third query that would be worthwhile to add in. eg
CREATE VIEW stem_alumni as
SELECT name, email
FROM students
WHERE graduated < 2019 AND major = 'Engineering'
UNION
SELECT name, email
FROM alumni
WHERE current_employer = 'Google'
The problem is when I try to add a third query in
UNION
SELECT name, email
FROM professors
WHERE department = 'Engineering'
it'll persist for maybe an hour, but then revert to just the original query.
I've run CREATE OR REPLACE VIEW... and dropping/recreate and get the same result.
How do I get an updated view definition to persist?
Adding more context
I created the view using DBeaver, a local SQL client using my specific Redshift credentials. The view is called by Periscope, our cloud-based BI tool using shared credentials. Querying the view in Periscope or separate DBeaver windows will eventually revert the view to its original definition.
Redshift shouldn't have a 'memory' of the view's prior DDL that it could revert to. I'm inclined to agree with the comments that something else is overwriting the updates to the view's DDL after you have committed them.
You should be able to see if something is overwriting the view, by querying the stl_query table:
SELECT q.starttime
, u.usename
, q.querytxt
FROM pg_user u
JOIN stl_query q ON u.usesysid = q.userid
WHERE POSITION('<view_name>' IN q.querytxt) > 0
ORDER BY q.starttime DESC
;
This table only contains recent query information (2-5 days according to the Redshift Documentation), so if you haven't experienced this behavior from the view within that timescale, you may need to force it to occur again in order to troubleshoot who/what is altering the DDL.
Additionally, if the view is being overwritten by a user other than yourself, you will need to query stl_query using a super user account (by default, non-super users will only be able to view information for queries that they themselves have executed).
Before anything, i must say this first: This table design is not my decision. We protest but to no avail, so please don't tell me, don't create a table like that.
We have a database with each table have a flag. This flag used to indicate which environment this row belong to, production or test data.
For server side, we have one variable which currently stored in ThreadLocal to indicate which environment this request belong to, same value as the flag in database.
Our requirement is that if my request belong to test environment then we must select only record belong to this environment. We would need to add a condition to every query we made to database, something like:
SELECT t FROM TABLE t WHERE t.flag = :environment
But we have to update every single query, update every object to set this flag before insert/update into database. This will require a lot of effort as our system already built long ago, not on progress. Also this will bring a lots of risk if someone forgot to add this to any new query.
So is there anyway to insert a condition to check this flag value for every query without have to manually edit the query string? Like an interceptor or something to put this condition in?
Which JPA provider?
With Hibernate, you could try using a #Filter.
Multitenancy could be another option, but probably an overkill in your scenario.
Finally, since you flagged the question with Oracle, perhaps the easiest approach would be to provide dedicated schemas (per environment) with views for every single table in your db, filtered by the flag column. Not sure if you're allowed to do that, though.
With some of the above, you would need a global entity listener to populate the flag field of your entities before they are persisted.
In the salesInvoice ssrs Report i have added a table called carTableEquipTmp which is not there by default, which I insert into along with the other tables(SalesinvoiceTmp and SalesinvoiceHeaderFooterTmp) in SalesInvoiceDP.InsertIntoSalesInvoiceTmp().
Even though my table carTableEquipTmp is getting successfully inserted into, the data doesn't show up on the report if i print a proforma report.
If i add test values to the carTableEquipTmp table in SalesInvoiceDP.processReport() they show up on the proforma invoice, but there's no way for me to get any parameters needed to set in the correct data into the table at this point. If i stop at this point in the debugger none of the data is present because processreport() is being called from a lower level in the code.
I think it might be a problem with maybe pack/unpack or that the proforma code runs from a server instance as the code run when it is proforma is quite different.
I can see that SalesInvoiceJournalPostBase.CreateReportData() creates an instance of salesInvoiceDP
salesInvoiceDP = new SalesInvoiceDP();
salesInvoiceDP.parmDataContract(salesInvoiceContract);
salesInvoiceDP.parmUserConnection(new UserConnection(true));
salesInvoiceDP.createData();
And that this might have something to do with it... but i still cant get the data i want in the carTableEquipTmp table.
So any idea on how to make Ax 2012 accept this new table i have added as it gets inserted into just like the other tables and there seems to be no problem...
I hope you guys can help.
The SalesInvoice report has two data classes you need to look at for the data provider, SalesInvoiceDP and SalesInvoiceDPBase. SalesInvoiceDPBase extends SrsReportDataProviderPreProcess, so there are a couple extra steps you need to take in order to add new datasources to the report.
In the salesInvoiceDP class, there is a method called useExistingReportData(), which re-inserts the pro-forma temp table data under a user connection, so the SrsReportDataProviderPreProcess framework will pick it up in your report. When the pro-forma process creates the report data, it doesn't insert with a user connection so it doesn't get added to the report. This method only gets called when the report is being run pro-forma.
You will need to add your temp table to this method, and follow the pattern for the other tables, so your code will look something like this:
//this is different from the buffer you insert your data with
CarTableEquipTmp localCarTableEquipTmp;
...
recordList = new RecordSortedList(tableNum(carTableEquipTmp));
recordList.sortOrder(fieldNum(carTableEquipTmp, RecId));
//You will need to add a field to relate your temp table
//to the current invoice journal, and insert it in
//InsertIntoSalesInvoiceTmp() if thats where you're inserting your table.
while select localCarTableEquipTmp
where localCarTableEquipTmp.JournalRecId == jourRecId
{
recordList.ins(localCarTableEquipTmp);
}
delete_from localCarTableEquipTmp
where localCarTableEquipTmp.JournalRecId == jourRecId;
recordList.insertDatabase(this.parmUserConnection());
This method re-inserts your data under the framework and deletes the original data. The data that was re-inserted will then get picked up by the framework and show in your report. If you open CarTableEquipTmp in the table browser, you will most likely see data still there from all the times you have tried running the report. This is why we have the delete_from operation after we re-insert the data. When data is inserted under a userConnection, it is automatically deleted when the report is finished
The other method you will want to modify is SalesInvoiceDP.setTableConnections(), and you will just need to add the following line:
CarTableEquipTmp.setConnection(this.parmUserConnection());
This will set the user connection for your table when running regular (not pro-forma). You will probably want to delete the data that is stored currently in your temp table using alt+F9 from the table browser.
Other than that it's all standard RDP stuff, but it sounds like you have that part working fine. Your temp table must be of type "Regular" for this to work.
Basically, I have a LINQ database context and its model. As usually, I create the DB in the SQL context if the DB does not exist (the context is a singleton and on every access to it, this is checked).
Everything works well if I add data to the DB on the first launch. But if I don't insert any data during the first start of the app, on successive launches I get
SqlCeException:The specified table does not exist [TableName]
I don't know how more specifically I can explain it, but the exception comes immediately whenever I do a LINQ query on the second launch of the app if I don't insert any data on the first launch. If i do insert some data during the first launch, all is fine for the rest of the app's life time. Why would it be a bad thing to create the DBs and introduce the DB context, but not insert any data?
Here's my LINQ DB model:
https://github.com/kypeli/Podcatcher/blob/master/wp7/Podcatcher/ViewModels/PodcastSubscriptionModel.cs
Here's where I get the exception on second start if I didn't insert any data on the first launch:
https://github.com/kypeli/Podcatcher/blob/master/wp7/Podcatcher/PodcastSqlModel.cs#L64
It also strikes me that there's no API call to check if a table exists or not in LINQ, so I would have to assume "this should just work" - but it doesn't.
Any ideas? Thanks! :)
Update: I verified analyzing the .sdf file that indeed there are no tables created if I don't insert any data upon first launch of the app. As I see it:
This is a bug in LINQ-to-SQL. It should not crash if there are no tables present, but know that it should create them. Or deal with the case and create tables only when data is inserted.
I would need to insert some dummy data into SQL always on first launch, or...
Check if a table exists, if not, react to it by forcing LINQ-to-SQL to create them. But how?
I've dealt with this problem also, I've fixed it this way:
get the data context:
dbDataContext = new DBDataContext(DBConnectionString);
if( dbDataContext.DatabaseExists() == true)
//then try to get an entity:
System.Data.Linq.Table<Entity> entities = dbDataContext.Tablename;
//try to get an element from the entity:
IEnumerator<Entity> enumEntity = entities.GetEnumerator();
entities.GetEnumerator(); will always raise the exception "Table not found."
Just use a try/catch and in the catch scope delete the db and recreate it, because your DB is empty anyway :)
dbDataContext.DeleteDatabase();
dbDataContext.CreateDatabase();
dbDataContext.SubmitChanges();
I'm trying to find some nice work arounds for the issues of computed columns in code first. Specifically, I have a number of CreatedAt datetime columns that need to be set to getdate().
I've looked at doing this via the POCO constructors, but to do that I must remove the Computed option (or it won't persist the data), however, there is no easy to way ensure the column is only set if we are inserting a record. So this would overwrite the CreatedAt each time we update.
I'm looking to create an alter script that can be called after the DropCreate that would go through and alter various columns to include the default value of getdate().
Is there an event to hook into something like OnDropCreateCompleted where I could then run additional SQL
What would be the best way handle the alter script? I am thinking just sending raw sql to the server that would run.
Is there another way to handle the getdate() issue that might be more graceful and more inline with code first that I'm missing?
Thanks
You can just make custom initializer derived from your desired one and override Seed method where you can execute any SQL you want to use - here is some example for creating such initializer.
If you are using migrations you can just the custom SQL to Up method.