Like the title says, I've been asked to come up with an estimate for retrofitting an existing asp application.
The current security mechanism controls access to different parts of the application (page-level restrictions), but has no mechanism for flagging individual records as restricted. Assigning rights to a user (using the existing, custom access management code) is no problem, but enforcing the rights is a different matter - each asp page has embedded sql - there's no use of stored procs, objects, etc.
Is the only solution to modify each table and query, or is there a better way? Any pointers, suggestions or prayers would be welcome.
This is classic asp, running on IIS6, against an oracle database.
Update: Here's a user scenario.
We have users, managers, directors, and VPs. The managers can see data created by users who report to them, but not users who report to other managers. Users can't see data created by any managers. Same thing with directors - they can see down, but their reports can't see up.
This sounds like an ideal time to implement row-level security. Oracle has a package DBMS_RLS that allows you to define arbitrary access policies that can be applied to one or more tables that limit what rows a particular user is allowed to see. Conceptually, when a user issues a query with no filters on a protected table, i.e.
SELECT *
FROM my_table
Oracle automatically and transparently inserts a WHERE clause defined by your security policy that limits the result set. You shouldn't need to make any changes to the SQL your application is executing.
Assuming you need maximum granularity, the ability to "grant" each and any row to any of very many users, then you have a many-to-many relation, yes?
So apply the following pattern:
Add a tables of users.
Then, for each restricted table, so the following:
Rename it tablename + "_base".
create a many-to-many table that
associates that table's id with a
user id, called tablename +
"allowed_user".
create a view with the name table
name that joins tablename_base to
table_name_allowed_user, with a
select* from tablename_base and
user_id from tablename_allowed_user.
This view should meet Oracle's
requirements rto be "inherently
updatable."
Now comes the hard part. You need to add "and user_id = $user_id" to every query. Find the various functions you're using to make queries. Wrap those function(s) in ones that gets the user id from the session and add that predicate.
One passable way to do this is to read select string, find the all "where"s (for subqueries there may be more that one), and replace it with "where (user = $user) and ". For queries that don't have a where, you'll need to insert this before any "group by" or "order by". This is fragile, so obviously you'll test that this works for all pages (you have an automated test for all pages, right?), and add hacks to cover special cases.
"update" statements won't have to change; "inserts" will presumably insert both to the view and then do a separate insert to the table's "allow_user" table with the id of the inserting user, to automatically grant teh inserting user acces to what he inserted.
If your number of users is more limited, or you're restricting types of users, you can go with a strategy of multiple views named for the user or type; then you'd replace tables names in the queries with the appropriate views.
Related
I'm using the Dynamics connector in Azure Data Factory.
TLDR
Does this connector support loading child records which need a parent record key passed in? For example if I want to create a contact and attach it to a parent account, I upsert a record with a null contactid, a valid parentcustomerid GUID and set parentcustomeridtype to 1 (or 2) but I get an error.
Long Story
I'm successfully connecting to Dynamics 365 and extracting data (for example, the lead table) into a SQL Server table
To test that I can transfer data the other way, I am simply loading the data back from the lead table into the lead entity in Dynamics.
I'm getting this error:
Failure happened on 'Sink' side. ErrorCode=DynamicsMissingTargetForMultiTargetLookupField,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=,Source=,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot find the target column for multi-target lookup field: 'ownerid'.
As a test I removed ownerid from the list of source columns it loads OK.
This is obviously a foreign key value.
It raises two questions for me:
Specifically with regards to the error message: If I knew which lookup it needed to use, how can I specify which lookup table it should validate against? There's no settings in the ADF connector to allow me to do this.
This is obviously a foreign key value. If I only had the name (or business key) for this row, how can I easily lookup the foreign key value?
How is this normally done through other API's, i.e. the web API?
Is there an XRMToolbox addin that would help clarify?
I've also read some posts that imply that you can send pre-connected data in an XML document so perhaps that would help also.
EDIT 1
I realised that the lead.ownertypeid field in my source dataset is NULL (that's what was exported). It's also NULL if I browse it in various Xrmtoolbox tools. I tried hard coding it to systemuser (which is what it actually is in the owner table against the actual owner record) but I still get the same error.
I also notice there's a record with the same PK value in systemuser table
So the same record is in two tables, but how do I tell the dynamics connector which one to use? and why does it even care?
EDIT 2
I was getting a similar message for msauto_testdrive for customerid.
I excluded all records with customerid=null, and got the same error.
EDIT 2
This link appears to indicate that I need to set customeridtype to 1 (Account) or 2 (Contact). I did so, but still got the same error.
Also I believe I have the same issue as this guy.
Maybe the ADF connector suffers from the same problem.
At the time of writing, #Arun Vinoth was 100% correct. However shortly afterwards there was a documentation update (in response to a GitHub I raised) that explained how to do it.
I'll document how I did it here.
To populate a contact with against a parent account, you need the parent accounts GUID. Then you prepare a dataset like this:
SELECT
-- a NULL contactid means this is a new record
CAST(NULL as uniqueidentifier) as contactid,
-- the GUID of the parent account
CAST('A7070AE2-D7A6-EA11-A812-000D3A79983B' as uniqueidentifier) parentcustomerid,
-- customer id is an account
'account' [parentcustomerid#EntityReference],
'Joe' as firstname,
'Bloggs' lastname,
Now you can apply the normal automapping approach in ADF.
Now you can select from this dataset and load into contact. You can apply the usual automapping approach, this is: create datasets without schemas. Perform a copy activity without mapping columns
This is the ADF limitation with respect to CDS polymorphic lookups like Customer and Owner. Upvote this ADF idea
Workaround is to use two temporary source lookup fields (owner team and user in case of owner, account and contact in case of customer) and with parallel branch in a MS Flow to solve this issue. Read more, also you can download the Flow sample to use.
First, create two temporary lookup fields on the entity that you wish to import Customer lookup data into it, to both the Account and Contact entities respectively
Within your ADF pipeline flow, you will then need to map the GUID values for your Account and Contact fields to the respective lookup fields created above. The simplest way of doing this is to have two separate columns within your source dataset – one containing Account GUID’s to map and the other, Contact.
Then, finally, you can put together a Microsoft Flow that then performs the appropriate mapping from the temporary fields to the Customer lookup field. First, define the trigger point for when your affected Entity record is created (in this case, Contact) and add on some parallel branches to check for values in either of these two temporary lookup fields
Then, if either of these conditions is hit, set up an Update record task to perform a single field update, as indicated below if the ADF Account Lookup field has data within it
I created a view in Redshift that unions two queries, and it works great. We've thought of a third query that would be worthwhile to add in. eg
CREATE VIEW stem_alumni as
SELECT name, email
FROM students
WHERE graduated < 2019 AND major = 'Engineering'
UNION
SELECT name, email
FROM alumni
WHERE current_employer = 'Google'
The problem is when I try to add a third query in
UNION
SELECT name, email
FROM professors
WHERE department = 'Engineering'
it'll persist for maybe an hour, but then revert to just the original query.
I've run CREATE OR REPLACE VIEW... and dropping/recreate and get the same result.
How do I get an updated view definition to persist?
Adding more context
I created the view using DBeaver, a local SQL client using my specific Redshift credentials. The view is called by Periscope, our cloud-based BI tool using shared credentials. Querying the view in Periscope or separate DBeaver windows will eventually revert the view to its original definition.
Redshift shouldn't have a 'memory' of the view's prior DDL that it could revert to. I'm inclined to agree with the comments that something else is overwriting the updates to the view's DDL after you have committed them.
You should be able to see if something is overwriting the view, by querying the stl_query table:
SELECT q.starttime
, u.usename
, q.querytxt
FROM pg_user u
JOIN stl_query q ON u.usesysid = q.userid
WHERE POSITION('<view_name>' IN q.querytxt) > 0
ORDER BY q.starttime DESC
;
This table only contains recent query information (2-5 days according to the Redshift Documentation), so if you haven't experienced this behavior from the view within that timescale, you may need to force it to occur again in order to troubleshoot who/what is altering the DDL.
Additionally, if the view is being overwritten by a user other than yourself, you will need to query stl_query using a super user account (by default, non-super users will only be able to view information for queries that they themselves have executed).
I'm new to database design and faced a problem while trying to design one. Here's my system:
There are 3 different softwares (SW_A, SW_B, SW_C) and 500-600 users. Each user can use any of the softwares and do some work. Each work is sent to the database approx. 6 times per minute. (Randomly for users and software)
I want a database to keep all user actions related to the softwares for archive and also want to keep track instant actions for all users related to these softwares. (Only the latest action for distinct users for each software)
So I designed sth. like that:
Table: SW_X_DATA_ARCHIVE
Columns :
UserID
Data_Date
SW_A_ActionID
Table: SW_X_INSTANT_DATA
Columns:
UserID
Data_Date
SW_X_ActionID
When a new data comes, it updates INSTANT_DATA table's related row if that UserID exists, inserts new row with UserID and data if not. Then inserts the same data to DATA_ARCHIVE table.
The question is: Is there a better way for this kind of work? Views maybe?
(I've read sth. about materialized views but not sure how to use it here.)
Thanks
Only the latest action for distinct users for each software
Your INSTANT table should have a unique composite key on (USERID,SOFTWAREID) so that it is not possible to have more than one row relating to a user's use of a software.
The ARCHIVE table does not require such a unique key or constraint.
You would insert into INSTANT if a row for (USERID, SOFTWARE) does not exist, or update the row if it already exists.
Typically, an insert/update trigger would be placed on the INSTANT table, to insert a row into ARCHIVE whenever a change happens to INSTANT.
EDIT: alternatively, you could maintain a single table, and have a view that selected only the latest action per user per software. You'd need to timestamp every action and place a composite index (non-unique) on (USERID,SOFTWAREID). The decision on which approach to take might be based on relative performance. I would tend to favor the trigger if the Latest Action information was needed frequently.
The scenario : an Oracle 11g database containing some sensitive user data that could result legal liabilities if disclosed to the wrong party.
The desired effect : only a certain user, connecting from a certain IP, can see the column that contains this sensitive user data
I am not sure that hidden columns or virtual columns are the right ways to do this. It seems that Fine-Grained Access Control could help. I am not sure of what is the best solution. The restriction by IP is probably done at the listener level?
The question :
How can we restrict the visibility of a column so it is only available only to a specific user? All the other users would never see the column, not even when doing a "DESC TABLE_WITH_SENSITIVE_DATA"
Thanks for any tips.
Simplest way to do this is to create a view on the table that does not contain all of the columns. Don't grant select on the table, but only on the view.
The "proper" way to do this is with Fine-Grained Access Control (Virtual Private Database), which can replace the contents of columns with a NULL if certain conditions are not met.
See the example here: http://docs.oracle.com/cd/B28359_01/network.111/b28531/vpd.htm#autoId17
You can probably build this sort of functionality yourself if you're feeling both impoverished and skilled.
Do you the ability to modify roles and create views? Perhaps you could create two separate views and grant access to two different roles for that table. All users that are restricted from seeing the sensitive data would belong to a "restricted" role and the others would have access to the "unrestricted" role. You would need to grant privileges on each view to the appropriate role.
It is important to note that there are restrictions on updating the underlying data associated with a view. As explained here, views that contain set operators, aggregates and GROUP BY DISTINCT and joins and not modifiable.
We use CRM 4.0 at our institution and have no plans to upgrade presently as we've spend the last year and a half customising and extending the CRM to work with our processes.
A tiny part of model is a simply hierarchy, we have a group of learning rooms that has a one-to-many relationship with another entity that describes the courses available for that learning room.
Another entity has a list of all potential and enrolled students who have expressed an interest in whichever course.
That bit's all straightforward and works pretty well and is modelled into 3 custom entities.
Now, we've got an Admin application that reads the rooms and then wants to show the courses for that room, but only where there are enrolled students.
In SQL this is simplified to:
SELECT DISTINCT r.CourseName, r.OtherInformation
FROM Rooms r
INNER JOIN Students S
ON S.CourseId = r.CourseId
WHERE r.RoomId = #RoomId
And this indeed is very close to the eventual SQL that CRM generates.
We use a Crm QueryEntity, a Filter and a LinkEntity to represent this same structure.
The problem now is that the CRM normalizes the a customize entity into a Base Table which has the standard CRM entity data that all share, and then an ExtensionBase Table which has our customisations. To Give a flattened access to this, it creates a view that merges both tables.
This view is what is used by the Generated SQL.
Now the base tables have indices but the view doesn't.
The problem we have is that all we want to do is return Courses where the inner join is satisfied, it's enough to prove there are entries and CRM makes it SELECT DISTINCT, so we only get one item back for Room.
At first this worked perfectly well, but now we have thousands of queries, it takes well over 30 seconds and of course causes a timeout in anything but SMS.
I'm given to believe that we can create and alter indices on tables in CRM and that's not considered to be an unsupported modification; but what about Views ?
I know that if we alter an entity then its views are recreated, which would of course make us redo our indices when this happens.
Is there any way to hint to CRM4.0 that we want a specific index in place ?
Another source recommends that where you get problems like this, then it's best to bring data closer together, but this isn't something I'd feel comfortable in trying to engineer into our solution.
I had considered putting a new entity in that only has RoomId, CourseId and Enrolment Count in to it, but that smacks of being incredibly hacky too; After all, an index would resolve the need to duplicate this data and have some kind of trigger that updates the data after every student operation.
Lastly, whilst I know we're stuck on CRM4 at the moment, is this the kind of thing that we could expect to have resolved in CRM2011 ? It would certainly add more weight to the upgrading this 5 year old product argument.
Since views are "dynamic" (conceptually, their contents are generated on-the-fly from the base tables every time they are used), they typically can't be indexed. However, SQL Server does support something called an "indexed view". You need to create a unique clustered index on the view, and the query analyzer should be able to use it to speed up your join.
Someone asked a similar question here and I see no conclusive answer. The cited concerns from Microsoft are Referential Integrity (a non-issue here) and Upgrade complications. You mention the unsupported option of adding the view and managing it over upgrades and entity changes. That is an option, as unsupported and hackish as it is, it should work.
FetchXml does have aggregation but the query execution plans still uses the views: here is the SQL generated from a simple select count from incident:
'select
top 5000 COUNT(*) as "rowcount"
, MAX("__AggLimitExceededFlag__") as "__AggregateLimitExceeded__" from (select top 50001 case when ROW_NUMBER() over(order by (SELECT 1)) > 50000 then 1 else 0 end as "__AggLimitExceededFlag__" from Incident as "incident0" ...
I dont see a supported solution for your problem.
If you are building an outside admin app and you are hosting CRM 4 on-premise you could go directly to the database for your query bypassing the CRM API. Not supported but would allow you to solve the problem.
I'm going to add this as a potential answer although I don't believe its a sustainable or indeed valid long-term solution.
After analysing the indexes that CRM had defined automatically, I realised that selecting more information in my query would be enough to fulfil the column requirements of an Index and now the query runs in less then a second.