I am trying to learn SSIS to create SharePoint packages for our company site. Through trial and error, I have learned to create basic packages that will extract specific data from existing SP source lists and populate others.
I found basic information on the web that has helped me thus far. Unfortunately, I cannot find information on an order of procedure for effectively using some of the SSIS containers
As an example, I know that a SP List Source must be followed by a Sort before a Merge Join Transformation. However, I am uncertain if a Derived Column should precede a Conditional Split, or if it really matters. Additionally, there are 2 connection points on Derived Column and Conditional Split containers, one Red and the other Blue. I assume they are used to connect a Derived Column to multiple Conditional Split containers, or vice versa, but I really don’t know for certain.
Can anyone help by providing me with a web source that doesn’t just tell me what a container does, but how to use it effectively. I’m ready for the next steps, but I don’t have reliable connections to get the information I need. BTW... I am using Visual Studio 2012 and SP 2013.
Thanks in advance for any help that can be given.
Related
I'm using Visual Studio Pro 2013 and want to use the Fuzzy Lookup task but there seems to be a bug that prevents the component from connecting to the reference tables.
A reference file and table is specified in the Connection tab which all seems fine but the Columns tab is also needed to create the reference links between different fields of the data however there is nothing there:
The error messages are as follows:
I've read elsewhere this was a known bug on older versions of SSIS from about 2005 - anyone know what the problem is here and how I can fix this?
I should add that the connection manager and the table seem fine as they have been used many times elsewhere in the project. I've tried recreating the data flow in a new document and even restarting my PC but this simply won't work. I should add that I've not used the Fuzzy Lookup before but have looked at several references and know that the column tab should be populated with data and not be an issue.
Many Thanks,
Kw
Not a bug, it's how the product works. From the manual, it specifies the reference table must be a table in sql server. A table in Access, therefore, is unsuitable for use in the fuzzy lookup component.
The transformation needs access to a reference data source that contains the values that are used to clean and extend the input data. The reference data source must be a table in a SQL Server database
Visual Studio 2013 has a feature that allows for performing a data compare between your SSDT project and a target database.
According to another post here on SO, there are certain requirements with regards to performing such a compare.
Those requirements taken into consideration, I want to do something like this as a part of our build and deployment process:
Publish any DB schema changes to the target database(s) to make sure that source and target have exactly the same tables, columns, SP's, etc. to comply with the requirements mentioned in the link above
Run a data compare and generate an update script, or publish any changes in the source DB directly to the target DB
Currently, I have a script which takes care of bullet no. 1 by doing a schema compare, using a DACPAC, via sqlpackage.exe. It does not look like it is possible to perform a data compare using sqlpackage, though, and I have not found any other alternatives yet. In VS 2010 it was possible to run a data compare via the command window, but I have not seen any documentation regarding this in VS 2013...
Thus, my question is if there exists an API and/or other tools that allows for a data compare to be run programmatically through e.g. a Powershell script.
It appears you are correct, for schema diff there is command line support as long as SSDT is installed on disk (more details here), but there is no programmatic interface yet for data compare and update.
We are looking to introduce odac into our application but I am running into a number of issues and I can't seem to find any solutions that fix our issues.
We are using an oracle database and trying to use ODAC 12c Release 1 (12.1.0.1.0) with Oracle Developer Tools for Visual Studio.
In our model we would like to have multiple schemas so we can perform cross schema queries. The schemas which we select in the filter for the database connection appear when we are creating the model. However when we try to update our model from the database only the default schema is visible. Sometimes this can be fixed by opening the relevant part of the database in Server Explorer in Visual Studio but this doesn't always work. This fix never works after we add multiple connection strings for the same model (depending on the location of the user will depend on which database the user gets directed to).
The next issue we are having is that we can't the return types of stored procedures to be auto-generate. I have tried to retrieve the column information but it is never able to retrieve the metadata. I have seen a few suggestions of modifying the stored procedures then getting the column information and reverting the stored procedures but this is something we would like to avoid. Also the suggestions don't seem to work on oracle databases (but that could be me, I don't have much experience with databases).
The final issue (this is a minor issue) is that I cannot figure out how to get the generate code to omit underscores from the classes/methods generated. This isn't a huge issue it is purely to make it easier migrating the code we have.
Sorry if a similar question has been posed before. There are a lot of deployment questions but none seemed to address my problem.
Anyway. I'm working with asp.net, C# and using Visual Studio.
The Organization I'm working in is changing rapidly. There are a lot of projects coming in the pipeline that will require multiple code changes and iterative deployments over the next few months. While working, these changes are always 'on the forefront', so sometimes I have to code certain parts of the same program multiple times.
Since these projects are all staggered, I can't just make one sweeping change all at once; I have to deploy and redeploy the same program multiple times, using only the changes that are required for that deployment.
If this is confusing, here's a simple example:
Application is being used on an Intranet. This application calls our Database, using Driver A.
There are two environments, test and production.
Certain Stored procedures have to be called with parameters that register 'Test' to allow certain other applications to run even with bad data (for testing purposes).
When deploying applications, these stored procedures have to be modified, removing Test parameters
We have an Operating System upgrade, allowing us to move to a much faster Driver B, but requires changes to be made to the code to use Driver B.
So that's two wholly different deployments where some code must be changed for Deployment 1 and other code must be changed for Deployment 2.
Currently I'm just using notepad for an overall change list, regular debugging break points and a multitude of in-code comments, and then I manually slog through the code to make sure that everything is changed. With hundreds of thousands of lines of code over multiple files, classes, objects, etc. this gets pretty tedious, as well as there being a good chance of missing something (causing it to break) or pushing wrong changes (causing it to either break or allow bad data).
Is there a tool that could be used to help in this situation? Preferably one that I can discern what needs to change for Deployment A and what needs to change for Deployment B? I'm also open to hearing other schools of thought as well (tips are definitely accepted!)
Sure, I understand your problem.
I would suggest a couple of things
Installers : Why don't you think of installers, there are loads of installers i.e Install shield, Wix, MSI installer.
These installers will give you flexibilty to update files which you need to update, i.e. Full Control.
But you need to choose the best of them, I have worked around MSI and Wix a lot, so I know this can sort your problem, however its your call.
Publish : I haven't played around much with this, I have just done website publish. However I know it does wonders, so try it also.
I usually create a solution folder in Visual Studio and put my DB scripts in them. I always use at least this set of scripts:
Drop model
Create model script
User functions
Stored procedures
Static data (lookup tables)
Test data (not deployed)
Then I simply combine them and run against an SQL Server so I'm able to recreate the whole DB in a single step (by combining these scripts into a single one and executing it).
Anyway. I've never used projects in either:
Visual Studio or
SQL Management Studio
I've tried creating SQL Server 2008 Database Project in Visual Studio 2010, but I'm somehow overwhelmed by all the possible server settings (which I prefer to stay default as set on the server anyway). So I'm a bit confused: Should I use this project template or should I just do the same thing I always did?
What do you use and why? What are advantages I may benefit from by using either?
If I were you I would continue to do it the way you are doing it. In fact I do! The advantages of having the actual .sql files right there in a folder for you to use/edit/look at in my opinion are far better than the advantages you get by using a DB project. DB Project would be used if you were doing something like Storage Reports, were you have to communicate with like 8 databases and compare then to 8 different databases and save result sets etc... Now don't get my wrong there are advantages of Database Projects, I just don't think they are actually doing much help when you have such a simple setup that works already.
Advantages of the SQL Server 2008 Database Project in VS10:
Not having to switch back and forth
from your current client you use to
communicate with your SQL server.
Decent Data and Schema compare tools.
Gives you a one-click way to reverse
engineer a database into source
control, and keep it up to date.
You can compare projects to physical
databases and vice-versa. (This makes it pretty easy to keep your database up to date, no matter where you make change it: file system database project, or in the physical database itself)
If the current tool your using is not specifically tailored to SQL Server, this one is.
Extremely helpful if you need to do
unit tests directly on the database
without using abstractions.
If you're looking for something a little less complicated, you might want to try SQL Source Control. This won't even require you to maintain scripts, as it doesn't this for you behind the scenes. It will, however, only work as a solution for you if you use either TFS or SVN. And it costs $295...
It has a 28-day trial period, so if you're happy to try it out, I'd be interested in your feedback.