I would like to now, if has a way to create automatically columns on Data Conversion (SISS) .
I have an Excel Source with a med19g (which represents 2019 year).
Next step, I have my Data Conversion:
As you see, med19g columns is on it.
So, next year will be added a med20g (represents 2020 year) column in Excel, and I'd like to find a way to add this column automatically or way to prevent that column on my solution (Data Conversion).
Does someone have any idea how I can I get it?
I'm using Visual Studio 2015
Thanks in advance
You will need to use "Flat file source" in the data flow to get the source file. After that, Use "Derived Column" under Data Flow Transformations category and add the desired field like below:
You can edit your output column name later by using Advanced Editor (Right click on Derived column)
If you need to check whether your source file has the same column or not already, you can add a "Script Component" (Source) between the "Get Flat File" and "Derived Column" components. In the script, check your column names and use a boolean value to decide if a column name is existing already or not. How you can do is explained clearly in this link:
https://dichotic.wordpress.com/2006/11/01/ssis-test-for-data-files-existence/
Related
I want to filter a column that spans from 2014-2019 to 2017-2018 in VS with SSIS.
I have tried different things but none seem to work.
Derived Column date in your example is likely what you're looking for.
The Week column is of a date type DT_DBDATE. Your string "2017-01-01" should be getting promoted to a data date type so the boolean check will identify if the lower bound is being met.
You'd either need to create a second derived column to check against the upper bound or as #vhoang indicates, change the logic to just extract the year from the date column.
YEAR([Week]) >= 2017 && YEAR([Week]) < 2019
Now, you have a column that flags each row as meets criteria or not (year is 2017 or 2018)
You will then need to do something with that. The SSIS something is called a Conditional Split. I would add a new path called OutOfConsideration and the logic there would be the inverse of our above Derived Column Derived Column date which is true if the year meets our criteria.
![Derived Column date]
Now connect your destination, or additional processing steps, to the Conditional Split's default output path. If you need to do processing on the invalid data, that'd be the OutOfConsideration path.
Finally, to get the best performance out of SSIS, only bring the rows into it that you need. If the source data is in a system that supports filtering, filter the data there. It is easy to click click click design SSIS packages but it is better long term for you to write custom queries to only bring the required columns and rows into the data flow. Less work for all around, lower maintenance cost, etc
Within the data load wizard that comes with APEX 18.1, after you choose your csv file to be uploaded you are offered a "TARGET COLUMN" drop down LOV which defaults to "DO NOT LOAD". It is possible to tell APEX which values you want in this LOV. I have done this.
My issue is, that this is quite laborious. Your users will not necessarily know which value you want them to pick from the LOV to map the related column when they are using a csv file with no header. As they are going to be doing.
Does anyone know how to change the "DO NOT LOAD" value in the LOV to another value? If I could get it to default to a column of my choosing, this would be great. Alternatively, there's a "SOURCE COLUMN" field in the wizard.
Getting the "SOURCE COLUMN" field to denote which column I wish users to map to the LOV value would be something also. Has anyone faced this before? Does anyone know if it is possible to do what I am suggesting as a work around? Thanks for looking and for your thoughts.
APEX does the column mapping automatically by checking the name of the column in the csv(the first row is names usually).
So if the names of the columns match in the table and the csv, it will connect them by itself. What you can also do is set column aliases.
If you want to edit the existing data load, you can go to Shared Components- Data Load definitions and pick the one you are using in there.
Then you can set column aliases there. But afaik you can only do one alias per column there.
I've made the mistake of using the 'Calculate and Replace Column' feature to replace the wrong column, and realized after the fact. The column I replaced corresponds to last names and is important. I would like to retrieve this column but maintain my other 15 or so data transformations. Ideally, I would like to remove this transformation, but I've come up empty so far. Here's what I've tried:
I tried adding the 'last name' column again from the same external source, using >Insert >Columns... I also tried renaming this column to avoid the data transformation. Unfortunately, this resulted in an entirely empty column, so it did not successfully match to the table or was affected by the transformation..
I checked the source information, and found exactly the 3-4 lines that I wish were not there. I thought it might be possible to edit this but haven't found a way. This seems like it would be the easiest.
Another idea I had was I could replace the data table with the same source, and repeat all of the transformations from the replace data table dialogue (excluding the bad one). This is my next plan of attack, but I figured I would come on here to see if there's an easier way first.
Thanks in advance!
Good News for YOU!!! #jeremyVollen.
It is possible to 'edit' your transformation per Tibco article 44098.
Resolution: If there are more then one transformations on a data table and you need to edit any of those transformation, follow the steps below:
Go To Edit >> Data Table Properties.
Select the desired data table inside which the transformation has been added and click on Refresh Data > With Prompt.
A new window will pop up which will allow you to make the desired changes in each of the transformations.
unfortunately it is NOT possible to reverse data table transformations.
it IS possible to undo the transformations with Edit>>Undo or CTRL+Z, but that's as far as it goes.
my strategy for dealing with this is (in accordance with your #3) to visit Edit>>Data Table Properties, select the table I'm interested in, select Source Information, then copy the contents of the textarea and paste it into notepad. then, I'll File>>Replace Data Table and start over from the beginning while keeping the notepad open so I don't miss any steps.
I realize it's not ideal, but there is unfortunately not another way.
I have installed and made some successful product imports in to Magento using Magmi, but as soon as I try to import any data where the spreedsheet columns have commas [,] Magmi will not perform the import.
For example when I save the data in this speadsheet as a CSV file Magmi successfully imports the data;
http://i.imgur.com/PpDt0PS.png
However, Magmi refuses to import the data in the table below, where you can see in column F I have added data that include 'commas'.
http://i.imgur.com/MtGJPCw.png
Can anyone advise. I am using an Apple Mac with OpenOffice to prepare and save my data.
Is the data not importing entirely, or is just the visibility column not being set?
Visibility is a Magento core attribute which Magmi can set by using exact numerical option id value.
Generally, the option values you want to use for the visibility field are as follows:
Not Visible Individually = 1
Catalog = 2
Search = 3
Catalog, Search = 4
So in your case, if you want to set these products to Catalog, Search, you can set the visibility column value to 4.
To double-check that the above mapping is correct for your instance of Magento, the easiest way is as follows:
Go edit any product
Look for the Visibility drop down field, and right click > inspect element
In the developer tools, take note the values associated to each label.
Below an example of the process and what to look for.
Axel is correct, you should set the data to the numerical value 4.
But I do also recommend you explore a better way to export CSV content from Open Office. You may have to start a new document because I find I only see the dialogue below once and then I never see it again. Create a new document, paste your data into it. Choose save-as, select CSV, and save it. Eventually you should see the dialogue below. Change the encoding to UTF 8, the text delimiter to " and tick the 'Quote all text cells' box.
Then you should be able to have any cells with commas or other things in them. Always ensure you CSV files are quoted. "like","this","so you, can","have commas, in them". It is worth inspecting your CSV file in a text editor to see the format is as expected before uploading it to MAGMI.
I've got a DataSet in VisualStudio 2005. I need to change the datatype of a column in one of the datatables from System.Int32 to System.Decimal. When I try to change the datatype in the DataSet Designer I receive the following error:
Property value is not valid. Cannot change DataType of a column once
it has data.
From my understanding, this should be changing the datatype in the schema for the DataSet. I don't see how there can be any data to cause this error.
Does any one have any ideas?
I get the same error but only for columns with its DefaultValue set to any value (except the default <DBNull>). So the way I got around this issue was:
Column DefaultValue : Type in <DBNull>
Save and reopen the dataset
Since filled Datatables do not entertain a change in the schema a workaround can be applied as follows:
Make a new datatable
Use datatable's Clone method to
create the datatable with the same
structure and make changes to that
column
In the end use datatable's ImportRow
method to populate it with data.
HTH
For those finding this via Google and you have a slightly different case where your table has got data and you add a new column (like me), if you create the column and set the datatype in separate statements you also get this same exception. However, if you do it in the same statement, it works fine.
So, instead of this:
var column = myTable.Columns.Add("Column1");
column.DataType = typeof(int); //nope, exception!
Do this:
var column = myTable.Columns.Add("Column1", typeof(int));
I have found a work around. If I delete the data column and add it back with the different data type, then it will work.
Close the DataSet in the visual designer
Right click the dataset, choose Open With...
Choose XML (Text) Editor
Find the column in the XML, in your dataset it will look something like:
<xs:element name="DataColumn1"
msprop:Generator_ColumnVarNameInTable="columnDataColumn1"
msprop:Generator_ColumnPropNameInRow="DataColumn1"
msprop:Generator_ColumnPropNameInTable="DataColumn1Column"
msprop:Generator_UserColumnName="DataColumn1"
type="xs:int"
minOccurs="0" />
Change the type="xs:int" to type="xs:decimal"
Save and close the XML editor
You may need to right click the DataSet again and choose Run Custom Tool
Its an old Question but it still can happen at VS 2019
Solution:
Change the DefaultValue to <DBNull>
Save the Dataset
Close the DataSet Designer
Re-Open the Designer
Now it should be possible to change the type without any problem.