When i open my csv file in an excel it looks like this -
Header
Value1
Value2
Value3
Value4
Value5
I want to know whether this file actually has commas in it? I am aware that if i have multiple columns i will see the commas
You can easily test that by opening the file in a text editor (e.g. Notepad on Windows). It will show the file as it is in text format, i.e., with commas present (if they are in the file). I would say that if it is single column, it won't have commas (but rather line breaks between the rows), but if you need to be sure just open it with a text editor.
https://www.ietf.org/rfc/rfc4180.txt
Given there is only one value in each record it would not have a comma given the spec.
Within the header and each record, there may be one or more
fields, separated by commas. Each line should contain the same
number of fields throughout the file. Spaces are considered part
of a field and should not be ignored. The last field in the
record must not be followed by a comma. For example:
Related
I have a file I am reading into a blob via datafactory.
Its formatted in excel. Some of the column headers have special characters and spaces which isn't good if want to take it to csv or parquet and then SQL.
Is there a way to correct this in the pipeline?
Example
"Activations in last 15 seconds high+Low" "first entry speed (serial T/a)"
Thanks
Normally, Data Flow can handle this for you by adding a Select transformation with a Rule:
Uncheck "Auto mapping".
Click "+ Add mapping"
For the column name, enter "true()" to process all columns.
Enter an appropriate expression to rename the columns. This example uses regular expressions to remove any character that is not a letter.
SPECIAL CASE
There may be an issue with this is the column name contains forward slashes ("/"). I accidentally came across this in my testing:
Every one of the columns not mapped contains forward slashes. Unfortunately, I cannot explain why this would be the case as Data Flow is clearly aware of the column name. It can be addressed manually by adding a Fixed rule for EACH offending column, which is obviously less than ideal:
ANOTHER OPTION
The other thing you could try is to pre-process the text file with another Data Flow using a Source dataset that has no delimiters. This would give you the contents of each row as a single column. If you could get a handle on the just first row, you could remove the special characters.
I would like to decode the content of csv in PLSQL.
The content of the csv as following.
"HEADER_ROW1_COL1,WITH_COMMA",HEADER_ROW1_COL2,my darling's ccc
DATA_ROW2_COL1,DATA_ROW2_COL2
DATA_ROW3_COL1,DATA_ROW3_COL2
CSV will split the column with a comma. If the cell contains a comma, CSV will enclosed the content of the cell with double quotes.
Threfore, if I want to read the content of the Excel file, I need to follow the steps as following
Read the content of the CSV by line
Split the text with comma
If the comma is enclosed by two double quotes, ignore the comma
Please advise.
Thanks
In a mapping we use delimited flat file having 3 columns.The column separated through comma. But i have a requirement that in between the column there is a column having 2 comma.So how should I process the column in the mapping?
You should have information quoted with "" so whatever is within " is skiped. this way you could differentiate between comma of a piece of information or as a column separator.
We don't know what have you tried, but count the number of commas for each line and separate accordingly (if possible).
I need to read a text file using spring batch process and bellow is a sample file
000115989 AB0001 BC00012 030114 010100 WITHDRAWL FROM SAVING 100.00
It doesn't have any column header and each column has a fixed length and delimited by two blank spaces.
Here I can't use DelimitedLineTokenizer for two blank spaces as columns can also have leading or trailing blank spaces .
Is there any work around so that I read each column with its specific length after that I can trim that.
Take a look at the FixedLengthTokenizer (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/transform/FixedLengthTokenizer.html). This allows you to set how lines are parsed by column instead of by delimiter.
I have got ASCII files and want to convert them into maybe excel or tab/csv delimited text file. The file is a table with field name and field attributes. It also includes index name, table name and field(s) to index if required depending on the software. I don't think it is necessary to think of this. Well, field name and field attributes are enough, I hope so. I just want the information hidden inside. Can you all experts help me to get this done.
The lines are something like this:
10000001$"WORD" WORD$10001890$$$$495.7$$$N$$
10000002$11-word-word word$10000002$$$$$$$Y$$
10000003$11-word word word$10033315$0413004$$$$$$N$$
10000004$11-word word word$10033315$$$$$$$Y$017701$
The general answer, before knowing your ascii file in details, operating system, and so on, would be:
1 - cut the top n-lines, that containg the information you don't want. Leave the filds names, if you want to.
2 - check if the fields are separated by a common character, for example, one comma ,
3 - import the file inside a spreadsheet program, like Excel or OpenOffice Calc. In OOCalc, choose to import the file, then select the correct separating character
that's all.