How do I format a YAML document like this so that PyYAML can parse it properly?
Data: Some data, here and a special character like ':'
Another line of data on a separate line
I know that the ':' character is special so I have to surround the whole thing in quotations like so:
Data: "Some data, here and a special character like ':'
Another line of data on a separate line"
And in order to add a new line, I have to add '\n':
Data: "Some data, here and a special character like ':'\n
Another line of data on a separate line"
Is there anyway to format the YAML document so I don't have to add the '\n's in order to have a new line?
For multi-line scalars, you can use blocks. The character | denotes the start of a block. Use:
Data: |
Some data, here and a special character like ':'
Another line of data on a separate line
If the extra newline that NullUserException's solutions is adding is a problem you should be using:
Data: |-
Some data, here and a special character like ':'
Another line of data on a separate line
Related
I have several YAML files that store SQL scripts in them (as multiline strings). I have a Python script that takes all of these scripts and aggregates them into a single table.
Whenever I make an update to a YAML file, it converts the SQL text to a regular string (with \n's to indicate line breaks). Is there a way to preserve the multiline formatting when I make updates to the YAML file?
For multi-line scalars, you can use blocks. The pipe symbol character | to denote the start of a block.
For example:
Data: |
Some data, here and a special character like ':'
Another line of data on a separate line
Also you can check the YAML Multiline
Table contains 3 columns
ID -integer
Name-varchar
Description-varchar
A file with .FILE extension has data with delimiter as |
Eg: 12|Ramu|"Ramu" is an architect
Command I am using to load data to db2:
db2 "Load CLIENT FROM ABC.FILE of DEL MODIFIED BY coldel0x7x keepblanks REPLACE INTO tablename(ID,Name,Description) nonrecoverable"
Data is loaded as follows:
12 Ramu Ramu
but I want it as:
12 Ramu "Ramu" is an architect
Take a look at how the format of delimited ASCII files is defined. The double quote (") is an optional delimited for character data. You would need to escape it. I have not tested it, but I would assume that you double the quote as you would do in SQL:
|12|Ramu|"""Ramu"" is an architect"
Delimited files (CSV) are defined in RFC 4180. You need to either use quotes for the entire field or none at all. Only in fields beginning and ending with a quote, other quotes can be used. They need to be escaped as shown.
Use the nochardel modifier.
If you use '|' as a column delimiter, you must use 0x7C and not 0x7x:
MODIFIED BY coldel0x7C keepblanks nochardel
I am trying to remove a newline characters from with in quotes in file
I am able to achieve that using the below code
awk -F"\"" '!length($NF){print;next}{printf("%s ", $0)}' filename.txt>filenamenew.txt
Note I am creating a new file filenamenew.txt is this avoidable can i do the command in place the reason I ask is because files are huge.
my file is pipe delimited
sample input file
"id"|"name"
"1"|"john
doe"
"2"|"second
name
in the list"
using the above code I get the following output
"id"|"name"
"1"|"john doe"
"2"|"second name in the list"
but I have a huge files and i see in some of the lines have ^M character in between quotes example
second sample input file
"id"|"name"
"1"|"john
doe"
"^M2"|"second^M^M
name
in the list"
o/p using above code
"id"|"name"
"1"|"john doe"
name in the list"
so basically if there is a ^M in the line that string is not being printed but i read online ^M is equal to \r so i used
tr -d'\r'< filename.txt
I also tried
awk-F"|"{sub(/^M/,"")}1
but it did not remove those characters (^M)
A little background on why i am doing this
I am extracting data from a relational table and loading into flat file and checking if the counts between table and file matched but since there is \n in columns count(*) vs wc-l in file is not matching.
final resolution:
i don't want to delete these unprintable characters in the long run but want to replace it with some character or value(so that counts between table and file matches) and then when i am loading it back to a table i want to again replace the value that i have added effectively as a place holder with \n or ^M what was originally present so that there is no tampering of data from my side.
Any suggestions is appreciated.
thanks.
I'm trying to load a csv that contains the character "|" without success
can i escape it or use other techinieue?
can you help?
thanks
If you are using '|' as your delimiter and some fields also contain '|', you can escape them as '\|'. (Or with some other character, if you've changed your escape character. But by default, '\'.)
If you have a lot of these, it might be easier to change your delimiter character. It doesn't have to be '|'. For example, you can do this:
=> COPY t1 FROM '/data/*.csv' DELIMITER '+';
You can use any ASCII value in the range E'\000' to E'\177', inclusive. See the documentation for COPY parameters.
I am trying to get variables into a block of text which will later be echoed to a file.
Problem is the $VAR is not getting converted into the variable's value ??
VAR="SOME Value"
read -d '' WPA <<"BLOCK"
Description='WIFI'
Interface=wlan0
Connection=wireless
IP=dhcp
Security=wpa
Key=$VAR
BLOCK
echo "$WPA"
Also, is it possible to append further text to the WPA Block ?
When you quote the delimeter of a heredoc, variables are not interpolated. Just drop the quotes:
read -d '' WPA <<BLOCK
Description='WIFI'
Interface=wlan0
Connection=wireless
IP=dhcp
Security=wpa
Key=$VAR
BLOCK
Why don't you just say
WPA="Description='WIFI'
Interface=wlan0
Connection=wireless
IP=dhcp
Security=wpa
Key=$VAR
"
?
There's not really a need to use read in your case.
If you want to echo append text to $WPA, do it like this:
WPA="$WPA
first appended line
second appended line
"
but be aware that you insert an extra newline this way - $WPA had a newline at the end and there's another one at the beginning of the new text. To avoid this, use
WPA="${WPA}first appended line
second appended line
"
The {} quotation delimits the variable name. Using
WPA="$WPAfirst appended line
would look for a variable named WPAfirst.
is it possible to append further text to the WPA Block ?
$WPA is just a normal shell variable (that happens to contain a multi-line string), so you can append to it with +=; e.g.:
WPA+=$'\nanother line\nand another'
If you wanted to append the content of another heredoc, assign it to a separate variable and append that to WPA (but, as #GuntramBlohm points out, you can just as easily assign/append a multi-line string directly).