Code:
#!/usr/bin/env python2
import sys
import ruamel.yaml
yaml_str = '''
---
project: ''
project_lead: &ptl
name: ''
company: ''
email: ''
id: ''
timezone: ''
primary_contact: *ptl
committers:
- <<: *ptl
- name: ''
email: ''
company: ''
id: ''
timezone: ''
- name: ''
email: ''
company: ''
id: ''
timezone: ''
'''
DATA = ruamel.yaml.round_trip_load(yaml_str, version=None, preserve_quotes=True)
indent = 4
block_seq_indent = 2
ruamel.yaml.round_trip_dump(DATA, sys.stdout, default_flow_style=True, indent=indent,
block_seq_indent=block_seq_indent)
Output:
project: ''
project_lead: &ptl
name: ''
company: ''
email: ''
id: ''
timezone: ''
primary_contact: *ptl
committers:
- <<: *ptl
- name: ''
email: ''
company: ''
id: ''
timezone: ''
- name: ''
email: ''
company: ''
id: ''
timezone: ''
ISSUE:
foo.yaml|10 col 3 error| wrong indentation: expected 4 but found 2 (indentation)
If I increase
block_seq_indent = 4
the values following it (email, company, id, timezone) don't follow its indentation.
Is my source YAML 'wrong'? Can I work around this? The ruamel module is great, and I'm able to add values programmatically, but I can't seem to get valid YAML in my output.
Your input has four positions indent on the (nested) block mapping that is the value for project_lead, and it has six indent on the sequence that is the value for committers, with an offset for the dash of four (in sequences the indent is counted to the beginning of the sequence element).
This does indeed go wrong if you use the old API as you do, but with the new API (and the appropriate values), you can do:
import sys
import ruamel.yaml
from cStringIO import StringIO
yaml_str = '''\
---
project: ''
project_lead: &ptl
name: ''
company: ''
email: ''
id: ''
timezone: ''
primary_contact: *ptl
committers:
- <<: *ptl
- name: ''
email: ''
company: ''
id: ''
timezone: ''
- name: ''
email: ''
company: ''
id: ''
timezone: ''
'''
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
yaml.indent(mapping=4, sequence=6, offset=4)
yaml.explicit_start = True
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)
buf = StringIO()
yaml.dump(data, buf)
assert buf.getvalue() == yaml_str
without getting an error.
Please note:
the default_flow_style=True has no effect in your example, they only would affect new mappings and lists, and I am not sure you want to do that for all added values.
I added yaml.explicit_start = True to get the leading ---
Your example starts with an empty newline (directly after the ''', that is as if you did yaml_str = '\n---\nproject: .....'). You cannot really get that output with ruamel.yaml() from a data structure, hence the backslash in my yaml_str.
although this might be considered a bug in the old API, I have no intention of fixing it.
Related
I want to build a pipeline function that replaces a value in a yaml file. For that I want to make both the
pattern and the replacement value variable. I have seen the env-variables-operators article in the yq docs, however I cannot find the relevant section.
I have a yaml file with the following content:
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "1.0.0"
I now want to build a pipeline function that will replace the value of the value key in the yaml.
I can do so with:
$ yq '.spec.source.helm.parameters[0].value = "2.0.0"' myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "2.0.0"
Now I want to make this command customizable.
What works:
$ VALUE=3.0.0
$ replacement=$VALUE yq '.spec.source.helm.parameters[0].value = env(replacement)' myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "3.0.0"
What doesn't work
$ VALUE=3.0.0
$ PATTERN=.spec.source.helm.parameters[0].value
$ replacement=$VALUE pattern=$PATTERN yq 'env(pattern) = env(replacement)'
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "1.0.0"
I have also tried to use strenv and wrapping the replacement pattern in quotes, but it is not working.
Can anyone help me with the correct syntax?
You can import data with env but not code. You could inject it (note the changes in the quoting), but this is bad practice as it makes your script very vulnerable:
VALUE='3.0.0'
PATTERN='.spec.source.helm.parameters[0].value'
replacement="$VALUE" yq "${PATTERN} = env(replacement)" myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "3.0.0"
Better practice would be to import the path in a form that is interpretable by yq, e.g. as an array and using setpath:
VALUE='3.0.0'
PATTERN='["spec","source","helm","parameters",0,"value"]'
replacement="$VALUE" pattern="$PATTERN" yq 'setpath(env(pattern); env(replacement))' myyaml.yml
A have the line in file acounts.list
['# name prueba7#prueba7.com', 'zimbraIsDelegatedAdminAccount: TRUE']
I would like to remove ['# name, all ' and '] What would the result be prueba7#prueba7.com, zimbraIsDelegatedAdminAccount: TRUE
I tried to create task to remove first ['# name
- lineinfile:
dest: /home/acounts.list
state: absent
regexp: "^\[\'\#\sname\s"
... but i got error. Could you please help with solution? Thank you.
use replace instead:
- name: replace
ansible.builtin.replace:
path: /home/acounts.list
regexp: "['\\[\\]]|# *name *"
replace: ''
you could write too: regexp: '[''\[\]]|# *name *'
['\\[\\]] means replace all ' or [ or ] (or == [])
so as [ and ] are special char, you ahve to protect by \ with " you have to protect \ too by another \ (not needed when surrounded by ')
| means or and the other expression is # *name * * means zero or more space
After upgrading a site with TYPO3 version 8.7 via v. 9.5 to version 10.4.2, the cache is "unstable" for lack of a better word. The problem is that after a few hours the cache contains faulty pages. That is, the links are either generated without the href attribute or with another language, i.e. /fr/ instead of /de/.
The fix: empty cache via install tool.
Approach:
stopped all back-end work, error persists,
checked and changed cache configuration to 'database', to no avail,
disabled caching 'no_cache = 1', no faulty pages are produced.
Question: What produces the faulty pages in the cache after a few hours? Are there possibly 'left-over-configurations' or settings from the version 8.7?
config.yaml
base: 'https://metu.de'
baseVariants: { }
errorHandling: { }
languages:
-
title: Deutsch
enabled: true
base: /de/
typo3Language: de
locale: de_DE.UTF8
iso-639-1: de
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
flag: de
languageId: '0'
-
title: English
enabled: true
base: /en/
typo3Language: default
locale: en_US.UTF8
iso-639-1: en
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: us
languageId: '1'
-
title: French
enabled: true
base: /fr/
typo3Language: default
locale: fr_FR.UTF8
iso-639-1: fr
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: fr
languageId: '2'
-
title: Italian
enabled: true
base: /it/
typo3Language: default
locale: it_IT.UTF8
iso-639-1: it
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: it
languageId: '4'
-
title: Russian
enabled: true
base: /ru/
typo3Language: default
locale: ru_RU.UTF8
iso-639-1: ru
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: ru
languageId: '5'
-
title: Spanish
enabled: true
base: /es/
typo3Language: default
locale: es_ES.UTF8
iso-639-1: es
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: es
languageId: '3'
rootPageId: 1
routes: { }
websiteTitle: Metu
I had the same issues when upgrading Typo3 from v 7.6 to v 10.4. Removing the old typoscript language configuration (config.linkVars, conditions with GP:L etc.) fixed them.
I'm having some issues with a Jenkins Job Builder YAML file which contains an attribute (message-content) with "{}" characters in it:
- job-template:
id: senderjob
name: '{job-prefix}{id}'
command: 'echo "executed with $PARAMETER"'
type: freestyle
properties:
- ownership:
enabled: true
owner: user1
- build-discarder:
num-to-keep: 100
parameters:
- string:
name: PARAMETER
default: 'param'
description: 'default parameter for message.'
# template settings
builders:
- shell: '{command}'
publishers:
- ci-publisher:
override-topic: VirtualTopic.abcd.message
message-type: 'Custom'
message-properties: |
release-name=$PARAMETER
PARAMETER=$PARAMETER
message-content: '{"release-name" : "1.0"}'
The error reported is:
jenkins_jobs.errors.JenkinsJobsException: release-name parameter missing to format {release-name : 1.0}
So it looks like it's trying to expand "release-name" with a parameter.
So I have added it as parameter:
- string:
name: release-name
default: '1.0'
description: 'default parameter for release.'
It still reports the same error. Should I include the parameter somewhere else or escape it ? I couldn't find any YAML escape for "{}" characters though.
Any idea?
You should add the following to the configuration file
[job_builder]
allow_empty_variables = True
Link: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template-1
I want to append a heredoc string after a specific heredoc string if file doesn't contain it already.
Eg.
Here are 2 files:
# file1
Description:
I am a coder
Username: user1
Password: password1
# file2
Description:
I am a coder
Username: user2
Password: password2
Address:
Email: user#gmail.com
Street: user street 19 A
I want to add:
Address:
Email: user#gmail.com
Street: user street 19 A
if file doesn't contain it already, and after:
Description:
I am a coder
So in the above files it will be added to the first one only. and that file will then look like this:
# file1
Description:
I am a coder
Address:
Email: user#gmail.com
Street: user street 19 A
Username: user1
Password: password1
How could I do this in Ruby?
The question is not well formulated - you are getting the concept "Here Docs" confused.
I'll leave some code wich I hope helps in your task, in some way
end_of_line_delimiter = "\n"
file1_arr = File.read('file1.txt').split(end_of_line_delimiter) #Array of lines
file1_has_address = file1_arr.index {|a_line| a_line =~ /^Address:/ }
unless file1_has_address
#file1 does not contain "Address:"
#Build address_txt
email = "some#email"
street = "some street"
address_txt = <<END
Address:
Email: #{email}
Street: #{street}
END
#Insert address_txt 2 lines after the "Description:" line
description_line_index = file1_arr.index {|a_line| a_line =~ /^Description:/ }
raise "Trying to insert address, but 'Description:' line was not found!" unless description_line_index
insert_line_index = description_line_index + 2
file1_arr.insert(insert_line_index, *address_txt.split(end_of_line_delimiter))
end
#file1_arr will now have any Address needed added
file1_txt = file1_arr.join(end_of_line_delimiter)
puts file1_txt
Please report back any success with the code :)