After upgrading a site with TYPO3 version 8.7 via v. 9.5 to version 10.4.2, the cache is "unstable" for lack of a better word. The problem is that after a few hours the cache contains faulty pages. That is, the links are either generated without the href attribute or with another language, i.e. /fr/ instead of /de/.
The fix: empty cache via install tool.
Approach:
stopped all back-end work, error persists,
checked and changed cache configuration to 'database', to no avail,
disabled caching 'no_cache = 1', no faulty pages are produced.
Question: What produces the faulty pages in the cache after a few hours? Are there possibly 'left-over-configurations' or settings from the version 8.7?
config.yaml
base: 'https://metu.de'
baseVariants: { }
errorHandling: { }
languages:
-
title: Deutsch
enabled: true
base: /de/
typo3Language: de
locale: de_DE.UTF8
iso-639-1: de
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
flag: de
languageId: '0'
-
title: English
enabled: true
base: /en/
typo3Language: default
locale: en_US.UTF8
iso-639-1: en
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: us
languageId: '1'
-
title: French
enabled: true
base: /fr/
typo3Language: default
locale: fr_FR.UTF8
iso-639-1: fr
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: fr
languageId: '2'
-
title: Italian
enabled: true
base: /it/
typo3Language: default
locale: it_IT.UTF8
iso-639-1: it
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: it
languageId: '4'
-
title: Russian
enabled: true
base: /ru/
typo3Language: default
locale: ru_RU.UTF8
iso-639-1: ru
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: ru
languageId: '5'
-
title: Spanish
enabled: true
base: /es/
typo3Language: default
locale: es_ES.UTF8
iso-639-1: es
websiteTitle: ''
navigationTitle: ''
hreflang: ''
direction: ''
fallbackType: strict
fallbacks: ''
flag: es
languageId: '3'
rootPageId: 1
routes: { }
websiteTitle: Metu
I had the same issues when upgrading Typo3 from v 7.6 to v 10.4. Removing the old typoscript language configuration (config.linkVars, conditions with GP:L etc.) fixed them.
Related
Given a docker compose file:
version: '3'
services:
service1:
image: image:name
environment:
- TEMP_ID=1928
volumes:
- type: bind
source: local/path/to
target: container/path/to
ports:
- 8900:8580
service2:
image: image:name
environment:
- TEMP_ID=1451
volumes:
- type: bind
source: local/path/to/1451
target: container/path/to
ports:
- 8901:8580
I am limited to writing a bash script to add a service to the above template based on its content. some of the values are being directly copied from the last service written in the services array, some fields needs modification.
I managed to extract and prepare all the values I need to add and I am stuck with creating the service object and add it to the existing file.
The script I have so far is:
#!/bin/bash
SERVICE_NAME=$1
TEMP_ID_ARG=$2
#extract data to copy from last configured client.
SERVICES=($(yq eval '.services | keys' docker-compose.yml))
LAST_SERVICE=${SERVICES[${#SERVICES[#]}-1]}
echo "adding user based on last service: $LAST_SERVICE"
IMAGE=$(yq -P -C eval '.services["'$LAST_SERVICE'"].image' docker-compose.yml)
ENVIRONEMNT_ENTRY="TEMP_ID=${TEMP_ID_ARG}"
TARGET_PATH_IN_CONTAINER=$(yq -P -C eval '.services["'$LAST_SERVICE'"].volumes[0].target' docker-compose.yml)
VOLUME_TYPE=$(yq -P -C eval '.services["'$LAST_SERVICE'"].volumes[0].type' docker-compose.yml)
LOCAL_PATH_TO_SOURCES=$(yq -P -C eval '.services["'$LAST_SERVICE'"].volumes[0].source' docker-compose.yml)
PATH_AS_ARRAY=($(echo $LOCAL_PATH_TO_SOURCES | tr "\/" " "))
PATH_AS_ARRAY[${#PATH_AS_ARRAY[#]}-1]=$TEMP_ID_ARG
NEW_PATH_TO_RESOURCE=$(printf "/%s" "${PATH_AS_ARRAY[#]}")
# extract port mapping, take first argument (exposed port) increment its value by 1 (no upper limitation)
# join back together with : delimiter.
PORT_MAPING_AS_ARRAY=($(yq -P -C eval '.services["'$LAST_SERVICE'"].ports[0]' docker-compose.yml | tr ":" " "))
# NO UPPER LIMITATION FOR PORT!!!
PORT_MAPING_AS_ARRAY[0]=$(expr $PORT_MAPING_AS_ARRAY + 1)
NEW_PORT_MAPPING=$(printf ":%s" "${PORT_MAPING_AS_ARRAY[#]}")
NEW_PORT_MAPPING=${NEW_PORT_MAPPING:1}
VOLUME_ENTRY=$(yq -P -C eval --null-input '.type = "'$VOLUME_TYPE'" | .source = "'$NEW_PATH_TO_RESOURCE'" | .target = "'$TARGET_PATH_IN_CONTAINER'"')
test=$(yq -P -C -v eval --null-input ' .'$SERVICE_NAME'.image = "'$IMAGE'" | .'$SERVICE_NAME'.environement = ["'$ENVIRONEMNT_ENTRY'"] | (.'$SERVICE_NAME'.volumes = ["'$VOLUME_ENTRY'"] | .'$SERVICE_NAME'.ports = ["'NEW_PORT_MAPPING'"])')
echo $test
when running, what I thought to be a working assembly command of all parts, returns the following error:
Error: cannot pass files in when using null-input flag
the expected output when calling add_service.sh service3 1234
given the above input file:
version: '3'
services:
service1:
image: image:name
environment:
- TEMP_ID=1928
volumes:
- type: bind
source: local/path/to
target: container/path/to
ports:
- 8900:8580
service2:
image: image:name
environment:
- TEMP_ID=1451
volumes:
- type: bind
source: local/path/to/1451
target: container/path/to
ports:
- 8901:8580
service3:
image: image:name
environment:
- TEMP_ID=1234
volumes:
- type: bind
source: local/path/to/1234
target: container/path/to
ports:
- 8902:8580
As my bash skills are not so strong I welcome any advice or better solution to my problem.
You should do it all with one yq call, passing the external values as variables; that will make the code safer and faster:
service_name="service3" \
temp_id="1234" \
environment="TEMP_ID=1234" \
yq eval '
.services[ .services | keys | .[-1] ] as $last |
.services[ strenv(service_name) ] = {
"image": $last.image,
"environment": [ strenv(environment) ],
"volumes": [ {
"type": $last.volumes[0].type,
"source": (
$last.volumes[0].source |
split("/") |
.[-1] = strenv(temp_id) |
join("/")
),
"target": $last.volumes[0].target
} ],
"ports": [ (
"" + $last.ports[0] |
split(":") |
.[0] tag = "!!int" |
.[0] += 1 |
join(":") |
. tag = ""
) ]
}
' docker-compose.yml
output:
version: '3'
services:
service1:
image: image:name
environment:
- TEMP_ID=1928
volumes:
- type: bind
source: local/path/to
target: container/path/to
ports:
- 8900:8580
service2:
image: image:name
environment:
- TEMP_ID=1451
volumes:
- type: bind
source: local/path/to/1451
target: container/path/to
ports:
- 8901:8580
service3:
image: image:name
environment:
- TEMP_ID=1234
volumes:
- type: bind
source: local/path/to/1234
target: container/path/to
ports:
- 8902:8580
I'm having some issues with a Jenkins Job Builder YAML file which contains an attribute (message-content) with "{}" characters in it:
- job-template:
id: senderjob
name: '{job-prefix}{id}'
command: 'echo "executed with $PARAMETER"'
type: freestyle
properties:
- ownership:
enabled: true
owner: user1
- build-discarder:
num-to-keep: 100
parameters:
- string:
name: PARAMETER
default: 'param'
description: 'default parameter for message.'
# template settings
builders:
- shell: '{command}'
publishers:
- ci-publisher:
override-topic: VirtualTopic.abcd.message
message-type: 'Custom'
message-properties: |
release-name=$PARAMETER
PARAMETER=$PARAMETER
message-content: '{"release-name" : "1.0"}'
The error reported is:
jenkins_jobs.errors.JenkinsJobsException: release-name parameter missing to format {release-name : 1.0}
So it looks like it's trying to expand "release-name" with a parameter.
So I have added it as parameter:
- string:
name: release-name
default: '1.0'
description: 'default parameter for release.'
It still reports the same error. Should I include the parameter somewhere else or escape it ? I couldn't find any YAML escape for "{}" characters though.
Any idea?
You should add the following to the configuration file
[job_builder]
allow_empty_variables = True
Link: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template-1
I am trying to put a together Puppet bolt plan in YAML format.
I got it working in .pp file and here is the plan
plan profiles::chg123456(
TargetSpec $nodes,
) {
apply($nodes) {
logrotate::rule {'proftpd':
path => ['/var/log/proftpd/*.log', '/var/log/xferlog', '/var/log/proftpd.system.log', '/var/log/sftp.log', '/var/log/sftp-xferlog',],
maxsize => '100m',
rotate_every => 'week',
compress => true,
ifempty => true,
missingok => true,
sharedscripts => true,
postrotate => 'test -f /var/lock/subsys/proftpd && /usr/bin/killall -HUP proftpd || :'
}
}
}
It worked and created /etc/logrotate.d/proftpd with all the correct settings.
Now I want to convert to YAML format but no idea how to do that.
Here is what I guessed but bolt plan show keep saying
$ bolt plan show
Parse error in step "chg123456":
No valid action detected (file: C:/Users/puppet/msys64/home/puppet/.puppetlabs/bolt/modules/profiles/plans/chg123456.yaml)
My YAML plan looks like follows
parameters:
nodes:
type: TargetSpec
steps:
- name: chg123456
target: $nodes
logrotate::rules:
proftpd:
path:
- '/var/log/proftpd/*.log'
- '/var/log/xferlog'
- '/var/log/proftpd.system.log'
- '/var/log/sftp.log'
- '/var/log/sftp-xferlog'
maxsize: '100m'
compress: true
ifempty: true
missingok: true
sharedscripts: true
postrotate: 'test -f /var/lock/subsys/proftpd && /usr/bin/killall -HUP proftpd || :'
return: $chg123456
What am I doing wrong?
Thanks
You'll want to use a resources step, and list the resources you want to use in yaml (documentation):
parameters:
nodes:
type: TargetSpec
steps:
- name: chg123456
target: $nodes
resources:
- logrotate::rules: proftpd
parameters:
path:
- '/var/log/proftpd/*.log'
- '/var/log/xferlog'
- '/var/log/proftpd.system.log'
- '/var/log/sftp.log'
- '/var/log/sftp-xferlog'
maxsize: '100m'
compress: true
ifempty: true
missingok: true
sharedscripts: true
postrotate: 'test -f /var/lock/subsys/proftpd && /usr/bin/killall -HUP proftpd || :'
return: $chg123456
In response to one comment, bolt plan convert is only used to convert yaml plans into Puppet plans, not the other way around.
Code:
#!/usr/bin/env python2
import sys
import ruamel.yaml
yaml_str = '''
---
project: ''
project_lead: &ptl
name: ''
company: ''
email: ''
id: ''
timezone: ''
primary_contact: *ptl
committers:
- <<: *ptl
- name: ''
email: ''
company: ''
id: ''
timezone: ''
- name: ''
email: ''
company: ''
id: ''
timezone: ''
'''
DATA = ruamel.yaml.round_trip_load(yaml_str, version=None, preserve_quotes=True)
indent = 4
block_seq_indent = 2
ruamel.yaml.round_trip_dump(DATA, sys.stdout, default_flow_style=True, indent=indent,
block_seq_indent=block_seq_indent)
Output:
project: ''
project_lead: &ptl
name: ''
company: ''
email: ''
id: ''
timezone: ''
primary_contact: *ptl
committers:
- <<: *ptl
- name: ''
email: ''
company: ''
id: ''
timezone: ''
- name: ''
email: ''
company: ''
id: ''
timezone: ''
ISSUE:
foo.yaml|10 col 3 error| wrong indentation: expected 4 but found 2 (indentation)
If I increase
block_seq_indent = 4
the values following it (email, company, id, timezone) don't follow its indentation.
Is my source YAML 'wrong'? Can I work around this? The ruamel module is great, and I'm able to add values programmatically, but I can't seem to get valid YAML in my output.
Your input has four positions indent on the (nested) block mapping that is the value for project_lead, and it has six indent on the sequence that is the value for committers, with an offset for the dash of four (in sequences the indent is counted to the beginning of the sequence element).
This does indeed go wrong if you use the old API as you do, but with the new API (and the appropriate values), you can do:
import sys
import ruamel.yaml
from cStringIO import StringIO
yaml_str = '''\
---
project: ''
project_lead: &ptl
name: ''
company: ''
email: ''
id: ''
timezone: ''
primary_contact: *ptl
committers:
- <<: *ptl
- name: ''
email: ''
company: ''
id: ''
timezone: ''
- name: ''
email: ''
company: ''
id: ''
timezone: ''
'''
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
yaml.indent(mapping=4, sequence=6, offset=4)
yaml.explicit_start = True
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)
buf = StringIO()
yaml.dump(data, buf)
assert buf.getvalue() == yaml_str
without getting an error.
Please note:
the default_flow_style=True has no effect in your example, they only would affect new mappings and lists, and I am not sure you want to do that for all added values.
I added yaml.explicit_start = True to get the leading ---
Your example starts with an empty newline (directly after the ''', that is as if you did yaml_str = '\n---\nproject: .....'). You cannot really get that output with ruamel.yaml() from a data structure, hence the backslash in my yaml_str.
although this might be considered a bug in the old API, I have no intention of fixing it.
i am trying to extract some email ids from a text file but when i run an awk i get a blank. The same seems to work when i try to grep for another pattern.
--------
Mess1288I: Message 'com.admin.AdminServices' on jvm 'MACHINE1' is running.
Additional thread instances: '0'
Deployed: '2/16/16 12:15 PM' in jar file '/www/deploy/dev/MACHINE1/AdminServicesDEV_2015-09-02_00-04-31.jar'
Last edited: '9/2/15 10:25 AM'
UUID: '848c53b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_RequestReceived'
'LoggingProgramName' = 'GetCustomerByAdmin'
Keywords:
--------
Mess1288I: Message 'com.authentication.AuthenticationService' on jvm 'MACHINE1' is running.
Additional thread instances: '0'
Deployed: '2/16/16 12:15 PM' in jar file '/www/deploy/dev/MACHINE1/AuthenticationServiceDEV_2014-11-06_07-18-34.jar'
Last edited: '11/6/14 6:47 PM'
UUID: '84b753b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_RequestSent'
'LoggingProgramName' = 'AuthenticationService'
Keywords:
--------
Mess1288I: Message 'com.authentication.MergeReply' on jvm 'MACHINE1' is running.
Additional thread instances: '0'
Deployed: '2/16/16 12:16 PM' in jar file '/www/deploy/dev/MACHINE1/AccountMergeDEV_2015-08-06_14-33-47.jar'
Last edited: '8/3/15 3:42 PM'
UUID: '1fda53b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_ResponseSent'
'LogLevel' = 'ERROR'
'LoggingProgramName' = 'MergeReply'
'MaxPerInterval' = '5'
'NotificationInterval' = '300'
Keywords:
--------
Mess1289I: Message 'com.authentication.Eligibility' on jvm 'MACHINE1' is stopped.
Additional thread instances: '0'
Deployed: '2/16/16 12:16 PM' in jar file '/www/deploy/dev/MACHINE1/AccountMergeDEV_2015-08-06_14-33-47.jar'
Last edited: '8/3/15 3:42 PM'
UUID: '5fda53b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_ResponseSent'
'LogLevel' = 'ERROR'
'LoggingProgramName' = 'CheckMergeEligibility'
'MaxPerInterval' = '5'
'NotificationInterval' = '300'
Keywords:
--------
Mess1288I: Message 'com.authentication.MergeRequest' on jvm 'MACHINE1' is running.
Additional thread instances: '0'
Deployed: '2/16/16 12:16 PM' in jar file '/www/deploy/dev/MACHINE1/AccountMergeDEV_2015-08-06_14-33-47.jar'
Last edited: '8/3/15 3:42 PM'
UUID: '7cda53b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_ResponseSent'
'LogLevel' = 'ERROR'
'LoggingProgramName' = 'MergeRequest'
'MaxPerInterval' = '5'
'NotificationInterval' = '300'
Keywords:
--------
Mess1289I: Message 'com.authentication.ForgotUserNameEmail' on jvm 'MACHINE1' is stopped.
Additional thread instances: '0'
Deployed: '2/16/16 12:16 PM' in jar file '/www/deploy/dev/MACHINE1/ForgotUsernameEmailDEV_2012-10-23_03-45-49.jar'
Last edited: '10/23/12 2:15 PM'
UUID: '1d7154b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'EmailFromAddress' = 'do-not-reply#hotmail.com'
'EmailSubject' = 'Retrieved User Name'
Keywords:
--------
Mess1288I: Message 'com.authentication.PasswordService' on jvm 'MACHINE1' is running.
Additional thread instances: '0'
Deployed: '2/16/16 12:16 PM' in jar file '/www/deploy/dev/MACHINE1/PasswordServicesDEV_2014-11-04_03-51-52.jar'
Last edited: '11/4/14 3:20 PM'
UUID: 'd9b154b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_RequestSent'
'LoggingProgramName' = 'PasswordService'
Keywords:
--------
Mess1288I: Message 'com.authentication.RegistrationEmail' on jvm 'MACHINE1' is running.
Additional thread instances: '0'
Deployed: '2/16/16 12:17 PM' in jar file '/www/deploy/dev/MACHINE1/RegistrationEmailDEV_2012-07-31_16-44-48.jar'
Last edited: '7/31/12 4:43 PM'
UUID: 'c3d154b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'FromEmailAddress' = 'do-not-reply#hotmail.com'
'LogLevel' = 'ERROR'
'MaxPerInterval' = '5'
'NotificationInterval' = '300'
Keywords:
--------
Mess1288I: Message 'com.authentication.RegistrationService' on jvm 'MACHINE1' is running.
Additional thread instances: '4'
Deployed: '2/16/16 12:17 PM' in jar file '/www/deploy/dev/MACHINE1/RegistrationServiceDEV_2015-10-29_05-29-56.jar'
Last edited: '10/29/15 3:55 PM'
UUID: '16f454b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_RequestSent'
'LoggingProgramName' = 'RegistrationService'
Keywords:
--------
Mess1288I: Message 'com.authentication.UserManagementService' on jvm 'MACHINE1' is running.
Additional thread instances: '4'
Deployed: '2/16/16 12:18 PM' in jar file '/www/deploy/dev/MACHINE1/UserManagementServiceDEV_2015-08-12_05-40-30.jar'
Last edited: '8/12/15 4:06 PM'
UUID: 'a1e455b2-5201-0000-0080-9c722b3eca55'
Start mode: 'Maintained'
Long description: ''
User-defined property names:
'ApplicationSupportGroup' = 'Integration'
'EVENTTYPE' = 'Integration_RequestSent'
'LoggingProgramName' = 'UserManagementService'
Keywords:
--------
Mess1290I: File 'Java_Utility.jar' is deployed to jvm 'MACHINE1'.
Deployed: '2/16/16 12:14 PM' in jar file '/www/deploy/dev/MACHINE1/JDEPCISInternalDEV_2012-03-01_11-56-03.jar'
Last edited: '3/1/12 11:22 PM'
Keywords:
The output i am trying to get is where ever there is AlertEmailList from the text file.
Machine1,/www/deploy/dev/MACHINE1/AccountMergeDEV_2015-08-06_14-33-47.jar,com.authentication.MergeReply,authenticationalerts#hotmail.com
Machine1,/www/deploy/dev/MACHINE1/AccountMergeDEV_2015-08-06_14-33-47.jar,com.authentication.Eligibility,authenticationalerts#hotmail.com
Machine1,/www/deploy/dev/MACHINE1/AccountMergeDEV_2015-08-06_14-33-47.jar,com.authentication.MergeRequest,authenticationalerts#hotmail.com
Machine1,/www/deploy/dev/MACHINE1/RegistrationEmailDEV_2012-07-31_16-44-48.jar,com.authentication.RegistrationEmail,authenticationalerts#hotmail.com
I have been able to get the output with status using awk
cat jvmdeets.txt|sed -n '/Mess129/q;p'|awk -v OFS=, -v r="^'|'$" '/Mess1288I|Mess1289I/ { o=$4; j=$8; s=$NF; gsub(r,"",o); gsub(r,"",j); sub(/\.$/,"",s) } /^Deployed:/ { p=$NF; gsub(r,"",p); print j, p, o, s }'|sort|uniq
But I have not been able to atleast get the email list by replacing the Deployed: in my awk statement with AlertEmailList.
I tried to atleast get the email lisit
cat jvmdeets.txt|awk -v OFS=, -v r="^'|'$" '/Mess1288I|Mess1289I/ { o=$4; j=$8; gsub(r,"",o); gsub(r,"",j) } /^AlertEmailList/ { p=$NF; print p }'
What am i doing wrong?
Note that you're looking with the start of line anchor ^, however the value is space padded and quoted
$ awk '/^AlertEmailList/' file
will return nothing
$ awk '/AlertEmailList/' file
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'AlertEmailList' = 'authenticationalerts#hotmail.com'
'AlertEmailList' = 'authenticationalerts#hotmail.com'
will return the lines.