On running molecule to test my Ansible changes. I was receiving an error with the state.yml file. As shown:
$ molecule test
ERROR: while scanning a simple key
in "/tmp/molecule/my_module/default/state.yml", line 8, column 1
could not find expected ':'
in "/tmp/molecule/my_module/default/state.yml", line 9, column 1
ERROR: Job failed: exit status 1
It appears on the state.yml file had got corrupted on the remove gitlab runner some how. I deleted the file and re-ran the test job, and this passed.
Related
i execute command :
groovy -cp SCRIPTS_DIR SCRIPTS_DIR/build.groovy >>output.txt 2>>error.txt
build.groovy file contains svn command and lot of mvn commands. During execution file output.txt is changing order and some rows are added in middle of output not at the end.
Example of output.txt:
Executing: svn export --revision 108 --force https://svn.server.eu:/svn/TEST/trunk 101
A /opt/eu/building/jenkins/workspace/test/101
A /opt/eu/building/jenkins/workspace/test/101/src
A /opt/eu/building/jenkins/workspace/test/101/src/scripts
A /opt/eu/building/jenkins/workspace/test/101/src/template
A /opt/eu/building/jenkins/workspace/test/101/src/template/tools
A /opt/eu/building/jenkins/workspace/test/101/MVN
A /opt/eu/building/jenkins/workspace/test/101/MVN/src
A /opt/eu/building/jenkins/workspace/test/101/MVN/src/main
A /opt/eu/building/jenkins/workspace/test/101/MVN/src/main/assembly
A /opt/eu/building/jenkins/workspace/test/101/common
A /opt/eu/building/jenkins/workspace/test/101/common/py
A /opt/eu/building/jenkins/workspace/test/101/PLAIN
A /opt/eu/building/jenkins/workspace/test/101/PLAIN/scripts
After few secunds when mvn commands are executed some of logs are between of rows related to svn export command:
Executing: svn export --revision 108 --force https://svn.server.eu:/svn/TEST/trunk 101
********************************************************************************
*** Check if root POM exists ***************************************************
********************************************************************************
*** Timestamp....................................: 30.03.2022 14:57:53
*** In minute since build start..................: 1
*** Root POM exists: MVN/pom.xml
********************************************************************************
*** Creating source.zip on Software Archive ************************************
********************************************************************************
*** Timestamp....................................: 30.03.2022 14:57:53
*** In minute since build start..................: 1
[zip] Building zip: /opt/eu/building/jenkins/workspace/test/source101.zip
********************************************************************************
*** Running resolve-props and lock-snapshots ***********************************
********************************************************************************
A /opt/eu/building/jenkins/workspace/test/101
A /opt/eu/building/jenkins/workspace/test/101/src
A /opt/eu/building/jenkins/workspace/test/101/src/scripts
A /opt/eu/building/jenkins/workspace/test/101/src/template
A /opt/eu/building/jenkins/workspace/test/101/src/template/tools
A /opt/eu/building/jenkins/workspace/test/101/MVN
A /opt/eu/building/jenkins/workspace/test/101/MVN/src
A /opt/eu/building/jenkins/workspace/test/101/MVN/src/main
A /opt/eu/building/jenkins/workspace/test/101/MVN/src/main/assembly
A /opt/eu/building/jenkins/workspace/test/101/common
A /opt/eu/building/jenkins/workspace/test/101/common/py
A /opt/eu/building/jenkins/workspace/test/101/PLAIN
A /opt/eu/building/jenkins/workspace/test/101/PLAIN/scripts
I execute this command on different linux systems and there output.txt is correct. Only on Solaris 10 i have this 'strange' behavior of stdout. Maybe it's related to output buffering but i'm not sure.
Any solution for this ?
I run my dbt command in a loop and save its contents to a .yml file. This works well and generates a schema in my .yml file accurately:
for file in models/l30_mart/*.sql; do
table=$(basename "$file" .sql)
dbt run-operation generate_model_yaml --args "{\"model_name\": \"$table\"}" > test.yml
done
However, in the example above, I am just saving the test.yml file in the root directory. When I try to save the file in another path for example models/l30_mart/test.yml, it doesn't work:
for file in models/l30_mart/*.sql; do
table=$(basename "$file" .sql)
dbt run-operation generate_model_yaml --args "{\"model_name\": \"$table\"}" > models/l30_mart/test.yml
done
In this case, when I open the test.ymlfile, I see this:
12:06:42 Running with dbt=1.0.1
12:06:43 Encountered an error:
Compilation Error
The schema file at models/l30_mart/test.yml is invalid because no version is specified. Please consult the documentation for more information on schema.yml syntax:
https://docs.getdbt.com/docs/schemayml-files
It's probably a syntax mistake. Do I need any quotes to specify the path? or brackets? Why can't I save the .yml file in the same folder? Similarly, something like this and try to save different files with the tablename, it also doesn't work:
for file in models/l30_mart/*.sql; do
table=$(basename "$file" .sql)
dbt run-operation generate_model_yaml --args "{\"model_name\": \"$table\"}" > models/l30_mart/$table.yml
done
In this case, the files either have this output:
20:39:44 Running with dbt=1.0.1
20:39:45 Encountered an error:
Compilation Error
The schema file at models/l30_mart/firsttable.yml is invalid because no version is specified. Please consult the documentation for more information on schema.yml syntax:
https://docs.getdbt.com/docs/schemayml-files
or this (eg in the second tablename.yml file):
20:39:48 Running with dbt=1.0.1
20:39:49 Encountered an error:
Parsing Error
Error reading dbt_4flow: l30_mart/firstablename.yml - Runtime Error
Syntax error near line 2
------------------------------
1 | 20:39:44 Running with dbt=1.0.1
2 | 20:39:45 Encountered an error:
3 | Compilation Error
4 | The schema file at models/l30_mart/firsttablename.yml is invalid because no version is specified. Please consult the documentation for more information on schema.yml syntax:
5 |
Raw Error:
------------------------------
mapping values are not allowed in this context
in "<unicode string>", line 2, column 31
Note that the secondtablename.yml mentions the firsttablename.yml.
I have created a pipeline which performs ansible-lint on $CI_PROJECT_DIR. The problem is the complete output is not shown in UI as compared to running on my local machine.
You can notice the difference in output for both.
Below is the output from my local machine (Ubuntu with ansible-lint installed)
**ansible-lint create-dir.yaml -v**
WARNING Overriding detected file kind 'yaml' with 'playbook' for given positional argument: create-dir.yaml
INFO Executing syntax check on create-dir.yaml (0.31s)
WARNING Listing 1 violation(s) that are fatal
syntax-check: 'file' is not a valid attribute for a Play
create-dir.yaml:4:3 [WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
ERROR! 'file' is not a valid attribute for a Play
The error appears to be in '/tmp/create-dir.yaml': line 4, column 3, but may be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
---
- name: Create a directory if it does not exist
^ here
Finished with 1 failure(s), 0 warning(s) on 1 files.
Below is output from Gitlab CI :
$ find ./ -not \( -name "*.ansible-lint" -o -name ".gitlab-ci.yml" \) \( -name "*yml" -o -name "*yaml" \) | xargs ansible-lint -v
WARNING Overriding detected file kind 'yaml' with 'playbook' for given positional argument: ./ansible/create-dir.yaml
INFO Executing syntax check on ansible/create-dir.yaml (0.39s)
Cleaning up project directory and file based variables 00:01
Job succeeded
I would like to know why there is difference in output and how to print the complete message on Gitlab CI
Job logs in GitLab CI/CD have a limited length. It's designed for operational reasons, as the jobs can create arbitrary outputs so one could - by mistake or by purpose - output even gigabytes or terabytes of text.
But if you scroll up in the output, you have the abillity to view the full logs by pressing 'Complete Raw'.
This is my setup:
I use AWS Batch that is running a custom Docker image
The startup.sh file is an entrypoint script that is reading the nth line of a text file and copying it from s3 into the docker.
For example, if the first line of the .txt file is 'Startup_00001/ Startup_000018 Startup_000019', the bash script reads this line, and uses a for loop to copy them over.
This is part of my bash script:
STARTUP_FILE_S3_URL=s3://cmtestbucke/Config/
Startup_FileNames=$(sed -n ${LINE}p file.txt)
for i in ${Startup_FileNames}
do
Startup_FileURL=${STARTUP_FILE_S3_URL}$i
echo $Startup_FileURL
aws s3 cp ${Startup_FileURL} /home/CM_Projects/ &
done
Here is the log output from aws:
s3://cmtestbucke/Config/Startup_000017
s3://cmtestbucke/Config/Startup_000018
s3://cmtestbucke/Config/Startup_000019
Completed 727 Bytes/727 Bytes (7.1 KiB/s) with 1 file(s) remaining download: s3://cmtestbucke/Config/Startup_000018 to Data/Config/Startup_000018
Completed 731 Bytes/731 Bytes (10.1 KiB/s) with 1 file(s) remaining download: s3://cmtestbucke/Config/Startup_000017 to Data/Config/Startup_000017
fatal error: *An error occurred (404) when calling the HeadObject operation: Key
"Config/Startup_000019 " does not exist.*
My s3 bucket certainly contains the object s3://cmtestbucke/Config/Startup_000019
I noticed this happens regardless of filenames. The last iteration always gives this error.
I tested this bash logic locally with the same aws commands. It copies all 3 files.
Can someone please help me figure out what is wrong here?
The problem was with EOL of the text file. It was set to Windows(CR LF). The docker image is running Ubuntu which caused the error. I changed the EOL to Unix(LF). The problem was solved.
For some reason I am having trouble with the make test statement while installing ```Vowpal Wabbit``. I am getting the following error:
RunTests: test 59: '/usr/bin/timeout 20 ../vowpalwabbit/vw -d train-sets/argmax_data -k -c --passes 20 --search_rollout oracle --search_alpha 1e-8 --search_task argmax --search 2 --holdout_off' failed (exitcode=1)
Anyone have a clue what this could be?
Just run the command which failed (in single quotes) directly from the test directory, and the reason would become obvious.
It is missing data file:
Reading datafile = test/train-sets/argmax_data
can't open: test/train-sets/argmax_data, error = No such file or directory
vw: std::exception
The issue was introduced in a recent check-in and should soon be fixed (hopefully).
Update (2014-05-31): fixed in the most recent commit.