When I create a new deploy in AWSCodeDeploy with GitHub I receive this fail message:
Error CodeScriptFailed
Script Namescripts/stop_server.sh
MessageScript at specified location: scripts/stop_server.sh run as user ubuntu failed with exit code 1
Log TailLifecycleEvent - ApplicationStop
Script - scripts/stop_server.sh
[stderr]su: user ubuntu does not exist
But, my instance is an Amazon Linux Instance and don't have a ubuntu user, anybody know anything about this?
The script that a try to run is:
# scripts/stop_server.sh
#!/bin/bash
forever stop .
My appspec.yml file:
version: 0.0
os: linux
files:
- source: /
destination: /home/ec2-user
hooks:
AfterInstall:
- location: scripts/install_dependencies.sh
timeout: 5
runas: root
ApplicationStart:
- location: scripts/start_server.sh
timeout: 5
runas: root
ApplicationStop:
- location: scripts/stop_server.sh
timeout: 5
runas: root
Codedeploy-agent version agent_version: OFFICIAL_1.0-1.1095_rpm
Application stop usually refers to appspec.yml in previous successful deployment archive. Either empty /opt/codedeploy-agent/deployment-archive/deployment-instructions/ or you may use BeforeInstall hook to execute stop script.
The code deploy create temporary application inside /opt/code-deploy/.... path. If the deployment fails normally it starts from the temporary directory next time. If you want to get rid of the error pointing by deployment, you should check the particular script file in the temporary directory and edit it.
Related
Is it possible to use short-lived credentials, with docker-compose, to run a bash scripted gcloud command?
Related posts that I attempted to use but they are 5+ years old and I've been led to believe that the gcloud auth command has changed during this time:
ERROR: (gcloud.auth.activate-service-account) Failed to activate the given service account. Please ensure provided key file is valid
gcloud auth activate-service-account [ERROR] Please ensure provided key file is valid
Setup
there is a lot going on but I've attempted to abbreviate to the relevant parts
Makefile
auth: ## commands for short lived auth
#gcloud config set project ${GCP_PROJECT}
#gcloud auth application-default login --impersonate-service-account="inst-dataflow-svc#${GCP_PROJECT}.iam.gserviceaccount.com"
#gcloud auth configure-docker $(REGION)-docker.pkg.dev
gcloud-flex-build: ## build & push base docker image
docker-compose build gcloud-build-flex-local
docker-compose run gcloud-build-flex-local
docker-compose.yaml
version: '3.4'
services:
gcloud-build-flex-local:
build:
dockerfile: docker/gcloud-build-flex-template.dockerfile
context: .
image: us-central1-docker.pkg.dev/gcp-project/dataflow-docker-registry/local-build/pubsub-to-gbq-build-flex-template
volumes:
- type: bind
source: ${HOME}/.config/gcloud/
target: /tmp
docker/gcloud-build-flex-template.dockerfile
FROM gcr.io/google.com/cloudsdktool/cloud-sdk:408.0.1
COPY docker/scripts/gcloud-build-flex-template.sh /app/gcloud-build-flex-template.sh
COPY dataflow/pubsub-to-gbq/pubsub-to-gbq-metadata /app/pubsub-to-gbq-metadata
WORKDIR /app
ENTRYPOINT "/app/gcloud-build-flex-template.sh"
/app/gcloud-build-flex-template.sh
#!/bin/bash
set -euo pipefail
SERVICE_ACCOUNT_EMAIL=inst-dataflow-svc#gcp-project.iam.gserviceaccount.com
GCP_PROJECT=gcp-project
export GOOGLE_APPLICATION_CREDENTIALS=/tmp/application_default_credentials.json
# debugging
echo $GOOGLE_APPLICATION_CREDENTIALS
ls -lah /tmp/
cat $GOOGLE_APPLICATION_CREDENTIALS
gcloud auth activate-service-account $SERVICE_ACCOUNT_EMAIL --project=$GCP_PROJECT --key-file=$GOOGLE_APPLICATION_CREDENTIALS
Execution
make auth
make gcloud-flex-build
Error
ERROR: (gcloud.auth.activate-service-account) The .json key file is not in a valid format.
make: *** [gcloud-flex-build] Error 1
stdout (abbreviated)
docker-compose build gcloud-build-flex-local
[+] Building 0.4s (9/9) FINISHED
...
docker-compose run gcloud-build-flex-local
drwxr-xr-x 17 root root 544 Dec 30 10:36 .
drwxr-xr-x 1 root root 4.0K Dec 30 10:40 ..
-rw------- 1 root root 591 Dec 30 10:36 application_default_credentials.json
{
"delegates": [],
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/inst-dataflow-svc#gcp-project.iam.gserviceaccount.com:generateAccessToken",
"source_credentials": {
"client_id": "alphanumeric string .apps.googleusercontent.com",
"client_secret": "alphanumeric string",
"refresh_token": "alphanumeric string",
"type": "authorized_user"
},
"type": "impersonated_service_account"
}
I can make it work via docker run by spoofing the credentials to include only the "source_credentials" object, passed in as a volume, but this same trick doesn't seem to work with docker-compose running a script inside a container...
There is a similar type of configuration mentioned in this document. This involves three major steps:
Create short-lived credentials for your service account and download
your service account keys.
Create the configuration files for making your docker environment up. Use
the above cred files for granting required permissions.
Once you have all the configuration files in place use your docker-compose
commands for making your environment up.
Follow this documentation for more details.
I have the following repo in pre-commit file .pre-commit-config.yaml
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
args: ["--compare"]
But it keeps giving me the error:
error: unrecognized arguments: .pre-commit-config.yaml
As it passes the filename as an argument to my python script. How can I prevent this?
to clean up your example a little bit -- and use files to only run when the necessary files change:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements --compare
files: ^requirements-dev.txt$
pass_filenames: false
note that I did a couple things:
args doesn't really make sense for local hooks, you can just put that in entry
pass_filenames (as you did) -- pre-commit is a framework based on passing filenames to executables, but you can turn that off
files: this will make it so the hook only gets triggered if requirements-dev.txt changes
alternatively (if you expect changes outside requirements-dev.txt to need to run this hook) you can drop files and use always_run: true
disclaimer: I'm the author of pre-commit
I spent quite some time figuring out what caused this and how to solve this. It's not documented well, eventually I fixed it by trial and error. We have to use pass_filenames: false in our hook:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
pass_filenames: false
args: ["--compare"]
I am using AWS CodePipeline for the first time and trying to figure out how to properly create my buildspec.yml file for my Laravel application. There are few resources on the internet.
I have the following in my buildspec.yml file currently:
version: 0.2
phases:
install:
commands:
- curl -s https://getcomposer.org/installer | php
- mv composer.phar /usr/local/bin/composer
- php --version
build:
commands:
- echo Build started on `date`
- echo Installing composer deps
- composer install
- cp extra/.env ./
- php artisan cache:clear
post_build:
commands:
- echo Build completed on `date`
artifacts:
type: zip
files:
- '**/*'
name: clyde-$(date +%Y-%m-%d)
The CodeBuild is successful and this does deploy to Elastic Beanstalk. I did change the configuration in Elastic Beanstalk so the root is /public (for Laravel). However, when I go to the URL, the first line of code run presents an error like below:
View [inc\navbar] not found. (View: /var/app/current/resources/views/layouts/app.blade.php)
This leads me to believe something is not built properly.
To make it work, it will need to use a complete Pipeline: CodeCommit-->CodeBuild-->CodeDeploy
Inside your Artifact bucket there will be two objects generated in the process:
s3://codepipeline-us-east-1-<001122334455>/SourceArtif/
s3://codepipeline-us-east-1-<001122334455>/BuildArtif/
The first one is obtained in the initial phase of the pipeline from CodeCommit.
The second one is created by CodeBuild. The resultant zip file will be exactly the same as that one from CodeCommit. So it seems, the CodeBuild is only testing but not saving the Artifact with results from the instructions specified in buildspec.yml.
The third phase, CodeDeploy will obtain the code from the Artifact and it will need to Build again via scripts referred by appspec.yml.
version: 0.0
os: linux
files:
- source: /
destination: /web/project/html
hooks:
BeforeInstall:
- location: scripts/install_dependencies.sh
timeout: 300
runas: root
AfterInstall:
- location: scripts/build_again.sh
timeout: 600
runas: user
ApplicationStart:
- location: scripts/start_application.sh
timeout: 300
runas: root
The build_again.sh file will need to include same commands you are using in buildspec.yml (build section), then your Laravel project should be working.
I have a RabbitMQ Docker container in a docker-compose.yml file.
rabbit:
image: rabbitmq:3-management
ports:
- "15672:15672"
- "5672:5672"
- "61613:61613"
- "15674:15674"
volumes:
- ./enabled_plugins:/etc/rabbitmq/enabled_plugins
labels:
- "traefik.enable=true"
- "traefik.docker.network=default"
- "traefik.ws.port=15674"
- "traefik.ws.frontend.rule=PathPrefixStrip:/stomp/"
- "traefik.web.port=15672"
- "traefik.web.frontend.rule=PathPrefixStrip:/rabbit/"
The problem is that when I do docker-compose up -d, there is an error.
$ docker-compose up -d
Removing deploji-server_rabbit_1
deploji-server_postgres_1 is up-to-date
Starting 820ca92c591a_deploji-server_server_1 ...
Recreating 563cd247aa16_deploji-server_rabbit_1 ...
deploji-server_traefik_1 is up-to-date
Recreating 563cd247aa16_deploji-server_rabbit_1 ... error
Starting 98268bf1b8a7_deploji-server_worker_1 ... done
ERROR: for 563cd247aa16_deploji-server_rabbit_1 Cannot start service rabbit: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/c/Projects/deploji-server/enabled_plugins\\\" to rootfs \\\"/mnt/sda1/var/lib/docker/overlay2/bfcfb23713669e206d402b6c3a183d772750b527f35e5d0372d4f6982ddeb56aStarting 820ca92c591a_deploji-server_server_1 ... done
tmq/enabled_plugins\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: for rabbit Cannot start service rabbit: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/c/Projects/deploji-server/enabled_plugins\\\" to rootfs \\\"/mnt/sda1/var/lib/docker/overlay2/bfcfb23713669e206d402b6c3a183d772750b527f35e5d0372d4f6982ddeb56a/merged\\\" at \\\"/mnt/sda1/var/lib/docker/overlay2/bfcfb23713669e206d402b6c3a183d772750b527f35e5d0372d4f6982ddeb56a/merged/etc/rabbitmq/enabled_plugins\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Encountered errors while bringing up the project.
There seems to be no problems in Linux environment, only on Windows. I assume the path for enabled_plugins file is incorrect. I've tried changing the mapping to something like
./enabled_plugins:/c/rabbitmq
and the container seems to start, but the server is not responding, so I guess the plugins from the enabled_plugins file were not properly installed.
Whole docker-compose.yml file can be seen here: https://github.com/maxmeister/deploji-server/blob/master/docker-compose.yml
Any advice on how could I make it run okay on my Windows machine? Thanks in advance.
I would suggest following the docs:
There are several options if use short syntax:
SHORT SYNTAX Optionally specify a path on the host machine
(HOST:CONTAINER), or an access mode (HOST:CONTAINER:ro).
You can mount a relative path on the host, that expands relative to
the directory of the Compose configuration file being used. Relative
paths should always begin with . or ...
volumes:
# Just specify a path and let the Engine create a volume
- /var/lib/mysql
# Specify an absolute path mapping
- /opt/data:/var/lib/mysql
# Path on the host, relative to the Compose file
- ./cache:/tmp/cache
# User-relative path
- ~/configs:/etc/configs/:ro
# Named volume
- datavolume:/var/lib/mysql
Related question and answers
I am experiencing some problems with RabbitMQ started as a service on Windows.
Operative System: Windows 8 (Microsoft Windows NT version 6.2 Server)
(build 9200)
Erlang: R16B03 (erts-5.10.4)
RabbitMQ: 3.2.2
Goal: create a RabbitMQ cluster with three servers: Srv1, Srv2, Srv3.
Note: I have carefully followed the official documentation
All the following operations are executed as user "Administrator".
FIRST SCENARIO: start RabbitMQ from command line as a background process
I used the command "rabbitmq-server -detached" on Srv1.
Result: a file ".erlang.cookie" is created under C:\Users\Administrator
The execution of the command "rabbimqctl status" is successful and gives me the current state of the node.
I can then copy the file .erlang.cookie in the same folder on Srv2 and Srv3 and successfully create a cluster.
SECOND SCENARIO: start RabbitMQ as a service (this is requirement I have)
Result: the file ".erlang.cookie" is created under C:\Windows.
When I type the command "rabbitmqctl status" another file .erlang.cookie is created under C:\Users\Administrator and I receive the following result:
C:\Program Files\Aspect\DashBoard\RabbitMQ\sbin>rabbitmqctl.bat status
Status of node 'rabbit#RABBITMQ-NODE4' ...
Error: unable to connect to node 'rabbit#RABBITMQ-NODE4': nodedown
DIAGNOSTICS
===========
nodes in question: ['rabbit#RABBITMQ-NODE4']
hosts, their running nodes and ports:
- RABBITMQ-NODE4: [{rabbit,49428},{rabbitmqctl3045334,49434}]
current node details:
- node name: 'rabbitmqctl3045334#rabbitmq-node4'
- home dir: C:\Users\Administrator
- cookie hash: 0DLAKf8pOVrGC016+6BDBw==
We know that this is ok because the two cookies are different.
So I copy the .erlang.cookie file from C:\Windows into C:\Users\Administrator and I try again the same command. This time I get:
C:\Program Files\Aspect\DashBoard\RabbitMQ\sbin>rabbitmqctl.bat status
Status of node 'rabbit#RABBITMQ-NODE4' ...
Error: unable to connect to node 'rabbit#RABBITMQ-NODE4': nodedown
DIAGNOSTICS
===========
nodes in question: ['rabbit#RABBITMQ-NODE4']
hosts, their running nodes and ports:
- RABBITMQ-NODE4: [{rabbitmqctl1178095,49471}]
current node details:
- node name: 'rabbitmqctl1178095#rabbitmq-node4'
- home dir: C:\Users\Administrator
- cookie hash: TIuqp21HOQSoUJT8JfgRQw==
C:\Program Files\Aspect\DashBoard\RabbitMQ\sbin>rabbitmqctl.bat status
Status of node 'rabbit#RABBITMQ-NODE4' ...
Error: {badarith,[{rabbit_vm,bytes,1,[]},
{rabbit_vm,'-mnesia_memory/0-lc$^0/1-0-',1,[]},
{rabbit_vm,mnesia_memory,0,[]},
{rabbit_vm,memory,0,[]},
{rabbit,status,0,[]},
{rpc,'-handle_call_call/6-fun-0-',5,
[{file,"rpc.erl"},{line,205}]}]}
Please notice the Error at the end: "badarith" in rpc.erl, line 205.
I think that the file is Erlang\lib\kernel-2.16.4\src\rpc.erl
The function is this one:
handle_call_call(Mod, Fun, Args, Gleader, To, S) ->
RpcServer = self(),
%% Spawn not to block the rpc server.
{Caller,_} =
erlang:spawn_monitor(
fun () ->
set_group_leader(Gleader),
Reply =
%% in case some sucker rex'es
%% something that throws
case catch apply(Mod, Fun, Args) of
{'EXIT', _} = Exit ->
{badrpc, Exit};
Result ->
Result
end,
RpcServer ! {self(), {reply, Reply}}
end),
{noreply, gb_trees:insert(Caller, To, S)}.
and line 205 is 'case catch apply(Mod, Fun, Args) of'
THIRD SCENARIO: start RabbitMQ as a named user to avoid it to create the file .erlang.cookie under C:\Windows
I set the RabbitMQ service to log on as the user "Administrator", this way it does not create the file under C:\Windows but only under C:\User\Administrator.
Result: when the service starts, the file ".erlang.cookie" is created only under C:\User\Administrator.
When I type the command "rabbitmqctl status" I get the same error as in the provious case (badarith...).
Now the question: I have not found any information about this error (badarith).
Could anyone give me a suggestion about how to troubleshoot/avoid this?