I am trying to deploy a Java Google App Engine project via my Jenkins server. The idea is that on a commit I run my tests and if all pass, I deploy to my Google project.
I have tried a variety of things and have gotten close with service account json files. I am currently running: mvn clean appengine:update -Dapplication-id=<app_id> -Dappengine.additionalParams=--service_account_json_key_file=<json_file>.
The problem I am running into is getting an error of:
You do not have permission to modify this app (app_id=...)
The service account has Google App Engine Deploy permissions and added Google App Engine Owner just to see if that fixed anything, but it didn't.
Previously, I tried to use the gcloud command line tool to authenticate prior to running maven but every time I did that I would have the build prompt to authenticate via a browser (which doesn't work in a CI environment).
Has anyone been able to build and deploy to Google App Engine from Jenkins?
TL;DR - Your steps and command look reasonable and should have worked. You might want to double check the following:
Service account has at least App Engine Deployer role assigned to it.
Maven command needs to use the credentials from the service account json file (look at the log line which looks something like this that includes --service_account_json_key_file=):
[INFO] Retrieving Google App Engine Java SDK from Maven
[INFO] Updating Google App Engine Application
[INFO] Running -A PROJECT_NAME -V 100 --oauth2 --service_account_json_key_file=/path/to/service_account.json update /Users/tuxdude/google-cloud-examples/google-app-engine/java/helloworld-email/appengine/helloworld/target/appengine-helloworld-1.0-SNAPSHOT
Steps that worked for me
I just tried these steps and it worked for me:
Create a new service account from IAM & Admin page in Google Cloud Console.
Grant just the 'App Engine Deployer' role.
Select the checkbox which says Furnish a new private key. Choose key type as json
Save the json file
Run the following maven command to deploy using the service account credentials:
mvn clean appengine:update -Dappengine.additionalParams=--service_account_json_key_file=path/to/service_account_key.json
If your path has any characters which could break argument tokenization, you can use this version:
mvn clean appengine:update -Dappengine.additionalParams=--service_account_json_key_file="path/to/service_account_key.json"
Related
I developed my first Go app and decided to deploy it to Google Cloud (App Engine). It was running perfectly fine locally but I run into a few issues during the deployment (I'm using Cloud SDK).
I eventually got to the point where I was able to successfully deploy it to GCP but I keep getting
when trying to access the app.
I cannot do any local development either because every time I run go run main.go, go get -u, go build I'm getting:
go: finding module for package github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/mysql
go: found github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/mysql in github.com/GoogleCloudPlatform/cloudsql-proxy v0.0.0-20200513164142-a9864b03c326
go: github.com/GoogleCloudPlatform/cloudsql-proxy#v0.0.0-20200513164142-a9864b03c326 requires
cloud.google.com/go#v0.56.0 requires
cloud.google.com/go/bigquery#v1.4.0 requires
cloud.google.com/go#v0.52.0 requires
cloud.google.com/go/bigquery#v1.0.1/go.mod: verifying module: cloud.google.com/go/bigquery#v1.0.1/go.mod: cannot authenticate record data in server response
My go.mod file looks like this:
module swanson
go 1.14
require github.com/gorilla/mux v1.7.4
and this is the structure of my app (main.go, models, and router are 3 different packages):
I'm new to Go development and never deployed a Go app to GCP so feeling quite lost, any help would be much appreciated.
I have tried deploying a go application in app engine and also running it locally and both worked.
I used
this application
in github, which you can clone with the command:
git clone https://github.com/GoogleCloudPlatform/golang-samples
The command I have used for running the application locally is :
go run helloworld.go
And the command returned the following message:
2020/05/22 11:34:16 Defaulting to port 8080
2020/05/22 11:34:16 Listening on port 8080
Then in the Cloud Shell I have opened the web preview in the port 8080 and it was working as expected.
For more information about the deployment, you can check this documentation.
I am trying to set up a CICD pipeline using Visual Studio > Visual Studio Team Services > Elastic Beanstalk Create version.
I have been able to check in my code OK, and kick off my build pipeline which contains the following step in place of 'publish artefact':
"Create Elastic Beanstalk Revision:"
This step is attached to an AWS IAM User with Administrator privileges. This step fails when I try to run my pipeline to deploy the ASP.NET application (Webforms, so not Core) via this method.
The error output is as follows:
2018-07-30T04:46:22.7765736Z ##[section]Starting: Create Elastic
Beanstalk Revision: Sparky 2018-07-30T04:46:22.7771363Z
============================================================================== 2018-07-30T04:46:22.7771634Z Task : AWS Elastic Beanstalk
Create Version 2018-07-30T04:46:22.7771964Z Description : Create an
application revision for deployment to an environment.
2018-07-30T04:46:22.7772192Z Version : 1.0.21
2018-07-30T04:46:22.7772403Z Author : Amazon Web Services
2018-07-30T04:46:22.7772908Z Help : Please refer to AWS
Elastic Beanstalk User
Guide
for more details on deploying applications with AWS Elastic Beanstalk.
2018-07-30T04:46:22.7773336Z
============================================================================== 2018-07-30T04:46:23.2641747Z ac747f99-1789-4d43-86c5-c8283d1a72c0
exists true 2018-07-30T04:46:23.2671026Z Deployment type set to aspnet
2018-07-30T04:46:24.8994140Z Determine S3 bucket
elasticbeanstalk-ap-southeast-2-153247006570 to store application
bundle 2018-07-30T04:46:24.9038683Z Upload of application bundle
failed with error: EISDIR: illegal operation on a directory, read {
Error: EISDIR: illegal operation on a directory, read
2018-07-30T04:46:24.9047409Z Uploading application bundle D:\a\1\a to
object Sparky/a-cicd_test.zip in bucket
elasticbeanstalk-ap-southeast-2-153247006570
2018-07-30T04:46:24.9048878Z ##[error]Error: EISDIR: illegal operation
on a directory, read 2018-07-30T04:46:24.9053846Z at Error
(native) errno: -4068, code: 'EISDIR', syscall: 'read' }
2018-07-30T04:46:24.9172250Z ##[section]Finishing: Create Elastic
Beanstalk Revision: Sparky
I could find very little (pretty much no) results online about this error. I'm not sure how to resolve it. Any ideas anyone? I know it's not IAM permissions as I am using ADMIN for the AWS User just for testing.
EDIT: Added image of build definition. (Note, I don't really know how to use version label output variables so just put something there, but I don't think that's the issue, this failure is something else entirely. I'm just following online example/tutorials for a basic deployment)
The build extension is https://aws.amazon.com/vsts/ and looking back over my screenshot and the instructions I was following here https://aws.amazon.com/blogs/developer/deploying-net-web-applications-using-aws-elastic-beanstalk-with-visual-studio-team-services/ I just realised a mistake! I didn't specify the file name in the web deploy archive.
I changed
$(build.artifactstagingdirectory)
To
$(build.artifactstagingdirectory)\SparkIdeaGenerator.zip
And the build succeeded! However, clearly I didn't understand the purpose of this build task, as it has only created an application revision in AWS, it hasn't actually deployed the updated code. This isn't much good, as I still need to go into the console and click 'Deploy'. This doesn't seem ideal. Here's what I mean:
Clearly I didn't understand the limits of this build task. I thought it would create the revision and deploy the code. It doesn't. There is only one other Elastic Beanstalk build task available in the toolset I downloaded, which is 'create application'. I don't want this, as I already have the application present, I Just want to update it. I will take a look further into this, as I need that full end-to-end automation, commit code, run build, deploy code, update site.
I will however mark the question as answered, as I have solved this specific question/error by specifically referencing a .zip with the package name of the solution itself.
The process is defined in: https://docs.aws.amazon.com/vsts/latest/userguide/tutorial-eb.html
It states that you use the zip file name along with the $(build.artifactstagingdirectory), like what was identified above.
This does appear to fix the issue.
Our containers are hosted using Google Container Registry, and I am using id "com.bmuschko.docker-java-application" version "3.0.7" to build and deploy docker containers. However, I run into permission issues whenever I try to pull the base image or push the image to GCR (I am able to get to the latter step by pulling the image and having it available locally).
I'm a little bit confused by how I can properly configure a particular GCloud account to be used whenever issuing any Docker related calls over a wire using the plugin.
As a first attempt, I've tried to create a task that precedes and build or push commands:
task gcloudLogin(type:Exec) {
executable "gcloud"
args "auth", "activate-service-account", "--key-file", "$System.env.KEY_FILE"
}
However, this simple wrapper doesn't work as desired. Is there currently a supported way to have this plugin work with GCR?
Got in touch with the maintainers of the gradle docker plugin and we have found this to be a valid solution.
I have tried deploying my meanjs on heroku.
I forked this https://github.com/meanjs/mean
1.) Login to heroku
2.) Deploy and connect github repositor
enabled automatic deploy CI
Click on Manual Deploy
On the build log it says "Bulid succeded"
My question is.
Why am I getting this application error?
When all I did was forked the repository and deployed it on heroku?
https://serkolgame.herokuapp.com/
Did you add a mongoDB to your app? Without it, the startup process is likely to fail.
Here are some options:
If you are using the default dev environment - then just add the mongodb connection string in development.js and restart the server
If you are using the prod environment - then you can use environment variables uri: process.env.MONGOHQ_URL or process.env.MONGOLAB_URI
This is assuming you have a mongoDB sandbox setup somewhere, if you don't, you'll first need a mongodb sandbox (get one from Heroku, Compose.io, or MongoLab).
I am using jenkins CI to build my project. After the build I had post steps shell script copy the war to WSO2 application servers /repository/deployment/server/webapps/ folder. But what I observed was sometimes ( not always ), the server failed to redeploy the web application. Most of the times the web application would be redeployed using the new war file copied by jenkins into the webapps folder. But sometimes this did not happen. At such times the only solution was to stop the wso2as server, delete the web applications folder leaving only the new war file in wenapps directory and start the wso2as server again. This deployed the new war file and testing could proceed.
But we need to run automated selenium tests. So for the tests to run we need the new war file to be deployed reliably on every jenkins build.
So I wrote a little script in the jenkins post step like below :
#!/bin/bash
/home/kk/wso2as-5.0.1/bin/wso2server.sh stop &&
sleep 1m &&
rm -rf /home/kk/wso2as-5.0.1/repository/deployment/server/webapps/Duster_App1* &&
cp /home/kk/.jenkins/workspace/Dusters_App1/target/Duster_App1-${MAVEN_VERSION}.war /home/kk/wso2as-5.0.1/repository/deployment/server/webapps/Duster_App1.war &&
/home/kk/wso2as-5.0.1/bin/wso2server.sh start &&
echo Starting wso2as
This script runs. The wso2as server is stopped, the webapp folder is deleted, the new war file is copied corectly in place, I get the 'Starting wso2as' message on jenkins console output, BUT the wso2as server does not start. It remains in the stopped state. I introduced the sleep 1m line after the wso2server.sh stop command thinking the server may be taking time to shut down and hence fails to start again by the time the wso2server.sh start command is run. But no. The server is stopped cleanly before the start command is run as I have verified by following the log file as well as by ps aux command.
So what could be preventing the server from starting again?
IMO writing a script to delete webapp folder and restart the server from time to time would not be a good idea. I would suggest you to do following steps and then try the deployment from your jenkins build server to AppServer repository/deployment/server/webapps.
1.) Stop the server
2.) Open the [AS_HOME]/repository/conf/tomcat/catalina-server.xml config file
3.) Now make the unpackWARs attribute in Host element "false"
4.) restart the server
By default AppServer would extract the WAR file inside webapps, and it is possible that this could cause intermittent issues when deploying the same webapp again and again. Please try this way and see.
The WSO2 servers are OSGI based. The modules offer also web services for all tasks you can do in the admin console. If you start the server as ./wso2server.sh -DosgiConsole and you can list all service interaces with osgi> listAdminServices. You will find i.e. interfaces to upload WAR, CAR and AAR archives. The upload is one stateless call, but the handling of the artifact upload is a little bit tricky and you have also to deal with the self signed certificates of the WSO2 server. You can do the upload in a script language of your choice or make a simple Jenkins plugin and integrate this into your build job.
If you want comfort and don't do it yourself, use this Jenkins post-build plug-in to deploy the WAR into the WSO2 AS via the deployment web service API: https://github.com/ma-ha/Jenkins-WSO2AS-Deployer