Trying to deploy BOSH director integrated with LDAP and running into problems. Here are my steps.
Clone latest repository:
git clone https://github.com/cloudfoundry/bosh-deployment
Create vars file and populate it.
Modify opsfile for ldap
File: ldap.yml
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.enabled?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.profile_type
value: "search-and-bind"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.url
value: "ldap://XXXX.XXXXXXX.XXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.userDNPatternDelimiter
value: ";"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.userDN
value: "XXXXX\\XXXXXXXXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.userPassword
value: "XXXXXXXXXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.searchBase
value: "dc=XXX,dc=XXXXXXX,dc=XXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.searchFilter
value: "cn={0}"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.passwordAttributeName
value: "userPassword"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.passwordEncoder
value: "org.cloudfoundry.identity.uaa.ldap.DynamicPasswordComparator"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.sslCertificate
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.ssl.skipVerification?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.ssl.tls
value: "none"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.mailAttributeName
value: "mail"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.mailSubstitute
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.mailSubstituteOverrridesLdap?
value: "false"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.referral
value: "follow"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groups.profile_type
value: "groups-map-to-scopes"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groups.searchBase
value: "dc=XXX,dc=XXXXXXXXX,dc=XXX"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groups.groupRoleAttribute
value: "member"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.groupSearchFilter
value: "member={0}"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.searchSubtree?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.maxSearchDepth
value: "1"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.emailDomain
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.attributeMappings
value:
user.attribute.name-of-attribute-in-uaa-id-token: name-of-attribute-in-ldap-record
user.attribute.name-of-other-attribute-in-uaa-id-token: name-of-other-attribute-in-ldap-record
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.storeCustomAttributes?
value: "true"
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.externalGroupWhitelist?
value: ""
- type: replace
path: /instance_groups/name=bosh/jobs/uaa/properties/uaa.ldap.add_shadow_user_on_login?
value: "true"
Run a deployment script:
#!/bin/bash
bosh create-env bosh-deployment/bosh.yml \
--state=state.json \
--vars-store=creds.yml \
--vars-file=vars.yml \
-o bosh-deployment/vsphere/cpi.yml \
-o bosh-deployment/uaa.yml \
-o bosh-deployment/ldap.yml \
-o bosh-deployment/misc/dns.yml \
-o bosh-deployment/misc/ntp.yml \
-o bosh-deployment/jumpbox-user.yml
Getting error:
Deployment manifest: '/Users/user1/workarea/bosh-1/bosh-deployment/bosh.yml'
Deployment state: 'state.json'
Started validating
Failed validating (00:00:00)
Parsing release set manifest '/Users/user1/workarea/bosh-1/bosh-deployment/bosh.yml':
Evaluating manifest:
Expected to find a map at path '/instance_groups/name=bosh/jobs/uaa' but found '[]interface {}'
Exit code 1
All works well if I remove ops file for LDAP integration (bosh-deployment/ldap.yml) - BOSH, UAA and etc get deployed. I don't understand why would it complain. UAA is creating a job in BOSH and I am passing ops file after uaa.yml file. It should have been already be able to create UAA job in BOSH deployment.
Any hints ?
Thanks so much!!
Answering my own question. I had to put in ?/- in the path so new element under ldap could be created. Once I put that in, I was able to deploy the environment with new OpsFile.
- type: replace
path: /instance_groups/name=bosh/jobs/name=uaa/properties/ldap?/-
value:
enabled: true
profile_type: "search-and-bind"
url: "ldap://ldap.localnet.com"
userDN: 'XXXX\\XXXXXXX'
userPassword: 'XXXXXXXXXX'
searchBase: "dc=local,dc=localnet,dc=com"
searchFilter: "cn={0}"
passwordAttributeName: "userPassword"
passwordEncoder: "org.cloudfoundry.identity.DynamicPasswordComparator"
sslCertificate: ""
ssl:
skipVerification: "true"
tls: "none"
mailAttributeName: "mail"
mailSubstitute: ""
mailSubstituteOverrridesLdap: "false"
referral: "follow"
groups:
profile_type: "groups-map-to-scopes"
searchBase: "dc=local,dc=localnet,dc=com"
groupRoleAttribute: "member"
groupSearchFilter: "member={0}"
searchSubtree: "true"
maxSearchDepth: "10"
emailDomain: ""
storeCustomAttributes: "true"
externalGroupWhitelist: ""
add_shadow_user_on_login: "true"
Related
HI Azure pipeline i am getting struck and having error while doing this
parameters:
- name: applications_module
displayName: Applications
type: boolean
default: 'false'
values: []
- name: management_module
displayName: Management
type: boolean
default: 'false'
values: []
- name: Creation
displayName: Creation
type: boolean
default: 'false'
values: []
variables:
- name: FilterValue
${{each item in parameters}}:
${{if contains(item.name, '_module')}}:
value: item.name # here I want to concatenate all the parameters names that have name _module but this statement is throwing an error
I am not able to its saying "'value' is already defined" so can anyone help me regarding this
using ##VSO we can do that
steps:
- ${{ each parameter in parameters }}:
- bash: echo '##vso[task.setvariable variable=allParametersString]$(allParametersString)${{ parameter.Key }}'
- script:
echo 'concatenated strings by comma .->$(allParametersString)'
I am brand new to concourse and am trying to use it to make a terraform-ci platform and cannot figure out why im getting this error on my very first pipeline, can anyone help out?
jobs:
- name: terraform-pipeline
serial: true
plan:
- aggregate:
- get: master-branch
trigger: true
- get: common-tasks
params: { submodules: [ terraform ] }
trigger: true
- task: terraform-plan
file: common-tasks/terraform/0.12.29.yml
input_mapping: { source: master-branch }
params:
command: plan
cache: true
access_key: ((aws-access-key))
secret_key: ((aws-secret-key))
directory: master-branch/terraform-poc/dev
resources:
- name: master-branch
type: git
source:
uri: https://github.com/rossrollin/terraform-poc
branch: master
- name: common-tasks
type: git
source:
uri: https://github.com/telia-oss/concourse-tasks.git
branch: master
Executing pipeline like so:
fly -t concourse-poc sp -p terraform-pipeline -c pipeline2.yml -v aws-access-key=''-v aws-secret-key=''
error: error unmarshaling JSON: while decoding JSON: no step configured
The aggregate step was deprecated in version 5.2.0 and removed in version 7.0.0.
You need to replace it with the new in_parallel step.
- - aggregate:
+ - in_parallel:
Removing '- aggregate:' and just running the resource get's inline fixes my issue.
I'm trying to pull source code from Github then build and push a docker image to docker hub using Tekton pipeline and Knative on Kubernetes cluster.
I'm following this link for the installation and setup of Tekton:
https://www.ibm.com/cloud/blog/build-a-knative-service-with-tekton-and-apache-openwhisk-nodejs-runtime
task-build.yaml
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: task-build
spec:
inputs:
resources:
- name: docker-source
type: git
params:
- name: TARGET_IMAGE_NAME
description: name of the image to be tagged and pushed
- name: TARGET_IMAGE_TAG
description: tag the image before pushing
default: "latest"
- name: DOCKERFILE
description: name of the dockerfile
- name: OW_RUNTIME_DEBUG
description: flag to indicate debug mode should be on/off
default: "false"
- name: OW_RUNTIME_PLATFORM
description: flag to indicate the platform, one of ["openwhisk", "knative", ... ]
default: "knative"
- name: OW_ACTION_NAME
description: name of the action
default: "foo"
- name: OW_ACTION_CODE
description: JavaScript source code to be evaluated
default: ""
- name: OW_ACTION_MAIN
description: name of the function in the "__OW_ACTION_CODE" to call as the action handler
default: "main"
- name: OW_ACTION_BINARY
description: flag to indicate zip function, for zip actions, "__OW_ACTION_CODE" must be base64 encoded string
default: "false"
- name: OW_HTTP_METHODS
description: list of HTTP methods, any combination of [GET, POST, PUT, and DELETE], default is [POST]
default: "[POST]"
- name: OW_ACTION_RAW
description: flag to indicate raw HTTP handling, interpret and process an incoming HTTP body directly
default: "false"
outputs:
resources:
- name: builtImage
type: image
steps:
- name: add-ow-env-to-dockerfile
image: "gcr.io/kaniko-project/executor:debug"
command:
- /busybox/sh
args:
- -c
- |
cat <<EOF >> ${inputs.params.DOCKERFILE}
ENV __OW_RUNTIME_DEBUG "${inputs.params.OW_RUNTIME_DEBUG}"
ENV __OW_RUNTIME_PLATFORM "${inputs.params.OW_RUNTIME_PLATFORM}"
ENV __OW_ACTION_NAME "${inputs.params.OW_ACTION_NAME}"
ENV __OW_ACTION_CODE "${inputs.params.OW_ACTION_CODE}"
ENV __OW_ACTION_MAIN "${inputs.params.OW_ACTION_MAIN}"
ENV __OW_ACTION_BINARY "${inputs.params.OW_ACTION_BINARY}"
ENV __OW_HTTP_METHODS "${inputs.params.OW_HTTP_METHODS}"
ENV __OW_ACTION_RAW "${inputs.params.OW_ACTION_RAW}"
EOF
- name: adapt-dockerfile-to-tekton
image: "gcr.io/kaniko-project/executor:debug"
command:
- sed
args:
- -i
- -e
- 's/COPY ./COPY .\/docker-source/g'
- ${inputs.params.DOCKERFILE}
- name: build-openwhisk-nodejs-runtime
image: "gcr.io/kaniko-project/executor:latest"
args: ["--destination=${inputs.params.TARGET_IMAGE_NAME}:${inputs.params.TARGET_IMAGE_TAG}", "--dockerfile=${inputs.params.DOCKERFILE}"]
When trying to build and push the image, am getting error:
conditions:
- lastTransitionTime: "2020-09-24T07:33:11Z"
"step-add-ow-env-to-dockerfile" exited with code 2 (image: "docker-pullable://gcr.io/kaniko-project/executor#sha256:0f27b0674797b56db08010dff799c8926c4e9816454ca56cc7844df228c53485"); for logs run: kubectl -n default logs task-run-helloworld-pod-5bbkx -c step-add-ow-env-to-dockerfile
reason: Failed
status: "False"
type: Succeeded
When checked the logs for error msg, I'm getting:
Error : /busybox/sh: syntax error: bad substitution
I have a YAML Cloudformation script which launches a single EC2 instance and runs some UserData upon startup.
I am using ami-0727f3c2d4b0226d5, a standard Ubuntu 18:04 LTS server.
Everything works fine provided the UserData is simple, eg -
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
echo "Hello World EC2!"
which gives me the following in the EC2 system log -
[[0;32m OK [0m] Started Apply the settings specified in cloud-config.
Starting Execute cloud user/final scripts...
[ 21.827930] cloud-init[1307]: + echo 'Hello World EC2!'
[ 21.832906] cloud-init[1307]: Hello World EC2!
but if I extend the UserData for some fairly normal- looking Ubuntu commands -
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
apt-get update
apt-get install -y ruby
echo "Hello World EC2!"
then (having torn down the original machine and restarted a new instance from scratch) the UserData process seems to hang with the following messages in the system log -
[ 29.606055] cloud-init[1304]: + apt-get install -y ruby
[ 29.675005] cloud-init[1304]: Reading package lists...
[ 29.828430] cloud-init[1304]: Building dependency tree...
[ 29.836236] cloud-init[1304]: Reading state information...
[ ... ]
[ ... ]
[ ... ]
[ 34.233706] cloud-init[1304]: Checking for services that may need to be restarted...done.
[ 34.254767] cloud-init[1304]: Checking for services that may need to be restarted...done.
[ 34.262182] cloud-init[1304]: Checking init scripts...
ie Checking init scripts ... never returns. Any thoughts on how to debug this situation / find out what is going wrong ?
TIA
[full YAML CF included]
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
AppName:
Type: String
InstanceType:
Type: String
Default: t2.micro
ImageId:
Type: String
Default: ami-0727f3c2d4b0226d5 # 18.04 LTS eu-west-1
KeyName:
Type: String
Outputs:
MyDNSName:
Value:
Fn::GetAtt:
- AppEC2Instance
- PublicDnsName
Description: "EC2 public DNS name"
MyIPAddress:
Value:
Fn::GetAtt:
- AppEC2Instance
- PublicIp
Description: "EC2 public IP address"
MyInstanceId:
Value:
Ref: AppEC2Instance
Description: "EC2 instance id"
Resources:
AppEC2Instance:
Properties:
IamInstanceProfile:
Ref: AppInstanceProfile
ImageId:
Ref: ImageId
InstanceType:
Ref: InstanceType
KeyName:
Ref: KeyName
SecurityGroupIds:
- Fn::GetAtt:
- AppSecurityGroup
- GroupId
SubnetId:
Ref: AppSubnet
Tags:
- Key: Name
Value:
Ref: AppName
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
apt-get update
apt-get install -y ruby
echo "Hello World EC2!"
Type: AWS::EC2::Instance
AppInstanceProfile:
Properties:
Path: /
Roles:
- Ref: AppInstanceRole
Type: AWS::IAM::InstanceProfile
AppInstanceRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Version: '2012-10-17'
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- ec2:DescribeTags # allow codedeploy to find machine
Effect: Allow
Resource: '*'
- Action: s3:* # allow machine to access deployables
Effect: Allow
Resource: '*'
- Action: logs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: # required
Fn::Join:
- '-'
- - Ref: AppName
- ec2
Type: AWS::IAM::Role
AppSecurityGroup:
Properties:
GroupDescription:
Ref: AppName
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
FromPort: '3000'
IpProtocol: tcp
ToPort: '3000'
VpcId:
Ref: AppVPC
Type: AWS::EC2::SecurityGroup
AppInternetGateway:
Type: AWS::EC2::InternetGateway
AppRoute:
DependsOn: AppInternetGateway
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: AppInternetGateway
RouteTableId:
Ref: AppRouteTable
Type: AWS::EC2::Route
AppRouteTable:
Properties:
VpcId:
Ref: AppVPC
Type: AWS::EC2::RouteTable
AppSubnet:
Properties:
CidrBlock: 172.31.0.0/20
MapPublicIpOnLaunch: true
VpcId:
Ref: AppVPC
Type: AWS::EC2::Subnet
AppSubnetRouteTableAssociation:
Properties:
RouteTableId:
Ref: AppRouteTable
SubnetId:
Ref: AppSubnet
Type: AWS::EC2::SubnetRouteTableAssociation
AppVPC:
Properties:
CidrBlock: 172.31.0.0/16
EnableDnsHostnames: true
EnableDnsSupport: true
InstanceTenancy: default
Type: AWS::EC2::VPC
AppVPCGatewayAttachment:
Properties:
InternetGatewayId:
Ref: AppInternetGateway
VpcId:
Ref: AppVPC
Type: AWS::EC2::VPCGatewayAttachment
AppCodeDeployApp:
Properties:
ApplicationName:
Ref: AppName
Type: AWS::CodeDeploy::Application
AppCodeDeployGroup:
Properties:
ApplicationName:
Ref: AppCodeDeployApp
DeploymentConfigName: CodeDeployDefault.AllAtOnce
DeploymentGroupName:
Ref: AppName
Ec2TagFilters: # lookup ec2 machine for deployment
- Key: Name
Type: KEY_AND_VALUE
Value:
Ref: AppName
ServiceRoleArn:
Fn::GetAtt:
- AppCodeDeployRole
- Arn
Type: AWS::CodeDeploy::DeploymentGroup
AppCodeDeployRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- codedeploy.amazonaws.com
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole
Path: /
Type: AWS::IAM::Role
Is it possible to define a parameter in a yaml (Open Shift template) file and use it in another parameter like:
kind: Template
apiVersion: v1
metadata:
name: myapp
objects:
parameters:
- name: FILE_VERSION
displayName: File Version
description: The File version
value: 1.2.2
required: true
- name: FILE_NAME
displayName: File Name
description: The File Name
value: prefix:${FILE_VERSION}
required: true
Right now it does not work when I read the above yaml part - ${FILE_VERSION} is not expanded.