Create folder on VCenter server using Ansible - ansible

Problem statement:
Given a folder name, Check if it exists on VCenter Server and if not create the same.
Clone a VM from template under this folder.
For example, I want to clone a VM under "Administrator Desktops" as shown in the following image:
Click here to see the image
The script below is cloning a VM from specified template and placing the VM into specified folder. But fails when that folder does not exist on VCenter server:
---
- hosts: localhost
connection: local
sudo: false
user: root
gather_facts: false
serial: 1
vars_files:
- createVmVars.yml
tasks:
- name: Deploying VM from template.
vsphere_guest:
vcenter_hostname: "{{vcenter_hostname}}"
username: "{{vcenter_username}}"
password: "{{vcenter_password}}"
guest: "{{guest_name}}"
from_template: yes
template_src: "{{template_src}}"
cluster: "{{cluster}}"
resource_pool: "{{resource_pool}}"
vm_extra_config:
folder: "{{folder_name}}"
Need help to make this script flexible so that when the folder does not exists, it should create the mentioned folder and then clone the VM under this folder.

You can use this Ansible module to create folder in vsphere:
#!/usr/bin/python
# -*- coding: utf-8 -*-
DOCUMENTATION = '''
---
author: "Shashank Awasthi"
module: vsphere_create_folder
short_description: Create a folder on VCenter if it does not exist
description:
- This module requires login to VCenter Server
- This module requires pysphere python module installed
- This module creates a folder on mentioned VCenter Server
- This module does not create any folder if the folder with the same name is already existing on the VCenter Server
- This module supports nesting upto only 2 levels
version_added: "1.2"
options:
host:
description:
- The vsphere Server on which the folder is to be created
required: true
login:
description:
- The login name to authenticate on VSphere
required: true
password:
description:
- The password to authenticate VSphere
required: true
folder_name:
description:
- The folder name which is to be created
required: true
parent_folder_name:
description:
The name of parent folder under which the folder_name is residing.
required: true
datacenter_name:
description:
- The name of the datacenter where the folder is to be created
required: true
examples:
- description: create the folder with name NewDeployments
code:
- local_action: vsphere_create_folder host=$eszserver login=$esxlogin password=$esxpassword folder_name=$folder_name
parent_folder_name=$parent_folder_name datacenter_name=$dc_name
notes:
- This module ought ot be run from a system which can access vsphere directly
'''
import sys
try:
import pysphere
from pysphere import *
from pysphere.resources import VimService_services as VI
except ImportError:
print "failed=true, msg=Pysphere Python module not available"
sys.exit(1)
def main():
module = AnsibleModule(
argument_spec = dict(
host = dict(requred = True),
login = dict(required = True),
password = dict(required = True),
folder_name = dict(required = True),
parent_folder_name = dict(required = True),
datacenter_name = dict(required = True)
)
)
host = module.params.get('host')
login = module.params.get('login')
password = module.params.get('password')
folder_name = module.params.get('folder_name')
parent_folder_name = module.params.get('parent_folder_name')
datacenter_name = module.params.get('datacenter_name')
server = pysphere.VIServer()
try:
server.connect(host,login,password)
except Exception, e:
module.fail_json(msg = 'Failed to connect to %s: %s' % (host, e))
def createFolder(vm_folder,folder_name):
try:
request = VI.CreateFolderRequestMsg()
_this = request.new__this(vm_folder)
_this.set_attribute_type(vm_folder.get_attribute_type())
request.set_element__this(_this)
request.set_element_name(folder_name)
server._proxy.CreateFolder(request)
except pysphere.ZSI.FaultException, e:
pass
try:
datacenters = server._get_datacenters()
dc = datacenters[datacenter_name]
dc_props = VIProperty(server,dc)
vm_folder = dc_props.vmFolder._obj
createFolder(vm_folder,parent_folder_name)
folders = server._retrieve_properties_traversal(property_names=['name'], from_node = dc, obj_type = 'Folder')
for f in folders:
if f.PropSet[0].Val == parent_folder_name:
vm_folder = f.Obj
break
createFolder(vm_folder,folder_name)
except Exception, e:
module.fail_json(msg = "failed to create folder: %s" % e)
module.exit_json(changed = True, folder = folder_name, parent_folder = parent_folder_name)
#<<INCLUDE_ANSIBLE_MODULE_COMMON>>
main()

Related

Pass Ansible variables into custom Ansible module

I have a custom module that resides in the library/ directory of my Ansible role. I can call the module from within my playbook, and the code executes correctly, but only if the values it expects are hardcoded in the module code itself. How can I pass values to the module from the playbook?
I've tried the following:
- name: Create repo and use specific KMS key
ecr_kms:
repositoryName: "new-ecr-repo"
encryptionConfiguration.kmsKey: 'my-kms-key-id"
and
- name: Create repo and use specific KMS key
ecr_kms:
repositoryName: "{{ repo_name }}"
encryptionConfiguration.kmsKey: "{{ kms_key_id }}"
Which I would expect to work, but neither does and, I get the following errors:
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid length for parameter repositoryName, value: 0, valid min length: 2
Invalid length for parameter encryptionConfiguration.kmsKey, value: 0, valid min length: 1
The service module I'm trying to use
The code of the custom module:
#!/usr/bin/python
from urllib import response
import boto3
from jinja2 import Template
from ansible.module_utils.basic import AnsibleModule
def create_repo():
client = boto3.client('ecr')
response = client.create_repository(
#registryId='',
repositoryName='',
imageTagMutability='IMMUTABLE',
imageScanningConfiguration={
'scanOnPush': True
},
encryptionConfiguration={
'encryptionType': 'KMS',
'kmsKey': ""
}
)
def main():
create_repo()
if __name__ in '__main__':
main()
You do need to make your module aware of the arguments you want it to accept, so, in your main function:
#!/usr/bin/env python
from ansible.module_utils.basic import AnsibleModule
def create_repo(repositoryName, kmsKey):
# Call to the API comes here
def main():
module = AnsibleModule(
argument_spec = dict(
repositoryName = dict(type = 'str', required = True),
kmsKey = dict(type = 'str', required = True),
)
)
params = module.params
create_repo(
params['repositoryName'],
params['kmsKey']
)
if __name__ == '__main__':
main()
More can be found in the relevant documentation: Argument spec.
With this, your taks would be:
- name: Create repo and use specific KMS key
ecr_kms:
repositoryName: "{{ repo_name }}"
kmsKey: "{{ kms_key_id }}"
PS, word of advice: avoid using a dot in a YAML key, that would just be making your life complicated for no actual good reason.

sap-cf-mailer VError: No service matches destination

I have a SAP CAP Nodejs application and I'm trying to send emails from the CAP application using sap-cf-mailer package.
I've created a destination service in the BTP as mentioned in the sample and when I'm trying to deploy the application to BTP it failed.
When I run the application locally using cds watch, it gives following error
VError: No service matches destination
This is my mta.yaml
## Generated mta.yaml based on template version 0.4.0
## appName = CapTest
## language=nodejs; multitenant=false
## approuter=
_schema-version: '3.1'
ID: CapTest
version: 1.0.0
description: "A simple CAP project."
parameters:
enable-parallel-deployments: true
build-parameters:
before-all:
- builder: custom
commands:
- npm install --production
- npx -p #sap/cds-dk cds build --production
modules:
# --------------------- SERVER MODULE ------------------------
- name: CapTest-srv
# ------------------------------------------------------------
type: nodejs
path: gen/srv
parameters:
buildpack: nodejs_buildpack
requires:
# Resources extracted from CAP configuration
- name: CapTest-db
- name: captest-destination-srv
provides:
- name: srv-api # required by consumers of CAP services (e.g. approuter)
properties:
srv-url: ${default-url}
# -------------------- SIDECAR MODULE ------------------------
- name: CapTest-db-deployer
# ------------------------------------------------------------
type: hdb
path: gen/db
parameters:
buildpack: nodejs_buildpack
requires:
# 'hana' and 'xsuaa' resources extracted from CAP configuration
- name: CapTest-db
resources:
# services extracted from CAP configuration
# 'service-plan' can be configured via 'cds.requires.<name>.vcap.plan'
# ------------------------------------------------------------
- name: CapTest-db
# ------------------------------------------------------------
type: com.sap.xs.hdi-container
parameters:
service: hana # or 'hanatrial' on trial landscapes
service-plan: hdi-shared
properties:
hdi-service-name: ${service-name}
- name: captest-destination-srv
type: org.cloudfoundry.existing-service
This is the js file of the CDS service
const cds = require('#sap/cds')
const SapCfMailer = require('sap-cf-mailer').default;
const transporter = new SapCfMailer("MAILTRAP");
module.exports = cds.service.impl(function () {
this.on('sendmail', sendmail);
});
async function sendmail(req) {
try {
const result = await transporter.sendMail({
to: 'someoneimportant#sap.com',
subject: `This is the mail subject`,
text: `body of the email`
});
return JSON.stringify(result);
}
catch{
}
};
I'm following below samples for this
Send an email from a nodejs app
Integrate email to CAP application
Did you create your default-.. json files? They are required to connect to remote services on your BTP tenant. You can find more info about this on SAP blogs like this one:
https://blogs.sap.com/2020/04/03/sap-application-router/
You could also use the sap-cf-localenv command:
https://github.com/jowavp/sap-cf-localenv
This tool is experimental,a s far as I know, this only works for the CF CLI V6. Higher version are fetching the service keys in another format, which leads to the command to fail.
Kind regards,
Thomas

Accessing Ansible variables in molecule test, TestInfra

I picked up molecule while researching around inspec and how to use it in ansible. I found molecule very cool and adopted it. I wanted to use it in 2 ways.
1- When developing a role or playbook
2- After a particular playbook have been run on production.
On number 1: I found this very useful question/ressponse on stackoverflow and that has helped me shape my thinking.I put my variable file for the role kafka under group_vars/all as suggested in the stackoverflow post
- kafka
- - molecule
- - - default
- - - - molecule.yml
- - - - playbook.yml
- - - - ...
- - - - group_vars
- - - - - all.yml
- - - - tests
- - - - - test_default.py
- - tasks
- - - main.yml
- - ....
test_default.py
import os
import testinfra.utils.ansible_runner
import pytest
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')
#pytest.fixture()
def AnsibleVars(host):
all_vars = host.ansible.get_variables()
return all_vars
def test_hosts_file(host):
f = host.file('/etc/hosts')
assert f.exists
assert f.user == 'root'
assert f.group == 'root'
def test_downloaded_binary(host, AnsibleVars):
# arch = host.file(AnsibleVars['kafka_archive_temp'])
result = host.ansible('debug','var=kafka_archive_temp')
arch = host.file(result['kafka_archive_temp'])
assert arch.exists
assert arch.is_file
def test_installation_directory(host,AnsibleVars):
# dir = host.file(AnsibleVars['kafka_final_path'])
result = host.ansible('debug','var=kafka_final_path')
dir = host.file(result['kafka_final_path'])
assert dir.exists
assert dir.is_directory
assert dir.user == AnsibleVars['kafka_user_on_os']
assert dir.group == AnsibleVars['kafka_group_on_os']
def test_user_created(host,AnsibleVars):
user = host.user(AnsibleVars['kafka_user_on_os'])
assert user.name == AnsibleVars['kafka_user_on_os']
assert user.group == AnsibleVars['kafka_group_on_os']
group_vars/all.yml
kafka_version: "2.2.1"
kafka_file_name: "kafka_2.12-{{ kafka_version }}.tgz"
kafka_user_on_os: kafka
kafka_group_on_os: kafka
kafka_zookeeper_service: zookeeper
kafka_service: kafka
kafka_log_folder: /var/log/kafka
kafka_zookeeper_port: 2181
kafka_archive_temp: "/tmp/{{ kafka_file_name }}"
kafka_final_path: "/usr/local/kafka/{{ kafka_version }}"
kafka_get_binaries_details:
- {
dest: "{{ kafka_archive_temp }}",
url: "http://www-us.apache.org/dist/kafka/2.2.1/kafka_2.12-2.2.1.tgz"
}
....
molecule verify
molecule verify
--> Validating schema /Users/joseph/Engineering/configuration-management-ansible/roles/kafka/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
└── default
└── verify
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /Users/joseph/Engineering/configuration-management-ansible/roles/kafka/molecule/default/tests/...
============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: /Users/joseph/Engineering/configuration-management-ansible/roles/kafka/molecule/default
plugins: testinfra-3.1.0
collected 8 items
tests/test_default.py ........ [100%]
============================== 8 passed in 18.34s ==============================
Verifier completed successfully.
However the method host.ansible.get_variables() could not resolve a variable inside another variable like : kafka_final_path: "/usr/local/kafka/{{ kafka_version }}".
I ended up using the following:
result = host.ansible('debug','var=kafka_final_path')
dir = host.file(result['kafka_final_path'])
to get the value of kafka_final_path.
Question 1.1: Looking at how there is a need of a little manipulation before a variable of a variable get be retrieved with all needed interpolation, I am wondering there is any better way of writing these tests?
Question 2.1: On number 2, I would like to create a different scenario for testing like for EC2 on AWS. On those playbooks, I use external variable files passed to ansible-playbook since they have higher precedence. I am wondering in that case how I would access those variables from the external vars_files in testinfra?

Giving ssh credentials for salt backend with gitfs in vagrant box

I use vagrant to test my salt configs. On one box I have a vagrant salt master and salt minion and on another box I have a salt minion.
I'm trying to switch over to use gitfs to fetch backend from my private repo.
This is my salt.master_config:
hash_type: sha256
auto_accept: True
roster_file: /srv/salt/roster
fileserver_backend:
- roots
- git
file_roots:
base:
- /srv/salt/environments/base/files
- /srv/salt/environments/base/states
- /srv/salt/users
gitfs_remotes:
- git#github.com:user/repo.git
- pubkey: /srv/salt/environments/base/files/.ssh/id_rsa.pub
- privkey: /srv/salt/environments/base/files/.ssh/id_rsa
- https://github.com/salt/users-formula.git
- https://github.com/salt/openssh-formula.git
The other gitfs remotes have worked in the past but the new one does not accept the ssh keys.
When running sudo salt '*' state.apply I get:
ERROR ] Error parsing configuration file: /etc/salt/master - mapping values are not allowed here
in "<string>", line 16, column 13:
- pubkey: /srv/salt/environments/base/fi
I have also tried using the Master Options from the vagrantup prosvisioning doc https://www.vagrantup.com/docs/provisioning/salt.html and added master_pub and master_key to my Vagrantfile:
master.vm.provision :salt do |salt|
salt.install_master = true
salt.master_pub = 'id_rsa.pub'
salt.master_key = 'id_rsa'
salt.install_type = 'stable'
salt.master_config = 'master'
salt.minion_config = 'salt-local'
end
But this is to ssh to the vagrant box and not actually used for gitfs.
How do you give ssh credentials for the vagrant config files?
Found the solution:
gitfs_provider: pygit2
gitfs_pubkey: /srv/salt/path/to/files/ssh/id_rsa.pub
gitfs_privkey: /srv/salt/path/to/files/ssh/id_rsa
gitfs_remotes:
- git#github.com:user/repo.git

Ansible Dict and Tags

I have a playbook creating EC2 by using a dictionary declared in vars: then registering the IPs into a group to be used later on.
The dict looks like this:
servers:
serv1:
name: tag1
type: t2.small
region: us-west-1
image: ami-****
serv2:
name: tag2
type: t2.medium
region: us-east-1
image: ami-****
serv3:
[...]
I would like to apply tags to this playbook in the simplest way so I can create just some of them using tags. For example, running the playbook with --tags tag1,tag3 would only start EC2 matching serv1 and serv3.
Applying tags on the dictionary doesn't seem possible and I would like to avoid doing multiplying tasks like:
Creatinge EC2
Register infos
Getting private IP from previously registered infos
adding host to group
While I already have a working loop for the case I want to create all EC2 at once, is there any way to achieve that (without relying on --extra-vars, which would need key=value) ? For example, filtering out the dictionary by keeping only what is tagged before running the EC2 loop ?
I doubt you can do this out of the box. And not sure this is good idea at all.
Because tags are used to filter tasks in Ansible, so you will have to mark all tasks with tags: always.
You can accomplish this with custom filter plugin, for example (./filter_plugins/apply_tags.py):
try:
from __main__ import cli
except ImportError:
cli = False
def apply_tags(src):
if cli:
tags = cli.options.tags.split(',')
res = {}
for k,v in src.iteritems():
keep = True
if 'name' in v:
if v['name'] not in tags:
keep = False
if keep:
res[k] = v
return res
else:
return src
class FilterModule(object):
def filters(self):
return {
'apply_tags': apply_tags
}
And in your playbook:
- debug: msg="{{ servers | apply_tags }}"
tags: always
I found a way to match my needs without touching to the rest so I'm sharing it in case other might have a similar need.
I needed to combine dictionaries depending on tags, so my "main" dictionary wouldn't be static.
Variables became :
- serv1:
- name: tag1
type: t2.small
region: us-west-1
image: ami-****
- serv2:
- name: tag2
type: t2.medium
region: us-east-1
image: ami-****
- serv3:
[...]
So instead of duplicating my tasks, I used set_fact with tags like this:
- name: Combined dict
# Declaring empty dict
set_fact:
servers: []
tags: ['always']
- name: Add Server 1
set_fact:
servers: "{{ servers + serv1 }}"
tags: ['tag1']
- name: Add Server 2
set_fact:
servers: "{{ servers + serv2 }}"
tags: ['tag2']
[..]
20 lines instead of multiply tasks for each server, change vars from dictionary to lists, a few tags and all good :) Now if I add a new server it will only take a few lines.

Resources