Run cmd commands with open source puppet - cmd

Can someone please help me with the open source puppet?
I want to provide a jar file to a windows client and execute the .jar file with the command line.
The .jar file is actually an update for an application which is running as a service.
I am poorly familiar with the puppet language but would guess something like this to execute the jar file:
exec { 'jar_execution':
command => 'cmd.exe /c java -jar foo.jar',
}
Should this be part of the manifest which could look like this?
service { 'fooservice':
name => foo_service,
ensure => running,
enable => true,
}
file { 'foo.jar':
path => 'C:/foo/temp/foo.jar',
ensure => file,
source => "puppet:///modules/foo/foo.jar",
}
exec { 'jar_execution':
command => 'cmd.exe /c java -jar C:/foor/temp/foo.jar',
}
And how does the agent actually run this command?

There are few architectural considerations:
Use archive resource (from archive module) instead of: file with your .jar in your module's .git history.
Have all artifacts (e.g. binary files) served by another service like Apache Archiva, Nexus or Artifactory or even Puppet Server itself. If the .jar is not developed by your company, you may want to use the authoritative source with maybe some internal caching.
Name your resources in a way that are global to your whole infrastructure, otherwise you may have events that notify themselves and produce undesired outcome.
Order of resources in puppet doesn't matter. You want to order the resources the way it makes sense to you and use the before, after, require, notify to ensure dependency.
I would recommend having binaries files outside of a module, as binaries are not supposed to be versioned. You would probably have another service that can serve files, or even your puppet infrastructure can provide those packages, in the similar way it provides the puppet-agent itself, regardless if you use Puppet OSS or Puppet Enterprise.
archive { 'C:/foo/temp/foo.jar':
source => https://location/to/your/.jar,
}
exec { 'C:/foo/temp/foo.jar': # notice in resource name recommended to use / not \
command => 'cmd.exe /c java -jar C:/foor/temp/foo.jar',
refreshonly => true # this one executes exec only if the file was re-downloaded (maybe it's signature has changed), or the file was removed from disk and was re-downloaded.
onlyif => 'test -f C:\foo\temp\foo.jar' # or some command that guards you to not run this .jar every time you run puppet
subscribe => Archive['C:/foo/temp/foo.jar'],
notify => Service['foo_service'] # most probably you need to restart the service after you ran the .jar, otherwise you wouldn't have added in your question.
}
service { 'foo_service':
ensure => running,
enable => true,
}
I notice that in your example you don't need to remove your .jar after being executed. In case you need that, another exec command can remove the file downloaded in the same

Related

Puppet - unable to execute ONLY ONCE ordered chain of Exec commands after notification

TLDR:
I can't configure ordered chain of Puppet "Exec" commands to run ONLY ONCE.
Details:
I want to use Vagrant and Puppet modules to setup VM with installed Redmine and some sample data loaded into it.
I'm using https://forge.puppetlabs.com/johanek/redmine and it works great - Redmine is installed and it works.
My goal:
Now I want to load sample data into Redmine using REST API:
Create 1 test project
Import 2 issues into this project
I want to run 2 simple "Exec", one after another and ONLY ONCE, but I can't achieve this, hence the question.
My current effort:
I've tried to subscribe to one of latest steps in redmine installation
subscribe => [Exec['rails_migrations']]
and then import data, but the first step "create-project1" always notifies second step "import-issues", so it creates duplicated data.
And if run vagrant provision few times, the "import-issues" creates duplicates of this issues.
Here is my code:
exec {'create-project1':
subscribe => [Exec['rails_migrations']],
path => ['/usr/bin', '/usr/sbin', '/bin'],
creates => "$redmine_install_dir/.data_loaded",
command => "curl WHICH_CREATES_PROJECT && touch $redmine_install_dir/.data_loaded",
notify => [Exec['import-issues']],
} ->
exec {'import-issues':
path => ['/usr/bin', '/usr/sbin', '/bin'],
command => "curl WHICH_IMPORTS_ISSUES",
refreshonly => true,
}
Question:
How to configure those Exec commands to run in chain and ONLY ONCE?
Im also thinking about extending this chain to 5 commands in near future, so keep that in mind.
you were almost there with 'ONLY ONCE' - Puppet has onlyif properties that you can include in your exec block to test if a file already exists or not.
you could then do something like
exec {'create-project1':
subscribe => [Exec['rails_migrations']],
path => ['/usr/bin', '/usr/sbin', '/bin'],
onlyif => "test ! -f $redmine_install_dir/.data_loaded"
command => "curl WHICH_CREATES_PROJECT && touch $redmine_install_dir/.data_loaded",
notify => [Exec['import-issues']],
which test on the existence of the $redmine_install_dir/.data_loaded- you should be able to play a bit with that to achieve what you want

Puppet - How to only run 'apt-get update' if a package needs to be installed or updated

I can't seem to figure out how to get Puppet to not run 'apt-get update' during every run.
The standard yet inefficient way:
The way I've been doing this is with the main Puppet manifest having:
exec { 'apt-get update':
path => '/usr/bin',
}
Then each subsequent module that needs a package installed has:
package { 'nginx':
ensure => 'present',
require => Exec['apt-get update'],
}
The problem with this is that, every time Puppet runs, Apt gets updated. This puts unnecessary load on our systems and network.
The solution I tried, but fails:
I looked in the Puppet docs and read about subscribe and refreshonly.
Refresh: exec resources can respond to refresh events (via notify, subscribe, or the ~> arrow). The refresh behavior of execs is non-standard, and can be affected by the refresh and refreshonly attributes:
If refreshonly is set to true, the exec will only run when it receives an event. This is the most reliable way to use refresh with execs.
subscribe
One or more resources that this resource depends on, expressed as resource references. Multiple resources can be specified as an array of references. When this attribute is present:
The subscribed resource(s) will be applied before this resource.
so I tried this in the main Puppet manifest:
# Setup this exec type to be used later.
# Only gets run when needed via "subscribe" calls when installing packages.
exec { 'apt-get update':
path => '/usr/bin',
refreshonly => true,
}
Then this in the module manifests:
# Ensure that Nginx is installed.
package { 'nginx':
ensure => 'present',
subscribe => Exec['apt-get update'],
}
But this fails because apt-get update doesn't get run before installing Nginx, so Apt can't find it.
Surely this is something others have encountered? What's the best way to solve this?
Puppet has a hard time coping with this scenario, because all resources are synchronized in a specific order. For each resource Puppet determines whether it needs a sync, and then acts accordingly, all in one step.
What you would need is a way to implement this process:
check if resource A (a package, say) needs a sync action (e.g., needs installing)
if so, trigger an action on resource B first (the exec for apt-get update)
once that is finished, perform the operation on resource A
And while it would be most helpful if there was such a feature, there currently is not.
It is usually the best approach to try and determine the necessity of apt-get update from changes to the configuration (new repositories added, new keys installed etc.). Changes to apt's configuration can then notify the apt-get upate resource. All packages can safely require this resource.
For the regular refreshing of the database, it is easier to rely on a daily cronjob or similar.
I run 'apt-get update' in a cron script on a daily basis, under the assumption that I don't care if it takes up to 24 hours to update OS packages via apt. Thus...
file { "/etc/cron.daily/updates":
source => "puppet:///modules/myprog/updates",
mode => 755
}
Where /etc/cron.daily/updates is, of course:
#!/bin/sh
apt-get -y update
Then for the applications, I just tell puppet something like:
# Ensure that Nginx is installed.
package { 'nginx':
ensure => latest
}
And done, once apt-get update runs, nginx will get updated to the latest version within the next twenty minutes or so (the next time puppet runs its recipe). Note that this requires you to have done 'apt-get update' in the initial image via whatever process you used to install puppet into the image (for example, if this is in CloudFormation, via the UserData section of the LaunchConfiguration). That is a reasonable requirement, IMHO.
If you want to do 'apt-get update' more often, you'll need to put a cron script into /etc/cron.d with the times you want to run it. I plopped it into cron.daily because that was often enough for me.
This is what you need to do - create an apt-get wrapper that would do apt-get update followed by calling a real apt-get (/usr/bin/apt-get) for install. Install the wrapper into a directory that will be in a PATH before apt-get.
Modify /usr/lib/ruby/vendor_ruby/puppet/provider/package/apt.rb and locate the line:
commands :aptget => "/usr/bin/apt-get"
( it will be right below has_features :versionenable, :install_options )
replace that line with:
commands :aptget => "apt-get"
You're done. For some boneheaded reason puppet insists on calling commands with absolute path rather than using a sane PATH variable.

Copy file from Puppet master to agent unless software is installed

execution of exec or package types by puppet master controllable
Installation of package X on linux, unless X has been installed already:
package { "X": }
&&
Installation of executable Y on Windows, unless Y has been installed already:
exec { "packageYInstalled":
command => "packageY /S",
require => "C:\\temp\\packageY",
unless => "packageYinstalled";
}
Execution of file type by puppet master uncontrollable as unless attribute is not allowed in puppet file type
puppet file attributes
file { "packageYCopiedToTempFolder": }
path => "C:\\temp\\packageY",
source => "puppet:///files/packageY";
}
Execute installers from shared (samba) folder instead of copy it first to agent system does not solve the issue
Puppet runs executed on multiple external systems
Executables, zips and or tar.gz packages are copied to the remote systems during every puppet run, while these files where removed after installation and software has been installed already
The way i tackle this, and there might be a better ways to do this :
Create a module for installing the product
In that module, write a custom fact for discovering the version installed
In the installer class, wrap everything in an 'if'
i.e.
class productx::install (
$version,
$installer_path,
) {
# productx_version is a fact
if ! $::productx_version {
do the install
}
}
You can do other neat stuff then, like audit the software in your environment

Make chef cookbook recipe only run once

So I use the following recipe:
include_recipe "build-essential"
node_packages = value_for_platform(
[ "debian", "ubuntu" ] => { "default" => [ "libssl-dev" ] },
[ "amazon", "centos", "fedora", "centos" ] => { "default" => [ "openssl-devel" ] },
"default" => [ "libssl-dev" ]
)
node_packages.each do |node_package|
package node_package do
action :install
end
end
bash "install-node" do
cwd Chef::Config[:file_cache_path]
code <<-EOH
tar -xzf node-v#{node["nodejs"]["version"]}.tar.gz
(cd node-v#{node["nodejs"]["version"]} && ./configure --prefix=#{node["nodejs"]["dir"]} && make && make install)
EOH
action :nothing
not_if "#{node["nodejs"]["dir"]}/bin/node --version 2>&1 | grep #{node["nodejs"]["version"]}"
end
remote_file "#{Chef::Config[:file_cache_path]}/node-v#{node["nodejs"]["version"]}.tar.gz" do
source node["nodejs"]["url"]
checksum node["nodejs"]["checksum"]
notifies :run, resources(:bash => "install-node"), :immediately
end
It successfully installed nodejs on my Vagrant VM but on restart it's getting executed again. How do I prevent this? I'm not that good in reading ruby code.
To make the remote_file resource idempotent (i.e. to not download a file already present again) you have to correctly specify the checksum of the file. You do this in your code using the node["nodejs"]["checksum"] attribute. However, this only works, if the checksum is correctly specified as the SHA256 hash of the downloaded file, no other algorithm (esp. not MD5) is supported.
If the checksum is not correct, your recipe will still work. However, on the next run, Chef will notice that the checksum of the existing file is different from the one you specified and will download the file again, thus notify the install node ressource and do the whole compile stuff.
With chef, it's important that recipes be idempotent. That means that they should be able to run over and over again without changing the outcome. Chef expects to be able to run all the recipes on a node periodically, and that should be ok.
Do you have a way of knowing which resource within that recipe is causing you problems? The remote_file one is the only one I'm suspicious of being non-idempotent, but I'm not sure offhand.
Looking at the Chef wiki, I find this:
Deprecated Behavior In Chef 0.8.x and earlier, Remote File is also
used to fetch files from the files/ directory in a cookbook. This
behavior is now provided by #Cookbook File, and use of Remote File for
this purpose is deprecated (though still valid) in Chef 0.9.0 and
later.
Anyway, the way chef tends to work, it will look to see if whatever "#{Chef::Config[:file_cache_path]}/node-v#{node["nodejs"]["version"]}.tar.gz" resolves to exists, and if it does, it should skip that resource. Is it possible that install-node deletes that file when it's finished installing? If so, chef will re-fetch it every time.
You can run a recipe only once overriding the run-list with -o modifier.
sudo chef-client -o "recipe[cookbook::recipe]"
-o RunlistItem,RunlistItem..., Replace current run list with specified items
--override-runlist
In my experience remote_file always runs when executing chef-client, even if the target file already exists. I'm not sure why (haven't dug into the Chef code to find the exact cause of the bug), though.
You can always write a not_if or only_if to control the execution of the remote_file resource, but usually it's harmless to just let it run every time.
The rest of your code looks like it's already idempotent, so there's no harm in running the client repeatedly.
There's an action you can specify for remote_file that will make it run conditionally:
remote_file 'target' do
source 'wherever'
action :create_if_missing
end
See the docs.
If you want to test whether your recipe is idempotent, you may be interested in ToASTER, a framework for systematic testing of Chef scripts.
http://cloud-toaster.github.io/
Chef recipes are executed with different configurations in isolated container environments (Docker VMs), and ToASTER reports various metrics such as system state changes, convergence properties, and idempotence issues.

How can I configure Module::Build to NOT install files as read-only?

I've encountered a scenario where I'm building a Perl module as part of another Build system on a Windows machine. I use the --install_base option of Module::Build to specify a temporary directory to put module files until the overall build system can use them. Unfortunately, that other Build system has a problem if any of its files that it depends on are read only - it tries to delete any generated files before rebuilding them, and it can't clean any read-only files (it tries to delete it, and it's read only, which gives an error.) By default, Module::Build installs its libraries with the read-only bit enabled.
One option would be to make a new step in the build process that removes the read-only bit from the installed files, but due to the nature of the build tool that will require a second temporary directory...ugh.
Is it possible to configure a Module::Build based installer to NOT enable that read-only bit when the files are installed to the --install_base directory? If so, how?
No, it's not a configurable option. It's done in the copy_if_modified method in Module::Build::Base:
# mode is read-only + (executable if source is executable)
my $mode = oct(444) | ( $self->is_executable($file) ? oct(111) : 0 );
chmod( $mode, $to_path );
If you controlled the Build.PL, you could subclass Module::Build and override copy_if_modified to call the base class and then chmod the file writable. But I get the impression you're just trying to install someone else's module.
Probably the easiest thing to do would be to install a copy of Module::Build in a private directory, then edit it to use oct(666) (or whatever mode you want). Then invoke perl -I /path/to/customized/Module/Build Build.PL. Or, (as you said) just use the standard Module::Build and add a separate step to mark everything writable afterwards.
Update: ysth is right; it's ExtUtils::Install that actually does the final copy. copy_if_modified is for populating blib. But ExtUtils::Install also hardcodes the mode to read-only. You could use a customized version of ExtUtils::Install, but it's probably easier to just have a separate step.

Resources