I am soooo confused when it comes to Chef / Berkshelf and need help and advice.
What I've found / read there's an underlining assumption with some things with Berkshelf and for the newbie there is a bit of a grey area that needs filling
Let me try to explain:
I followed the typical Chef path
Create Chef-repo in user directory
C:\Users\itsmeofcourse\chef-repo
then hooked that into an internal git-repo
and happily writing basic cookbooks for Windows and uploading everything into that git-repo
as it stands every cookbook exists under the "cookbook" folder in my chef-repo.
C:\Users\itsmeofcourse\chef-repo
/cookbook
I've then followed the path of writing wrapper cookbooks around community cookbooks, so it would look like
client_iis - depends upon
department_iis - depends upon
global_iis - depends upon
iis - community cookbook
this allows us make IIS changes at certain different levels within our infrastructure.
Now where documentation I feel falls down, is everyone is saying move your cookbooks out of the "cookbook" folder
so what I understand, "your" chef-repo will exist in a git-repo but just for changes to sub-folders like environments / data bags / roles / certificates etc ? and the cookbook are then separate projects is that correct or not ?
Where do you move your cookbooks to ? anywhere on you machine / within your user %home%?
How does Chef know where these are stored or do you have to amend your "knife.rb" and point to a certain directory ?
so it would look like
knife.rb
cookbook_path ["c:/cookbooks"]
C:\Users\itsmeofcourse\chef-repo :github => repo_1
c:/cooksbooks
/base :github => repo_2
/iis :github => repo_3
/sql :github => repo_4
/client_iis :github => repo_
/department_iis :github => repo_3
Can I ask what am I missing
or do you place a berksfile in the root of my chef-repo and then do what ? to manage everything in my cookbook folder ?
I have read through https://github.com/berkshelf/berkshelf/issues/535
please can someone help
Yes, it's correct and you would usually move your cookbooks to a separate repository.
One gotcha I had, was while reading this article: http://www.prashantrajan.com/posts/2013/06/leveling-up-chef-best-practices/ and the "Single Repo Per Cookbook" part. Read the whole thing, it's good!
It seems like you are not missing anything. Moving your cookbooks out of cookbooks directory, means creating separate repos per cookbook and depending on them using top level Berksfile (in the root of your chef-repo).
For a typical vagrant+chef repo for a web app (called coolwebapp), I would usually have:
.
+-- cmp-cookbooks
| +-- cmp-coolwebapp (this is only cookbook stored in this repo, and this repo exists because of this cookbook)
+-- data_bags
| +-- users
| | +-- mysql.json
| | +-- os.json
| | +-- admins.json
| +-- private_keys
| +-- deployment.json
+-- environments
| +-- production.rb
| +-- staging.rb
| +-- qa.rb
| +-- integration.rb
| +-- local.rb
+-- nodes (but this should not be stored in your repo I guess)
| +-- ip_here.json
| +-- other_ip_here.json
+-- Berksfile
+-- Vagrantfile
Berksfile would contain:
cookbook "cmp-coolwebapp", "~>0.3.0", path: "./cmp-cookbooks/cmp-coolwebapp"
cookbook "cmp-provisioning", "~>0.7.0", git: _priv_provisioning_cookbook_repo_
cookbook "cmp-role-db", "~>0.7.0", git: _priv_role1_cookbook_repo_
cookbook "cmp-role-www", "~>0.8.0", git: _priv_role2_cookbook_repo_
cookbook "cmp-role-devops", "~>0.7.0", git: _priv_role3_cookbook_repo_
"cmp" stands for our company name. Our cookbooks are stored in private repos, and are being maintained individually.
Cookbook cmp-role-www for example, would have mostly community cookbooks as dependencies in Berksfile, and our own cmp-apache2, cmp-nginx, cmp-varnish wrapper cookbooks stored in its repo.
Answering your last question" "How does Chef know where these are stored or do you have to amend your "knife.rb" and point to a certain directory ?"
If you manage your cookbook dependencies with Berkshelf, you can include cookbook from any location you prefer:
cookbook "artifact", path: "/Users/reset/code/artifact-cookbook"
cookbook "mysql", git: "https://github.com/opscode-cookbooks/mysql.git", branch: "foodcritic"
cookbook "rightscale", git: "https://github.com/rightscale/rightscale_cookbooks.git", rel: "cookbooks/rightscale"
The last one is useful when you store several company cookbook in one repository.
http://berkshelf.com/ "Source options" section.
Related
eg.
The directory structure of the recipes is as follows:
--my_cookbook
|-- recipes
|- abc.rb
|- xyz.rb
|-- attributes
|-- templates
|- random.xml.erb
|-- test
Now let's assume we have a resource as follows in abc.rb
... # Line 20
template '/some_location/random.xml' do
source 'random.xml.erb'
owner 'root'
group 'root'
mode '0644'
notifies :start, 'service[vicious_service]', :immediately
end
... # Line 28
Now we have the declaration of vicious_service which is as follows:
service 'vicious_service' do
action [:enable, :start]
end
The question now is, Can we have the declaration of vicious_service somewhere in xyz.rb ? Or is it mandatory for us to declare it in abc.rb ?
Yes, what recipe things are in doesn't matter to Chef beyond the original loading phase. Everything ends up in a big array called the "resource collection". You can find more details at https://coderanger.net/two-pass/.
I want to leverage chef-metal and chef-zero with my existing cookbooks and chef-repo (already leveraging berkshelf and vagrant for dev)
I started with the example provided at https://github.com/opscode/chef-metal#vagrant
I've got a vagrant_linux.rb
require 'chef_metal_vagrant'
vagrant_box 'CentOS-6.4-x86_64' do
url 'http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.4-x86_64-v20130427.box'
end
with_machine_options :vagrant_options => {
'vm.box' => 'CentOS-6.4-x86_64'
}
I also have dev_server.rb
require 'chef_metal'
with_chef_local_server :chef_repo_path => '~/workspace/git/my-chef-repo'
machine 'dev_server' do
tag 'dev_server'
recipe 'myapp'
converge true
end
If I put my myapp cookbook under ~/workspace/git/my-chef-repo/cookbooks, the above works fine
using the following command, I've got a vagrant managed vm named dev_server converging (applying myapp recipe)
chef-client -z vagrant_linux.rb dev_server.rb
But now, I'd like to keep my cookbooks folder empty and use berkshelf,
It does not look supported by chef-zero at the moment, is it ?
How could I do that ?
You can pass :cookbook_path that contains multiple paths as an Array like so: https://github.com/opscode/ec-metal/blob/master/cookbooks/ec-harness/recipes/vagrant.rb#L12-L13
with_chef_local_server :chef_repo_path => repo_path,
:cookbook_path => [ File.join(repo_path, 'cookbooks'),
File.join(repo_path, 'vendor', 'cookbooks') ]
Then you can use berks to vendor upstream cookbooks into a different path (vendor/cookbooks/), while putting your own cookbooks into cookbooks/ like so: https://github.com/opscode/ec-metal/blob/master/Rakefile#L114
berks vendor vendor/cookbooks/
The "berks vendor" command is how I generally do that--use "berks vendor" and add the vendored path to your cookbook path.
I'm using puppet as my provisioner in one of my vagrant project. I'm trying to add a module for a custom bash_profile.
The module_path for puppet is set to:
puppet.module_path = "puppet/modules"
The class for my bash_profile module looks like this:
class bash_profile
{
file
{
"/home/vagrant/bash_profile":
ensure => present,
source => "puppet:///modules/bash_profile/files/bash_profile"
}
}
Here's the file structure for my puppet structure:
puppet
| manifests
| | phpbase.pp // my main manifest file that has includes for modules
| modules
| | bash_profile
| | | files
| | | | bash_profile // the actual bash_profile file I want to ensure is present on my VM
| | | manifests
| | | | init.pp // the init file included for the bash_profile class
When I run the provisioning for vagrant, I get the error
err: /Stage[main]/Bash_profile/File[/home/vagrant/bash_profile]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/bash_profile/files/bash_profile at /tmp/vagrant-puppet-1/modules-0/bash_profile/manifests/init.pp:8
I'm not sure why it can't retrieve the information. The path seems to be correct. Can anyone see what I'm missing?
Yes, you are not supposed to include the literal files/ in the URL. Instead, it should just be
puppet:///modules/bash_profile/bash_profile
You may also receive this error with recurse => true if your module name is invalid. For instance, if you have this module structure:
modules
├── my-example
│ └── files
│ └── example
│ └── test.txt
and this resource:
file { "/tmp/example":
ensure => directory,
recurse => true,
source => "puppet:///modules/my-example/example",
}
you'll get this error:
==> default: Info: Could not find filesystem info for file 'my-example/example' in environment production
==> default: Error: /Stage[main]/Main/Node[default]/File[/tmp/example]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///my-example/example
The fix is to rename the module—for instance, naming it my_example fixes it. The rules for module names are documented but easy to miss.
Things to care about
The Puppet URI format puppet:///modules/name_of_module/filename
The fileserver directory to be present in the module directory
This video is an shows step-by-step guide to resolve the error
I have currently my octopress blog up and running on my digital ocean droplet and everything works fine. But it is in the root folder of the droplet and I want to move it to a blog folder. Is that just moving all content to that subfolder, or is there more work to do to get this done to be aware of?
According to the Octopress documentation, if you are deploying in a subdirectory you'll need to update the following files:
_config.yml
config.rb
Rakefile
If you're deploying to a subdirectory on your site, or if you're using
Github's project pages, make sure you set up your urls correctly in
your configs. You can do this almost automatically:
rake set_root_dir[your/path]
# To go back to publishing to the document root
rake set_root_dir[/] Then update your _config.yml and Rakefile as follows:
# _config.yml
url: http://yoursite.com/your/path
# Rakefile (if deploying with rsync)
document_root = "~/yoursite.com/your/path"
To manually configure deployment to a subdirectory, you'll change _config.yml, > config.rb and Rakefile. Here's an example for deploying a site to the /awesome
subdirectory:
# _config.yml
destination: public/awesome
url: http://example.com/awesome
subscribe_rss: /awesome/atom.xml
root: /awesome
# config.rb - for Compass & Sass
http_path = "/awesome"
http_images_path = "/awesome/images"
http_fonts_path = "/awesome/fonts"
css_dir = "public/awesome/stylesheets"
# Rakefile
public_dir = "public/awesome"
# If deploying with rsync, update your Rakefile path
document_root = "~/yoursite.com/awesome"
source: http://octopress.org/docs/deploying/subdir/
I have a Sinatra-based app that runs fine locally.
I moved it to a nginx-based server with Passenger and now all my links to files in my apps /public are returning 404 errors. The primary app runs, is able to access the HAML templates in /view, which render correctly. The files exist and permissions are correct; I can open and edit them so I know they're there.
In my HAML templates I'm referring to the files that I can't access like this:
%script{ :src => 'js/jquery.js' }
%link{ "rel" => "stylesheet", "href" => "styles/input.css" }
My config.ru has gone through a lot mutations while I try to find the problem. Currently I have:
require 'sinatra'
require './peering_template.rb'
root_dir = File.dirname(__FILE__)
# disable :run
# set :root, root_dir
# set :views, File.join(File.dirname(__FILE__), 'views')
# set :environment, (ENV['RACK_ENV'] ? ENV['RACK_ENV'].to_sym : :development)
run Sinatra::Application
The app exists in /home/apps/peering_template.
The web space is /home/webapps.
There is a soft-link in /home/webapps like this: peering_template -> /home/apps/peering_template/public/.
/home/webapps/
`-- peering_template -> /home/apps/peering_template/public/
The pertinent part of nginx.conf for this config is:
server {
listen 3000;
server_name my_servers_name;
root /home/webapps;
passenger_enabled on;
passenger_base_uri /peering_template;
}
Obviously, my server's name is different.
The pertinent part from nginx' error.log is like this:
"/home/webapps/js/jquery.js" failed (2: No such file or directory), request: "GET /js/jquery.js HTTP/1.1"
As near as I can tell this fits the directions for an "nginx and passenger configuration using sub-URIs". What am I missing?
/home/apps/peering_template/
|-- config.ru
|-- lib
| |-- bgp-config.rb
| |-- ios-xr-config.rb
| |-- ipv4_ipv6_grammar.rb
| `-- ipv4_ipv6_grammar.treetop
|-- nginx.conf
|-- peering_template.rb
|-- public
| |-- js
| | |-- jquery-1.6.min.js
| | |-- jquery-ui-1.8.12.custom.zip
| | |-- jquery.js -> jquery-1.6.min.js
| | `-- scripts.js
| |-- peering_template_tool.htm
| `-- styles
| `-- input.css
|-- spreadsheets
| |-- Peering Template-AMS-IX.xlsx
| `-- Peering Template-IOS-XR-ASH1.xlsx
|-- tmp
| `-- always_restart.txt
`-- views
|-- index.haml
`-- output.haml
I'm not sure if it matters but this is on a CentOS release 5.3 (Final) host, running nginx/1.0.0 and passenger (3.0.7).
In the original question I wrote:
I moved it to a nginx-based server with Passenger and now all my links to files in my apps /public are returning 404 errors. The primary app runs, is able to access the HAML templates in /view, which render correctly. The files exist and permissions are correct; I can open and edit them so I know they're there.
That was my clue. On my fourth pass or so through the Passenger docs I ran into a section that talked about errors with /public assets:
The second and highly recommended way is to always use Rails helper methods to
output tags for static assets. These helper methods automatically take care of
prepending the base URI that you’ve deployed the application to. For images
there is image_tag, for JavaScript there is javascript_include_tag and for CSS
there is stylesheet_link_tag. In the above example you would simply remove the
HTML tag and replace it with inline Ruby like this:
So, that got me searching for similar helpers for Sinatra. I found in Sinatra's extensions page:
sinatra-url-for construct absolute paths and full URLs to actions in a Sinatra application
sinatra-static-assets implements image_tag, stylesheet_link_tag, javascript_script_tag and link_tag helpers. These helpers construct correct absolute paths for applications dispatched to sub URI.
And that got me searching Sinatra's docs because it sparked a memory, and I relearned Sinatra's built-in "url" method:
Generating URLs
For generating URLs you should use the url helper method, for instance, in
Haml:
%a{:href => url('/foo')} foo
It takes reverse proxies and Rack routers into account, if present.
This method is also aliased to to (see below for an example).
By using the static asset methods, or Sinatra's own url helper it fixed the problem.
The root in your nginx config should be the public (or some other) directory, not the root of the entire rails application:
root /home/webapps/public;
Now put all your static files inside that directory, and Passenger will be smart enough to resolve the config.ru Rack up file from the parent directory automatically, but serve the files from the public directory if they exist through nginx.
For what it's worth, as well, you shouldn't need anything but the require of your application ruby file, and and the Sinatra init method in your rack file. Here's one I use in another app:
require 'application'
run Sinatra::Application
One other little note, best practice to stick a / in front of any URLs that reference these static files to ensure they're reachable wherever the page URL ends up, eg. ...:src => '/js/jquery.js'...
Edit:
I think there's a fundamental problem with the way your app is setup on the server. In my mind, it should look something like this:
/app
whatever.rb
/public
...
The nginx config should point to app/public as the root, and the public directory should not be a symlink.
With all that in mind, perhaps the root should just be set directly to /home/apps/peering_template/public?