Category Archives: vagrant

Getting Started with Chef on Windows Server – Part 3a – Packer, Vagrant, and Vagrant Omnibus

Introduction – HashiCorp Atlas

It’s been a fair few months since my last post in this series (or at all for this matter) and I haven’t made anywhere near the progress with Chefifying (definitely a word) my environment as I would like due to new more urgent projects.

Since my last post, HashiCorp has announced Atlas as a tech preview, which (in part) replaces VagrantCloud which I referenced in part 3. HashiCorp are placing Atlas as a solution to a large portion of your DevOps deployment lifecycle.

how-it-works-4a283ee2

This is powered by four components:

  • Consul (monitoring & service discovery)
  • Packer (VM template management)
  • Vagrant (Repeatable development environment creation)
  • Terraform (Cloud agnostic infrastructure-as-code)

Atlas looks to be the wrapper that ties all these together to make it easy to manage vagrant boxes, terraform deployments, packer images and consul configurations in a single place as a coherent pipeline.

Terraform looks very interesting to me as it’s placed in a CloudStack style space but looks much lighter weight. But currently we’re only interested in Vagrant and by extension Vagrant boxes and Vagrant Cloud.

So what happened to Vagrant Cloud?

vagrantcloudandatlas

Well the good news is that all existing VagrantCloud Boxes and URLs still work, so in theory Part 3 will still work for everyone!

However, for some reason, when you search Atlas for “Windows” the kensykora/windows_2012_r2_standard box we were using no longer shows up!

windows search on hasicorp atlas

No idea why as it still exists, but anyway this gave me a good excuse to play with Packer!

Rolling your own Windows 2012 R2 Trial Vagrant box with Packer & Joe Fitzgerald

Why? Well, initially I thought kensykora’s Vagrant box had disappeared which was what caused me to look into this in the first place, but there are a few additional reasons:

  1. kensykora’s box took a very long time to download (I imagine because the chap was dutifully hosting it on his own hardware in which case power to him and we shouldn’t complain about conveniences we get for free!)
  2. Packer allows you to roll your own boxes from the Windows Trial ISOs, meaning you’re not dependent on other people keeping their boxes up-to-date with fresh trials
  3. Packer is a useful tool in its own right. It allows you to create your own golden images (even if they’re just Windows with the latest updates installed) and deploy them pretty much anywhere, AWS, VMWare, VirtualBox, Azure, you name it.

Like pretty much every tool in this space it has its origins in in the nix world, but fortunately all the hard work of creating Windows packages has been done for us already by Joe Fitzgerald in his freaking awesome Windows Packer GitHub repo.

There’s a lot of stuff going on with Packer, but we’re not trying to explore Packer in depth just yet. All we’re trying to do is roll a Windows 2012 R2 trial Vagrant box with the minimum amount of fuss.

You will need

  1. VirtualBox
  2. Vagrant
  3. Some knowledge of the above two (the previous two parts of this blog series would be helpful! :))

Step 1) Install Packer

Well… I say install, more like extract and add to your path environment variable.

Download the windows version from https://packer.io/downloads.html and extract it anywhere you like, and add that path to your PATH environment variable like so:

[Environment]::SetEnvironmentVariable("path", "$($env:path);C:\Users\smartin\Documents\packer", "machine")

Where “C:\Users\smartin\Documents\packer” is the path you extracted packer to.
You could of course set it using the GUI like a normal person if you prefer!

Step 2) Download and extract Joe’s Packer-Windows repo

Go to https://github.com/joefitzgerald/packer-windows and hit the “Download Zip” button.

download zip github

Then extract the Zip to wherever you like (though preferably on a hard disk with some space as this directory will be where the ISO is downloaded to AND where the resultant box file is created).

Step 3)  Disable Windows Updates, Enable Head, and Build

Okay you don’t have  to disable Windows updates, in fact, if you’re building a golden image for a production environment you absolutely should not do this step. However, if you’re just trying to get a 2012 R2 Vagrant box built as fast as possible (i.e. in less than an hour!) then follow Joe’s steps to disable Windows Updates, which are:

Open the directory to which you extracted packer-windows-master.

packer-windows-master

Find answer_files\2012_r2\Autounattend.xml and uncomment the section that starts “WITHOUT WINDOWS UPDATES” and comment out the section that says “WITH WINDOWS UPDATES”. It should look like the following screenshot once you’re done:

comment out windows updates

This will cut out a lot of time for the packer build!

Next, open up packer-windows-master\windows_2012_r2.json and change the virtualbox-iso headless value to false. This isn’t strictly necessary, but helps you see what’s going on and appreciate how awesome it is that you don’t have to do all this by hand!

headless false

Save, that, open a PowerShell window and CD to your packer-windows-master directory and run:

packer validate windows_2012_r2.json

then once that returns successfully

packer build windows_2012_r2.json

This will take a while to run as it does the following:

  1. Downloads the Windows 2012 R2 Trial ISO
  2. Installs Windows 2012 R2 Trial to a temporary VirtualBox VM
  3. Packages that VM up into a Vagrant box and dumps it into your packer-windows-master directory

packer build

*snore*

finished building vagrant box

This will throw errors about VMwareapplication not existing, but that’s fine, we don’t want a VMware template.

Finished? Then let’s proceed to register it.

Step 4) Register the box with Vagrant

Now, you’ve not had to do this in any of the previous guides because we always used Vagrant Cloud for our Vagrant boxes. But this time we need to tell Vagrant about the new box we’ve just created!

vagrant box add C:\Users\smartin\Downloads\packer-windows-master\windows_2012_r2_virtualbox.box --name "smartin-2012r2"

add vagrant box

The name is completely arbitrary, do change it for your purposes, and the location obviously has to be amended to reflect where your packer-windows-master directory is!

 Step 5) Install Vagrant Omnibus and Create a Vagrant Project

Vagrant Omnibus is a Vagrant Plugin that ensures Chef is installed on your Vagrant box as part of the provisioning process (much neater than my messy PowerShell script in Part 3).

vagrant plugin install vagrant-omnibus

install vagrant omnibus
With that done you can create a new folder (wherever you like) for your vagrant project. Mine’s called vagrant-base-2012R2 but it really doesn’t matter.

In that new folder, create a file named VagrantFile (no extension) and paste the following into it:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # All Vagrant configuration is done here. The most common configuration
  # options are documented and commented below. For a complete reference,
  # please see the online documentation at vagrantup.com.

  config.vm.provider "virtualbox" do |v|
	  v.gui = true
	  v.memory = 1024
      v.cpus = 1
  end
  # Every Vagrant virtual environment requires a box to build off of.
  config.vm.box = "smartin-2012r2"
  
  # Install the latest chef
  config.omnibus.chef_version = :latest
   
end

Most of this should be familiar from previous parts, but the important bit to change is the name in config.vm.box as this must match whatever name you passed as part of vagrant box add.

Once that’s created, VAGRANT UP!

vagrant up #1 vagrant up #2Et Voila! One repeatable build from ISO > Vagrant Box with Chef installed!

Getting Started with Chef on Windows Server – Part 3 – Vagrant, Windows, and Managed Chef

In the previous two parts (Intro and Chef Server & Bootstrapping) we used a plain old VirtualBox VM with Windows 2012 R2 as our Chef client, which required downloading VHDs, registering them as individual VMs and then installing Chef manually. Part 2 even required that you still had your old VM from the first session lying around in order to start where you left off!

This is not very chicken farm of us, and, I’ve since learned, really doing it the hard and old-fashioned way. So what’s the easy way?

Vagrant

Vagrant is a tool for building complete development environments. With an easy-to-use workflow and focus on automation, Vagrant lowers development environment setup time, increases development/production parity, and makes the “works on my machine” excuse a relic of the past.

About Vagrant

For anyone that has familiarity with AWS, I describe Vagrant as (loosely) CloudFormation for VirtualBox (other hypervisors are supported!).

It allows you to easily spin up an environment based on any template found on VagrantCloud.com, bootstrap it, test it, throw it away and start again.

So go and download it, I’m sure you’ve already got VirtualBox installed, but if not, download that too.

vagrantup

Exercise

Prerequisites

  1. Vagrant
  2. Virtualbox
  3. Chef Client/DK
  4. Some awareness of what Chef is
  5. Some familiarity with VirtualBox
  6. Some familiarity with scripting/cmdline

We’re going to use Vagrant to setup a Windows 2012 R2 virtual machine, install Chef client on it, and apply a basic cookbook. Once you’ve done this you’ll have a great platform for creating and testing your own cookbooks without having to manage redeploying VMs manually.

1) Setup Managed Chef

For the purposes of this trial run of Chef inside Vagrant, we’re going to use Managed Chef.

Managed Chef is Chef hosted by OpsCode, sorry Chef (the company), relieving you of the necessity to setup your own server and host it yourself. If you’re interested in setting up your own Chef Server, see Getting Started with Chef on Windows Server – Part 2 – Chef Server & Bootstrapping.

Visit manage.opscode.com and register for a free account (up to 5 nodes).

manage.opscode

Once you’ve signed in, download the starter kit and extract the contents to a new directory called “vagrant-chef-windows” somewhere in your My Documents folder.

Important: It is imperative that you create this folder in your My Documents, or some other subfolder within your user’s home directory. Vagrant, Chef, and other tools which have their roots in Linux, use the current working directory and sometimes the user’s home directory in order to figure out where to look for their configuration files. Always be aware of your CWD when executing Vagrant and Chef commands, as it’s surprisingly important!

Download Starter Kit

chef-repo

Now we’re setup, you’re ready to start with Vagrant!

2) Setup Windows Chef with Vagrant

Windows in Vagrant is pretty tried and tested now it seems. Although support for Windows hosts was only officially added in April 2014, it was a plugin for quite a while before that.

Nonetheless, the selection of “Boxes” (VM templates) on vagrantcloud.com is pretty limited right now, presumably due to licensing concerns.

vagrantcloudsearch

The most popular Windows 2012 R2 box is currently one provided by OpenTable, however it seems to have issues with password expiry, so, we’ll go with the second most popular, the one by kensykora.

If you open up the link to that box, you’ll see a handy command in a textbox, ready for you to copy out.

vagrantcloudcommand

Copy that command, open a new PowerShell window on your computer, create a new folder in your My Documents called “vagrant-chef-windows”, then execute the command:

vagrant init kensykora/windows_2012_r2_standard

vagrantinitThis creates a Vagrantfile in the directory in which you’ve executed the command.

2.1) Setup Initial Vagrant Configuration

Open the Vagrantfile in your favourite text editor, and replace the contents with the following:

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
	# Every Vagrant virtual environment requires a box to build off of.
	config.vm.box = "kensykora/windows_2012_r2_standard"
	
	# Forward ports
	config.vm.network "forwarded_port", guest: 80, host: 8080
	
	config.vm.provider "virtualbox" do |vb|
		# Don't boot with headless mode
		vb.gui = true
	end

	# Shell Provisioning
	config.vm.provision "shell" do |shell|
		shell.path = "install-chef.ps1"
	end
	
end

The configuration file is Ruby based, and does several things.

  1. Provisions the VM based on kensykora/windows_2012_r2_standard (downloading it if necessary)
  2. Forwards port 80 in the guest machine to port 8080 on your machine (the host)
  3. Pops up a Virtualbox window with the guest’s console for simplicity’s sake
  4. Executes install-chef.ps1 in the guest

Take a few moments to pair up the list above with the lines in the configuration file, once you have, you’ll wonder “where the hell is it getting install-chef.ps1 from?”. At the moment, it isn’t.

2.2) Use PowerShell Bootstrapping to Instal Chef

Create a new file in your vagrant-chef-windows directory called install-chef.ps1 and populate it with the following:

$progressPreference = 'silentlyContinue';
$chefInstaller = 'C:\vagrant\chef-windows-11.16.2-1.windows.msi';
$chefInstallerUri = "https://opscode-omnibus-packages.s3.amazonaws.com/windows/2008r2/x86_64/chef-windows-11.16.2-1.windows.msi";
 
if(!(test-path $chefInstaller)){
    Write-Host "$(Get-Date) Downloading Chef...";
    Invoke-WebRequest -Uri $chefInstallerUri -outfile $chefInstaller;
}
 
 
if(!(Test-Path "C:\chef")){
    Write-Host " $(Get-Date) Installing Chef";
    Start-Process -Wait -FilePath 'C:\\Windows\\system32\\msiexec.exe' -ArgumentList @('-i',$chefInstaller,'/quiet','/log','C:\\tmp\\chef-client-install.log')
    Write-Host " $(Get-Date) Installation Complete"
}else{
    Write-Host " $(Get-Date) Chef is already installed!";
}

Ideally, we wouldn’t need to do this as Chef would already be installed in the Box we got from VagrantCloud.com, however, at the time of writing there are no Windows 2012 R2 boxes with Chef pre-installed.
Your folder should now look like this:
folder with install chef.ps1

2.3) Power On – Vagrant Up

Now, ensure you’re in your vagrant-chef-windows folder in the PowerShell console, then execute:

vagrant up

vagrant up #1

It will scurry off, download the kensykora 2012 R2 box (not shown as I already had it), power up a new VM and execute your ps1. Once complete, you should have a VirtualBox console pop up and allow you to sign in (right ctrl + del = Ctrl + Alt + Delete).

Username: Vagrant
Password: vagrant

2012 vagrant VMIf you login, you’ll see C:\chef exists, and if you browse into C:\vagrant, you’ll see that the entirety of your vagrant-chef-windows folder is available within the VM!

see c vagrant

This is important because almost all file paths you’ll set in your Vagrantfile configuration will be relative to this directory.

2.4) Setup Vagrant Chef Provisioning Configuration

Now it’s time to actually use Chef. But we’re not going to just open up a PS console inside the VM and run chef-client. Oh no, we’re going to use Vagrant’s chef-client provisioning functionality!

That means that every time we deploy a new VM, our PS1 file will install Chef, then Vagrant will run chef-client for us, with the configuration we’ve defined in the Vagrantfile.

Add the following lines to the end ofyour Vagrantfile (but before the final “end”).

        # Chef Provisioning
	config.vm.provision "chef_client" do |chef|
	 chef.chef_server_url = "https://api.opscode.com/organizations/orgname"
	 chef.node_name = "node20141019"
	 chef.validation_client_name = "orgname-validator"
	 chef.validation_key_path = "chef-repo\\.chef\\orgname-validator.pem"
	 chef.add_recipe "learn_chef_iis"
	end

You will, of course, need to replace orgname with your organisation name on the highlighted lines, and amend the node_name if you like.

Your Vagrantfile should now look like this:

Final Vagrantfile

This code uses the Chef Client we’ve already installed and the orgname-validator.pem which came with our Starter Kit in order to add this guest as a node to our managed Chef environment.

2.5) Upload the Cookbook

But wait, we haven’t got the cookbook learn_chef_iis (a simple Windows/IIS example used by the learnchef.com/windows walkthroughs)! CD into your chef-repo directory and execute:

knife cookbook site download learn_chef_iis

download learn_chef_iis

Now extract the resulting tar.gz into your cookbooks subdir.

learn_chef_iis extracted

And finally, upload it to your managed Chef environment.

 knife upload learn_chef_iis

knife upload learn_chef_iis

2.6) Vagrant Provision

Excellent! The cookbook’s ready to go. Now CD up a level into your vagrant directory and run:

vagrant provision

vagrant provision

Vagrant has now kicked off a chef-client run with the learn_chef_iis cookbook as its runlist. Once it’s finished (and in combination with the forwarded port we setup earlier) you should now be able to open your favourite browser on your host machine and go to http://localhost:8080 and see…

localhost8080Voila!

You’re seeing the results of the IIS webserver that Chef configured in the VirtualBox that Vagrant deployed and bootstrapped for you! *Phew*

2.7) Redeploy from Scratch

Now for the moment of truth. Delete the node from the managed Chef environment, destroy the VM and redeploy a fresh one based on the configuration we’ve provided!

delete node

vagrant destroy -f
vagrant up

vagrant destroy

Wait a little while for Vagrant and Chef to finish doing their thing and you should be able to go back to localhost:8080 again and see exactly the same thing on a fresh VM!

You can use this environment to test the custom cookbooks we created in Part 1, but I’ll leave that to you to figure out in combination with what we’ve done today!

Further Reading

Chef Manage

Vagrant Chef Client Provisioner

Vagrant Getting Started

Vagrant Cloud