Building Hyper-V lab environments based on PowerShell DSC
In this article we will look at how the PowerShell Desired State Configuration (DSC) feature in Windows PowerShell can be leveraged for building lab environments. If you are unfamiliar with DSC, have a look at the DSC overview page on MSDN. As stated there: DSC provides a set of Windows PowerShell language extensions, new Windows PowerShell cmdlets, and resources that you can use to declaratively specify how you want your software environment to be configured.
Background
PowerShell DSC was first released back in 2014 integrated in PowerShell 4.0 as part of the Windows Server 2012 R2 release. Since then, it has first and foremost been leveraged in Windows-based server deployments. However, now that PowerShell is open sourced and available on Linux, it means that DSC is also available on that platform. It can also be leveraged on Nano Server (Microsoft’s container optimized operating system) and Windows Containers, thus it is clearly an important part of Microsoft`s strategy going forward. There is also a number of creative community projects leveraging DSC, one of them which we are going to look at in this article. Imagine if we could leverage the same configuration data structure as DSC for other purposes, such as building a lab environment. At the end, the configuration structure is “just” a PowerShell hash table which can optionally be stored in a PowerShell data file (psd1). This is exactly what the community based Lability module does, in addition to leveraging DSC for configuring the virtual machines being created. The module is specifically written for the Hyper-V virtualization platform, but parts of it can be leveraged in other environments as-is. The module leverages the concept of injecting DSC configurations into the virtual machines hard disk file, typically a VHD or VHDX file. This concept is explained and demonstrated in detail in this Channel 9 video.
Getting started with Lability
The Lability module is available from the PowerShell Gallery, meaning you can use PowerShellGet to install it: Install-Module -Name Lability As we can see, there is a number of useful functions available:
Get-LabHostDefault returns the default settings for the Lability environment:
Using Set-LabHostDefault we can customize the settings, for example pointing paths to an external hard drive or another custom location. Similarly, we have default settings for virtual machines which can be retrieved/customized using Get-VMLabDefault/Set-VMLabDefault:
Whether you choose to customize those settings or not, the next step is to run Start-LabHostConfiguration which will create the directories specified in the configuration if they do not exist, as well as verify if the Hyper-V role is installed (and install it if not):
In its simplest form, that is all that is necessary to configure before you can start creating your first lab environment based on Lability. First, create DSC configuration files (.mof and .meta.mof) for the nodes in your new lab environment. This is accomplished in exactly the same way as you do when configuring the DSC agent (Local Configuration Manager) and creating DSC configurations, and is not something that is created by Lability, it just consumes the DSC configuration files. If we are going to create a simple lab with a domain controller (DC1) and an application server (APP1), we would need to make sure the DSC configuration files exists in Lability`s default configuration directory specified in (Get-LabHostDefault).ConfigurationPath:
Here is a complete end-to-end example for getting started with Lability which should work on any Windows 10 computer capable of running Hyper-V:
At this point, the lab host is configured and we are ready to create out first lab environment (you can create as many as you want). The next step is to create configuration data specifying the details Lability needs to know about our first lab environment, such as virtual machines and virtual networks. Here is an example for setting up a basic lab environment:
This example assumes that the before mentioned DSC configuration files (MOF-files for APP1 and DC1) exists in (Get-LabHostDefault).ConfigurationPath.
If you wanted to use a custom directory for the DSC configuration files, you would need to add the –Path parameter to Start-LabConfiguration. When the above example is executed, Lability will start to download the necessary media files (Windows Server 2016 Evaluation ISO in this example), create virtual machines, virtual switches and so on as specified in the configuration data:
Side note: If you for some reason do not want to configure the virtual machines using DSC, you can specify Start-LabConfiguration -SkipMofCheck. As you can see in the example above, there is a lot of Lability-specific settings (note that the Lability_ prefix is optional). In order to know what settings Lability uses, the best way is to look into the help system. Lability has a number of very comprehensive about_ files, the main one being: Get-Help about_Lability
The above is just a small part of the complete help topic, you can read the complete file here. An excerpt from the help file:
There is also a number of other helpful about_ files which provides great built-in documentation for the module:
- about_ConfigurationData
- about_CustomResources
- about_Bootstrap
- about_Media
- about_Networking
By default, Lability downloads evaluation media from Microsoft, but you can also use your own media. More details and instructions on how to register your own sysprepped VHD(X)-files can be found in the about_Media help topic. This is all we will cover on the basics in this article, as there already is a couple of other great articles published which goes into more details:
- Building A Lab using Hyper-V and Lability – The End to End Example by Ryan Yates
- Lability - The ultimate Hyper-V lab tool (Powershell DSC) by James Booth
As a practical example, I recently had the need to provision a lab environment for testing Storage Spaces Direct in Windows Server 2016. A complete lab environment with 1 domain controller and 3 storage nodes was provisioned in about 15 minutes on my local laptop running Windows 10:
An Active Directory domain was installed, and all features necessary for this particular lab was installed on the storage nodes, saving a lot of time compared to a manual effort. The whole configuration for setting up the Storage Spaces Direct lab end to end using Lability is available here on my GitHub account.
Inner workings
Under the hood, Lability actually uses the xHyper-V DSC module to provision virtual machines and create virtual networks:
This way, it also gets a lot of functionality “for free”. For example, currently the xHyper-V module does not support adding data disks to virtual machines. When this functionality is added at some point in the future, it means that Lability will also get that new feature. Without doing any changes to the Lability module. One of the main goals behind the module is not to reinvent the wheel, and rather contribute to create general DSC resources which can be leveraged by Lability as well as benefit everyone else. You might be concerned that Lability will overwrite any DSC configuration you already have on your lab host computer. At least I was, since I have a large configuration which is configuring my IT Pro computer with things such as Visual Studio Code extensions, software via Chocolatey and so on. In early versions, Lability used to apply a permanent DSC configuration to the local computer. However, that is not an issue anymore as the author realized this was a bad idea. Now, Lability do not apply any permanent DSC configuration to the lab host computer, it rather calls DSC resources directly. Invoke-DscResource was introduced in Windows Management Framework 5 for this exact purpose, allowing third party management products such as Ansible, Chef and Puppet leverage DSC resources without interacting with the Local Configuration Manager or call any other native DSC cmdlets.
Lability actually calls this functionality via CIM/WMI on order to also support Windows Management Framework 4, that is why there is an internal InvokeDscResource function in the Lability module:
Using Lability in Azure virtual machines
Microsoft announced nested virtualization support becoming available on Azure with the Dv3 and Ev3 virtual machines series on July 13, 2017, which was announced the same day.
This makes it possible to run Hyper-V on virtual machines in Azure. Most people would think of running Windows Server with Hyper-V in Azure, which makes perfect sense for scenarios such as Windows Containers and setting up Hyper-V lab environments without any physical hardware.
However, it is also possible to run Windows 10 in Azure - it is not supported but it works without any issues I am aware of. In fact, if you have an MSDN subscription, you already have access to a Windows 10 Enterprise evaluation image from the Azure Markedplace.
I have a virtual machine running Windows 10 in Azure, using the Standard D8s v3 (8 cores, 32 GB memory) series.
I use the Auto-shutdown feature for making sure I do not forget to shutdown the VM, since it can be quite expensive running the VM series that supports nested virtualization for a longer period of time in a personal lab subscription:
That is the nice thing about using a cloud service for this scenario, you can shut down the environment when it is not in use. If I suddenly need to test something that requires a large amount of memory, I can scale the VM up to a Standard_E64_v3 getting 432 GB RAM for the few hours I need it and then scale it back down to a more price friendly series. The same goes for disk space, I can add both HDD and SSD capacity on demand.
Thus, this makes it a very good alternative or addition to running lab environments on local laptops or other physical hardware.
One tip regarding disk performance if you are going to use Lability on Windows 10 running in Azure: Use Storage Spaces to aggregate disks in order to get better IOPS. For example, the initial data disk I configured for my Azure Windows 10 VM running Lability is an SSD disk providing 7 500 IOPS. That means all virtual machines running on that disk will share that capacity. If I setup 4 disks like this and add them to a Storage Space, I get 4 x the performance (30 000 IOPS). Another thing to remember if you do this is to turn off host caching for disks you will use for Storage Spaces. That and other things to keep in mind is available in the related links in the Resources section at the bottom of this article.
Summary
Personally I work a lot with Azure Automation DSC, and I`m seeing an opportunity to bootstrap virtual machines only with the DSC meta configuration (.meta.mof) generated by Azure Automation. That way, the configurations and all others settings (credentials, global variables, etc) can be dynamically retrieved from Azure Automation. I find Lability to be a very useful resource to have in your tool belt, as there is so many scenarios it enables. Some that comes to mind is building standardized lab environments for classrooms, as well as spinning up an isolated lab for testing a specific part of a production environment. Virtual Engine (a hosted online lab provider) is actually using Lability (in conjunction with Ravello) for creating all of their demo and lab environments. Microsoft Test Lab Guides (TLGs) is another scenario which easily could be implemented in Lability by adding the specifications from a TLG into a DSC configuration. Actually, there is a discussion thread in the module repository regarding lab configuration sharing, where one of the suggestions is to add a PowerShellGet provider for this purpose. If this becomes a reality and the community starts to share lab configurations for TLGs and other scenarios, it will become even easier to create lab environments for various scenarios. If you are not using Hyper-V and still would like to benefit from using Lability configuration data, there is a separate LabilityBootstrap module available: The LabilityBootstrap module enables manual deployment of (virtual) machines using Lability configuration data. This makes it possible to use the same lab/testing configuration documents outside of the standard Lability local Hyper-V host deployment model; for example using VMware Workstation or Oracle VirtualBox.
In a conversation with Iain Brighton, the author of Lability, he states that There is lots of potential with Direct Access and integration testing and When combined with nested Hyper-V, we have enormous potential. If you have other ideas for enhancements, you can engage with Iain and other contributors in the Lability repository on GitHub. Note that there is a separate dev branch, where unreleased functionality can be inspected and tested.
At the end of the article, we looked at how Microsoft Azure can be used to host virtual machines running Windows 10 in order to leverage Lability in a scalable cloud platform.