governance

Joe Carlyle

How to – Azure Policy via ARM Template

If you’ve worked with Azure for a while, you would know that one of the most efficient methods of deployment is ARM templates and one of the most powerful services is Azure Policy. What you might not know, is that you can combine the two for efficient, iterative and defined deployments.

A great point I saw recently on Twitter was that a lot of technical posts highlight features and how to use them but rarely go into why you should use them. Conscious of that, here are a couple of points on why I think you should make use of Policy via Template (PvT):

  • Quick deployment time – hilariously quick.
  • Repeatable defined structures – the exact policy definition, applied to the exact scope, with no possibility of user error.
  • Confident flexibility – Templates are idempotent; need to update the definition? Update the template, deploy the update, job done.

So if the “why” makes sense to you, let’s move onto the “how”. If it doesn’t, let me know! I’d love to hear your horror stories/use cases…

Templates can be deployed in several ways, for the sake of simplicity, I’m going to use two tools here. Visual Studio Code and Powershell. Currently you can only deploy subscription scope resources via Powershell or CLI.

There are some other differences to note. The schema for the template must be:

https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#

When deploying the template, it must be deployed to a location and given a name (the name of the template will be used if none is specified), that combination is then immutable for that location. So if you need to change location, you need to use a new name etc.

Now, let’s create our template. For this post, I am going to use an existing Template Definition and scope it to my Subscription. While you can pass the Template Parameters via Powershell Variable, for this post I am going to define them as a Template Variable. This is tricky piece of logic as they must be defined as a nested, object array. I also define the policyID via Variable. For existing definitions, you can get this via the Portal, or Powershell command

Get-AzPolicyDefinition | select PolicyDefinitionId -ExpandProperty properties | where displayName -Match "allowed locations"
{
    "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {},
    "variables": {
      "policyName":"Allowed Locations",
      "policyDefinitionID":"/providers/Microsoft.Authorization/policyDefinitions/e56962a6-4747-49cd-b67b-bf8b01975c4c",
      "policyParameters":{
        "listOfAllowedLocations":{
          "value":["NorthEurope","WestEurope"]}}
    },
    "resources": [
        {
            "type": "Microsoft.Authorization/policyAssignments",
            "name": "[variables('policyName')]",
            "apiVersion": "2018-03-01",
            "properties": {
                "scope": "[subscription().id]",
                "policyDefinitionId": "[variables('policyDefinitionID')]",
                "parameters": "[variables('policyParameters')]"
            }
        }
    ]
}

Now are Policy deployment is defined and ready for use, we deploy using Powershell:

New-AzDeployment -Name "pvtDeployment" -Location northeurope -TemplateFile 'C:\Users\username\Documents\WDA\PvT.json'

You should receive a succeeded message within your shell and you can verify via the Portal. As it was a subscription level deployment, head to your Subscription blade and check the Deployments tab. You should see the Template listed as the same name as you ran for the deployment.

You can then confirm your settings via heading to Azure Policy and the Assignments blade. You will see your Policy Definition assigned at the scope you set, using the Parameters you set.

Just to go back to an early point on why you’d use this option. Look at the duration of the deployment in the above screengrab – 1 second. You simply cannot beat that!

This can obviously be used for much more complex deployments, for example, defining your own policy inline and deploying via template. The possibilities are endless with one current exception; Subscription is highest scope you can currently use, hopefully Management Groups are on the roadmap and therefore the scaling capability is excellent.

As always, if there are any questions or suggestions, please get in touch!

Joe Carlyle

How to – Choose an Azure Naming Convention

When working with Azure, as with many cloud based services, one of the difficulties you may have to deal with is naming standards. Common sense would lead you to applying a single naming convention to all resources. However, in Azure, as the name of the resource can actually be used for many different things, one single convention is quite difficult to achieve. Especially if your convention requires any real degree of complexity.

The most simple reason for having a good naming convention? You cannot rename resources in Azure.

In one of my earlier posts, I discussed the benefits of using the Azure Architecture Center, we visit this again for naming conventions. The general recommendations are as follows:

So let’s look at those in some more detail.

Short and Simple

This somewhat explains itself. However there are technical justifications too. For example, a Windows VM name has a maximum character limit of 15. Keeping it simple allows you to re-use the same logic regardless of resource type, location or service.

Affix some affixes

Where possible, make use of a prefix or a suffix to add clarity to your resource names. the most common are based on environment (prod, dev, test) or instance (A, B, C or 01, 02, 03).

While trying to do the above, make sure to reference back to the matrices of support.

Policy

Depending on whether your environment is brand new, or you’re trying to implement some control on an existing environment; applying your naming convention via Policy can speed things up. You can choose all of the usual options when it comes to policy, my preferred method is to create several policies scoped to resource types that deny deployment if naming convention doesn’t match. This avoids conflicts with any auto-deployed resources. Then a catch all policy to audit everything for compliance. An example policy for VMs is below:

{
     "properties": {
         "displayName": "VM naming pattern",
         "description": "Require naming patterns for VMs.",
         "mode": "All",
         "policyRule": {
             "if": {
                 "allOf": [
                     {
                         "field": "type",
                         "match": "Microsoft.Compute/virtualMachines"
                     },
                     {
                         "not": {
                             "field": "name",
                             "match": "az-????-##"
                         }
                     }
                 ]
             },
             "then": {
                 "effect": "deny"
             }
         }
     }
 }

This checks all of the fields listed, so for Virtual Machines, where Name is Not a Match for “az-????-##” then Deny. The match pattern is defined using standard Azure Policy conditions.

Tagging

Tags can form a one to many relationship for resources. Tagging all resources that are part of a project, or have a commonality adds additional metadata to resources without adding increased complexity to your naming convention. Tagging should be viewed as a complimentary option to a good naming convention rather than an alternative. They can also be applied via Policy, saving you time and effort!

As always if there are any questions, or suggestions on how you do naming in Azure, get in touch and I’ll add them to this post!

Joe Carlyle

Azure Policy – Where to Start?

One of the positives of Azure is that it can offer you so many possibilities when it comes to deployment options. However, if you don’t implement the correct governance, this can very quickly become a negative. Historically, Cloud has had difficulties when it comes to sprawl; Azure Policy is a service that will help prevent that.

Azure Policy isn’t only a tool for prevention either. With the right policies, you can audit and enhance your environment in terms of efficiency, security and compliance. This gives you greater insight into your Azure deployment and confidence in your requirements.

Governance in Azure is addressed in many ways; a good place to start for overall strategy is the Azure Architecture Center. It has specific sections on Governance as well as overall design guidelines.

So how does Azure Policy work? At its core, it is an assessment service. You create policies with specific rules and scopes. Once the policies are active, they audit all resources in the included scope for compliance. Policies can range in complexity; you can use the default templates or create a custom one to meet your needs.

There are two core areas when dealing with Azure Policy:

Policy Definition

Every policy must have a definition. The definition contains all the details of the conditions under which it’s enforced. It also has the defined effect that occurs if the conditions are met. Definitions are created using JSON and the full structure is defined here. You will need familiarity with this if you are going to write your own custom policies.

Allowed Locations Policy Example:

{
    "properties": {
        "mode": "all",
        "parameters": {
            "allowedLocations": {
                "type": "array",
                "metadata": {
                    "description": "The list of locations that can be specified when deploying resources",
                    "strongType": "location",
                    "displayName": "Allowed locations"
                },
                "defaultValue": [ "westus2" ]
            }
        },
        "displayName": "Allowed locations",
        "description": "This policy enables you to restrict the locations your organization can specify when deploying resources.",
        "policyRule": {
            "if": {
                "not": {
                    "field": "location",
                    "in": "[parameters('allowedLocations')]"
                }
            },
            "then": {
                "effect": "deny"
            }
        }
    }
}
Policy Assignment

A policy assignment is a policy definition that has been assigned to take place within a specific scope. Assignments range from Management Groups to a single Resource Group. The scope of the assignment refers to all the Resource Groups, Subscriptions, or Management Groups that the Definition is assigned to. Inheritance is enabled for all assignments. Therefore, a policy applied to a resource group is also applied to all resources in that resource group. However, you can include exclusions as a sub-scope of the assignment. For example, a Definition is assigned to a Subscription; all Resource Groups inherit the Definition but you need a single Resource Group excluded. Rather than redo the Assignment for each Resource Group, you can simply exclude it from the Subscription assignment.

Your First Policy

Now that you understand what Azure Policy is, let’s get started with our first policy. For this example, I’m going to prevent Public IP addresses being deployed within a Subscription. This is something I commonly add to IaaS projects that are connected to a local LAN.

Once you’ve logged in to the Azure Portal, make your way to the Azure Policy service, I normally use the search bar as below as it’s quick!

Once you’re on the Overview blade, a handy option for your first time is to click the Getting Started option. This details the steps to take and we’re going to start by browsing default Definitions so click that View Definitions option as below:

This will bring you to the Definitions blade. You will see a lot of built-in policies. To simplify things, click the search bar and enter “not allowed”, this will bring up the Definition we will use, then go ahead and click on the Policy name “Not allowed resource types” as below:

You’re now in the Definition page, where you can see the exact structure in JSON format. We’re going to jump straight to Assignment from here by clicking “Assign” as below:

This will bring you the assignment blade. Our first step is to set a scope. I’m going to go ahead and choose my Subscription and a Resource Group, then click “Select” as below:

We’re going to leave most of the settings as they are, but you can where you can set Exclusions below. We’re going to click the drop down arrow as highlighted:

This opens a huge list of resource providers and types. Thankfully, there is a search bar, so type in “public” to narrow the list and tick the checkbox for Microsoft.Network/publicIPAddresses, then click away from the list as below:

We’re almost there! You can see that publicIPAddresses are now defined as a parameter. So click the blue “Assign” button as below:

Now your policy is assigned, we need to give it a couple of minutes to propagate. Now, when I try to create a Public IP resource in my scoped Resource Group (I’ve used POSH in Cloudshell, but deployment method doesn’t matter) I’m told I cannot as it’s disallowed by policy:

You’ve now successfully applied your first Azure Policy! As you can see, even though this is a single Definition it is still very powerful. Your options to layer Definitions and apply custom ones allow for full control of your environment with very little effort. Azure Policy should be high on your list of priorities for your Azure deployments.

As always, if there are any questions, please get in touch!

Joe Carlyle

How to – Reduce your Azure IaaS Costs

A regular starting point for most people when first using Azure, or any public cloud, is a virtual machine. Depending on your environment, VMs can be one of the most expensive resources. It’s no surprise that this can be a strong negative when considering a move to cloud.

Before anything is deployed, it’s important that you are aware of the tools that Microsoft make available to help you estimate your costs in advance. This can help both understand and avoid unwanted surprises with your bill.

First up is the Azure Pricing Calculator, with a bit of work, you can achieve an acceptably accurate cost estimate for an environment. I normally choose the default settings when it comes to pricing options (such as PAYG) as it gives me the most expensive and therefore safest estimate for a quick quote. If you have access to other consumption offers, ensure you are signed in so you can access their rates.

For this post I’m going to use a single VM estimate to display cost and changes. As it’s a single VM I have chosen a beast – M128m

Once you have your worst case estimate, it’s time to start making some adjustments to get that price down as low as possible. To do this, I recommend the following three options.

  1. Reserved Instances
  2. Automation
  3. Hybrid Benefit

First up, and most straight forward – Reserved Instances. They are a billing object that allows you to save money over a fixed period of time by paying for the usage up-front. From the screen grab you can see the savings can be approx. 64% for a three-year reserved instance. I have an old post that is still valid on RIs over here.

Again, you will pay the entire price up front, but look at the difference it makes to the monthly rate for our beast:

Next, modifying your usage hours using Automation. Now, this doesn’t have to be using Azure Automation and its Start/Stop solution as there are alternative like over on Azure MVP, Gregor Suttie’s blog. Whatever method you choose, update your usage hours in the cost calculator to see your savings, for this post I’m going to first remove weekends (average 8 days a month = 192 hours) and cut the remaining workdays in half (538/2). So instead of 730 hours, we get 269 hours and the appropriate reduction in price to our beast:

One thing to note at this point, if you’re using Reserved Instances, there is no point in using Automation to save on costs. RIs cover the full usage for the period.

Finally, the simplest to implement but arguably most complex option, Azure Hybrid Benefit. This is a licensing option that allows you to reuse your on-prem licenses in Azure. This is an option that can only be used in Azure and therefore a unique cost saving method. Applying it is simply a tickbox within your VM blade. Microsoft have a calculator to help you work out the licensing side of things, I’d recommend leaning on your LSP for this part as it can be a bit complicated and you need to make sure you’re compliant. You can see the savings below for our beast:

You’re probably already thinking it, can I layer these together and save even more? Absolutely.

Check out the reduction to the price of the beast if we apply AHB and a three year RI:

So what are you waiting for, head over to your Azure tenant and start saving some money on those VMs ASAP. As always, if there are any questions, get in touch!

Joe Carlyle

Azure Sentinel – Where to start?

First announced back in late February, Azure Sentinel is the first cloud native SIEM service from a major provider. SIEM (security information and event management) is a primary component in any security service. Sentinel aims to leverage cloud specific benefits like elastic scale and AI to allow customers detect and respond to security incidents as quickly and efficiently as possible.

The workflow of Azure Sentinel can be broken into four steps:

Azure Sentinel core capabilities

1. Collect

Sentinel allows you to collect data at scale from multiple users, devices, applications and infrastructure, hosted in Azure, on-premises, and even in multiple clouds. This means you can aggregate all security data using industry standard log formatting. With built-in integration, you can enable collection for features such as Office 365 or Azure AD within seconds.

2. Detect

Having all of your data collected to Sentinel allows for more simple analysis and detection at scale than was previously possible. This more efficient triage, and the capability to leverage Microsoft Machine Learning allows you to be more productive, minimise false positives and react to those high accuracy alerts as early as possible.

3. Investigate

Sentinel allows you to visualise and resolve alerts using the same dashboards. Proactively hunting for incidents can be automatic or scripted into a set of queries. Microsoft have provided some to get you started too based on their analysis and response teams.

4. Respond

Continuing the efficiency seen in previous steps, Sentinel allows you to orchestrate and automate responses to incidents. Allowing you to automatically handle repeat and/or known incidents.

So now that you know what it is, the next step is to put it in action and see if it can be of use to you and your client/business. Currently still in preview, Sentinel is free to use, which is always good and allows you to assess the service without any significant financial impact. Bear in mind, you will pay for the Log Analytics workspace which stores the data!

First, you’ll need to enable Sentinel and a workspace, this can be done via the portal and a walkthrough is here. Then, you need to connect some services to start streaming data to Sentinel. As you can see below, there are multiple options and you can choose which logs/data is sent to Sentinel too.

Once your data connector is active, you can make use of the built in dashboards to visualise it. Below is a subsection of the Azure AD sign-in log dashboard, which is available immediately via Sentinel. You can also create your own custom dashboards, there is a guide with samples here.

Now that is being collected and you have visuals, your next step is analysis. The first thing you will need to do is to create Detection Rules. These are essentially Log Analytics queries with alerting parameters wrapped around them. Microsoft offer sample queries on Github which are updated regularly. Alternatively, you can simply write your own to meet your needs.

The results of your Detection Rules are then fed into the Cases section of Sentinel. Here you can triage, investigate and remediate incidents. The cases are created dynamically from the parameters you set for Detection Rules such as severity and entity mapping. As such, be prepared to have to tweak those thresholds and alert patterns a bit. I have Sentinel running within several customer tenants, and am still not 100% happy with my detection rules yet. Always remember to update the status of your case too, in progress, resolved etc.

Finally, you should set up some Playbooks to respond to your alerts. A Playbook is simply a set of procedures that you can run from Azure Sentinel. They help automate and orchestrate your responses to alerts, and you can run them manually or ideally set them up to run automatically in response to certain alerts. They are based on Logic Apps, which means all of the same actions are available via Sentinel. One quick note, there is a charge for Logic Apps and therefore Playbooks, so ensure you understand your costs first.

When creating a Playbook, regardless of it is going to be run automatically by Sentinel or manually by you, you should first define your scenario. My preferred approach here is to come up with “If-This-Then-That” loops and apply them as needed. This is another section that will take some tweaking over time. In my experience, I only run Playbooks manually initially, then start to add automated triggering once I’m happy with the alert and response. Docs have a nice sample alert-response playbook with messaging and actions which is a great place to start.

Another function which I haven’t covered here is Hunting. I haven’t spent enough time with this feature yet to give a detailed opinion but you can read more on it over on Docs.

So, if you haven’t given Sentinel a try yet, I’d recommend you review the quickstarts and deploy in your tenant for one or two of the data sources like Azure AD. While it’s in Preview, it is a great chance to assess it relevant to your tenant and hopefully gain some greater insight and response capability too.

As always, if there any questions or if you have any problems with your Sentinel, get in touch!