How to – Use Azure Firewall IP Groups

If you’re familiar with Azure Firewall you would know that the introduction of an IP Group resource is most welcome. IP Groups are still in preview at the moment, so as usual be cautious on production environments as there is no SLA. However, it’s always nice to try out a service to see if it can work for you, or make your life easier.

IP Groups themselves are a relatively simple resource. They can contain a single IP address, multiple IP addresses, or one or more IP address ranges. They can then be used for DNAT, Network, or Application rules in Azure Firewall.

They currently have some interesting limitations that are a little bit confusing at first. From Docs:

For 50 IP Groups or less, you can have a maximum of 5000 individual IP addresses each per firewall instance. For 51 to 100 IP Groups, you can have 500 individual IP address each per firewall instance.

What this means is that while your rules should already be scoped accurately, you may need to use a couple of extra IP groups if you’re working with large address ranges. A simple example is a /16 will simply not work in an IP Group, /20 is basically your limit per IP Group.

I actually tried this on my own sub and it appears to actually work for now. Expect that to change as preview progresses.

If you’ve worked with Azure Firewall, I’m sure you’ve already thought of several places these rules can really help. For me, it was within Network Rule Collections.

However, as the service is in preview, there are a few aspects to be ironed out. Unfortunately, one of those is the ability to add an IP Group as a destination within a network rule when using the Portal. See below

UPDATE: As expected, this is now resolved! However, read on to see how to do this at scale.

At this point, I am going to flag extreme caution if your Azure Firewall is in production and you are trying this. It is very easy to overwrite all of your collections, take your time and export them before making any changes!

I’m a Windows guy, so I’m going to explain how to do this with Powershell, but it also works for CLI. Similarly, I’m showing a Network Rule, same process works for Application Rules.

First up, you need to all of the details for your Azure Firewall as we will work with it’s config as a variable and finally update it.

#Get the AFW I want to edit
$afw = Get-AzFirewall -Name wda-afw-test -ResourceGroupName rg-wda-afw

#Save current Network Rule Collection to a variable for reference
$oldcol = $afw.NetworkRuleCollections

#Get the IP Group I want to use
$ipg = Get-AzIpGroup -Name wda-group1 -ResourceGroupName rg-wda-afw

#Create my new network rule
$newrule = New-AzFirewallNetworkRule -Name "rule2" -Protocol TCP -SourceAddress * -DestinationIpGroup $ipg.Id -DestinationPort 445

Now this is where it can get a bit tricky. Collections are stored as nested arrays. My AFW has two collections, I want to add my new rule to the second one which means I need to reference index 1. See the collections below, the one we’ll be editing is “collection2” which currently only has “rule2”

#view all collections
$afw.NetworkRuleCollections

#view the specific collection rules using place in array
$afw.NetworkRuleCollections[1].Rules | ft

#add my new rule to my collection
$afw.NetworkRuleCollections[1].AddRule($newrule)

#if you like, check it has updated as desired
$afw.NetworkRuleCollections[1].Rules | ft

#If as expected, update AFW
Set-AzFirewall -AzureFirewall $afw

The last command can take a minute or two to complete. Once it has, you can see the rule is now added to my collection2. The Portal will display it correctly, but you cannot edit correctly with the glitch.

And that’s it! You’ve successfully added an IP Group as a destination to your Azure Firewall. Again, please be careful, the above is only a guide and I cannot be responsible for your Azure Firewall ūüôā

How to – Troubleshoot Azure Firewall

Networking in Azure is one of my favourite topics. As my work has me focus primarily on Azure Virtual Datacenter builds, networking is key. When Microsoft introduced Azure Firewall (AFW), I was excited to see a platform based option as a hopeful alternative to the traditional NVAs. Feature wise in preview, AFW lacked some key functionality. Once it went GA a lot of the asks from the community were rectified however there are still some outstanding issues like cost, but all of that is for a blog post for another day!

AFW is used in a lot of environments. It’s simple to deploy, resilient and relatively straight forward to configure. However, once active in the environment, I noticed that finding out what is going wrong can be tricky. Hopefully this post helps with that and can save you some valuable time!

I don’t know about you, but the first thing I always check when trying to solve a problem is the most simple solution. For AFW that check is to make sure it’s not stopped. Yes that’s right you can “stop” AFW. It’s quick and easy to do via shell:

# Stop an existing firewall

$azfw = Get-AzFirewall -Name "FW Name" -ResourceGroupName "RG Name"
$azfw.Deallocate()
Set-AzFirewall -AzureFirewall $azfw

But how do you check if it has been stopped? Very simply, via the Azure Portal. On the overview blade for AFW it shows provisioning state. If this is anything but “Succeeded” you most likely have an issue.

So, how do you enable it again should you find your AFW deallocated? Again, quite simply via shell, however, it must be allocated to the original resource group and subscription. Also, while it deallocates almost instantly, it takes roughly the same amount of time to allocate AFW as it does to create one from scratch.

# Start a firewall

$azfw = Get-AzFirewall -Name "FW Name" -ResourceGroupName "RG Name"
$vnet = Get-AzVirtualNetwork -ResourceGroupName "RG Name" -Name "VNet Name"
$publicip = Get-AzPublicIpAddress -Name "Public IP Name" -ResourceGroupName " RG Name"
$azfw.Allocate($vnet,$publicip)
Set-AzFirewall -AzureFirewall $azfw

So, your AFW is active and receiving traffic via whatever method (NAT, Custom Route Tables etc.) and you have created rules to allow traffic as required. Don’t forget all traffic is blocked by default until you create rules.

By default the only detail you can get from AFW are metrics. These can show a small range of traffic with no granular detail, such as rules hit count.


To get detailed logs, like other Azure services, you need to enable them. I recommend doing this as part of your creation process. In terms of what to do, you have to add a diagnostic setting. There are two logs available, and I recommend choosing both.

  • AzureFirewallApplicationRule
  • AzureFirewallNetworkRule

In terms of where to send the logs, I like the integration offered by Azure Monitor Logs and there is a filtered shortcut right within the AFW blade too.

Once enabled, you should start seeing logs flowing into Azure Monitor Logs within five to ten minutes. One aspect that can be viewed as a slight negative is that logs are sent in JSON. As a result most of the interesting data you want is part of an object array:

{
  "category": "AzureFirewallNetworkRule",
  "time": "2018-06-14T23:44:11.0590400Z",
  "resourceId": "/SUBSCRIPTIONS/{subscriptionId}/RESOURCEGROUPS/{resourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/{resourceName}",
  "operationName": "AzureFirewallNetworkRuleLog",
  "properties": {
      "msg": "TCP request from 111.35.136.173:12518 to 13.78.143.217:2323. Action: Deny"
  }
}

So, when running your queries, you need to parse that data. For those who have strong experience in Kusto, this will be no problem. For those who don’t, Microsoft thankfully provide guidance on how to parse both logs including explanatory comments

For ApplicationRule log

AzureDiagnostics
| where Category == "AzureFirewallApplicationRule"
//using :int makes it easier to pars but later we'll convert to string as we're not interested to do mathematical functions on these fields
//this first parse statement is valid for all entries as they all start with this format
| parse msg_s with Protocol " request from " SourceIP ":" SourcePortInt:int " " TempDetails
//case 1: for records that end with: "was denied. Reason: SNI TLS extension was missing."
| parse TempDetails with "was " Action1 ". Reason: " Rule1
//case 2: for records that end with
//"to ocsp.digicert.com:80. Action: Allow. Rule Collection: RC1. Rule: Rule1"
//"to v10.vortex-win.data.microsoft.com:443. Action: Deny. No rule matched. Proceeding with default action"
| parse TempDetails with "to " FQDN ":" TargetPortInt:int ". Action: " Action2 "." *
//case 2a: for records that end with:
//"to ocsp.digicert.com:80. Action: Allow. Rule Collection: RC1. Rule: Rule1"
| parse TempDetails with * ". Rule Collection: " RuleCollection2a ". Rule:" Rule2a
//case 2b: for records that end with:
//for records that end with: "to v10.vortex-win.data.microsoft.com:443. Action: Deny. No rule matched. Proceeding with default action"
| parse TempDetails with * "Deny." RuleCollection2b ". Proceeding with" Rule2b
| extend 
SourcePort = tostring(SourcePortInt)
|extend
TargetPort = tostring(TargetPortInt)
| extend
//make sure we only have Allowed / Deny in the Action Field
Action1 = case(Action1 == "Deny","Deny","Unknown Action")
| extend
    Action = case(Action2 == "",Action1,Action2),
    Rule = case(Rule2a == "",case(Rule1 == "",case(Rule2b == "","N/A", Rule2b),Rule1),Rule2a), 
    RuleCollection = case(RuleCollection2b == "",case(RuleCollection2a == "","No rule matched",RuleCollection2a),RuleCollection2b),
    FQDN = case(FQDN == "", "N/A", FQDN),
    TargetPort = case(TargetPort == "", "N/A", TargetPort)
| project TimeGenerated, msg_s, Protocol, SourceIP, SourcePort, FQDN, TargetPort, Action ,RuleCollection, Rule

For NetworkRule log

AzureDiagnostics
| where Category == "AzureFirewallNetworkRule"
//using :int makes it easier to pars but later we'll convert to string as we're not interested to do mathematical functions on these fields
//case 1: for records that look like this:
//TCP request from 10.0.2.4:51990 to 13.69.65.17:443. Action: Deny//Allow
//UDP request from 10.0.3.4:123 to 51.141.32.51:123. Action: Deny/Allow
//TCP request from 193.238.46.72:50522 to 40.119.154.83:3389 was DNAT'ed to 10.0.2.4:3389
| parse msg_s with Protocol " request from " SourceIP ":" SourcePortInt:int " to " TargetIP ":" TargetPortInt:int *
//case 1a: for regular network rules
//TCP request from 10.0.2.4:51990 to 13.69.65.17:443. Action: Deny//Allow
//UDP request from 10.0.3.4:123 to 51.141.32.51:123. Action: Deny/Allow
| parse msg_s with * ". Action: " Action1a
//case 1b: for NAT rules
//TCP request from 193.238.46.72:50522 to 40.119.154.83:3389 was DNAT'ed to 10.0.2.4:3389
| parse msg_s with * " was " Action1b " to " NatDestination
//case 2: for ICMP records
//ICMP request from 10.0.2.4 to 10.0.3.4. Action: Allow
| parse msg_s with Protocol2 " request from " SourceIP2 " to " TargetIP2 ". Action: " Action2
| extend
SourcePort = tostring(SourcePortInt),
TargetPort = tostring(TargetPortInt)
| extend 
    Action = case(Action1a == "", case(Action1b == "",Action2,Action1b), Action1a),
    Protocol = case(Protocol == "", Protocol2, Protocol),
    SourceIP = case(SourceIP == "", SourceIP2, SourceIP),
    TargetIP = case(TargetIP == "", TargetIP2, TargetIP),
    //ICMP records don't have port information
    SourcePort = case(SourcePort == "", "N/A", SourcePort),
    TargetPort = case(TargetPort == "", "N/A", TargetPort),
    //Regular network rules don't have a DNAT destination
    NatDestination = case(NatDestination == "", "N/A", NatDestination)
| project TimeGenerated, msg_s, Protocol, SourceIP,SourcePort,TargetIP,TargetPort,Action, NatDestination

Using either query gives you clear readable that you can filter. One tip however, is to add a sort command to the end of the queries, normally I use by TimeGenerated to show me the latest data. So to condense and add that for the NetworkRule query above, it would look like:

AzureDiagnostics
| where Category == "AzureFirewallNetworkRule"
| parse msg_s with Protocol " request from " SourceIP ":" SourcePortInt:int " to " TargetIP ":" TargetPortInt:int *
| parse msg_s with * ". Action: " Action1a
| parse msg_s with * " was " Action1b " to " NatDestination
| parse msg_s with Protocol2 " request from " SourceIP2 " to " TargetIP2 ". Action: " Action2
| extend SourcePort = tostring(SourcePortInt),TargetPort = tostring(TargetPortInt)
| extend Action = case(Action1a == "", case(Action1b == "",Action2,Action1b), Action1a),Protocol = case(Protocol == "", Protocol2, Protocol),SourceIP = case(SourceIP == "", SourceIP2, SourceIP),TargetIP = case(TargetIP == "", TargetIP2, TargetIP),SourcePort = case(SourcePort == "", "N/A", SourcePort),TargetPort = case(TargetPort == "", "N/A", TargetPort),NatDestination = case(NatDestination == "", "N/A", NatDestination)
| project TimeGenerated, msg_s, Protocol, SourceIP,SourcePort,TargetIP,TargetPort,Action, NatDestination
| sort by TimeGenerated desc

Using parsed data, you can immediately see all the traffic hitting AFW and, for example, filter on options such as Action to see only denied traffic.

Microsoft also provide pre-cooked visualisation should you prefer it, you can download from here – https://raw.githubusercontent.com/Azure/azure-docs-json-samples/master/azure-firewall/AzureFirewall.omsview – then import into Azure Monitor. The detail is great for quick glance work, I really like the ApplicationRule breakout

Application rule log data

That about sums it up. Hopefully you are now informed and equipped to troubleshoot traffic issues in your Azure Firewall instance. As always, if there are any questions, please get in touch!

If you need more info on how to enable logs – https://docs.microsoft.com/en-us/azure/firewall/tutorial-diagnostics

Log and metrics concepts – https://docs.microsoft.com/en-us/azure/firewall/logs-and-metrics

Azure Firewall – Where to Start?

About a year ago, Microsoft introduced the first release of Azure Firewall. Since then, and since its general release the service has grown and the features have matured.

To begin, let’s understand what Azure Firewall is? At its core it’s a managed, network security service that protects your Azure Virtual Network resources. It functions as a stateful firewall-as-a-service and offers built-in high availability and scalability. This means you can centrally control, enforce and log all of your network traffic. It fully integrates with Azure Monitor too which means all of the usual logging and analytical goodness.

If the above sounds like something you’d like to use, or at least try, in your Azure environment, read on! To start, let’s break out what can be configured within Azure Firewall and which features could be useful for you.

When deploying an Azure Firewall, you need a couple of things in advance. It needs a dedicated subnet, specifically named “AzureFirewallSubnet” and the minimum size it can be is a /26. It also needs at least one Static Public IP. The Public IP must be on the Standard tier. My recommendation here is to look at creating a Public IP Prefix in advance of creating your Azure Firewall. That way, if you need to delete it and redeploy, you can continue to use the same Public IP again and again. If you want to use multiple Public IPs, it supports up to 100.

So, let’s look at what Azure Firewall (AFW) can do for you on your Virtual Network and then consider some deployment options.

Access

Using your single, or multiple Public IP addresses, AFW allows both source and destination NATing. Meaning it can support multiple inbound ports, such as HTTPS over 443 to different resources. Outbound SNAT helps greatly with services that require white-listing. If you are using multiple Public IPs, AFW randomly picks one for SNAT, so ensure you include all of them in your white-listing requirements.

Protection

AFW uses a Microsoft service called Threat Intelligence filtering. This allows Azure Firewall to alert and deny traffic to and from known malicious IPs and domains. You can turn this setting off, set it to just alert or to both alert and deny. All of the actions are logged.

Filtering

Finally, for filtering, AFW can use both Network Traffic and Application FQDN rules. This means that you can limit traffic to only those explicitly listed within the rule collections. For example, an application rule that only allows traffic to the FQDN – www.wedoazure.ie

A visual representation of the above features is below:

Firewall overview

Now that you understand AFW, let’s look at how to configure to your needs. Normally I would go into the deployment aspect, but it is excellently documented already and relatively easy to follow. However, there are some aspects of the configuration that warrant further detail.

Once deployed, you must create a Custom Route Table to force traffic to your AFW. In the tutorial, it shows you how to create a route for Internet traffic (0.0.0.0/0), however you may want the AFW to be your central control point for your vnet traffic too. Don’t forget, traffic between subnets is not filtered by default. Routing all traffic for each subnet to AFW could allow you to manage which subnet can route where centrally. For example, if we have three subnets, Web, App and DB. A single route table applied to each subnet can tunnel all traffic to AFW. On the AFW you can then allow Web to the Internet and the App subnet. The App subnet can access Web and DB but not Internet and finally the DB subnet can only access the App subnet. This would all be achieved with a single Network Rule collection.

Similarly you can allow/block specific FQDNs with an Application Rule collection. In the tutorial, a single FQDN is allowed. This means that all others are blocked as that is the default behaviour. This might not be practical for your environment and the good news is, you can implement the reverse. With the right priority order, you can allow all traffic except for blocked FQDNs.

A feature you may also want to consider trying is destination NATing. This thankfully has another well documented tutorial on Docs.

Finally, and in some cases most importantly, let’s look at price. You are charged in two ways for AFW. There is a price per-hour-per-instance. That means if you deploy and don’t use it for anything, you will pay approx. ‚ā¨770 per-month (PAYG Calculator). On top of that, you will pay for both data inbound and outbound that is filtered by AFW. You’re charged the same price either direction and that’s approx. ‚ā¨14 per-Tb-per-month. Depending on your environment and/or requirements this price could be OK or too steep. My main advice is to ensure you understand it before deploying!

As always, if there are any questions please get in touch!