Azure Firewall is ever growing in popularity as a choice when it comes to perimeter protection for Azure networking. The introduction of additional SKUs (Premium and Basic) since its launch have made it both more functional while also increasing its appeal to a broader environment footprint.
For anyone who has used Azure Firewall since the beginning, troubleshooting and analysis of your logs has always had a steep-ish learning curve. On one hand, the logs are stored in Log Analytics and you can query them using Kusto, so there is familiarity. However, without context, their formatting can be challenging. The good news is, this is being improved with the introduction of a new format.
Previously logs we stored using the Azure Diagnostics mode, with this update, we will now see the use of Resource-Specific mode. This is something that will become more common across many Azure resources, and you should see it appear for several in the Portal already.
What difference will this make for Azure Firewall? This will mean individual tables in the selected workspace are created for each category selected in the diagnostic setting. This offers the following improvements:
- Makes it much easier to work with the data in log queries
- Makes it easier to discover schemas and their structure
- Improves performance across both ingestion latency and query times
- Allows you to grant Azure RBAC rights on a specific table
For Azure Firewall, the new resource specific tables are below:
- Network rule log – Contains all Network Rule log data. Each match between data plane and network rule creates a log entry with the data plane packet and the matched rule’s attributes.
- NAT rule log – Contains all DNAT (Destination Network Address Translation) events log data. Each match between data plane and DNAT rule creates a log entry with the data plane packet and the matched rule’s attributes.
- Application rule log – Contains all Application rule log data. Each match between data plane and Application rule creates a log entry with the data plane packet and the matched rule’s attributes.
- Threat Intelligence log – Contains all Threat Intelligence events.
- IDPS log – Contains all data plane packets that were matched with one or more IDPS signatures.
- DNS proxy log – Contains all DNS Proxy events log data.
- Internal FQDN resolve failure log – Contains all internal Firewall FQDN resolution requests that resulted in failure.
- Application rule aggregation log – Contains aggregated Application rule log data for Policy Analytics.
- Network rule aggregation log – Contains aggregated Network rule log data for Policy Analytics.
- NAT rule aggregation log – Contains aggregated NAT rule log data for Policy Analytics.
So, let’s start with getting logs enabled on your Azure Firewall. You can’t query your logs if there are none! And Azure Firewall does not enable this by default. I’d generally recommend enabling logs as part of your build process and I have an example of that using Bicep over on Github, (note this is Diagnostics mode, I will update it for Resource mode soon!) However, if already built, let’s look at simply doing this via the Portal.
So on our Azure Firewall blade, head to the Monitoring section and choose “Diagnostic settings”
We’re then going to choose all our new resource specific log options
Next, we choose to send to a workspace, and make sure to switch to Resource specific.
Finally, give your settings a name, I generally use my resource convention here, and click Save.
It takes a couple of minutes for logs to stream through, so while that happens, let’s look at what is available for analysis on Azure Firewall out-of-the-box – Metrics.
While there are not many entries available, what is there can be quite useful to see what sort of strain your Firewall is under.
Hit counts are straight forward, they can give you an insight into how busy the service is. Data Processed and Throughput are also somewhat interesting from an analytics perspective. However, it is Health State and SNAT that are most useful in my opinion. These are metrics you should enable alerts against.
For example, an alert rule for SNAT utilisation reaching an average of NN% can be very useful to ensure scale is working and within limits for your service and configuration of IPs.
Ok, back to our newly enabled Resource logs. When you open the logs tab on your Firewall, if you haven’t disabled it, you should see a queries screen pop-up as below:
You can see there are now two sections, one specifically for Resource Specific tables. If I simply run the following query:
AZFWNetworkRuleI get a structured and clear output:
To get a comparative output using Diagnostics Table, I need to run a query similar to the below:
// Network rule log data
// Parses the network rule log data.
AzureDiagnostics
| where Category == "AzureFirewallNetworkRule"
| where OperationName == "AzureFirewallNatRuleLog" or OperationName == "AzureFirewallNetworkRuleLog"
//case 1: for records that look like this:
//PROTO request from IP:PORT to IP:PORT.
| parse msg_s with Protocol " request from " SourceIP ":" SourcePortInt:int " to " TargetIP ":" TargetPortInt:int *
//case 1a: for regular network rules
| parse kind=regex flags=U msg_s with * ". Action\\: " Action1a "\\."
//case 1b: for NAT rules
//TCP request from IP:PORT to IP:PORT was DNAT'ed to IP:PORT
| parse msg_s with * " was " Action1b:string " to " TranslatedDestination:string ":" TranslatedPort:int *
//Parse rule data if present
| parse msg_s with * ". Policy: " Policy ". Rule Collection Group: " RuleCollectionGroup "." *
| parse msg_s with * " Rule Collection: " RuleCollection ". Rule: " Rule
//case 2: for ICMP records
//ICMP request from 10.0.2.4 to 10.0.3.4. Action: Allow
| parse msg_s with Protocol2 " request from " SourceIP2 " to " TargetIP2 ". Action: " Action2
| extend
SourcePort = tostring(SourcePortInt),
TargetPort = tostring(TargetPortInt)
| extend
Action = case(Action1a == "", case(Action1b == "",Action2,Action1b), split(Action1a,".")[0]),
Protocol = case(Protocol == "", Protocol2, Protocol),
SourceIP = case(SourceIP == "", SourceIP2, SourceIP),
TargetIP = case(TargetIP == "", TargetIP2, TargetIP),
//ICMP records don't have port information
SourcePort = case(SourcePort == "", "N/A", SourcePort),
TargetPort = case(TargetPort == "", "N/A", TargetPort),
//Regular network rules don't have a DNAT destination
TranslatedDestination = case(TranslatedDestination == "", "N/A", TranslatedDestination),
TranslatedPort = case(isnull(TranslatedPort), "N/A", tostring(TranslatedPort)),
//Rule information
Policy = case(Policy == "", "N/A", Policy),
RuleCollectionGroup = case(RuleCollectionGroup == "", "N/A", RuleCollectionGroup ),
RuleCollection = case(RuleCollection == "", "N/A", RuleCollection ),
Rule = case(Rule == "", "N/A", Rule)
| project TimeGenerated, msg_s, Protocol, SourceIP,SourcePort,TargetIP,TargetPort,Action, TranslatedDestination, TranslatedPort, Policy, RuleCollectionGroup, RuleCollection, RuleObviously, there is a large visual difference in complexity! But there are also all of the benefits as described earlier for Resource Specific. I really like the simplicity of the queries. I also like the more structured approach. For example, take a look at the set columns that are supplied on the Application Rule table. You can now predict, understand, and manipulate queries with more detail than ever before. You can check out all the new tables by searching “AZFW” on this page.
Finally, a nice sample query to get you started. One that I use quite often when checking on new services added, or if there are reports of access issues. The below gives you a quick glance into web traffic being blocked and can allow you to spot immediate issues.
AZFWApplicationRule
| where Action == "Deny"
| distinct Fqdn
| sort by Fqdn ascAs usual, if there are any questions, get in touch!
wedoAzure 