!!! Unfortunately I have moved to Ubiquiti for my entire networking soltion and I am using the Ubiquiti Solution for Microsoft Sentinel. Therefore will no longer be updating this project. Apologies for any inconvenience. !!!
pfSense/OPNsense syslog, Suricata & HA-Proxy to Azure Sentinel
As there is no out-of-the-box Azure Sentinel connector for either pfSense or OPNsense this project fills that void and allows you full control over your logs.
It can be used to collect syslog messages from pfSense or OPNsense, parse them using Logstash GROK, add additional context to the log messages such as GeoIP information and then send them to Azure Sentinel.
Changes
2021.04
-
pfSense Workbook v0.2.1
-
Added Tabs and split out visuals:
- Firewall
- Unbound
- Services
- Inbound
- Outbound
- Threat Hunting.
-
Added Time Slice for common services
-
Added Unbound lookups
-
Moved Lateral Movement and Subnets to
Threat Hunting
-
-
Added pfSense Workbook v0.1
-
Added ability to drop GROK Failures before sending to Azure Sentinel (Thanks @a3ilson)
2021.03
- Updated configuration based on pfELK changes
- Moved to Microsoft Logstash Azure-Log Analytics Plugin
Credits
This project is only possible with the work carried out by a3ilson and his pfELK project to parse the pfSense log files.
Configuration
For Deployments please use the Logstash Guide
This project exposes the following pfSense/OPNsense data points to Azure Sentinel:
| Data points in Azure Sentinel | |||||
|---|---|---|---|---|---|
| _timestamp_t | _version_s | destination_as_ip_s | destination_as_number_d | destination_as_organization_name_s | |
| destination_geo_city_name_s | destination_geo_continent_code_s | destination_geo_country_code3_s | destination_geo_country_iso_code_s | destination_geo_country_name_s | |
| destination_geo_dma_code_d | destination_geo_ip_s | destination_geo_latitude_d | destination_geo_location_lat_d | destination_geo_location_lon_d | |
| destination_geo_longitude_d | destination_geo_postal_code_s | destination_geo_region_iso_code_s | destination_geo_region_name_s | destination_geo_timezone_s | |
| destination_ip_s | destination_port_s | destination_service_sdhcp_operation_s | dhcpd_release_s | dhcpv4_client_ip_s | |
| dhcpv4_client_mac_s | dhcpv4_option_hostname_s | dhcpv4_server_ip_s | ecs_version_s | event_action_s | |
| event_created_t | event_reason_s | Flags_s | icmp_type_s | interface_alias_s | |
| interface_name_s | log_original_s | log_syslog_priority_s | network_direction_s | network_iana_number_s | |
| network_name_s | network_packets_s | network_transport_s | network_type_s | observer_ip_s | |
| observer_name_s | observer_product_s | observer_serial_number_s | observer_type_s | option_s | |
| pf_app_s | pf_icmp_echo_id_s | pf_icmp_echo_sequence_s | pf_icmp_unreachport_destination_ip_s | pf_icmp_unreachport_port_s | |
| pf_icmp_unreachport_protocol_s | pf_ipv4_ecn_s | pf_ipv4_flags_s | pf_ipv4_offset_s | pf_ipv4_packet_id_s | |
| pf_ipv4_tos_s | pf_ipv4_ttl_s | pf_ipv6_class_s | pf_ipv6_flow_label_s | pf_ipv6_hop_limit_s | |
| pf_packet_length_s | pf_protocol_id_s | pf_protocol_type_s | pf_tcp_ack_number_s | pf_tcp_flags_s | |
| pf_tcp_options_s | pf_tcp_sequence_number_s | pf_tcp_window_s | pf_transport_data_length_s | priority_s | |
| process_name_s | process_pid_s | rule_alias_s | rule_classification_s | rule_description_s | |
| rule_reference_s | rule_ruleset_s | rule_uuid_s | rule_version_s | source_as_ip_s | |
| source_as_number_d | source_as_organization_name_s | source_geo_city_name_s | source_geo_continent_code_s | source_geo_country_code3_s | |
| source_geo_country_iso_code_s | source_geo_country_name_s | source_geo_dma_code_d | source_geo_ip_s | source_geo_latitude_d | |
| source_geo_location_lat_d | source_geo_location_lon_d | source_geo_longitude_d | source_geo_postal_code_s | source_geo_region_iso_code_s | |
| source_geo_region_name_s | source_geo_timezone_s | source_ip_s | source_port_s | source_service_s | |
| tags_s | pf | GeoIP_Source | GeoIP_Destination | vpn_log_message_s |
KQL
Using the Azure Sentinel KQL we can break down this data into readable formats
Linux Performance Monitoring
Optionally you can also install the Linux OMS Agent to collect performance stats from the Logstash log collector. This is useful for creating analytics rules that will monitor for disk space alerts and excessive CPU usage.