All Projects → metal-stack → firewall-controller

metal-stack / firewall-controller

Licence: MIT license
A kubernetes controller running on bare-metal firewalls, creating nftables rules, configures suricata, collects network metrics

Programming Languages

go
31211 projects - #10 most used programming language
Makefile
30231 projects
Smarty
1635 projects
Dockerfile
14818 projects

Projects that are alternatives of or similar to firewall-controller

Intelspy
Perform automated network reconnaissance scans
Stars: ✭ 134 (+262.16%)
Mutual labels:  services
Awesome Ad Free
Curated list of ad-free alternatives to popular services on the web
Stars: ✭ 235 (+535.14%)
Mutual labels:  services
daemons.el
An Emacs UI for managing init system services
Stars: ✭ 87 (+135.14%)
Mutual labels:  services
Ioc
🦄 lightweight (<1kb) inversion of control javascript library for dependency injection written in typescript
Stars: ✭ 171 (+362.16%)
Mutual labels:  services
Storaji
📒 The light/responsive inventory management system available on Windows, macOS and Linux.
Stars: ✭ 222 (+500%)
Mutual labels:  services
rc
Structured system configuration (I moved from NixOS to GuixSD)
Stars: ✭ 97 (+162.16%)
Mutual labels:  nftables
Pysoa
A Python library for writing (micro)services and their clients
Stars: ✭ 74 (+100%)
Mutual labels:  services
prickly-pete
A script using Docker to quickly bring up some honeypots exposing lots of services. For research, reconnaissance, and fun. (DISCLAIMER may not be fun, not to be taken internally, aim away from face)
Stars: ✭ 29 (-21.62%)
Mutual labels:  ids
Haven Platform
Haven is an open source Docker container management system. It integrates container, application, cluster, image, and registry management in one single place.
Stars: ✭ 223 (+502.7%)
Mutual labels:  services
angus
Simple and powerful Rest Service Framework
Stars: ✭ 13 (-64.86%)
Mutual labels:  services
Mode
Python AsyncIO Services
Stars: ✭ 204 (+451.35%)
Mutual labels:  services
Express Starter
It's a hackathon-starter fork, but designed to use PostgreSQL by default (or MySQL)
Stars: ✭ 215 (+481.08%)
Mutual labels:  services
dictionary-renderer-toolkit
The Dictionary Renderer Toolkit provides documentation and tools for the creation of custom dictionary styles for use with the dictionary renderer.
Stars: ✭ 18 (-51.35%)
Mutual labels:  services
Serverless Monorepo App
Serverless Monorepo App
Stars: ✭ 136 (+267.57%)
Mutual labels:  services
Kv-z
Online Quiz App using Firebase
Stars: ✭ 17 (-54.05%)
Mutual labels:  services
Homebrew Services
🚀 Manage background services with macOS' launchctl daemon manager
Stars: ✭ 1,656 (+4375.68%)
Mutual labels:  services
acfs
API client for services
Stars: ✭ 13 (-64.86%)
Mutual labels:  services
UTMFW
UTM Firewall on OpenBSD
Stars: ✭ 104 (+181.08%)
Mutual labels:  ids
mezon
Mezon is a simple php framework wich will help you to create business applications.
Stars: ✭ 35 (-5.41%)
Mutual labels:  services
SampleMusicPlayer
Simple music player
Stars: ✭ 59 (+59.46%)
Mutual labels:  services

Firewall Controller

This controller is installed on a bare-metal firewall in front of several kubernetes worker nodes and responsible to reconcile a ClusterwideNetworkPolicy to nftables rules to control access to and from the kubernetes cluster. It allows also to control the traffic rate going through, to limit network resources for restricted usage scenarios. Nftable and node metrics are exposed with the nftables-exporter and node-exporter, the ips are visible as service and endpoint from the kubernetes cluster.

Additional an IDS is managed on the firewall to detect known network anomalies. suricata is used for this purpose. Right now, only basic statistics about the amount of scanned packets is reported. In a future release, access to all alarms will be provided.

Architecture

Architecture

Automatically generated ingress rules

For every Service of type LoadBalancer in the cluster, the corresponding ingress rules will be automatically generated.

If loadBalancerSourceRanges is not specified, incomig traffic to this service will be allowed for any source ip adresses.

Configuration

Firewall Controller is configured with 2 CRDs: firewalls.metal-stack.io and clusterwidenetworkpolicies.metal-stack.io. Both are namespaced and must reside in the firewall namespace. The firewalls CRD is typically written from the gardener-extension-provider-metal, the clusterwidenetworkpolicy should be provided by the deployment of your application.

Example Firewall CRD:

apiVersion: metal-stack.io/v1
kind: Firewall
metadata:
  namespace: firewall
  name: firewall
spec:
  # Interval of reconcilation if nftables rules and network traffic accounting
  interval: 10s
  # Ratelimits specify on which physical interface, which maximum rate of traffic is allowed
  ratelimits:
  # LogAcceptedConnections specifies whether accepted connections should be logged by the firewall in addition to dropped/rejected connections
  logAcceptedConnections: false
  # The name of the interface visible with ip link show
  - interface: vrf104009
    # The maximum rate in MBits/s
    rate: 10
  # Internalprefixes defines a list of prefixes where the traffic going to, or comming from is considered internal, e.g. not leaving into external networks
  # given the archictecture picture above this would be:
  internalprefixes:
  - "1.2.3.0/24
  - "172.17.0.0/16"
  - "10.0.0.0/8"

Example ClusterwideNetworkPolicy:

apiVersion: metal-stack.io/v1
kind: ClusterwideNetworkPolicy
metadata:
  namespace: firewall
  name: clusterwidenetworkpolicy-sample
spec:
  egress:
  - to:
    - cidr: 1.1.0.0/24
      except:
      - 1.1.1.0/16
    - cidr: 8.8.8.8/32
    ports:
    - protocol: UDP
      port: 53
    - protocol: TCP
      port: 53
    - protocol: TCP
      port: 8080
      # Optional, if specified this is the way to specify a port range from port to endPort
      endPort: 8088 

DNS policies configuration

ClusterwideNetworkPolicy resource allows you to define DNS based Egress policies as well. They allow you to filter Egress traffic based either on DNS name or by matching names to the provided pattern.

To filter by specific domain name you need to provide matchName field:

apiVersion: metal-stack.io/v1
kind: ClusterwideNetworkPolicy
metadata:
  namespace: firewall
  name: clusterwidenetworkpolicy-fqdn
spec:
  egress:
  - toFQDNs:
    - matchName: example.com
    ports:
    - protocol: UDP
      port: 80
    - protocol: TCP
      port: 80

If you want to filter FQDNs that are matching certain pattern, you can use matchPattern field, which supports * wildcard. Following example allows traffic to port 80 of all resources in the .example top-level domain:

apiVersion: metal-stack.io/v1
kind: ClusterwideNetworkPolicy
metadata:
  namespace: firewall
  name: clusterwidenetworkpolicy-fqdn-pattern
spec:
  egress:
  - toFQDNs:
    - matchPattern: *.example
    ports:
    - protocol: UDP
      port: 80
    - protocol: TCP
      port: 80

By default, DNS info is collected from Google DNS (with address 8.8.8.8:53). The preferred DNS server can be changed by specifying the dnsServerAddress field in Firewall resource:

apiVersion: metal-stack.io/v1
kind: Firewall
metadata:
  namespace: firewall
  name: firewall
spec:
  ...
  dnsServerAddress: 8.8.4.4:53
  ...

Status

Once the firewall-controller is running, it will report several statistics to the Firewall CRD Status: This can be inspected by running:

kubectl describe -n firewall firewall

The output would look like:

Status:
  Last Run:  2020-06-17T13:18:58Z
  Stats:
    # Network traffic in bytes separated into external and internal in/out/total
    Devices:
      External:
        In:     91696
        Out:    34600
        Total:  0
      Internal:
        In:     0
        Out:    0
        Total:  2678671
    # IDS Statistics by interface
    Idsstats:
      vrf104009:
        Drop:              1992
        Invalidchecksums:  0
        Packets:           4997276
    # nftable rule statistics by rule name
    Rules:
      Accept:
        BGP unnumbered:
          Counter:
            Bytes:    0
            Packets:  0
        SSH incoming connections:
          Counter:
            Bytes:    936
            Packets:  16
        accept established connections:
          Counter:
            Bytes:    21211168
            Packets:  39785
        accept icmp:
          Counter:
            Bytes:    0
            Packets:  0
        accept traffic for k8s service kube-system/vpn-shoot:
          Counter:
            Bytes:    360
            Packets:  6
      Drop:
        drop invalid packets:
          Counter:
            Bytes:    52
            Packets:  1
        drop invalid packets from forwarding to prevent malicious activity:
          Counter:
            Bytes:    0
            Packets:  0
        drop invalid packets to prevent malicious activity:
          Counter:
            Bytes:    0
            Packets:  0
        drop packets with invalid ct state:
          Counter:
            Bytes:    0
            Packets:  0
        drop ping floods:
          Counter:
            Bytes:    0
            Packets:  0
      Other:
        block bgp forward to machines:
          Counter:
            Bytes:    0
            Packets:  0
        count and log dropped packets:
          Counter:
            Bytes:    2528
            Packets:  51
        snat (networkid: internet):
          Counter:
            Bytes:    36960
            Packets:  486

Prometheus integration

There are two exporters running on the firewall to report essential metrics from this machine:

  • node-exporter for machine specific metrics like cpu, ram and disk usage, see node-exporter for details.
  • nftables-exporter for nftables metrics, see nftables-exporter

Both exporters are exposed as services:

kubectl get svc -n firewall
NAME                TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
nftables-exporter   ClusterIP   None         <none>        9630/TCP   13h
node-exporter       ClusterIP   None         <none>        9100/TCP   13h

These services are in front of virtual endpoints:

kubectl get ep -n firewall
NAME                ENDPOINTS         AGE
nftables-exporter   10.3.164.1:9630   13h
node-exporter       10.3.164.1:9100   13h

You can scrape these services in you prometheus installation to get the metrics.

To check you can run:

curl nftables-exporter.firewall.svc.cluster.local:9630/metrics
curl node-exporter.firewall.svc.cluster.local:9100/metrics

Firewall Logs

It is also possible to tail for the dropped packets with the following command (install stern from stern ):

stern -n firewall drop

The output will look like:

droptailer-6d556bd988-4g8gp droptailer 2020-06-17 13:23:27 +0000 UTC {"ACTION":"Drop","DPT":"4000","DST":"1.2.3.4","ID":"54321","IN":"vrf104009","LEN":"40","MAC":"ca:41:f9:80:fa:89:aa:bb:0e:62:8c:a6:08:00","OUT":"vlan179","PREC":"0x00","PROTO":"TCP","RES":"0x00","SPT":"38464","SRC":"2.3.4.5","SYN":"","TOS":"0x00","TTL":"236","URGP":"0","WINDOW":"65535","timestamp":"2020-06-17 13:23:27 +0000 UTC"}
droptailer-6d556bd988-4g8gp droptailer 2020-06-17 13:23:34 +0000 UTC {"ACTION":"Drop","DPT":"2362","DST":"1.2.3.4","ID":"44545","IN":"vrf104009","LEN":"40","MAC":"ca:41:f9:80:fa:89:aa:bb:0e:62:8c:a6:08:00","OUT":"","PREC":"0x00","PROTO":"TCP","RES":"0x00","SPT":"40194","SRC":"2.3.4.5","SYN":"","TOS":"0x00","TTL":"242","URGP":"0","WINDOW":"1024","timestamp":"2020-06-17 13:23:34 +0000 UTC"}
droptailer-6d556bd988-4g8gp droptailer 2020-06-17 13:23:30 +0000 UTC {"ACTION":"Accept","DPT":"650","DST":"1.2.3.4","ID":"12399","IN":"vrf104009","LEN":"40","MAC":"ca:41:f9:80:fa:89:aa:bb:0e:62:8c:a6:08:00","OUT":"vlan179","PREC":"0x00","PROTO":"TCP","RES":"0x00","SPT":"40194","SRC":"2.3.4.5","SYN":"","TOS":"0x00","TTL":"241","URGP":"0","WINDOW":"1024","timestamp":"2020-06-17 13:23:30 +0000 UTC"}
droptailer-6d556bd988-4g8gp droptailer 2020-06-17 13:23:34 +0000 UTC {"ACTION":"Accept","DPT":"2362","DST":"1.2.3.4","ID":"44545","IN":"vrf104009","LEN":"40","MAC":"ca:41:f9:80:fa:89:aa:bb:0e:62:8c:a6:08:00","OUT":"","PREC":"0x00","PROTO":"TCP","RES":"0x00","SPT":"40194","SRC":"2.3.4.5","SYN":"","TOS":"0x00","TTL":"242","URGP":"0","WINDOW":"1024","timestamp":"2020-06-17 13:23:34 +0000 UTC"}
droptailer-6d556bd988-4g8gp droptailer 2020-06-17 13:23:10 +0000 UTC {"ACTION":"Accept","DPT":"63351","DST":"1.2.3.4","ID":"11855","IN":"vrf104009","LEN":"40","MAC":"ca:41:f9:80:fa:89:aa:bb:0e:62:8c:a6:08:00","OUT":"vlan179","PREC":"0x00","PROTO":"TCP","RES":"0x00","SPT":"54589","SRC":"2.3.4.5","SYN":"","TOS":"0x00","TTL":"245","URGP":"0","WINDOW":"1024","timestamp":"2020-06-17 13:23:10 +0000 UTC"}
droptailer-6d556bd988-4g8gp droptailer 2020-06-17 13:23:51 +0000 UTC {"ACTION":"Accept","DPT":"8002","DST":"1.2.3.4","ID":"17539","IN":"vrf104009","LEN":"40","MAC":"ca:41:f9:80:fa:89:aa:bb:0e:62:8c:a6:08:00","OUT":"","PREC":"0x00","PROTO":"TCP","RES":"0x00","SPT":"47615","SRC":"2.3.4.5","SYN":"","TOS":"0x08","TTL":"239","URGP":"0","WINDOW":"1024","timestamp":"2020-06-17 13:23:51 +0000 UTC"}

You can forward the droptailer logs to any log aggregation infrastructure you have in place.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].