5. Monitoring, Logging and Runtime Security

Perform behavioral analytics of syscall process

Falco Overview and Installation

Use Falco to Detect Threats

• rules.yaml https://falco.org/docs/reference/rules/supported-fields/

• - rule: Detect shell inside a container
    desc: Alert if a shell such as bash is open inside the container
    condition: container and proc.name in (linux_shells)
    output: Bash opened (user=%user.name container=%container.id)
    priority: WARNING
  - list: linux_shells
    items: [bash, zsh, ksh, sh, csh]
  - macro: container
    condition: container.is != host

Falco Configuration Files

• /etc/falco/falco.yaml

• journalctl -fu falco

• kill -1 $(cat /var/run/falco.pid)

Mutable vs Immutable Infrastructure

Ensure Immutability of Containers at Runtime

• readOnlyRootFileSystem: true (can be done with Pod Security Standards)

  • you can use an init container with RW access and than the app container is RO (same pod)

• volumeMounts: for the writable dir (/var/run emptyDir:{})

• run as user and non root

• remove bash/shell

  • command: (this will change the container start command)

  • startupProbe:

Use Audit Logs to monitor access to Kubernetes API

Not enabled by default!

• audit-policy.yaml

  • audit stages

    • RequestReceived

    • ResponceStarted

    • ResponceComplete

    • Panic

  • audit level

    • None

    • Metadata

    • Request

    • RequestResponce

• backend

  • log audit (JSON file)

  --audit-log-path=/var/log/kubernetes/audit/audit.log

  --audit-policy-file=/etc/kubernetes/audit-polocy.yaml

  --audit-log-maxage=10 (days)

  --audit-log-maxbackup=5

  --audit-logmaxsize=100 (MB)

    volumeMounts:
      - mountPath: /etc/kubernetes/audit-policy.yaml
        name: audit
        readOnly: true
      - mountPath: /var/log/kubernetes/audit/
        name: audit-log
        readOnly: false
  volumes:
  - name: audit
    hostPath:
      path: /etc/kubernetes/audit-policy.yaml
      type: File
  
  - name: audit-log
    hostPath:
      path: /var/log/kubernetes/audit/
      type: DirectoryOrCreate

  • ElasticSearch

  • Fluentd

  • FileBeat