Skip to content

Notification Options

flowwler fires notifications on every escalation state transition and on every manual mitigation action via the REST API.

Configuration

Targets are configured globally (all groups) and/or per group:

notifications:
  timeout: 10s          # per-target send timeout (default: 10s)
  retry_count: 2        # retries on failure (default: 0)
  targets:
    - type: teams
      name: ops-channel
      url: "https://outlook.office.com/webhook/..."

groups:
  - name: datacenter-a
    notifications:
      targets:
        - type: telegram
          name: noc-bot
          token: "123456:ABC..."
          chat_id: "-1001234567890"

Per-group targets receive only events for that group. Global targets receive events for all groups. Both lists are evaluated for every event - a target in both will receive events twice if not deduplicated by name.

Event filter

Any target can be restricted to a subset of event types:

- type: webhook
  name: activate-only
  url: "https://example.com/hook"
  events: [activated, escalated]   # omit field to receive all events

Valid event types: activated, escalated, holddown, cleared.

Event schema

All backends receive the same Event struct. Human-readable backends (Telegram, Pushover, Teams, Slack) format BPS and PPS; machine-readable backends (webhook, PagerDuty, Alertmanager) carry raw integers.

Field Type Description
type string activated, escalated, holddown, or cleared
attack_id string 16-byte hex ID, stable across all events for one attack
timestamp RFC3339 Time the event was fired
victim_ip string IP address of the attack target
group_name string Group name ("manual" for REST API-triggered events)
rule_name string Rule name if a named rule triggered mitigation; omitted for group-level
level int Current escalation level (0 on cleared)
prev_level int Previous level (populated on escalated)
mitigation_type string blackhole, subnet-blackhole, flowspec, or "" for alert-only levels
flowspec_action string FlowSpec action: discard, rate-limit, or redirect. Present on all event types when the mitigation type is flowspec; omitted otherwise.
notification_only bool true when the level has no mitigation configured (alert-only). Omitted when false.
bps uint64 Attack traffic in bits per second
pps uint64 Attack traffic in packets per second
threshold_bps uint64 Configured BPS threshold that triggered the level (omitted if not set)
threshold_pps uint64 Configured PPS threshold that triggered the level (omitted if not set)
discard_bps uint64 Traffic already being dropped by routers (bits per second)
flowspec_rules array FlowSpec rules announced for this event (omitted for non-FlowSpec mitigations; see below)
suggested_flowspec object FlowSpec rule suggestions derived from flow data (omitted when flowspec_rules is present or no traffic pattern is dominant; see below)

FlowSpec rule fields (each entry in flowspec_rules)

Field Type Description
dst_ip string Destination IP matched by this rule (always the victim IP)
src_ip string Source IP matched by this rule (omitted when no source filter)
protocol uint8 IP protocol number (0/omitted = any; 6=TCP, 17=UDP, 1=ICMP)
dst_ports []uint16 Destination ports (empty/omitted = any). A single-element list may carry an explicit 0 (e.g. an hping3 --destport 0 flood) — a real match criterion, not "any". More than one element is an OR'd match list.
src_ports []uint16 Source ports (empty/omitted = any)
min_pkt_len uint16 Minimum packet length in bytes (0/omitted = no match)
max_pkt_len uint16 Maximum packet length in bytes (0/omitted = no match)
pkt_len_approximate bool Always false/omitted for a live FlowSpec rule notification. min_pkt_len/max_pkt_len, when present, is always real exporter data or an explicit pkt_len match config — a purely derived value is never attached to an announced route. The equivalent field in suggested_flowspec (below) can be true, since a suggestion is reviewed by an operator before ever being applied.
action string discard, rate-limit, or redirect

Known gap: live FlowSpec rules can also be automatically narrowed by ICMP type/code (e.g. echo request 8/0), and GET /api/v1/flowspec/rules surfaces that as icmp_types/icmp_codes on the REST rule object (see rest-api.md). That detail is not currently carried into notification payloads — flowspec_rules entries above never include ICMP-type/code criteria, even when the live rule has them. Check the REST API for the full match criteria of an active rule; do not infer them from a chat/ticket notification alone. Separately, TCP flags and the is-fragment match component (used when building the live BGP route) are not currently exposed as fields on FlowSpecRuleInfo/GET /api/v1/flowspec/rules at all, and so are also absent from notifications — this is a pre-existing gap on both surfaces, not specific to notifications.

Multiple attack sources

When FlowSpec is the active mitigation type, the escalation engine selects the top-N attack sources by volume (up to max_rules, default 10) and announces one BGP FlowSpec NLRI per source in a single activation call. One notification is fired per state transition (not one per source). That single event contains all rules in flowspec_rules, so every backend will display the full set of matching sources and their match criteria in one message.

FlowSpec suggestions (suggested_flowspec)

For events where flowspec_rules is absent (alert-only levels, blackhole mitigations, or any event where the active mitigation is not FlowSpec), flowwler automatically analyses the live flow distribution and attaches a suggested_flowspec object when a dominant traffic pattern is found. This allows NOC engineers to see what FlowSpec rules would be effective even when the active mitigation type does not use FlowSpec.

The suggestion uses the same source-selection and consolidation logic as a live FlowSpec mitigation. The ranking metric (BPS or PPS) is chosen based on which threshold was proportionally more exceeded - a PPS-only threshold always ranks by PPS; a BPS-only threshold always ranks by BPS; when both are configured, the metric with the greater ratio of observed/threshold wins.

suggested_flowspec is omitted when: - flowspec_rules is already present (live FlowSpec mitigation is active) - No source-level flow records exist for the victim IP - The dominant pattern covers less than 20% of total attack traffic (too distributed to suggest)

suggested_flowspec object fields:

Field Type Description
ranked_by string "bps" or "pps" - the metric used to rank and select rules
unique_source_count int Total number of distinct source IPs observed before any consolidation
spoofed_sources bool true when the source distribution looks randomised (≥ 50 unique sources each contributing < 5 Mbps on average). Indicates per-source FlowSpec rules will be ineffective; destination-only or pattern-only rules are preferable.
rules array Suggested rules, ranked by the metric in ranked_by

Each entry in rules:

Field Type Description
src_ip string Source IP matched by this rule (omitted once the rule set is consolidated to a source-agnostic pattern)
protocol uint8 IP protocol number (0 = any; 6=TCP, 17=UDP, 1=ICMP)
dst_ports []uint16 Destination ports (empty/omitted = any; a single explicit 0 is a real match, not "any")
src_ports []uint16 Source ports (empty/omitted = any)
min_pkt_len uint16 Minimum packet length in bytes (0 = no constraint)
max_pkt_len uint16 Maximum packet length in bytes (0 = no constraint)
pkt_len_approximate bool true when min_pkt_len/max_pkt_len is a bytes/packets-derived approximation rather than real exporter data (false/omitted otherwise)
coverage_pct uint8 Percentage of total inbound BPS (or PPS) this rule would cover

Suggestions do not currently carry icmp_types/icmp_codes — the suggestion engine doesn't compute ICMP match dimensions, even when the underlying traffic is ICMP.

Backends

alertmanager

Sends firing/resolved alerts to a Prometheus Alertmanager instance.

- type: alertmanager
  name: ops-am
  url: "http://alertmanager:9093"
  critical_level: 2

Fires on activated, escalated, and holddown; resolves on cleared. The attack_id is included as a label. Severity (warning/critical) is controlled by critical_level.

jira

Opens a Jira issue when an attack is detected and resolves it when the attack clears. Escalation and hold-down events are appended as comments on the existing issue.

- type: jira
  name: noc-jira
  url: "https://mycompany.atlassian.net"
  email: "noc@example.com"          # Jira user email (for Basic auth with API token)
  api_token: "xxxxxxxxxxxxxxxxxxxx"  # Atlassian API token (not your account password)
  project_key: "NOC"                 # Jira project key
  issue_type: "Incident"             # optional; default: "Task"
  resolve_transition: "Done"         # optional; transition name to close the issue (default: "Done")
  issue_priority: "Normal"           # optional; default: "Normal". Escalation events bump it to "High".
  tags: [network, ddos-response]     # optional; extra labels appended to every issue

Setup:

  1. Log in to id.atlassian.com and generate an API token.
  2. Create or identify the target Jira project and note its key (e.g. NOC).
  3. Check the available transition names on your issue workflow - the value of resolve_transition must match exactly (case-insensitive). Common names: Done, Resolved, Close Issue.

Behaviour by event type:

Event Action
activated (first time) Creates a new issue with a description table (victim, group, level, traffic, FlowSpec rules if applicable). Priority is set to Normal (or the value of issue_priority). Labels default to flowwler and ddos; configuring tags replaces those defaults.
activated (re-activation) If traffic recovered during hold-down (same attack_id), appends a comment instead of opening a duplicate issue: "Traffic recovered during hold-down - mitigation remains active."
escalated Adds a comment: level, new traffic rate, mitigation type. Updates the issue priority to High.
holddown Adds a comment noting the attack has subsided.
cleared Adds a final comment with the peak traffic, then transitions the issue using resolve_transition.

Issue state is tracked in memory (keyed by attack_id). If the daemon is reloaded while an attack is active the new target instance will not find the open issue, so escalation and cleared events for that attack will be silently skipped for the Jira backend only.

pagerduty

Sends events to the PagerDuty Events v2 API. Automatically resolves on cleared.

- type: pagerduty
  name: ops-pd
  integration_key: "abc123def456..."
  critical_level: 2   # levels >= this value use "critical" severity; below uses "warning"

Setup: Create a service in PagerDuty with an Events API v2 integration and copy the integration key.

The attack_id is used as the dedup_key, so all events for the same attack are grouped into one PagerDuty incident. A cleared event resolves the incident automatically. For FlowSpec mitigations the custom_details object includes a flowspec_rules array with one structured entry per announced NLRI (same fields as the event schema above).

pushover

Sends a push notification via the Pushover API. Message priority is derived from the escalation level.

- type: pushover
  name: ops-po
  user_key: "uXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
  app_token: "aXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
  priority:           # optional; omit to use defaults below
    0: -1             # cleared/holddown: quiet
    1: 0              # level 1: normal
    2: 1              # level 2: high (bypasses quiet hours)
    3: 2              # level 3+: emergency (repeats every 60s until acknowledged)

Default priority mapping when priority is not set:

Event / Level Priority Behaviour
cleared / holddown -1 Quiet
Level 1 0 Normal
Level 2 1 High
Level 3+ 2 Emergency

Priority range: -2 (silent) to 2 (emergency). Emergency priority requires acknowledgement in the Pushover app and retries every 60 seconds for up to 1 hour.

Message body includes Rule: <name> after the group name when a named rule triggered the mitigation. For group-level mitigations the rule line is omitted. A Threshold: BPS ≥ … / PPS ≥ … line appears after the level number when a BPS or PPS threshold is configured for that escalation level. For FlowSpec mitigations a FlowSpec Rules (N): block is appended before the Attack ID, listing each announced source rule on its own line.

slack

Sends a message to a Slack channel via an Incoming Webhook. Uses Block Kit with a colored left-border attachment: red for activated/escalated, orange for holddown, green for cleared.

- type: slack
  name: ops-slack
  url: "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX"

Setup:

  1. Go to api.slack.com/apps and create a new app (or use an existing one).
  2. Under Features → Incoming Webhooks, toggle Activate Incoming Webhooks on.
  3. Click Add New Webhook to Workspace, pick the target channel, and copy the webhook URL.
  4. Paste the URL as url in the target config.

Message layout (Block Kit attachment with colored sidebar):

DDoS activated: 203.0.113.10
─────────────────────────────────────────
Victim          203.0.113.10
Group           datacenter-a
Rule            udp-amplification   ← only shown when a named rule triggered the mitigation
Level           1
Threshold       BPS ≥ 1.20 Gbps    ← omitted if no threshold configured for this level
Mitigation      flowspec
BPS             1.20 Gbps
PPS             75.00 Kpps
FlowSpec Rules (2):                 ← only shown for FlowSpec mitigations
  #1: `dst=203.0.113.10 src=1.2.3.4 proto=UDP dport=53 → discard`
  #2: `dst=203.0.113.10 src=5.6.7.8 proto=UDP dport=53 → discard`
─────────────────────────────────────────
Attack ID: `a3f1c2d4e5b6a7f8`

teams

Sends a MessageCard to an MS Teams channel via an Incoming Webhook. The card header color reflects the event type: red for activated/escalated, orange for holddown, green for cleared.

- type: teams
  name: ops-teams
  url: "https://outlook.office.com/webhook/XXXXXXXX/IncomingWebhook/YYYYYYYY/ZZZZZZZZ"

Setup:

  1. In Teams, open the channel you want to post to.
  2. Click the three-dot menu next to the channel name → Connectors.
  3. Find Incoming Webhook, click Configure, give it a name, and copy the URL.
  4. Paste the URL as url in the target config.

Note: Microsoft is deprecating Office 365 Connectors in favour of Power Automate Workflows. If your tenant has disabled legacy connectors, create a Workflow using the "Post to a channel when a webhook request is received" template and use the Workflow's webhook URL instead - the MessageCard payload is compatible.

Card fields:

Field Value
Victim victim IP
Group group name
Rule rule name (omitted if group-level)
Level escalation level
Threshold configured threshold, e.g. BPS ≥ 1.20 Gbps (omitted if not configured)
Mitigation mitigation type
BPS human-readable (e.g. 1.20 Gbps)
PPS human-readable (e.g. 75.00 Kpps)
FS Rule #1, #2, … one fact per FlowSpec rule, e.g. dst=203.0.113.10 src=1.2.3.4 proto=UDP dport=53 → discard (omitted for non-FlowSpec mitigations)
Attack ID hex attack identifier

telegram

Sends an HTML-formatted message via the Telegram Bot API.

- type: telegram
  name: noc-bot
  token: "123456:ABCdefGHIjklMNOpqrSTUvwxYZ"
  chat_id: "-1001234567890"

Setup:

  1. Create a bot via @BotFather and copy the token.
  2. Add the bot to your channel or group and obtain the chat_id (use getUpdates or a helper bot).

Message format:

DDoS Alert: activated
Victim: 203.0.113.10
Group: datacenter-a
Rule: udp-amplification   ← only shown when a named rule triggered the mitigation
Level: 1
Threshold: BPS ≥ 1.20 Gbps   ← omitted if no threshold is configured for this level
Mitigation: flowspec
BPS: 1.20 Gbps
PPS: 75.00 Kpps
FlowSpec Rules (2):           ← only shown for FlowSpec mitigations
  #1: dst=203.0.113.10 src=1.2.3.4 proto=UDP dport=53 → discard
  #2: dst=203.0.113.10 src=5.6.7.8 proto=UDP dport=53 → discard
Attack ID: a3f1c2d4e5b6a7f8

zammad

Opens a Zammad ticket when an attack is detected and closes it when the attack clears. Escalation and hold-down events are appended as internal notes on the existing ticket.

- type: zammad
  name: noc-zammad
  url: "https://zammad.example.com"
  api_token: "xxxxxxxxxxxxxxxxxxxx"  # Zammad API token
  group: "Users"                     # Zammad ticket group
  customer: "noc@example.com"        # customer email for new tickets
  close_state: "closed"              # optional; default: "closed"
  tags: [flowwler, ddos-response]    # optional; tags set on the ticket at creation

Setup:

  1. In Zammad, go to Profile → Token Access and create a token with the ticket.agent role.
  2. Identify the target group name (visible under Manage → Groups).
  3. Supply a customer email - this must be an existing Zammad user or an email that Zammad can auto-create.
  4. Check the available state names under Manage → Ticket States - the close_state value must match exactly (case-sensitive). Common names: closed, resolved.

Behaviour by event type:

Event Action
activated (first time) Creates a new ticket with a plain-text description (victim, group, level, traffic, FlowSpec rules if applicable). Priority is set to 2 normal. Tags default to flowwler and ddos; configuring tags replaces those defaults.
activated (re-activation) If traffic recovered during hold-down (same attack_id), appends an internal note instead of opening a duplicate ticket: "Traffic recovered during hold-down - mitigation remains active."
escalated Appends an internal note: new level, traffic rate, mitigation type. Updates the ticket priority to 3 high.
holddown Appends an internal note that the attack has subsided.
cleared Appends a final note with the peak traffic, then sets the ticket state to close_state.

Authentication uses Authorization: Token token=<api_token> (Zammad's token scheme - not HTTP Basic). Ticket state is tracked in memory (keyed by attack_id). If the daemon is reloaded while an attack is active the new target instance will not find the open ticket, so escalation and cleared events for that attack will be silently skipped for the Zammad backend only.

webhook

HTTP POST with a JSON body matching the event schema above.

- type: webhook
  name: my-hook
  url: "https://example.com/flowwler-hook"
  headers:
    Authorization: "Bearer secret"
  events: [activated, cleared]

The headers map is merged into every request. BPS and PPS are raw integers in the JSON body. For FlowSpec mitigations the body also includes a flowspec_rules array containing one object per announced NLRI (see schema above).

Testing a target

Use the REST API to send a synthetic test notification to any configured target without waiting for a real attack:

curl -s -X POST http://[::]:9731/api/v1/notifications/targets/my-target/test | jq .

The request bypasses event-type filters and sends a canned activated event. An optional JSON body overrides victim_ip (default 192.0.2.1) and group_name (default test). The target's health state is updated on success or failure and is visible in GET /api/v1/notifications/targets. See REST API - POST /api/v1/notifications/targets/{name}/test for full details.

Delivery guarantees

Each target is called concurrently in its own goroutine with a configurable timeout (default 10s). Failed sends are retried up to retry_count times (default 0). Events that exhaust all retries are dropped and logged at warn level. There is no queue or persistence - if the process restarts during an attack, in-progress events are lost.