EEM as a Guardrail

This is Part 4 of a 5-part series on Cisco EEM. So far we’ve used EEM reactively — Part 2 to heal, Part 3 to capture. This part flips the posture from reactive to preventive: EEM as a guardrail that catches mistakes before they land and makes every change on the device accountable. If you work anywhere audited — PCI-DSS, SOX, HIPAA — this is the part that earns EEM a permanent place in your baseline.

Most automation is about doing things. Guardrails are about catching things — the fat-fingered write erase, the change nobody logged, the quiet drift away from your hardened baseline that an auditor finds before you do. This is a different use of EEM from the previous parts, and arguably the highest-value one in a regulated environment, because it turns the device itself into a witness and a gatekeeper.

A guardrail isn’t a lock. It doesn’t stop a determined admin with enable, and it isn’t a substitute for TACACS+ command authorization or RBAC. What it does is make the dangerous thing harder to do by accident, impossible to do invisibly, and automatically reported when it happens. In compliance terms: it adds detective and deterrent controls at the device level that complement the preventive controls you already have upstream. Auditors like layered controls, and "the device logs and attributes every configuration change to a named user in real time" is a sentence that makes an assessment go smoothly.

Let’s build that.

Guardrail 1: Intercepting a destructive command

The canonical example is write erase (or erase startup-config), the command that wipes a device’s saved configuration. There’s almost never a good reason to run it interactively on a production box, and a spectacularly bad day when someone does on the wrong device. The event cli trigger lets EEM see the command as it’s typed, before it executes, and interpose.

event manager applet GUARD-WRITE-ERASE
 event cli pattern "write erase" sync yes skip yes
 action 1.0 syslog priority alerts msg "EEM GUARD: 'write erase' entered by $_cli_username from $_cli_tty - BLOCKED"
 action 2.0 puts "=================================================="
 action 3.0 puts "  BLOCKED by EEM guardrail."
 action 4.0 puts "  'write erase' is disabled on this device."
 action 5.0 puts "  Contact NetOps if this is genuinely required."
 action 6.0 puts "=================================================="

The mechanics that matter:

• sync yes makes the applet run synchronously — the command is held until the applet completes, so the applet’s outcome can decide the command’s fate.
• skip yes is the key: it tells IOS to skip execution of the original command entirely. The write erase never runs. You’ve converted a destructive command into a logged, blocked event with a helpful message. (If you want to allow it after logging, use set $_exit_status 1 to permit execution instead — the applet becomes an audit trail rather than a block.)
• $_cli_username and $_cli_tty capture who typed it and from where. This is the accountability core: the alert-severity syslog names the user and their line, and ships that to your SIEM. Even if you choose to allow the command, you now have an attributed record that it happened.

The same pattern generalizes to any command you want to gate or record: reload without a reason, format flash:, no aaa new-model (someone disabling AAA is a red flag worth alerting on immediately), clearing counters an auditor relies on. Pick your list deliberately — over-guarding turns into friction — but the destructive and security-relevant commands are strong candidates.

A caution: event cli guardrails are a safety net, not a security boundary. A knowledgeable admin can craft variations that don’t match your pattern, and EEM applets can be removed by someone with enable. Treat these as protection against accident and as an audit/deterrent layer — not as something that stops a malicious insider. Your actual authorization control is TACACS+ command authorization; the guardrail complements it.

Guardrail 2: Attributing every configuration change

Here’s a compliance requirement nearly every framework shares in some form: know who changed what, and when. IOS emits a syslog message every time someone leaves configuration mode with an actual change:

%SYS-5-CONFIG_I: Configured from console by <user> on vty0 (192.168.99.20)

That message is the hook for a real-time change-attribution guardrail. Every config change triggers an applet that snapshots the config, records the user, and notifies your monitoring/SIEM — turning the device into an active participant in your change record rather than something you reconcile after the fact.

event manager applet GUARD-CONFIG-CHANGE
 event syslog pattern "%SYS-5-CONFIG_I" ratelimit 10
 action 1.0 syslog priority notifications msg "EEM GUARD: config change detected - $_syslog_msg"
 action 2.0 cli command "enable"
 action 3.0 set fname "flash:cfg-change-$_event_pub_sec.txt"
 action 4.0 cli command "term length 0"
 action 5.0 cli command "show clock | append $fname"
 action 6.0 cli command "show running-config | append $fname"
 action 7.0 cli command "term length 24"
 action 8.0 cli command "copy $fname scp://netops@192.168.37.50/config-changes/ " pattern "assword"
 action 9.0 cli command "<use-key-auth>"
 action 10.0 syslog priority notifications msg "EEM GUARD: post-change config snapshot saved and offloaded"

How this earns its keep:

• It fires the moment a change is committed, capturing the resulting config with a timestamp — so you have a point-in-time record independent of your NMS’s polling cycle. If someone makes a change and reverts it before your config-management tool polls, this still caught it.
• Paired with the %SYS-5-CONFIG_I message (which names the user and source IP), you get attribution plus content — who, when, from where, and the resulting state.
• Shipping to a remote store means the record lives off the device, where it survives a wipe and can’t be edited by whoever made the change. That off-box, tamper-evident trail is exactly what an auditor wants to see.

This complements rather than replaces a proper config-management/NCM system (like SolarWinds NCM or similar) — but it adds a device-side, real-time layer that catches things between polling intervals and doesn’t depend on the management network being up.

A refinement worth adding: also trigger on %SYS-6-LOGOUT or archive configure events, and consider enabling the IOS configuration archive with logging enable under archive, which gives you native config diffs. EEM can then capture the diff rather than the whole config, which is far more useful in a change record. More on that next.

Guardrail 3: Native config archive + EEM = attributed diffs

IOS has a built-in configuration archive that, when enabled, keeps versioned copies of running-config and can log the exact commands each user entered. This is the single highest-value thing you can turn on for change accountability, and EEM makes it actionable.

First, the archive itself (this is native IOS, not EEM):

archive
 path flash:archive/$h-config
 maximum 14
 write-memory
 log config
  logging enable
  logging size 500
  hidekeys
  notify syslog contenttype plaintext

What this gives you:

• log config / logging enable records every configuration command entered, with the username and timestamp — a per-command audit trail, not just "something changed."
• hidekeys keeps secrets out of that log (important — you don’t want passwords in a plaintext audit log).
• notify syslog emits each logged command as a syslog message, which becomes an EEM hook and a SIEM feed.
• write-memory archives a fresh copy on every copy run start.

Now EEM can react to the per-command log messages for high-sensitivity changes. For example, alert immediately if anyone touches an access-list, an AAA command, or SNMP config — the things that, changed wrongly, open a hole:

event manager applet GUARD-SENSITIVE-CONFIG
 event syslog pattern "%PARSER-5-CFGLOG_LOGGEDCMD.*(access-list|aaa |snmp-server|username|crypto )" ratelimit 5
 action 1.0 syslog priority critical msg "EEM GUARD: sensitive config command logged - $_syslog_msg"
 action 2.0 syslog priority critical msg "EEM GUARD: review required - security-relevant change"

This is genuinely powerful in a PCI context: a change to an ACL that protects the cardholder data environment, or to the AAA config that controls access, generates an immediate critical alert naming the command and the user. You’re not waiting for a quarterly review to find it — you know within seconds. The %PARSER-5-CFGLOG_LOGGEDCMD message is what the archive’s log config produces; matching specific sensitive keywords in it lets you tier your alerting by risk.

Guardrail 4: Baseline drift detection

Change attribution catches changes as they happen. Drift detection catches the state having quietly diverged from your intended baseline — whether from an unlogged change, a failed automation run, or a device that was never brought fully into standard. This is a scheduled guardrail: on a timer, compare current state to expected and alert on divergence.

The simplest useful version checks that specific required baseline elements are still present. Here’s a drift check that verifies a handful of hardening lines exist and alerts if any are missing:

event manager applet GUARD-BASELINE-DRIFT
 event timer cron cron-entry "0 6 * * *"
 action 1.0 cli command "enable"
 action 2.0 cli command "show running-config all | include ^service password-encryption"
 action 3.0 string match "*service password-encryption*" "$_cli_result"
 action 4.0 if $_string_result eq "0"
 action 5.0  syslog priority critical msg "EEM DRIFT: service password-encryption MISSING from baseline"
 action 6.0 end
 action 7.0 cli command "show running-config | include aaa new-model"
 action 8.0 string match "*aaa new-model*" "$_cli_result"
 action 9.0 if $_string_result eq "0"
 action 10.0  syslog priority critical msg "EEM DRIFT: aaa new-model MISSING - AAA may be disabled"
 action 11.0 end
 action 12.0 syslog priority notifications msg "EEM DRIFT: daily baseline check complete"

The logic pattern here — run a show ... | include, test the result with string match, alert with an if/end block — is the workhorse of drift detection. You extend it by adding a check per baseline element you care about: NTP configured, SNMPv3 (not v2c) in use, logging destination present, login banner set, specific ACLs applied to VTY, and so on.

For a PCI-aligned baseline, the high-value drift checks are the ones that map to requirements: AAA present, encryption of secrets, secure management protocols (SSH not Telnet, SNMPv3 not v2c), logging enabled and pointed at your collector, and NTP synchronized. A device silently drifting off any of those is a finding waiting to happen — and now it pages you at 06:00 instead of surprising you during an assessment.

For richer comparisons than include checks, EEM can diff the running config against a stored golden copy — but that’s often better handled by dropping to Tcl (Part 5) or by your NCM platform doing full-config compliance. The EEM include-check approach shines for the critical must-exist elements where you want device-side, real-time-ish assurance independent of the management plane.

Guardrail 5: Unauthorized-access alerting

A security-relevant guardrail: react to authentication failures and successful logins from unexpected places. Excessive failed logins can indicate a brute-force attempt; a successful enable from an unusual source deserves at least a log. EEM can watch the auth syslog and escalate.

event manager applet GUARD-AUTH-FAILURES
 event syslog pattern "%SEC_LOGIN-4-LOGIN_FAILED" occurs 5 period 60
 action 1.0 syslog priority critical msg "EEM GUARD: 5+ login failures in 60s - possible brute force"
 action 2.0 cli command "enable"
 action 3.0 set fname "flash:authfail-$_event_pub_sec.txt"
 action 4.0 cli command "show clock | append $fname"
 action 5.0 cli command "show logging | include LOGIN | append $fname"
 action 6.0 cli command "show users | append $fname"
 action 7.0 syslog priority critical msg "EEM GUARD: auth-failure burst captured to $fname - review source"

The occurs 5 period 60 clause is the important part: the applet fires only when the pattern matches 5 times within 60 seconds, not on every single failure. That’s what distinguishes a fat-fingered password (one or two failures) from a genuine brute-force pattern (many in a short window). This built-in occurrence/period counting is one of EEM’s most underused features and it’s perfect for security event thresholds — no external correlation engine required for the device-local case.

The guardrail deployment checklist

• Guardrails complement, never replace, real controls. TACACS+ command authorization, RBAC, and change management are your primary controls. EEM guardrails are detective/deterrent layers on top. Don’t let a guardrail create false confidence.
• Ship everything to your SIEM/syslog collector. A guardrail whose alerts only land in the local buffer is half a control. The off-box, tamper-evident trail is the point — especially for audit.
• hidekeys and no plaintext secrets. Any guardrail that logs or archives config must not leak credentials into logs. Verify your archive and captures respect this.
• Don’t over-guard. Blocking or alerting on too many commands creates friction and alert fatigue, which gets guardrails disabled. Scope to genuinely destructive and security-relevant actions.
• Protect the guardrails themselves. An applet that can be removed with enable is only as strong as your access control. Consider alerting on changes to EEM config itself (event syslog pattern matching event manager changes) so someone disabling a guardrail trips an alarm.
• Map checks to requirements. In a regulated environment, tie each drift check and each attribution guardrail to the specific control it supports. That mapping is what makes them defensible in an audit and easy to justify keeping.
• Test the alert path end to end. Trigger each guardrail deliberately and confirm the alert actually arrives at the SIEM, not just the local log. An untested alert path is an assumption.

Why this matters more than it looks

The reactive uses of EEM in the earlier parts save you time and sleep. Guardrails save you from categories of incident — the accidental wipe, the untracked change, the silent drift, the brute-force nobody noticed. In a regulated environment they do something more: they turn each device into an active enforcer and witness of your control framework, generating the attributed, off-box, real-time evidence that audits are built on.

The mental shift is from "EEM reacts to problems" to "EEM prevents and records them." Same primitives — events, actions, syslog, thresholds — pointed at the goal of making the wrong thing hard to do by accident and impossible to do invisibly. That’s a guardrail, and every device you own can have one.

---

What command would you block or alert on first in your environment? And has an auditor ever found drift you wish you’d caught yourself? Those stories are exactly why device-side guardrails earn their keep.

Read my articles on DEV.to