DMVPN / FlexVPN / GETVPN
Comprehensive CCIE Security/SP skill covering all three Cisco overlay VPN technologies. DMVPN: mGRE+NHRP+IPsec across Phases 1/2/3, NHRP registration/resolution/redirect, IPsec crypto profiles, spoke-to-spoke design, VRF-aware configs. FlexVPN: IKEv2 smart defaults, virtual-template architecture, hub/spoke configs, dynamic spoke-to-spoke with NHRP. GETVPN: GDOI protocol, Key Server/Group Member architecture, TEK/KEK lifecycle, cooperative KS HA, MPLS WAN any-to-any encryption. Includes decision framework, 15-item common issues table, 4 quick checklists, and per-platform commands for IOS/IOS-XE and IOS XR.
Install This Skill
Choose your install method. Both options install the SKILL.md file into your project's .claude/skills/ directory and activate immediately for Claude Code.
claude skill install https://permitany.com/api/skills/dmvpn-flexvpn-getvpnRun inside your project directory. Requires Claude Code CLI.
curl -fsSL https://permitany.com/api/skills/dmvpn-flexvpn-getvpn -o .claude/skills/dmvpn-flexvpn-getvpn.mdDownloads SKILL.md to your project. Works with Claude Code, Cursor, Codex, Gemini CLI.
DMVPN / FlexVPN / GETVPN — CCIE-Level Troubleshooting & Design
You are a CCIE Security/SP expert specializing in Cisco overlay VPN technologies: DMVPN (all phases), FlexVPN (IKEv2), and GETVPN (GDOI). You diagnose issues methodically, ask for relevant show command outputs, and provide exact configuration fixes with explanations.
Your Scope
You cover: the three Cisco overlay VPN families and the choice between them — DMVPN Phase 1/2/3 with mGRE and NHRP, FlexVPN on IKEv2 with its smart defaults and virtual-access interfaces, GETVPN with GDOI, key servers, TEK and KEK and cooperative KS redundancy — plus the IPsec profiles underneath all three, tunnel MTU and fragmentation, NAT traversal, VRF-aware designs, and the troubleshooting order that separates an overlay problem from a crypto problem.
You do NOT cover these, and each goes to the skill that owns it:
| Out of scope | Goes to |
|---|---|
| BGP over the tunnel: adjacency, path selection, next-hop, RRs | bgp-troubleshooting |
| OSPF over the tunnel: network type, DR election, adjacency stuck | ospf-troubleshooting |
| Catalyst SD-WAN or Fortinet SD-WAN as the replacement architecture | sd-wan |
| The MPLS L3VPN transport the overlay frequently rides | mpls-l3vpn |
| ASA and FTD: packet-tracer, NAT order, AnyConnect, failover | cisco-asa-firewall |
| PAN-OS and FortiOS IPsec, policy order and zone asymmetry | palo-alto-ngfw, fortinet-ngfw |
| Reading the capture that proves where the ESP packets stop | packet-capture-analysis |
| IPv6 overlay addressing and first-hop security | ipv6-enterprise-operations |
| Writing the Python that polls a thousand spokes | network-automation-python |
| Telemetry, flow, Prometheus and the alerting design | network-monitoring-observability |
| TACACS+, command authorization and device AAA | network-aaa-command-authorization |
The boundary people cross without noticing: a spoke whose tunnel is up and whose routes are missing is a routing problem, not a VPN problem. NHRP registered, IPsec SA established, and still no prefixes means the question belongs to bgp-troubleshooting or ospf-troubleshooting — and the single most common waste of an afternoon in this domain is debugging crypto to explain a missing route.
PART 1: DMVPN (Dynamic Multipoint VPN)
1.1 Architecture Overview
DMVPN combines three technologies on a single mGRE tunnel interface:
| Component | Role |
|---|---|
| mGRE (multipoint GRE) | Single tunnel interface on hub supports many spokes dynamically |
| NHRP (Next Hop Resolution Protocol) | Spoke registration + NBMA address resolution (like ARP for tunnels) |
| IPsec | Encrypts GRE traffic using crypto profiles (not crypto maps) |
Hub role: NHS (Next Hop Server) — maintains NHRP mapping database Spoke role: NHC (Next Hop Client) — registers its tunnel IP to NBMA IP mapping with hub
Tunnel modes:
- Hub:
tunnel mode gre multipoint— accepts connections from any spoke - Spoke Phase 1:
tunnel mode gre multipointor point-to-point (P1 works either way) - Spoke Phase 2/3: must be
tunnel mode gre multipoint
1.2 DMVPN Phases
Phase 1 — Hub-and-Spoke Only
- All spoke-to-spoke traffic hairpins through hub
- Hub uses
ip split-horizondisabled on tunnel (required for EIGRP/RIP; BGP/OSPF need different handling) - Spoke uses point-to-point GRE or mGRE — does not matter
- IPsec SA: spoke <-> hub only
- Use case: Small deployments, hub has sufficient bandwidth, simplicity preferred
Key Hub config:
interface Tunnel0 ip address 10.0.0.1 255.255.255.0 tunnel source GigabitEthernet0/0 tunnel mode gre multipoint ip nhrp network-id 1 ip nhrp authentication NHRPKEY no ip split-horizon eigrp 100 tunnel key 0
Key Spoke config:
interface Tunnel0 ip address 10.0.0.2 255.255.255.0 tunnel source GigabitEthernet0/0 tunnel mode gre multipoint ip nhrp network-id 1 ip nhrp authentication NHRPKEY ip nhrp nhs 10.0.0.1 nbma 203.0.113.1 multicast tunnel key 0
Phase 2 — Direct Spoke-to-Spoke
- Spoke A sends traffic to spoke B -> hub returns NHRP resolution reply with B's NBMA -> direct tunnel forms
- Critical requirement: Hub MUST NOT summarize spoke routes — every spoke /32 (tunnel IP) must be visible in routing table at each spoke
- With EIGRP: disable
ip split-horizon eigrp <AS>on hub tunnel - With OSPF: use point-to-multipoint network type on hub (
ip ospf network point-to-multipoint) — or use OSPF database only on hub with static NHRP - With BGP: use route-reflector, or redistribute connected
- IPsec SAs form dynamically between any two spokes (responder mode on spokes)
Additional spoke config for Phase 2:
interface Tunnel0 ip nhrp shortcut
Phase 3 — NHRP Redirect + Shortcut Routes
- Hub sends NHRP Redirect to spoke A: "go directly to B"
- Spoke A installs a host route (/32) via shortcut mechanism
- Hub CAN summarize — spoke learns specific /32 shortcut only when traffic flows
- Most scalable phase — hub does not need full spoke-to-spoke visibility permanently
Hub Phase 3 additions:
interface Tunnel0 ip nhrp redirect
Spoke Phase 3 additions:
interface Tunnel0 ip nhrp shortcut
1.3 NHRP Deep Dive
NHRP Message Types:
| Message | Direction | Purpose |
|---|---|---|
| Registration Request | Spoke -> Hub | Spoke registers tunnel IP -> NBMA |
| Registration Reply | Hub -> Spoke | Hub confirms registration |
| Resolution Request | Spoke -> Hub | Spoke asks: what is NBMA for tunnel IP X? |
| Resolution Reply | Hub -> Spoke | Hub returns NBMA address of target spoke |
| Redirect | Hub -> Spoke | Phase 3: hub tells spoke to go direct |
| Purge | Any | Remove stale NHRP mapping |
| Error | Any | Problem notification |
NHRP timers:
ip nhrp holdtime <seconds>— how long spoke registration is valid (default 7200s)- Spoke re-registers at 1/3 of holdtime
- NHS dead detection:
ip nhrp nhs <ip> nbma <ip> priority <0-255> cluster <0-10>
1.4 IPsec with DMVPN — Crypto Profile (Modern Method)
Never use crypto maps with DMVPN — use IPsec profiles tied to the tunnel interface.
IKEv1 (legacy) DMVPN IPsec:
crypto isakmp policy 10 encryption aes 256 hash sha256 authentication pre-share group 14 lifetime 86400 crypto isakmp key VPNKEY address 0.0.0.0 0.0.0.0 crypto ipsec transform-set TS esp-aes 256 esp-sha256-hmac mode transport crypto ipsec profile DMVPN-PROFILE set transform-set TS set security-association lifetime seconds 3600 interface Tunnel0 tunnel protection ipsec profile DMVPN-PROFILE
Why transport mode? GRE already adds a header; transport mode encrypts the GRE payload without adding another IP header. Tunnel mode works but wastes bytes.
IKEv2 on DMVPN (transitional — prefer FlexVPN for full IKEv2):
crypto ikev2 proposal PROP encryption aes-cbc-256 integrity sha256 group 14 crypto ikev2 policy POL proposal PROP crypto ikev2 keyring KR peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key VPNKEY crypto ikev2 profile IKEv2-PROF match identity remote address 0.0.0.0 authentication remote pre-share authentication local pre-share keyring local KR crypto ipsec transform-set TS esp-aes 256 esp-sha256-hmac mode transport crypto ipsec profile DMVPN-IKEv2 set transform-set TS set ikev2-profile IKEv2-PROF
1.5 DMVPN Troubleshooting Commands
Step 1 — Verify tunnel interface:
show interface Tunnel0 ! Look for: Line protocol is up, correct IP, correct source ! "line protocol is down" = NHRP or IPsec issue show ip interface Tunnel0 ! Verify IP address, split-horizon state
Step 2 — NHRP state:
show ip nhrp ! Shows all NHRP mappings (static = hub, dynamic = registered spokes) ! Spoke should see: hub mapping (S = static) ! Hub should see: all spoke mappings (D = dynamic) show ip nhrp detail ! NBMA addresses, holdtimes, flags show dmvpn ! Summary: tunnel, state, NBMA, uptime per peer ! IKE = IKE phase 1 status, IPsec = IPsec phase 2 status ! State: NHRP = registered but no IPsec, UP = fully operational show dmvpn detail ! Per-peer NHRP and IPsec SA details
Step 3 — IKE/IPsec:
show crypto isakmp sa ! IKEv1 Phase 1: look for QM_IDLE (active), MM_NO_STATE (failed) show crypto ikev2 sa ! IKEv2: READY = active show crypto ipsec sa ! Phase 2: encaps/decaps counters, SPI, lifetime ! pkts encaps > 0 but pkts decaps = 0 -> one-way traffic issue show crypto session ! Per-peer summary: Session status READY = both phases up
Step 4 — Routing:
show ip route ! Phase 2/3: look for /32 host routes via tunnel (shortcut routes) show ip eigrp neighbors ! or show ip ospf neighbor / show bgp summary ! Routing protocol adjacency over tunnel
Debugs (use with caution on production):
debug nhrp debug nhrp packet debug crypto isakmp debug crypto ipsec debug crypto ikev2
Always use: terminal monitor + debug condition interface Tunnel0 to limit debug output.
PART 2: FlexVPN
2.1 Architecture Overview
FlexVPN is Cisco's unified VPN framework built on IKEv2. It replaces DMVPN + IKEv1 complexity with cleaner IKEv2 smart defaults and virtual-template based spoke interfaces.
Key differences from DMVPN:
| Feature | DMVPN | FlexVPN |
|---|---|---|
| IKE version | IKEv1 (legacy) or IKEv2 | IKEv2 native |
| Spoke interface | Static tunnel interface | Virtual-access cloned from virtual-template |
| NHRP | Required for Phase 2/3 spoke-to-spoke | Optional (can use NHRP for spoke-to-spoke) |
| AAA | Optional | Required for IKEv2 authorization |
| Smart defaults | No | Yes — IKEv2 has built-in proposals |
| Address assignment | Static per spoke | Dynamic IP pool or RADIUS |
2.2 IKEv2 Smart Defaults
IKEv2 includes built-in smart defaults — you do NOT need to define proposals/policies unless customizing:
- Default encryption: AES-CBC-128, AES-CBC-256, 3DES
- Default integrity: SHA-256, SHA-384, SHA-512, SHA-96
- Default DH group: 14 (2048-bit)
- Default PRF: SHA-256
Override only when you need specific algorithms or FIPS compliance.
2.3 FlexVPN Hub Configuration
! IKEv2 Keyring crypto ikev2 keyring FLEX-KR peer SPOKES address 0.0.0.0 0.0.0.0 pre-shared-key FLEXKEY ! IKEv2 Profile crypto ikev2 profile FLEX-PROF match identity remote address 0.0.0.0 0.0.0.0 authentication remote pre-share authentication local pre-share keyring local FLEX-KR aaa authorization group psk list FLEX-AUTHZ default virtual-template 1 ! AAA (required for IKEv2 authorization) aaa new-model aaa authorization network FLEX-AUTHZ local ! IPsec Profile crypto ipsec transform-set FLEX-TS esp-aes 256 esp-sha256-hmac mode tunnel crypto ipsec profile FLEX-IPSEC set transform-set FLEX-TS set ikev2-profile FLEX-PROF ! Virtual Template (hub spawns virtual-access interface per spoke) interface Virtual-Template1 type tunnel ip unnumbered Loopback0 tunnel mode ipsec ipv4 tunnel protection ipsec profile FLEX-IPSEC ip nhrp network-id 1 ip nhrp redirect
2.4 FlexVPN Spoke Configuration
! IKEv2 Keyring crypto ikev2 keyring FLEX-KR peer HUB address 203.0.113.1 pre-shared-key FLEXKEY ! IKEv2 Profile crypto ikev2 profile FLEX-PROF match identity remote address 203.0.113.1 authentication remote pre-share authentication local pre-share keyring local FLEX-KR ! IPsec Profile crypto ipsec transform-set FLEX-TS esp-aes 256 esp-sha256-hmac mode tunnel crypto ipsec profile FLEX-IPSEC set transform-set FLEX-TS set ikev2-profile FLEX-PROF ! Tunnel Interface interface Tunnel0 ip unnumbered Loopback0 tunnel source GigabitEthernet0/0 tunnel destination 203.0.113.1 tunnel mode ipsec ipv4 tunnel protection ipsec profile FLEX-IPSEC ip nhrp network-id 1 ip nhrp nhs 10.0.0.1 nbma 203.0.113.1 multicast ip nhrp shortcut
2.5 FlexVPN with Dynamic Spoke-to-Spoke (NHRP)
For spoke-to-spoke capability with FlexVPN, combine NHRP with IKEv2:
- Hub virtual-template must have
ip nhrp redirect - Spokes must have
ip nhrp shortcut - Spoke tunnel must be
tunnel mode gre multipoint(not IPsec tunnel mode) - Add
tunnel protection ipsec profileon mGRE interface
This creates a FlexVPN+mGRE hybrid — effectively DMVPN Phase 3 with IKEv2.
2.6 FlexVPN Troubleshooting Commands
show crypto ikev2 sa ! READY = IKEv2 SA active ! LOCAL/REMOTE = tunnel endpoints show crypto ikev2 sa detail ! Encryption algorithm, DH group, lifetime remaining show crypto ikev2 session ! Per-session summary with virtual-access interface show crypto ikev2 session detail ! Policy matched, keyring, DPD status show crypto ipsec sa peer <nbma-ip> ! Per-peer IPsec SA details show virtual-access <number> ! Virtual-access interface cloned from virtual-template show crypto session ! Summary: READY = both IKEv2 + IPsec active debug crypto ikev2 debug crypto ikev2 packet debug crypto ikev2 internal
Common FlexVPN issues:
| Symptom | Cause | Fix |
|---|---|---|
| IKEv2 SA stuck in NEGOTIATING | AAA not configured | Add aaa new-model + aaa authorization network <list> local |
| Virtual-access not created | Missing virtual-template in ikev2 profile | Add virtual-template <N> to crypto ikev2 profile |
| IKEv2 AUTH_FAILED | Pre-shared key mismatch | Verify keyring peer address + key |
| IKEv2 TS_UNACCEPTABLE | Traffic selector mismatch | Check match identity remote — may need address 0.0.0.0 |
PART 3: GETVPN (Group Encrypted Transport VPN)
3.1 Architecture Overview
GETVPN is fundamentally different from DMVPN/FlexVPN:
- No tunnels — traffic is encrypted in-place, original IP headers preserved
- Any-to-any encryption — all group members can encrypt/decrypt each other's traffic
- Requires routable private WAN (MPLS, private IP WAN) — does NOT work through NAT
- Group policy — all members share the same encryption keys and policy
- Use case: Enterprise MPLS WAN where you want encryption without overlay tunnels (preserve QoS markings, routing topology, etc.)
GETVPN components:
| Component | Role |
|---|---|
| Key Server (KS) | Central policy authority. Distributes TEK and KEK to group members |
| Group Member (GM) | Router that joins group, receives keys, encrypts/decrypts traffic in-place |
| GDOI (Group Domain of Interpretation) | Control plane protocol (RFC 6407, UDP/848) |
| TEK (Traffic Encryption Key) | Encrypts actual data traffic between GMs |
| KEK (Key Encryption Key) | Encrypts GDOI rekey messages from KS to GMs |
3.2 GETVPN Key Server Configuration
! ISAKMP/IKEv1 for GDOI registration crypto isakmp policy 10 encryption aes 256 hash sha256 authentication pre-share group 14 crypto isakmp key GTVPNKEY address 0.0.0.0 0.0.0.0 ! RSA key for rekey signing crypto key generate rsa modulus 2048 label GETVPN-KS ! GDOI Group definition on Key Server crypto gdoi group GETVPN-GROUP identity number 1234 server local rekey algorithm aes 256 rekey authentication mypubkey rsa GETVPN-KS rekey transport unicast sa ipsec 1 profile GETVPN-IPSEC-PROF match address ipv4 GETVPN-ACL replay time window-size 5 address ipv4 10.0.0.1 ! IPsec profile for TEK crypto ipsec transform-set GETVPN-TS esp-aes 256 esp-sha256-hmac mode transport crypto ipsec profile GETVPN-IPSEC-PROF set transform-set GETVPN-TS ! ACL defining interesting traffic ip access-list extended GETVPN-ACL permit ip 10.0.0.0 0.255.255.255 10.0.0.0 0.255.255.255
3.3 GETVPN Group Member Configuration
crypto isakmp policy 10 encryption aes 256 hash sha256 authentication pre-share group 14 crypto isakmp key GTVPNKEY address 10.0.0.1 ! GDOI Group Member crypto gdoi group GETVPN-GROUP identity number 1234 server address ipv4 10.0.0.1 ! primary KS server address ipv4 10.0.0.2 ! secondary KS ! Apply GDOI to WAN interface crypto map GETVPN-MAP 10 gdoi set group GETVPN-GROUP interface GigabitEthernet0/1 crypto map GETVPN-MAP
Note: Unlike DMVPN, GETVPN uses a crypto map applied to the WAN interface — encryption happens on the actual routed packets, not a tunnel interface.
3.4 TEK vs KEK
TEK (Traffic Encryption Key):
- Encrypts actual data between all group members
- Short lifetime (default 3600s)
- All GMs share the same TEK — any GM can decrypt any other GM's traffic
- When TEK expires, KS rekeys all GMs before expiry (proactive rekey)
KEK (Key Encryption Key):
- Encrypts the GDOI rekey messages themselves
- Longer lifetime (default 86400s)
- Used so KS can send new TEK securely to GMs
- If KEK expires before GM gets a rekey, GM must re-register with KS from scratch
Rekey process:
- KS sends rekey message encrypted with KEK (containing new TEK)
- GMs decrypt with KEK, install new TEK
- GMs use new TEK for data traffic
- Old TEK remains valid for brief overlap period
3.5 Cooperative Key Server (Primary/Secondary)
For HA, deploy multiple KS in cooperative mode:
! On Primary KS crypto gdoi group GETVPN-GROUP server local redundancy local priority 200 peer address ipv4 10.0.0.2 ! On Secondary KS crypto gdoi group GETVPN-GROUP server local redundancy local priority 100 peer address ipv4 10.0.0.1
- Primary KS holds authoritative policy and distributes keys
- Secondary KS syncs from primary; takes over if primary fails
- GMs register to whichever KS responds
3.6 GETVPN Troubleshooting Commands
On Key Server:
show crypto gdoi ks ! KS status, group name, registered GMs, rekey counters show crypto gdoi ks members ! List of registered GMs with IP, registration time, rekey status show crypto gdoi ks policy ! Active TEK/KEK policy, transform set, ACL show crypto gdoi ks cooperative ! Co-operative KS status — PRIMARY or SECONDARY role show crypto isakmp sa ! IKEv1 SA for KS-GM GDOI registration channel
On Group Member:
show crypto gdoi ! Summary: GM status, group, KS address, TEK/KEK details show crypto gdoi gm ! GM detailed: registered KS, TEK/KEK lifetimes, rekey counters show crypto gdoi gm acl ! ACL downloaded from KS (what traffic is being encrypted) show crypto ipsec sa ! IPsec SAs created from GDOI policy show crypto map ! GDOI crypto map applied to WAN interface
Debugs:
debug crypto gdoi debug crypto gdoi ks debug crypto gdoi gm debug crypto isakmp
PART 4: Technology Decision Framework
When to Use Which Technology
Do you need encryption over MPLS/private WAN?
-> Any-to-any, no tunnels, preserve routing/QoS topology?
YES -> GETVPN
NO -> Internet/public WAN?
YES -> New deployment or IKEv2 required?
YES -> FlexVPN
NO -> DMVPN (IKEv1)
NO -> GETVPN or P2P IPsec VTIDecision summary:
| Scenario | Recommended |
|---|---|
| Internet-based hub-and-spoke, new deployment | FlexVPN (IKEv2, cleaner, more scalable) |
| Internet-based hub-and-spoke, legacy IOS | DMVPN Phase 3 |
| Direct spoke-to-spoke over internet | DMVPN Phase 2/3 or FlexVPN+mGRE |
| MPLS WAN, preserve routing topology, any-to-any | GETVPN |
| Cloud-based SD-WAN overlay | Cisco SD-WAN (Viptela) |
| Simple site-to-site, no dynamic spokes | P2P IPsec (VTI) |
PART 5: Common Issues — Master Troubleshooting Table
| # | Technology | Symptom | Cause | Exact Fix |
|---|---|---|---|---|
| 1 | DMVPN | Spoke tunnel up but show dmvpn shows no NHRP entries | NHRP authentication mismatch | ip nhrp authentication <key> must be identical on hub and spoke (case-sensitive) |
| 2 | DMVPN | Spoke registers to hub but spoke-to-spoke fails (Phase 2) | Hub is summarizing spoke routes | Remove summary-address on hub tunnel or change to Phase 3 with ip nhrp redirect + ip nhrp shortcut |
| 3 | DMVPN | IPsec never forms (NHRP maps exist but state stays NHRP) | Crypto proposal mismatch | Check show crypto isakmp sa — MM_NO_STATE means Phase 1 failed. Align encryption, hash, DH group, lifetime |
| 4 | DMVPN | Spoke behind NAT cannot form IPsec with hub | NAT-T not enabled (UDP/4500 blocked) | Enable NAT-T: crypto isakmp nat-traversal 30 on hub. Open UDP/4500 on firewall. |
| 5 | DMVPN | GRE keepalives failing | keepalives enabled on mGRE (not supported) | Remove keepalive from tunnel interface — mGRE does not support GRE keepalives |
| 6 | DMVPN | NHRP shows mapping but wrong NBMA | Spoke NBMA IP changed (DHCP reconnnect) | NHRP self-corrects on re-registration. Lower ip nhrp holdtime for faster recovery. |
| 7 | DMVPN | Routing protocol flaps over DMVPN | MTU mismatch (GRE+IPsec overhead) | Set ip mtu 1400 and ip tcp adjust-mss 1360 on tunnel interface |
| 8 | FlexVPN | IKEv2 SA will not form — "No proposal chosen" | IKEv2 proposal mismatch | Use smart defaults (delete custom proposals) or align encryption/integrity/DH group |
| 9 | FlexVPN | Virtual-access interface not created | aaa authorization network not configured | Add aaa new-model + aaa authorization network <list> local + reference list in ikev2 profile |
| 10 | FlexVPN | Spoke connects but gets wrong IP | IP pool exhausted or misconfigured | Check show ip local pool — resize pool or fix ip address-pool config |
| 11 | GETVPN | GM stuck in "Registering" | ISAKMP Phase 1 failure to KS | show crypto isakmp sa on GM — check KS IP reachable, pre-shared key matches |
| 12 | GETVPN | GM registered but traffic not encrypted | Crypto map not applied to WAN interface | Apply crypto map <name> to correct WAN interface facing MPLS |
| 13 | GETVPN | GETVPN traffic fails after NAT device inserted | GETVPN incompatible with NAT | GETVPN preserves original IP headers — NAT changes them, breaking IPsec. Remove NAT or use DMVPN instead |
| 14 | GETVPN | Cooperative KS both claim PRIMARY | KS-to-KS connectivity issue | Verify IKEv1 between KS peers. Check show crypto gdoi ks cooperative — connectivity flap causes split-brain |
| 15 | GETVPN | All GMs re-register simultaneously | KEK expired before rekey reached all GMs | Increase rekey lifetime or investigate unicast rekey delivery failures. Switch to multicast rekey if possible |
PART 6: IOS XR Notes
IOS XR supports DMVPN and FlexVPN but with different configuration syntax:
GETVPN on IOS XR: Not natively supported in standard IOS XR. Use IOS XE for GETVPN deployments.
DMVPN on IOS XR:
interface tunnel-ip0 ipv4 address 10.0.0.1 255.255.255.0 tunnel source GigabitEthernet0/0/0/0 tunnel mode gre multipoint ! nhrp network-id 1 nhrp authentication NHRPKEY nhrp nhs 10.0.0.1 nbma 203.0.113.1 multicast
Key IOS XR differences:
- Interface:
tunnel-ip<N>(notTunnel<N>) - NHRP:
nhrpcommands under interface (noipprefix) - Debug:
debug nhrpworks;show nhrp(noipprefix)
IOS XR FlexVPN:
crypto ikev2 profile FLEX-PROF match identity remote address 0.0.0.0/0 authentication remote pre-shared-key FLEXKEY authentication local pre-shared-key FLEXKEY crypto ipsec profile FLEX-IPSEC set ikev2-profile FLEX-PROF set transform-set FLEX-TS
PART 7: VRF-Aware DMVPN
DMVPN supports front-door VRF (fVRF) and inside VRF (iVRF):
- fVRF: The physical WAN interface VRF — where the NBMA addresses live
- iVRF: The tunnel VRF — where tunnel IPs and routed traffic live
- Spokes behind MPLS CE routers often need fVRF for the PE-facing interface
Hub config with fVRF:
interface Tunnel0 ip vrf forwarding INSIDE-VRF ip address 10.0.0.1 255.255.255.0 tunnel source GigabitEthernet0/0 tunnel vrf FRONT-DOOR-VRF tunnel mode gre multipoint ip nhrp network-id 1
Verify VRF-aware DMVPN:
show ip nhrp vrf <vrf-name> show crypto isakmp sa vrf <vrf-name> show crypto ipsec sa vrf <vrf-name>
PART 8: Monitoring & Operational Commands Reference
Daily Health Checks
! DMVPN hub — check all spoke connectivity show dmvpn | include (Intf|Peer|NHRP|ATTRB) ! Count registered spokes show ip nhrp | count Dynamic ! IPsec SA utilization show crypto ipsec sa summary ! Check for SAs with zero traffic (potential zombie SAs) show crypto ipsec sa | include (peer|pkts encaps|pkts decaps) ! GETVPN KS — check registered GMs show crypto gdoi ks members | count GM ! Check rekey failures show crypto gdoi | include (rekey|fail)
Performance & Scale
| Technology | Max Spokes (typical IOS XE) | SA overhead |
|---|---|---|
| DMVPN Phase 1 | ~1000 per hub | Hub-spoke SAs only |
| DMVPN Phase 2/3 | ~500 per hub | Spoke-to-spoke SA overhead |
| FlexVPN | ~1000+ per hub | Virtual-access per spoke |
| GETVPN | ~10,000 GMs per KS | Shared TEK — no per-pair SA overhead |
GETVPN scales far better than DMVPN for large flat networks because all GMs share a single TEK — no N-squared SA problem.
Quick Checklists
Checklist: DMVPN Spoke Not Registering
[ ] 1. Tunnel interface up/up: show interface Tunnel0 [ ] 2. Spoke can ping hub NBMA IP: ping <hub-NBMA> [ ] 3. NHRP auth key matches hub exactly: ip nhrp authentication <key> [ ] 4. NHRP NHS points to correct hub tunnel IP + NBMA: ip nhrp nhs <tunnel-ip> nbma <nbma-ip> [ ] 5. NHRP network-id matches hub: ip nhrp network-id <same-number> [ ] 6. GRE key matches (if used): tunnel key <same-value> [ ] 7. iVRF/fVRF consistent if VRFs in use [ ] 8. ACL not blocking GRE (IP proto 47) or IKE (UDP/500) or NAT-T (UDP/4500) [ ] 9. show ip nhrp — does hub see spoke registration? [ ] 10. debug nhrp — what error does hub return?
Checklist: DMVPN IPsec Not Forming
[ ] 1. NHRP registration working (pre-req): show dmvpn — state should progress past NHRP [ ] 2. IKE Phase 1 (ISAKMP): show crypto isakmp sa — look for QM_IDLE [ ] 3. Proposal match: encryption, hash, DH group, lifetime identical hub <-> spoke [ ] 4. Pre-shared key matches (wildcard on hub, specific on spoke OK) [ ] 5. NAT-T enabled if spoke is behind NAT: crypto isakmp nat-traversal [ ] 6. Transform set mode = transport (for GRE tunnel): mode transport [ ] 7. IPsec profile applied to tunnel: tunnel protection ipsec profile <name> [ ] 8. show crypto ipsec sa — any encaps/decaps? [ ] 9. debug crypto isakmp — what phase fails?
Checklist: No Spoke-to-Spoke Traffic (Phase 2/3)
Phase 2: [ ] 1. Both spokes registered to hub (show dmvpn on hub) [ ] 2. Hub has specific /32 route for each spoke tunnel IP (no summarization) [ ] 3. Split-horizon disabled on hub tunnel (EIGRP: no ip split-horizon eigrp <AS>) [ ] 4. Spoke has ip nhrp shortcut on tunnel interface [ ] 5. show ip nhrp — does initiating spoke have dynamic entry for target spoke? [ ] 6. Spoke-to-spoke IPsec forms: show crypto session on spoke after traffic attempt Phase 3 (additional): [ ] 7. Hub tunnel has ip nhrp redirect [ ] 8. Hub CAN summarize — spoke gets /32 shortcut only after first packet [ ] 9. Shortcut route in routing table: show ip route — look for H (NHRP) routes
Checklist: GETVPN GM Not Encrypting Traffic
[ ] 1. GM registered with KS: show crypto gdoi gm — status = Registered [ ] 2. ISAKMP SA between GM and KS: show crypto isakmp sa [ ] 3. TEK downloaded: show crypto gdoi gm — TEK count > 0 [ ] 4. Crypto map applied to correct interface: show crypto map [ ] 5. Traffic matches downloaded ACL: show crypto gdoi gm acl [ ] 6. No NAT device between GMs (GETVPN incompatible with NAT) [ ] 7. show crypto ipsec sa — encaps/decaps incrementing? [ ] 8. KS reachable from GM WAN interface
PermitANY Resources
Config Templates (https://permitany.com/configs)
- /configs/cisco-ios/dmvpn-hub — mGRE hub with NHRP, IPsec profile and Phase 3 redirect
- /configs/cisco-ios/dmvpn-spoke — spoke with NHS mapping, shortcut and tunnel protection
- /configs/cisco-ios/flexvpn — IKEv2 hub/spoke with smart defaults and virtual-templates
- /configs/cisco-ios/getvpn — key server and group member with GDOI, TEK and KEK
- /configs/cisco-ios/vpn-ipsec — the classic site-to-site crypto map, for the leg that is not overlay
Automation Scripts (https://permitany.com/scripts)
- /scripts/dmvpn-health-check — tunnel, NHRP and IPsec state across the whole spoke estate in one pass
- /scripts/nhrp-cache-inspector — the NHRP cache, incomplete entries and stale mappings
- /scripts/getvpn-ks-monitor — registered GMs, rekey success and cooperative KS role
- /scripts/ike-sa-analyzer — IKE and IPsec SA state, proposals and mismatches
- /scripts/vpn-tunnel-monitor — up/down and flap detection with alerting
- /scripts/vpn-crypto-audit — weak proposals, DH groups and lifetimes still in production
- /scripts/path-mtu-analyzer — the black hole that only appears on large packets over the tunnel
AI Prompts (https://permitany.com/prompts)
- /prompts/vpn-troubleshooter — structured DMVPN/FlexVPN/GETVPN/AnyConnect diagnostic conversation
- /prompts/dmvpn-design-advisor — phase choice, hub count and routing protocol for a DMVPN build
- /prompts/vpn-design-advisor — DMVPN vs FlexVPN vs GETVPN vs SD-WAN, with the trade-offs named
Skills (https://permitany.com/skills)
- /skills/bgp-troubleshooting — the overlay routing protocol most of these designs run
- /skills/ospf-troubleshooting — the other one, and the network-type trap over mGRE
- /skills/sd-wan — what replaces these designs, and when it should not
- /skills/packet-capture-analysis — proving where the ESP packets actually stop
Working Style
When someone brings you an overlay VPN problem:
- Establish which technology and, for DMVPN, which phase. Phase 2 and Phase 3 fail differently and are fixed differently, and people routinely describe a Phase 3 design as Phase 2.
- Separate the three layers before touching anything: underlay reachability to the NBMA address, the overlay (GRE/NHRP), and crypto. Work them in that order. Crypto is the layer everyone starts with and the layer that is least often at fault.
- Ask for
show dmvpn detail,show ip nhrpandshow crypto ipsec satogether. One of the three on its own will mislead you — a tunnel line protocol stays up on an mGRE interface with no peers at all. - Give the exact command and the exact config lines, not the concept. "Check NHRP" is not an answer;
show ip nhrp nhs detailand the expectedREflag is one. - State the cost of the design you recommend. GETVPN with 10,000 GMs and DMVPN Phase 2 with 500 spokes are not the same operational commitment, and a key server has no natural failure domain boundary.
- Check MTU before declaring success. A tunnel that passes ping and fails file transfers is
ip mtuandip tcp adjust-mss, and it is the defect that gets closed as "intermittent". - Never recommend a change to a hub without naming the spoke impact. On this topology the hub is the blast radius.
And the question that resolves most of these cases faster than anything else:
Does the tunnel come up when you remove tunnel protection from the profile?If it does, you have a crypto problem and the overlay is fine. If it does not, you have been debugging IPsec for an hour on a GRE or NHRP fault — and that is the single most common way this domain wastes an afternoon.
Further Learning
Recommended books to go deeper on this topic.
CCNP Security Virtual Private Networks SVPN 300-730 Official Cert Guide
Joseph Muniz, Steven Chimes, James Risler · Cisco Press · The only Cisco-approved SVPN study guide — covers DMVPN, FlexVPN, GETVPN, AnyConnect, and IKEv2 in depth
IKEv2 IPsec Virtual Private Networks — Understanding and Deploying IKEv2, IPsec VPNs, and FlexVPN in Cisco IOS
Graham Bartlett, Amjad Inamdar · Cisco Press · The definitive IKEv2 and FlexVPN reference — crypto fundamentals, keyring, profiles, DVTI, spoke-to-spoke
Network Security Technologies and Solutions
Yusuf Bhaiji · Cisco Press · CCIE Professional Development · Comprehensive CCIE Security reference — IPsec, IKE, VPN, AAA, PKI, firewalls all in one volume