Initial public release — aci-sim v0.16.0

aci-sim is a stateful REST simulator of a 2-site Cisco ACI fabric
(per-site APIC + ND/NDO management planes) — for testing ACI automation
(Ansible cisco.aci / cisco.mso, aci-py, custom REST clients) and CI gates
without real hardware.

Highlights: APIC + NDO REST surface served from one in-memory MIT built
from a declarative topology.yaml; port mode + sandbox (real per-device IPs
on :443) run modes; LLDP/CDP neighbor visibility; NDO->APIC deploy mirror
(multi-site templates materialize onto the target sites' APIC stores);
real-APIC fvBD default attributes; file-backed state save/restore; an
`aci-sim` CLI (validate/show/graph/run/new/init/lldp); and an 888-test suite.

History squashed for the public release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-05 20:06:36 +10:00
co-authored by Claude Fable 5
commit 7e9a175ce6
117 changed files with 31769 additions and 0 deletions
+579
View File
@@ -0,0 +1,579 @@
# autoACI ⇄ APIC/NDO REST CONTRACT (what the simulator must satisfy)
> Distilled from a read-only audit of `~/autoACI/backend` (services/aci_connector.py,
> ndo_connector.py, routers/topology.py, routers/ndo.py, routers/fabric_build.py,
> services/acipy_exec.py, plugins/*). This is the authoritative contract the sim
> emulates. Do NOT re-explore autoACI; trust this file. If something here is
> ambiguous, prefer the behavior that makes autoACI render.
autoACI connects to **three host identities**, each `https://{host}` with `verify=False`:
- **Site-A APIC**, **Site-B APIC** (each logged in separately; autoACI tracks `sites[].{host,asn}` and prefixes node IDs by site when >1 site), and
- **ND / NDO** (separate auth + endpoints).
---
## 1. APIC — authentication
- `POST /api/aaaLogin.json` body `{"aaaUser":{"attributes":{"name":<user>,"pwd":<pwd>}}}`.
`name` may be a bare username (`admin`) or **domain-qualified**
(`apic:<domain>\<user>` or `<domain>\<user>`, e.g. `apic:local\admin`) — PR-15:
aci-py's APIC connector sends the domain-qualified form by default, matching
real APIC's local-login-domain behavior. `auth.py::_bare_username()` strips an
optional `apic:<domain>\`/`<domain>\` prefix before the credential compare, so
both forms authenticate identically; the domain segment itself is not
validated (this sim has no concept of configured login domains — any domain
string is accepted). Credentials (post-strip) are checked against
`SIM_USERNAME`/`SIM_PASSWORD` (default `admin`/`cisco`; see README §Configuration) —
a mismatch (either form) returns a **401 APIC error envelope**, never a bare token.
**Admin-account wizard (topology `auth:` section):** `aci-sim init`'s Step 0
writes an `auth: {username, password}` section into `topology.yaml`
(`topology/schema.py`'s `Auth` model); `aci-sim run` resolves
`SIM_USERNAME`/`SIM_PASSWORD` with this precedence (highest first): (1) the
caller's shell already has `SIM_USERNAME` set — wins outright, `auth:` is
ignored; (2) `topology.yaml` has an `auth:` section — its `username`/
`password` are injected; (3) neither — nothing injected, this sim falls back
to the hardcoded `admin`/`cisco` default documented above. `cli.py`'s
`resolve_admin_credentials()` implements this precedence and is unit-tested
directly (`tests/test_admin_account.py`). **NDO is NOT affected**
`ndo/app.py`'s `POST /api/v1/auth/login`/`POST /login` (§7 below) accept
any credential by design and never validate anything against
`auth.ndo_username`/`ndo_password`; those two fields (set only when the
wizard's `ndo_same_account` answer is "no") are stored for informational
parity only. Response on
success: `{"imdata":[{"aaaLogin":{"attributes":{"token":"<t>","refreshTimeoutSeconds":"600"}}}]}`.
Token also returned as `APIC-cookie` cookie; subsequent requests send that cookie.
Sessions expire after `refreshTimeoutSeconds` (600s); every query/write route
requires a live, unexpired `APIC-cookie` or returns a **403 APIC error envelope**
(`text="Token was invalid (Error: Token timeout)"`, `code="403"`). A malformed
login body (wrong shape / non-JSON) returns a **400 APIC error envelope**, never
FastAPI's raw `{"detail": [...]}` validation-error shape.
- `GET /api/aaaRefresh.json` → requires a valid existing `APIC-cookie`; renews its
expiry to a full `refreshTimeoutSeconds` and returns the same aaaLogin envelope
(autoACI refreshes every ~240s). Without/with an invalid cookie → 403 envelope.
- `POST /api/aaaLogout.json` body `{"aaaUser":{"attributes":{"name":<user>}}}`
invalidates the server-side session in addition to clearing the cookie.
- During/after login autoACI probes: `GET /api/class/topSystem.json?query-target-filter=eq(topSystem.role,"controller")&page-size=1`
and `GET /api/mo/uni/userprofile-<user>.json?query-target=subtree&target-subtree-class=aaaUserDomain`.
## 2. APIC — the three query shapes + full param grammar
1. **Class query**`GET /api/class/{class}.json` with any of:
- `page-size` (always; default 500, clamped to `[1, 5000]`), `page` (always; 0-indexed, must be `>= 0`).
A non-numeric or out-of-range `page`/`page-size` yields a **400 APIC error envelope**
(`code="107"`), never a bare 500 or silent truncation.
- `query-target-filter={expr}` (filter grammar §4)
- `rsp-subtree=children|full` + `rsp-subtree-class={c1,c2,...}` (modern subtree)
- `query-target=subtree` + `target-subtree-class={c1,c2,...}` (legacy subtree)
2. **MO/DN query**`GET /api/mo/{dn}.json` with optional `query-target=subtree` + `target-subtree-class={...}`.
DNs contain slashes inside brackets: `subnet-[10.0.0.1/24]`, `pathep-[eth1/9]`, `rsdomAtt-[uni/phys-X]` — bracket-aware DN parsing required.
`GET/POST/DELETE /api/node/mo/{dn}.json` is a full alias of `/api/mo/{dn}.json`
(PR-10) — real APIC serves both; the sim's API Inspector / `aci_rest`-module
callers may use either interchangeably with identical behavior.
3. **Node-scoped class query**`GET /api/node/class/{dn}/{class}.json` with optional `query-target-filter`.
Used for node-local `faultInst` (e.g. `.../topology/pod-1/node-101/faultInst.json`).
**Fabric-wide form:** `GET /api/node/class/{class}.json` (no `topology/pod-X/node-Y`
DN prefix at all — just the bare class name) is also supported and is equivalent to
`GET /api/class/{class}.json` (same result set, real APIC behavior). The sim
distinguishes the two by whether the path segment after `/api/node/class/` contains
a `/`: a node-scoped DN always does, the fabric-wide form never does.
**Response envelope (always):**
```json
{"imdata":[ {"<class>":{"attributes":{...}, "children":[ {"<child>":{"attributes":{...}}} ]}} ], "totalCount":"<N>"}
```
- `children` present only when a subtree mode was requested (children/full nest children;
`rsp-subtree-class`/`target-subtree-class` filter which descendant classes appear).
- `totalCount` is a STRING; used for pagination (client loops `page` until a short page).
- Subtree responses may return children before parents → consumers do **two-pass parsing**; the
sim may emit any order but MUST include all requested descendants.
## 2a. APIC — query subscriptions + websocket push
All three query shapes in §2 accept `?subscription=yes`. The response is
the normal §2 envelope PLUS a top-level `subscriptionId` string sitting
alongside `imdata`/`totalCount` (NOT nested inside them):
```json
{"imdata":[...], "totalCount":"<N>", "subscriptionId":"<id>"}
```
Without `?subscription=yes`, the response has no `subscriptionId` key —
byte-identical to a pre-subscription-feature query. This is a strict
backward-compat requirement, not a default choice.
**Scope**: a class-query subscription (`/api/class/{cls}.json`) watches
every MO of that class fabric-wide. A DN-scoped subscription
(`/api/mo/{dn}.json`, `/api/node/class/{dn}/{cls}.json`) watches that DN
and its entire subtree.
**Websocket**: `GET /socket<token>``<token>` is the same `APIC-cookie`
token from `aaaLogin` (no separator, literal path concatenation, matching
real APIC). Unknown/expired token → connection rejected before accept. One
live connection per token.
**Push event** (sent on the matching token's websocket after a write commits):
```json
{"subscriptionId":["<id>",...], "imdata":[{"<class>":{"attributes":{...,"status":"created|modified|deleted","dn":"<dn>"}}}]}
```
- `subscriptionId` is a list — every subscription on that token's session
that matched the change (a client can hold multiple overlapping subs).
- `status` is `created` (DN didn't previously exist), `modified` (DN
already existed), or `deleted` (DELETE, or POST body `status="deleted"`).
- A DELETE on a DN with children emits one event per removed MO in the
subtree (root + all descendants), each carrying its own class/dn/status,
so both class-scoped and DN-scoped subscribers see everything actually
removed.
**Refresh**: `GET /api/subscriptionRefresh.json?id=<id>` — always 200
(`{"imdata":[],"totalCount":"0","subscriptionId":"<id>"}`), including for
an unknown/expired id (real APIC does not error a stale refresh). Default
subscription TTL is 90s, matching real APIC, renewed by this call.
**Lifecycle**: disconnecting the websocket immediately drops every
subscription owned by that connection's token — a subsequent matching
change is silently skipped (no error), same as a DN/class nobody
subscribed to.
Implementation: `aci_sim/rest_aci/subscriptions.py` (in-memory
registry + notify) and `aci_sim/rest_aci/app.py` (route wiring). See
README §10a for the user-facing walkthrough.
## 3. APIC — write path (Fabric Build apply)
- `POST /api/mo/{dn}.json` body `{"<class>":{"attributes":{"name":...,"dn":...,"status":"created|modified|deleted"?}, "children":[...]}}`.
Response `{"imdata":[{"<class>":{"attributes":{"dn":...,"status":"created"}}}],"totalCount":"1"}` (HTTP 200).
Nested `children` in the body must be upserted too (recursively). `status:"deleted"` removes.
- Writes must **upsert into the live MIT** so a later GET reflects them (read-back verification).
- **Atomic (all-or-nothing):** the entire body (root + every nested child, recursively) is shape-validated
*before* any store mutation happens. A malformed node anywhere in the tree (e.g. a `children` entry that
isn't a single-key `{className: {...}}` object) rejects the whole POST with a 400 APIC error envelope and
leaves the store completely untouched — including MOs earlier in traversal order that would otherwise
have been written first. Mirrors real APIC POST semantics.
- **Canonical RN synthesis:** a child posted without an explicit `dn` gets its RN derived from the class's
real RN prefix (e.g. `fvBD``BD-{name}`, `fvAEPg``epg-{name}`, `fvSubnet``subnet-[{ip}]`), not a
generic `{class}-{name}` guess — so it lands on the same DN a canonical GET or a later explicit-`dn` POST
would use (no orphan duplicates). Classes without a known template fall back to `{class}-{name}`.
- **Node registration reaction:** `initial_fabric_discovery`/`add_leaf_switch_pair`/`add_spine_switch`
POST `fabricNodeIdentP` at `uni/controller/nodeidentpol/nodep-{id}`. The sim reacts by
materializing a full node-registration MO set, not just a bare `fabricNode`: `fabricNode`,
`topSystem` (loopback + OOB addresses via the same scheme every boot-seeded node gets),
`healthInst`, `fabricLink` (+enriched `lldpAdjEp`/`cdpAdjEp` and their `lldpInst`/`lldpIf`/
`cdpInst`/`cdpIf` containers, v0.12.0 — same `build/neighbors.py` helper the boot-time
builders use, so a dynamically-registered node's neighbors are queryable the identical way)
cabling the new node to every spine in its site, and `l1PhysIf`/`ethpmPhysIf` port inventory
(fabric uplinks + host-access ports) — so a dynamically-registered node renders identically
to one seeded from `topology.yaml`. The
reaction fires for a `fabricNodeIdentP` found **anywhere** in the POST body's validated write
plan, not only when it is the top-level class (a `fabricNodeIdentP` nested under a parent, e.g.
`uni/controller`, also triggers it). Pod is derived from the target site (`site.pod`), never
hardcoded — `fabricNodeIdentP`/nodeidentpol DNs carry no pod of their own. `POST /_sim/add-leaf`
shares this same enrichment.
- Multi-site playbooks push NDO schemas/templates/deploy (see §7) then per-site APIC MOs.
## 4. APIC — filter grammar (query-target-filter values)
- `eq(<class>.<attr>,"<val>")` exact match (numbers unquoted).
- `ne(<class>.<attr>,"<val>")` not-equal (a missing attr is treated as ≠, i.e. matches).
- `wcard(<class>.<attr>,"<substr>")` substring/wildcard (commonly on `.dn`).
- `gt|lt|ge|le(<class>.<attr>,"<val>")` compare — numeric when both sides parse as
numbers (e.g. `gt(fabricNode.id,"200")`), else lexical.
- `bw(<class>.<attr>,"<lo>","<hi>")` inclusive range.
- `and(<c1>,<c2>,...)`, `or(<c1>,<c2>,...)` compose. Nesting allowed. Zero-argument
`and()`/`or()` are rejected (not "match everything" / "match nothing").
- `query-target=subtree` scopes the match set to root+descendants FIRST, then applies
`query-target-filter` to every scoped MO (root and descendants alike) — a filter on
an attribute only descendants carry (e.g. `faultInst.severity` under a `fabricNode`
root) still reaches matching descendants; it does not pre-filter the root out of the
scope before expansion.
- A malformed expression (unknown operator, truncated call, trailing tokens after a
complete expression, e.g. two comma-joined filters or an unbalanced trailing `)`)
yields a **400 APIC error envelope** (`code="107"`), never a 500.
- Examples seen verbatim:
`eq(fabricNode.role,"spine")`, `wcard(fvCtx.dn,"tn-Tenant1/")`,
`and(eq(fvBD.name,"X"),wcard(fvBD.dn,"tn-T/"))`,
`or(eq(faultInst.severity,"critical"),eq(faultInst.severity,"major"))`,
`ne(fvTenant.name,"common")`, `gt(fabricNode.id,"200")`, `bw(fabricNode.id,"100","200")`.
## 5. APIC — error shapes / HTTP codes
- 200 success. 400 → ACIQueryError (also: malformed aaaLogin body, bad page params,
strict-filter-parse errors). 401 → bad aaaLogin credentials
("Authentication failed: invalid username or password"). 403 → ACIAuthError —
missing/unknown/expired `APIC-cookie` on any query/write route or on
aaaRefresh ("Token was invalid (Error: Token timeout)").
Error body: `{"imdata":[{"error":{"attributes":{"text":"<msg>","code":"<c>"}}}]}`.
The `/_sim/*` control-plane router and `/api/aaaLogin.json`/`aaaLogout.json`
are intentionally exempt from the 403 auth gate (test tooling + the login
flow itself depend on them being reachable pre-auth).
- **`GET /api/mo/{dn}.json` on a nonexistent DN → 200 + empty envelope
(`{"imdata":[],"totalCount":"0"}`), NOT 404** (PR-9 FEATURE 6, corrects a
pre-PR-9 design choice that returned 404/ACINotFoundError here). Verified
against upstream cisco.aci `plugins/module_utils/aci.py`'s `api_call()`:
any GET response with `status != 200` is unconditionally fatal
(`fail_json`) — there is no "404 means not-found, that's fine" special
case. Every `state=present` module does a `get_existing()` GET *before*
building its diff, so on a brand-new object that GET's DN does not exist
yet; a 404 there fails the very first task of any create playbook
(`fatal: APIC Error 103: Unable to find the object specified`). 404 is
reserved for genuinely malformed requests, not absent-but-well-formed
objects — matching real APIC. `/api/class/*` and `/api/node/class/*`
already returned 200-empty for a query matching nothing; this brings
`/api/mo/*` in line for the "DN itself absent" case too.
- **Any unmatched `/api/*` path → 400 APIC error envelope, never FastAPI's
`{"detail":...}` shape** (PR-10). Real APIC never emits FastAPI's default
404 body; `cisco.aci` cannot parse it and surfaces it as
`"APIC Error None: None"`, `status=-1`. Body:
`{"imdata":[{"error":{"attributes":{"text":"Invalid request path: <path>","code":"400"}}}],"totalCount":"1"}`.
Implemented as a 404 exception-handler fallback scoped to `/api/*` — it
only fires once every real route (including the `/api/node/mo` alias)
has already failed to match, so it can never shadow one.
## 6. APIC — class catalog (102) + attributes actually read
Group by concern; every listed attribute is read by some plugin/router — include them.
(Header count corrected from a stale "63" to the actual number of distinct
classes catalogued below, verified by direct count — see PR-3b-batch1.
✅ marks classes seeded by PR-3b-batch1 (previously catalogued here but never
built, so their class queries returned empty and the depending autoACI
plugins rendered nothing); 🆕 marks classes seeded by PR-3b-batch2/3 (same
gap, closed in this batch); everything else was already built pre-batch-1.
**Batch-2/3 changelog note (this revision):**
- `vpcRsMbrIfs`**corrected to `pcRsMbrIfs`**. The catalogued name was
wrong: autoACI's `vpc_status.py` actually queries `pcRsMbrIfs` (the real
ACI class for a port-channel member-port relation, child of `pcAggrIf`,
RN `rsmbrIfs-[{ifDn}]`), never `vpcRsMbrIfs` — verified read-only against
the plugin source before building. Built as `pcRsMbrIfs` in
`build/fabric.py`; see docs/DESIGN.md "Batch-2" section.
- `infraNodeIdentP`**removed from this catalog, NOT built.** No autoACI
plugin/router references it at all (verified: a fabric-wide grep across
`~/autoACI/backend/plugins/*.py` for `infraNodeIdentP` returns zero
matches — unlike `fabricNodeIdentP`, which the Fabric Build write-path
reaction genuinely needs). Real ACI's `infraNodeIdentP` is a *node
provisioning-policy* object (`uni/infra/nodep-{id}`, part of interface
selector/AEP scoping for a specific node), a materially different concept
from `fabricNodeIdentP`'s Fabric Membership node registration
(`uni/controller/nodeidentpol/nodep-{id}`) — the two are easy to confuse
by name alone. Building an `infraNodeIdentP` that mirrors
`fabricNodeIdentP`'s semantics (as an earlier draft of this catalog
implied) would be a faithfulness regression: a wrong object with a
plausible-looking name, satisfying no real consumer. Decision: leave it
out of scope entirely rather than build a wrong object.
**Fabric/topology:** `fabricNode`(id,name,role∈{spine,leaf,controller},model,serial,version,adSt,fabricSt,dn) ·
`fabricLink`(dn contains `/lnk-{n1port}-to-{n2port}/`, n1,n2,operSt,operSpeed,linkType) ·
`topSystem`(dn,fabricDomain,version,role,state,address/oobMgmtAddr) ·
`fabricHealthTotal`(cur) · `healthInst`(cur,min,max,prev) ·
`vpcDom`(id,dn,peerSt) · `vpcIf`(dn,name,operSt) · `pcAggrIf`🆕(dn,name,id,operSt) · `pcRsMbrIfs`🆕(tDn,parentSKey) · `infraWiNode`🆕(health,state) ·
`firmwareCtrlrRunning`🆕(version,node) — PR-9, one per controller at `{ctrl_dn}/ctrlrfwstatuscont/ctrlrrunning`.
**Mgmt tenant — OOB management (OOB-gateway PR):** real APIC models each
node's OOB management address + default gateway in the special `mgmt`
tenant, never on `topSystem` itself (`topSystem.oobMgmtAddr` carries only the
address — see above). Prior to this PR, `aci-sim init`'s wizard collected an
OOB gateway per site but discarded it entirely (no schema field, no MO).
`build/mgmt.py` now builds the minimal faithful scaffolding, one tree per
site: `fvTenant`(name="mgmt", dn=`uni/tn-mgmt`) →
`mgmtMgmtP`(name="default", dn=`uni/tn-mgmt/mgmtp-default`) →
`mgmtOoB`(name="default", dn=`uni/tn-mgmt/mgmtp-default/oob-default`) → one
`mgmtRsOoBStNode`(tDn,addr,gw) per node — switches AND controllers — at
`{oob-default dn}/rsooBStNode-[topology/pod-{p}/node-{id}]`, where
`tDn=topology/pod-{p}/node-{id}`, `addr=<node's OOB IP>/<mask>` (same
`oob_ip(pod,node_id)` address `topSystem.oobMgmtAddr` uses; mask from
`fabric.oob_subnet` if set, else `/24` — see build/mgmt.py docstring for why
not `fabric.oob_pool`'s `/16`), and `gw=site.oob_gateway` (empty string if
unset — never invented/derived). No other mgmt-tenant children (`mgmtInB`,
`mgmtRsOoBCtx`, `mgmtInstP`, ...) are built — no autoACI/aci-py chain
audited for this sim reads them; same scope judgment as
`fabric.inb_subnet`'s store-only status (README §6c).
**Faults:** `faultInst`(dn,severity∈{critical,major,minor,warning},code,lc,type,subject,descr,created,lastTransition).
Fault DNs live under node-local shards; node-scoped query via `/api/node/class/{nodeDn}/faultInst.json`.
**Interfaces:** `l1PhysIf`(id,adminSt,descr,mtu,mode) · `ethpmPhysIf`(dn has `phys-[eth{s}/{p}]`,operSt,operSpeed,usage,lastLinkStChg,resetCtr) · `ethpmFcot`🆕(typeName,guiName,vendorName,vendorSn,actualType).
PR-19: the ISN/IPN spine uplink `l1PhysIf.mtu` is now `isn.mtu`-driven (default 9150) instead of the general fabric `9216` hardcode every other port still uses.
**Neighbors/underlay:** `lldpAdjEp`(dn,sysName,mgmtIp,portIdV,portIdT,chassisIdT,chassisIdV,sysDesc,capability,id,ttl) ·
`cdpAdjEp`(dn,sysName,mgmtIp,devId,portId,platId,ver,cap) ·
`lldpInst`🆕(dn,adminSt,name,holdTime,initDelayTime,txFreq,ctrl,optTlvSel) · `lldpIf`🆕(dn has `if-[eth{s}/{p}]`,id,adminRxSt,adminTxSt,operRxSt,operTxSt,mac,portDesc,portVlan,sysDesc,wiring) ·
`cdpInst`🆕(dn,adminSt,name,holdIntvl,txFreq,ver,ctrl) · `cdpIf`🆕(dn has `if-[eth{s}/{p}]`,id,adminSt,operSt) ·
`isisAdjEp`(dn,operSt,sysId,lastTrans,numAdjTrans) · `isisDom` · `ospfAdjEp`(dn,operSt,id,peerIp/addr,nbrId,area) · `ospfIf`✅(name,area,state,helloIntvl,deadIntvl) · `arpAdjEp`✅(ip,mac,ifId,physIfId,operSt).
LLDP/CDP fidelity (v0.12.0): `lldpAdjEp`/`cdpAdjEp` are now enriched with real-APIC-shaped fields
(`portIdV`/`portId` carry the NEIGHBOR's own port; `chassisIdV` is a deterministic per-node MAC,
see `build/neighbors.py:node_mac`; `sysDesc`/`platId`/`ver` describe the neighbor's model+version, or
"Cisco APIC"/`APIC-SERVER-M3` for a controller neighbor) — additive only, pre-existing `sysName`/
`mgmtIp` are unchanged. `lldpInst`/`cdpInst`/`lldpIf`/`cdpIf` are newly materialized containers (one
inst per switch node with any adjacency, one `*If` per adjacency-bearing local port) — this also fixes
a bug where a deep-root subtree query (`GET .../sys/lldp/inst.json?query-target=subtree&
target-subtree-class=lldpAdjEp`) returned `totalCount=0` because the root DN had no MO of its own.
`aci-sim lldp` (CLI) prints a `show lldp neighbors`-style table from these fields (`--cdp` for CDP,
`--json` for machine-readable output).
PR-19: `ospfIf.area`/`ospfAdjEp.area` are now `isn.ospf_area`-driven (default `"0.0.0.0"`) instead of a hardcoded literal.
PR-20: `ospfIf.helloIntvl`/`deadIntvl` are new, driven by `isn.ospf_hello`/`isn.ospf_dead` (defaults 10/40, real ACI defaults) — previously not carried on this MO at all.
**Overlay/BGP:** `bgpInst`(dn,asn) · `bgpPeer`(dn,asn,addr,type,peerRole) ·
`bgpPeerEntry`(dn,addr,operSt∈{established,...},rtrId,lastFlapTs,connEst,connDrop,type) ·
`bgpPeerAfEntry`(dn,type/afId,acceptedPaths,pfxSent,tblVer) · `bgpVpnRoute`🆕(pfx,rd) · `bgpPath`🆕(nh,asPath,localPref,metric,origin,type).
For ISN: each spine `/sys/bgp/inst` subtree must contain `bgpPeer` entries whose addr is a **/32** and whose ASN ≠ the local fabric ASN (autoACI infers the inter-site EVPN cloud from this).
**COOP/EPM:** `coopPol`,`coopInst`(operSt) · `epmMacEp`(addr,ifId,flags,createTs,encap) · `epmIpEp`(addr,ifId,flags,createTs).
**Routing tables/control:** `uribv4Route`✅,`uribv4Nexthop`✅ · `rtctrlProfile`✅,`rtctrlCtxP`✅,`rtctrlSubjP`✅,`rtctrlMatchRtDest`✅,`rtctrlSetComm`✅,`rtctrlSetPref`✅.
**Tenant (control):** `fvTenant`(name,descr,dn) · `fvCtx`(name,pcEnfPref,bdEnforcedEnable,ipDataPlaneLearning,pcEnfDir,pcTag,scope) ·
`fvBD`(name,arpFlood,unicastRoute,ipLearning,unkMacUcastAct,unkMcastAct,multiDstPktAct,epMoveDetectMode,limitIpLearnToSubnets,mtu,mac,intersiteBumTrafficAllow,type) ·
PR-20: `fvBD.mac` now reflects `bd.mac` (per-BD) falling back to `fabric.default_bd_mac` (fabric-wide, default `"00:22:BD:F8:19:FF"` — the exact literal already hardcoded here pre-PR-20, so unset topology.yaml fields produce byte-identical output).
`fvSubnet`(ip,scope) · `fvAp`(name) · `fvAEPg`(name,prefGrMemb,floodOnEncap,pcEnfPref,isAttrBasedEPg,pcTag) ·
`fvCEp`(mac,ip,encap,fabricPathDn) · `fvIp`(addr) · `fvIfConn`(dn has tn-/ap-/epg-, encap) · `fvEpP`🆕(epgPKey,pcTag,scopeId).
**Rels:** `fvRsBd`(tnFvBDName) · `fvRsCtx`(tnFvCtxName) · `fvRsBDToOut`(tnL3extOutName) ·
`fvRsDomAtt`(tDn,instrImedcy) · `fvRsPathAtt`✅(tDn,encap,mode,instrImedcy) · `fvRsCons`(tnVzBrCPName) · `fvRsProv`(tnVzBrCPName).
**Contracts:** `vzBrCP`(name,scope,prio,targetDscp) · `vzSubj`(name,revFltPorts) · `vzFilter`(name) ·
`vzEntry`(name,etherT,prot,dFromPort,dToPort,sFromPort,sToPort,stateful) · `vzRsSubjFiltAtt`(tnVzFilterName) · `vzRsSubjGraphAtt`✅ · `actrlRule`🆕(scopeId,sPcTag,dPcTag,fltId,action,direction,prio,ctrctName,operSt).
**L3Out (control):** `l3extOut`(name,enforceRtctrl) · `l3extLNodeP`(name) · `l3extRsNodeL3OutAtt`(tDn,rtrId) ·
`l3extLIfP`(name) · `l3extRsPathL3OutAtt`(tDn,encap,addr,ifInstT) · `l3extInstP`(name,prefGrMemb) ·
`l3extSubnet`(ip,scope) · `l3extRsL3DomAtt`(tDn) · `l3extRsEctx`(tnFvCtxName,tDn) · `l3extMember`✅(side,addr) ·
`bgpPeerP`(addr,peerCtrl,keepAliveIntvl,holdIntvl) · `bgpAsP`(asn) · `bgpExtP` · `ospfExtP`✅ · `ipRouteP`✅(ip,pref) · `ipNexthopP`✅(nhAddr).
PR-20: `bgpPeerP.keepAliveIntvl`/`holdIntvl` are new, driven by `l3out.csw_peer.keepalive`/`.hold` (defaults 60/180, real ACI defaults) — previously not carried on this MO at all.
**Access policy:** `infraAttEntityP`(AAEP,name) · `infraAccPortGrp`,`infraAccBndlGrp`✅(name,lagT) · `infraAccPortP`✅(name) ·
`infraHPortS`✅(name,type) · `infraPortBlk`✅(fromPort,toPort,fromCard) · `infraRsAttEntP`(tDn) · `infraRsAccBaseGrp`✅(tDn) ·
`infraRsDomP`(tDn) · `infraRsVlanNs`(tDn) · `physDomP`(name) · `l3extDomP`(name) · `fvnsVlanInstP`(name,allocMode) · `fvnsEncapBlk`(from,to) ·
`vmmDomP`🆕(name) · `vmmCtrlrP`🆕(name,hostOrIp,rootContName,dvsName,dvsVersion) · `vmmUsrAccP`🆕(name) — PR-19, Tier-2
`access.vmm_domains[]`; `vmmCtrlrP`/`vmmUsrAccP` only emitted when `vcenter_ip` is set. Pure addition — no
existing plugin/production-chain reads these (see docs/DESIGN.md's PR-19 section for the verified
`bind_epg_to_vmm_domain` DN-string-only finding).
**Infra write targets (Fabric Build):** `fabricNodeIdentP`🆕(nodep-{id},serial,nodeId,name,role) · `infraInfra`,`infraAttEntityP`, VLAN pools/AAEPs.
(`infraNodeIdentP` intentionally removed from this catalog — see the batch-2/3 changelog note above.)
**Capacity/backup:** `eqptcapacityPolUsage5min`(polUsage,polUsageCap,polUsageCum,polUsageCapCum) · `configExportP`(name) · `configJob`(operSt,executeTime).
## 7. NDO / MSO contract
- `POST /api/v1/auth/login` `{"userName","userPasswd","domain":"local"}``{"token":...}` (Bearer header after).
**Deliberately lenient**: accepts ANY credential and never validates the returned
token on subsequent requests (client-compat for `cisco.mso`/aci-py, which expect a
login to just work). This is unchanged by the admin-account wizard (§1) — NDO
shares the APIC admin account by default but is never actually checked against it.
- `GET /api/v1/platform/version``{"version":"4.2.2"}`.
- `GET /mso/api/v1/sites``{"sites":[{id,name,platform,urls[]}]}`. **`name` is
the topology's `fabric_name`** (e.g. `"LAB1-IT-ACI"`), NOT the short
internal site name (`"LAB1"`) — PR-10, verified against a real-gear
`cisco.mso.mso_tenant` run that rejected the short form
(`"Site 'LAB1-IT-ACI' is not a valid site name"` when only `"LAB1"` was
exposed). Internal site-id joins (tenant/schema site associations) key off
the topology's short name regardless of what `name` reports here.
- `GET /mso/api/v1/tenants` (also `GET /api/v1/tenants` — PR-10, backs
`cisco.mso.ndo_template`'s `lookup_tenant()` prereq lookup, same data) →
`{"tenants":[{id,name,displayName,siteAssociations:[{siteId}]}]}`.
- `GET /mso/api/v1/schemas``{"schemas":[{id,displayName,name,templates:[{name}]}]}`.
- `GET /mso/api/v1/schemas/list-identity` (PR-10) → `{"schemas":[{id,displayName}]}`
— lightweight enumeration every `cisco.mso` schema module calls first to
resolve a schema `displayName``id` (verified against `ansible-mso`'s
`plugins/module_utils/mso.py` `lookup_schema()`, which calls
`query_objs("schemas/list-identity", key="schemas", displayName=schema)`).
**Must be declared before** the parameterized `/schemas/{id}` route below,
or `"list-identity"` gets matched as a schema id instead.
- `GET /mso/api/v1/schemas/{id}` → full schema; template shape:
`templates[].{name,templateType,vrfs[],bds[],anps[].epgs[],contracts[],filters[],externalEpgs[]}` and top-level `sites[].{templateName,siteId}`.
- vrf: `{name,displayName,vrfRef:"/vrfs/NAME",vzAnyEnabled,ipDataPlaneLearning}`
- bd: `{name,vrfRef:"/vrfs/NAME",subnets:[{ip}],l2Stretch,intersiteBumTraffic,l2UnknownUnicast,arpFlood,unicastRouting,epMoveDetectMode,dhcpLabels:[{ref:UUID}]}`
- epg: `{name,bdRef:"/bds/NAME",preferredGroup,proxyArp,uSegEpg,intraEpg,contractRelationships:[{relationshipType,contractRef:"/contracts/NAME"}]}`
- contract: `{name,scope,filterRelationships:[{filterRef:"/filters/NAME"}]}` filter: `{name,entries:[{name,ipProtocol,dFromPort,dToPort,etherType}]}`
- externalEpg: `{name,vrfRef,l3outRef:"/l3outs/NAME",type,subnets:[{ip}]}`
- `GET /mso/api/v1/sites/fabric-connectivity``{"sites":[{id/siteId,status,connectivityStatus}]}` (or list).
- `GET /mso/api/v1/schemas/{id}/policy-states``[{status,drift}]` or `{policyStates:[...]}`.
- `GET /mso/api/v1/templates/summaries``[{templateId,templateName,templateType}]` (tenantPolicy carries DHCP;
`templateName` required — PR-13 — since `ansible-mso`'s `MSOTemplate` resolves templates by name+type).
- `GET /mso/api/v1/templates/{id}``{tenantPolicyTemplate:{template:{dhcpRelayPolicies:[{uuid,name}],dhcpOptionPolicies:[{uuid,name}]}}}`.
- **Template writes (PR-13):** `POST /templates` (create — payload
`{displayName,templateType,<typeContainer>:{template:{...},sites:[{siteId}]}}`,
e.g. `tenantPolicyTemplate`), `PATCH /templates/{id}` (JSON-Patch ops,
e.g. `add /tenantPolicyTemplate/template/dhcpRelayPolicies/-`), `DELETE
/templates/{id}`. **Both bare (`/api/v1/...`) and `/mso`-prefixed
(`/mso/api/v1/...`) forms are backed by the same store** — see §7c.
- `GET /templates/objects?type={dhcpRelay|dhcpOption|epg|externalEpg}&{uuid=...|name=...}`
(both prefixes) — cross-template object lookup; `dhcpRelay`/`dhcpOption`
search every tenant-policy template's policy lists, `epg`/`externalEpg`
search every schema template's `anps[].epgs[]`/`externalEpgs[]`. Returns
a list when only `type` (or `type`+`name`) is given, a single object (or
`{}`) when `uuid` is given.
- `GET /mso/api/v1/schemas/service-node-types` (also bare) →
`{"serviceNodeTypes":[{id,displayName}]}` (Firewall/Load Balancer/Other).
- `GET /api/v1/audit-records?count=N` (fallback `/mso/api/v1/audit-records`) → `{auditRecords:[{timestamp,user,action,description,details}]}` (or records/imdata/list).
- **Writes:** `POST /mso/api/v1/schemas`, `POST /mso/api/v1/deploy` → accept + store; return an id/status. Cross-plane rule: NDO tenants/VRFs/BDs must match APIC MIT names.
- Errors: 401 → NDOAuthError; else propagate. Router returns `{"detail":...}`.
### 7a. NDO schema write round-trip (PR-11)
The multi-site (`MS`) aci-ansible playbooks create a per-tenant-region schema (`<tenant>-<region>`, e.g. `"MS-TN1-LAB0"`) during `create_tenant`'s MSO play, then PATCH it from every subsequent playbook (`create_bd`, `create_application`, ...). Verified against `ansible-mso`'s `plugins/module_utils/{mso,schema}.py` and a real multi-site E2E run against ACI hardware.
- **Sequence:** `GET schemas/list-identity` (displayName→id) → `GET schemas/{id}` (full doc) → `PATCH schemas/{id}` with a JSON-Patch-shaped op list `[{"op":"add"/"replace"/"remove","path":...,"value":...}]`. `POST /mso/api/v1/schemas` creates a schema (optionally already carrying its first template — `mso_schema_template.py`'s "schema doesn't exist yet" branch); `PATCH` applies ops against the stored doc and returns the updated doc. Implemented in `aci_sim/ndo/patch.py` (`apply_json_patch`), wired into `aci_sim/ndo/app.py`'s `create_schema`/`patch_schema` routes.
- **Path addressing — three conventions cisco.mso modules use, all supported:**
1. Standard RFC 6902: numeric index, or `"-"` to append.
2. Named-segment lookup against a `name`/`displayName` field — most template-level objects (`bds`/`anps`/`epgs`/`contracts`/`filters`/`externalEpgs`/`vrfs`), e.g. `/templates/LAB1/bds/bd-App1_LAB1`.
3. A synthetic **composite key** `"{siteId}-{templateName}"` unique to the top-level `sites[]` array — every `mso_schema_site_*` module builds this literally (`site_template = "{0}-{1}".format(site_id, template)`), e.g. `/sites/1-LAB1/bds/-`.
4. A **`*Ref`-name** lookup for site-local child objects, which carry no `name` field of their own — only a `bdRef`/`vrfRef`/etc string pointing back at the template object — yet are still addressed by that object's bare name: `/sites/1-LAB1/bds/bd-App1_LAB1/subnets/-`.
- **Collection-default normalization:** real NDO always serves back every collection key (`vrfs`/`bds`/`anps`/`contracts`/`filters`/`externalEpgs`/`intersiteL3outs`/`serviceGraphs`, and nested `subnets`/`epgs`/`contractRelationships`/...) as `[]` even when a client's create/add payload omitted them. `normalize_template()`/`normalize_site()` backfill these on every create and every PATCH — a missing key is a hard client-side crash (`'NoneType' object is not iterable`, bare `KeyError`), not a 4xx, so this has to be done proactively rather than reactively.
- **`*Ref` string normalization:** several write payloads (e.g. `mso_schema_site_bd.py`'s `bdRef=dict(schemaId=...,templateName=...,bdName=...)`) send a *dict* form of a `*Ref` field, but real NDO stores/serves it as the canonical **string** `/schemas/{schemaId}/templates/{templateName}/{category}/{name}` — confirmed because a sibling module (`custom_mso_schema_site_bd_subnet.py`) does `bd_ref_string in [v.get('bdRef') for v in ...]` and `', '.join(...)` on the stored values; a dict there crashes with `TypeError: sequence item 0: expected str instance, dict found`. `apply_json_patch` stringifies `bdRef`/`vrfRef`/`l3outRef`/`filterRef`/`contractRef`/`anpRef`/`serviceGraphRef` dicts on every `add`/`replace` op.
- **ND user-class fallback:** `GET /api/config/class/remoteusers` / `/localusers``{"items":[{"spec":{"loginID",...,"id"|"userID"}}]}`. `cisco.mso.mso_tenant`'s `lookup_users()`/`lookup_remote_users()` query the modern `/nexus/infra/api/aaa/v4/*` routes first and fall back to these legacy routes when both come back empty; a 404 here (no route at all) aborts with `"ND Error: Unknown error no error code in decoded payload"` since the error body has neither `code` nor `messages`.
- **Bare `GET /api/v1/sites`** (PR-11) — `cisco.mso.ndo_template`'s TenantPol prereq lookup hits this un-prefixed path; same data as `/mso/api/v1/sites`.
- **`PUT`/`POST /mso/api/v1/tenants{,/{id}}`** — `mso_tenant`'s create-vs-update branch: every tenant this sim seeds already exists in `GET /mso/api/v1/tenants`, so a real run always takes the `PUT .../tenants/{id}` (update) path.
- **Schema-template deploy:** `GET schemas/{id}/validate` (legacy + `ndo_schema_template_deploy`, discarded return value, only status matters), `GET execute|status/schema/{id}/template/{name}` (legacy `mso_schema_template_deploy`), `POST /mso/api/v1/task` `{schemaId,templateName,isRedeploy}` (`ndo_schema_template_deploy` — the module aci-ansible's `mso-model` role actually invokes; confirmed against the collection installed on real ACI hardware, which differs from `ansible-mso`'s `master`-branch module source).
- **Site-local ANP/EPG auto-mirroring + self-referencing anpRef/epgRef (PR-12).** Two compounding gaps, both required to close `bind_epg_to_static_port`:
1. **Template-level self-refs.** Real NDO stores a self-referencing `anpRef` string on every template-level ANP object itself (and `epgRef` on every EPG) — confirmed because `ansible-mso`'s `mso_schema_template_anp.py`/`_anp_epg.py` explicitly strip it client-side before an idempotency comparison (`if "anpRef" in mso.previous: del mso.previous["anpRef"]`) — dead code unless the server actually sends one back. `MSOSchema.set_site_anp()`/`set_site_anp_epg()` (`ansible-mso`'s `module_utils/schema.py`) key their site-local lookup off exactly this field (`template_anp.details.get("anpRef")`); without it, the lookup compares every site-anp's `anpRef` against `None` and never matches. `normalize_template()` (`aci_sim/ndo/patch.py`) now backfills `anpRef`/`epgRef` on every `anps[]`/`epgs[]` entry it normalizes (`/schemas/{id}/templates/{t}/anps/{a}` and `/schemas/{id}/templates/{t}/anps/{a}/epgs/{e}` respectively), and takes an optional `schema_id` param to build them; `build_ndo_model()` (`aci_sim/ndo/model.py`) calls it at boot too, so a topology tenant that ships with ANPs/EPGs already configured doesn't skip this.
2. **Site-local mirroring.** Real NDO 4.x auto-populates a schema's site-local `sites[].anps[].epgs[]` the moment a template-level ANP/EPG is added to a template that already has a site attached (`mso_schema_site.py` already ran in `create_tenant`'s flow, per PR-11). `apply_json_patch` replicates this: an `add /templates/{t}/anps/-` op auto-creates a matching `{anpRef, epgs:[]}` entry in every `sites[]` row whose `templateName == t`; an `add /templates/{t}/anps/{a}/epgs/-` op auto-creates a matching **full-shaped** site-EPG `{epgRef, subnets:[], staticPorts:[], staticLeafs:[], domainAssociations:[]}` entry under that mirrored site-anp. `build_ndo_model()` performs the same mirroring at boot for topology-seeded schemas. Both are idempotent (safe on playbook retries). The full child-collection shape is mandatory: every sibling `mso_schema_site_anp_epg_*` module reads its own array with a **bare subscript** (not `.get()`), so any missing key is a hard `KeyError` client-side — `mso_schema_site_anp_epg_domain.py` bare-subscripts `domainAssociations` (`domains = [dom.get("dn") for dom in ...["epgs"][epg_idx]["domainAssociations"]]`), `_staticleaf` reads `staticLeafs`, `_staticport` reads `staticPorts`, `_subnet` reads `subnets`. A real hardware MS-TN1 run regressed on `bind_epg_to_physical_domain`/`_vmm_domain` with `KeyError: 'domainAssociations'` when the mirrored site-EPG shipped without it (pre-PR-12 those binds passed only because no site-EPG existed at all, so the module took a non-crashing branch). Single source of truth for the shape is `SITE_OBJECT_DEFAULTS["epgs"]`, consumed by both `_new_site_epg()` (mirror path) and `normalize_site()` (backfill path).
Both `*Ref` strings use NDO's canonical form — `anpRef`: `/schemas/{id}/templates/{t}/anps/{a}` (`_REF_NAME_FIELDS`); `epgRef` uniquely nests **two** name segments, `/schemas/{id}/templates/{t}/anps/{a}/epgs/{e}` (confirmed against `ansible-mso`'s `module_utils/mso.py` `epg_ref()` — handled by a dedicated `_stringify_refs` branch since the generic single-category table only fits one name field). `_REF_LOOKUP_KEYS` also gained `epgRef` so a site-epg (ref-only, no `name` of its own) resolves by its bare EPG name the same way site-local bds/anps already did (PR-11).
Without either fix, `mso_schema_site_anp_epg_staticport.py`'s own "create site anp/epg if missing" fallback fires instead (its own code comment: *"Coverage misses this two conditionals when testing on 4.x and above"* — i.e. it's dead code on real hardware) and ultimately crashes with `'NoneType' object has no attribute 'details'` — confirmed on a real MS-TN1 `bind_epg_to_static_port` run on ACI hardware before this fix (schema=MS-TN1-LAB0, anp=app-Web_LAB1, epg=epg-web, site=LAB1-IT-ACI, static vpc port `ipg-vpc-LegacySW01` vlan-100), now resolved end-to-end.
### 7c. Tenant-policy-template DHCP surface + schema serviceGraphs (PR-13)
Closes the last two NDO write-surface gaps blocking the MS suite's DHCP and service-graph tasks. Verified against `ansible-mso`'s `plugins/modules/{ndo_template,ndo_dhcp_relay_policy,ndo_schema_template_bd_dhcp_policy}.py`, `plugins/module_utils/{mso,template}.py`, the `mso-model` role's `library/custom_mso_schema_service_graph.py`, and real multi-site E2E runs against ACI hardware.
- **Bare vs `/mso`-prefixed split, and why it matters for templates.** `ansible-mso`'s `MSOModule.request()` builds its URL differently depending on connection mode: tasks using the persistent `ansible.netcommon.httpapi` connection (`ansible_network_os: cisco.nd.nd`) route through `NDO_API_VERSION_PATH_FORMAT = "/mso/api/{v}/{path}"`; tasks carrying `delegate_to: localhost` (no persistent httpapi socket) fall through `MSOModule`'s direct-HTTP branch, which never adds the `/mso` prefix at all — `self.url = "{0}api/{1}/{2}".format(base_uri, api_version, path)`. `prereq_tenantpol.yml`'s `cisco.mso.ndo_template` task uses `delegate_to: localhost`, so it hits `GET/POST https://host/api/v1/templates*` (BARE) while `create_dhcp_relay`'s `cisco.mso.ndo_dhcp_relay_policy` and `bind_dhcp_relay_to_bd`'s `cisco.mso.ndo_schema_template_bd_dhcp_policy` (no `delegate_to`) hit the `/mso`-prefixed form. Both forms are registered against the SAME mutable template store (`aci_sim/ndo/app.py`'s `_get_template_summaries`/`_create_template`/`_patch_template`/`_get_template_objects`, looped over `("/mso/api/v1", "/api/v1")`), matching the existing `/api/v1/tenants``/mso/api/v1/tenants` / `/api/v1/sites``/mso/api/v1/sites` pattern PR-10/PR-11 established for the identical `delegate_to` split.
- **Template create/lookup sequence.** `ndo_template`'s `MSOTemplate.__init__` first queries `templates/summaries?templateName=...&templateType=...` (name+type filter; `schemaName`/`schemaId` kwargs are `None` and skipped by `query_objs()`) — every summary entry therefore needs a `templateName` field, not just `templateId`/`templateType` (the boot-seeded tenantPolicy template summary was retrofitted with one). If nothing matches, it `POST`s `{displayName, templateType, tenantPolicyTemplate:{template:{tenantId}, sites:[{siteId}]}}` to `templates` (no `/summaries` suffix) — the sim assigns a `templateId`, stores the full doc, and mirrors a summary entry so the very next `templates/summaries` lookup (by any caller, either path prefix) resolves it.
- **DHCP relay policy add + uuid backfill.** `ndo_dhcp_relay_policy`'s add PATCHes `add /tenantPolicyTemplate/template/dhcpRelayPolicies/-` with `{name, providers, description?}` — no `uuid` (real NDO assigns one server-side). A provider's `epgRef` is the target schema-template EPG's `uuid` (`MSOSchema.set_template_anp_epg().details.get("uuid")`) — schema-template EPGs/externalEpgs never carried a `uuid` field before this PR; `normalize_template()` now backfills a stable one (`_normalize_object`'s `epgs`/`externalEpgs` branches). `ndo_schema_template_bd_dhcp_policy` (`bind_dhcp_relay_to_bd`) resolves a relay/option policy's uuid via `GET templates/objects?type={dhcpRelay|dhcpOption}&name=...`, requiring both a `tenantId` (matched client-side against the policy-owning template) and a non-empty `uuid` on the returned entry, or it `fail_json`s — the sim's `_patch_template` backfills a stable uuid on every dhcpRelayPolicies/dhcpOptionPolicies entry that lacks one. The same `templates/objects` route also serves `type=epg`/`type=externalEpg` queries (searching every schema template's `anps[].epgs[]`/`externalEpgs[]`), used by `ndo_dhcp_relay_policy`'s own uuid→name resolution on query/present.
- **Site-level `serviceGraphs` + full-detail `GET /schemas`.** The mso-model role's "Create service graph" task uses `custom_mso_schema_service_graph.py`, a pre-`MSOTemplate`/`MSOSchema` community module (`version_added: '2.8'`) that bare-subscripts straight into the RAW `GET /schemas` list response: `mso.get_obj('schemas', displayName=schema)` then `schema_obj.get('templates')[idx]['serviceGraphs']` and, once a site-local device binding step runs, `schema_obj.get('sites')[site_idx]['serviceGraphs']`. The template-level bare-subscript was already covered (PR-11's `TEMPLATE_COLLECTION_KEYS`), but (a) the sim's `GET /mso/api/v1/schemas` previously returned only a trimmed `{id,displayName,name,templates:[{name}]}` summary — insufficient for this module's bare-subscript pattern, which needs the full nested doc (`bds`/`anps`/`serviceGraphs`/`sites`) — and (b) the SITE-level `serviceGraphs` key was never backfilled at all. Both fixed: `GET /schemas` now returns full schema detail per entry (real NDO's plain list endpoint already does this — it's precisely why the lighter `schemas/list-identity` endpoint exists as a separate optimization); `SITE_TOP_LEVEL_DEFAULTS` (`aci_sim/ndo/patch.py`) backfills `serviceGraphs` (and `contracts`, see below) as top-level keys on every schema `sites[]` entry via `normalize_site()`. The module also needs `GET schemas/service-node-types` (Firewall/Load Balancer/Other → stable id) and a `tenantId` field on the schema template itself (used on its "service graph already exists, add a node" branch) — both added.
- **Site-local contracts mirroring (deeper layer, found chasing GAP 2 to green on real ACI hardware).** Once the `serviceGraphs` KeyError was fixed, a real hardware MS-TN2 `create_tenant` pass-2 run (`-e automate_contract_graph=true -e automate_site_redirect=true`) progressed to the mso-model role's raw `cisco.mso.mso_rest`-driven "Atomic PATCH — bind service-graph redirect on ALL fabrics in one request" task, which PATCHes `add /sites/{siteId}-{template}/contracts/{contract}/serviceGraphRelationship` — addressing a SITE-LOCAL contract by bare name, mirroring the same addressing convention site-local bds/anps already use. The sim never mirrored template-level contract adds into `sites[].contracts[]` at all (unlike anps/epgs, mirrored since PR-12), so `apply_json_patch`'s `_find_by_name` raised `PatchError: path segment 'con-Firewall_LAB0' not found in list` — confirmed verbatim on the real run. `_mirror_template_contract_to_sites()` (PATCH-time, parallel to `_mirror_template_anp_to_sites`) and an equivalent boot-time mirror in `build_ndo_model()` close this: an `add /templates/{t}/contracts/-` op (or a topology-seeded template that already has contracts) now auto-creates a matching `{contractRef}` entry in every `sites[]` row whose `templateName == t`.
- **Verified on real ACI hardware end-to-end:** MS-TN1 `create_dhcp_relay` (08) and `bind_dhcp_relay_to_bd` (09) both reach `failed=0`; MS-TN2's identical DHCP chain (08/09) also `failed=0`; MS-TN2 `create_tenant` pass-2 (01b, `-e automate_contract_graph=true -e automate_site_redirect=true`) reaches `failed=0` — including the atomic per-fabric service-graph redirect PATCH. No regression observed on MS-TN1/MS-TN2's already-green tasks (01-07, 10, 01a) or a SF-suite spot-check (create_tenant, create_dhcp_relay) re-run on the same sandbox.
### 7d. Site-local BD auto-mirroring for aci-py (PR-15)
Closes a schema-PATCH 400 mid `create_tenant`/`create_bd`, found running `aci-py` (a pure-Python Ansible-to-ACI compiler, distinct from the `ansible-mso`/`cisco.mso` collection §7a-7c document) against the sandbox on real ACI hardware. Same family of gap as PR-12's ANP/EPG mirroring and PR-13's contract mirroring — this closes the missing BD sibling.
- **Why `mso_schema_site_bd` always sends `replace`, never `add`.** aci-py's `mso_schema_site_bd` shim (mirroring `ansible-mso`'s `mso_schema_site_bd.py`) PATCHes a site-local BD shadow with `op: replace` unconditionally — its own docstring/comment documents why: real NDO 4.x auto-creates the site BDDelta shadow the instant a template-level BD is added to a template that already has a site attached, so by the time the site-BD module runs the shadow already exists and a fresh `add` would 409 ("Multiple BDDelta entries") on real hardware; the module does a GET-then-replace instead.
- **The gap.** The sim mirrored template-level ANP/EPG adds into `sites[].anps[].epgs[]` (PR-12) and contract adds into `sites[].contracts[]` (PR-13) but never mirrored template-level BD adds into `sites[].bds[]` at all. So the very first `replace /sites/{siteId}-{t}/bds/{bd}` against a schema that had only ever seen the template-level `add /templates/{t}/bds/-` hit `_find_by_name`'s "not found" branch: `PatchError: replace: 'bd-FW_LAB0' not found at '/sites/1-LAB1-LAB2/bds/bd-FW_LAB0'` — confirmed verbatim against a real hardware aci-py `create_tenant`/`create_bd` run (MS-TN2) before this fix.
- **Fix.** `_mirror_template_bd_to_sites()` (`aci_sim/ndo/patch.py`, PATCH-time, parallel to `_mirror_template_anp_to_sites`/`_mirror_template_contract_to_sites`): an `add /templates/{t}/bds/-` op now auto-creates a matching site-local BD shadow (`_new_site_bd()`: `{bdRef, hostBasedRouting: False, subnets: []}` — the full child-collection shape `mso_schema_site_bd_subnet.py` bare-subscripts) in every `sites[]` row whose `templateName == t`. `build_ndo_model()` (`aci_sim/ndo/model.py`) performs the equivalent mirroring at boot, so a topology.yaml tenant that ships with BDs already configured also starts with the site shadow pre-mirrored (matching the same boot-time treatment PR-12/PR-13 gave ANPs/EPGs/contracts). Idempotent (safe on playbook/driver retries) — `_find_by_name` skips a site that already carries the bdRef.
- **Verified on real ACI hardware end-to-end:** aci-py's `scripts/live_chain.py` driver against `environments/lab-multisite-sim` — MS-TN2 now reaches **13/13 steps rc=0** (was 9 ok / 4 fail before this fix: `create_tenant` pass1, `create_bd`, `create_tenant` pass2, and `migrate_existing_vlan_gateway` all 400'd on exactly this `replace: 'bd-*' not found` error); MS-TN1 stays **11/11 rc=0** (1 skip — no `set_primary_fw_state` var file for TN1's combo — no regression).
### 7e. Site-local BD mirror-vs-site-module dedup (PR-16)
Closes a site-local BD *duplication* left behind by PR-15's mirror, found running the Acme App1-Net tenant's `create_tenant`/`create_bd` playbooks against a real NDO schema on ACI hardware (schema `App1-Net-LAB0`).
- **The symptom.** After `create_tenant` (whose template-BD `add` fires `_mirror_template_bd_to_sites`, PR-15) and `create_bd` (whose `cisco.mso.mso_schema_site_bd`/`custom_mso_schema_site_bd_subnet` tasks PATCH the site-local shadow), `sites[0].bds` on the live schema carried **two** entries for the same template BD `App1-Net_VLAN10`: one empty mirror (`bdRef=".../templates/LAB1/bds/App1-Net_VLAN10"`, `subnets=[]`) and one carrying the real subnet (`10.10.10.1/24`). Real NDO has exactly one site-BD row per template BD.
- **Root cause #1 — the append path skipped dedup entirely.** `apply_json_patch`'s `op: add` handling has three sub-branches keyed on the last path token: `"-"` (append), a numeric index (`insert`), or a name (resolved via `_find_by_name`, which already deduped named-segment adds). Only the named-segment branch deduped. `mso_schema_site_bd.py`'s "BD does not exist yet" branch (confirmed against the pre-existing `test_patch_add_site_bd_by_composite_key`, PR-11) PATCHes `add /sites/{siteId}-{template}/bds/-` — the **append** form — so it always appended unconditionally, regardless of whether the template-add mirror already created a site-local shadow for the same BD.
- **Root cause #2`_find_by_name`'s own ref-lookup only recognized the STRING ref form.** Even where dedup *was* attempted (the mirror's own idempotency check, `_find_by_name(site["bds"], bd_name)`), it iterated each item's `bdRef` and matched only `isinstance(ref_val, str)`. Several real cisco.mso write payloads send `bdRef` as a **dict** (`{schemaId, templateName, bdName}`, or a bare `{bdName: ...}` with no schema/template context) — an existing site-local entry left in that shape was invisible to any subsequent name-based lookup.
- **Fix — dedup a NAMELESS shadow strictly by its single IDENTITY ref.** A site-local shadow (`sites[].bds`/`anps`/`epgs`/`contracts` entry) carries no `name`/`displayName` of its own; its identity is exactly one `*Ref` (`bds``bdRef`, `anps``anpRef`, `epgs``epgRef`, `contracts``contractRef`, in `_IDENTITY_REF_BY_COLLECTION`). `_bare_ref_name(ref_val)` extracts the bare object name from that identity ref regardless of shape (string trailing segment, or the dict form's own name field via `_REF_NAME_FIELDS`), so a dict-form `bdRef` (a site-module payload's shape, possibly a bare `{bdName: ...}`) and a string-form `bdRef` (the mirror's canonical shape) resolve to the same bare name. `_find_by_name`'s ref-lookup now fires **only for nameless items** and **only over the identity-ref keys** (never a property ref like `vrfRef`). `_find_shadow_by_identity(items, value, collection)` is the append-path counterpart: it dedups **only** a nameless value, **only** by the identity ref for that collection (`tokens[-2]`), and `apply_json_patch`'s `add`+`"-"` branch calls it before appending — a match **replaces** in place (cisco.mso's GET-then-write idiom, real NDO's one-row-per-object invariant), a genuinely new shadow appends. Because the machinery is keyed by collection→identity-ref, it covers site-local ANPs/contracts too with no per-type change.
- **Why identity scoping is load-bearing (regression the first cut of this PR shipped and an independent full-chain run caught).** An earlier cut resolved a value's identity by iterating **all** `_REF_LOOKUP_KEYS` — including `vrfRef`/`l3outRef`. But every template BD under one L3-attached VRF shares the same `vrfRef` (e.g. `vrf-L3_LAB0`), and a template-level BD `add` (a NAMED object, not a shadow) was also run through the same dedup. Result: two distinct template BDs both "matched" via their shared `vrfRef`, so each `add /templates/{T}/bds/-` overwrote the previous one and every template's BDs collapsed to a **single** entry — the real Acme `create_bd` failed with `"Provided BD 'bd-App2-Net_VLAN11_LAB0' does not exist. Existing BDs: bd-App3-Hst_VLAN12_LAB0"` (only the last-added BD survived per template). The invariant the corrected code enforces: **a named object is matched ONLY by name/displayName, never by any `*Ref`; ref-matching is ONLY for nameless shadows and ONLY via the ONE identity ref for that list** — a property ref is a property, not an identity. Guarded by `test_distinct_template_bds_sharing_vrfref_do_not_collapse` and three sibling tests.
- **Verified on real ACI hardware end-to-end:** Acme App1-Net tenant's `create_tenant.yml` + `create_bd.yml` both `failed=0` against the sim; the live NDO schema `App1-Net-LAB0` shows template BD counts `LAB1-LAB2=6`, `LAB1=14`, `LAB2=10` (30 total, no template collapse) and exactly one site-local BD entry per template BD in each site's `bds[]` (no duplicate).
## 8. Topology view (`GET /api/topology`) — exact needs
Per site autoACI issues: `fabricNode`, `fabricLink`, `vpcDom`, `fabricHealthTotal` (paged);
per-node `query_dn("topology/pod-{pod}/node-{id}/sys/health")` → healthInst.cur;
fabric-wide `faultInst` (severity filter) — counts per node parsed from `/node-{id}/` in DN;
session tables `ospfAdjEp`, `bgpPeer`, `bgpPeerEntry`, `bgpPeerAfEntry`;
multi-site ISN: per spine `query_dn("topology/pod-1/node-{spineId}/sys/bgp/inst", subtree=True, subtree_class="bgpPeer")`
→ detects /32 peers with asn≠local → synthesizes an ISN cloud node + spine→ISN links.
Node detail `GET /api/topology/node/{id}`: `ethpmPhysIf`,`l1PhysIf` (via `/sys` subtree), node-scoped `faultInst`,
`fvIfConn` (EPGs on ports), health. `pod` is parsed from `fabricNode.dn` (`pod-{N}`) — do NOT assume pod-1 everywhere.
Link adjacency from `fabricLink.n1/n2` + DN `/lnk-{p1}-to-{p2}/`. Roles from `fabricNode.role`.
## 9. Plugins — 46 files, all must get believable data
Every class in §6 is read by some plugin. Control-plane plugins (tenant/bd/epg/vrf/contract/l3out/access/
capacity) read config MOs. Data-plane plugins (bgp_health, routing_state, fabric_bgp_evpn, adjacency_health,
interface_status/flaps, ep_tracker, fault_summary, health_score) read operational MOs. Many do two-pass
parent→child parsing and use `eq/wcard` filters + subtree. The generic query engine + a consistent MIT
covers all of them without per-plugin code.
## 10. Fabric Build (mostly local; write path is dry-run-first)
Catalog/prereq-gate/common-vars/generate-vars are LOCAL to autoACI (no APIC). Real apply goes through a
hidden `/api/_internal/run-playbook` that compiles templates via aci-py → `POST /api/mo/{dn}.json` (single-fabric)
and/or NDO `schemas`+`deploy` (multi-site), `--check` (dry-run) by default. To test a playbook end-to-end:
run in apply mode → sim upserts the MOs → verify via read-back. ~22 playbooks are pure ACI/NDO REST
(create_*/bind_*/initial_fabric_*/add_leaf/add_spine/provision_*); the N5K/N7K SSH migration family is OUT OF SCOPE.
## 11. Ansible-compatibility (PR-9 — cisco.aci fidelity)
The owner's primary public use case is running **Ansible `cisco.aci` playbooks**
against this simulator directly (not just autoACI). Everything below was
verified read-only against upstream `plugins/module_utils/aci.py`
(https://github.com/CiscoDevNet/ansible-aci) — the shared HTTPAPI/module
base every `cisco.aci.aci_*` module calls into.
**GET-before-write existence check (FEATURE 6, live blocker).** Every
`state=present`/`state=absent` module calls `get_existing()` (a GET) before
building its diff or delete. `api_call()`'s response handling treats ANY
`status != 200` as fatal via `fail_json` — GET has no "404 = not found,
that's fine" carve-out. So `GET /api/mo/{dn}.json` on a nonexistent-but-
well-formed DN **must** be `200 {"imdata":[],"totalCount":"0"}`, never 404
(§5 above) — otherwise the first task of any `state=present` playbook
against a not-yet-created object fails immediately.
**`DELETE /api/mo/{dn}.json` (`state=absent`).** `delete_config()`:
```python
if not self.existing and not self.suppress_previous:
return
elif not self.module.check_mode:
self.api_call("DELETE", self.url, None, return_response=False)
```
The module itself is idempotent at the CLIENT level — it only ever issues
DELETE after its own GET confirmed the object exists, so it never actually
sends a DELETE for an already-absent DN. This sim implements the route as
server-side idempotent too (200 + empty envelope whether the DN existed or
not) rather than 404-on-missing: (a) it matches this codebase's existing
MIT delete semantics (`MITStore.upsert`'s `status="deleted"` branch is
unconditional, no existence check) and (b) it's the safer default for any
OTHER client that DOES call DELETE without a prior GET. An existing DN's
entire subtree is removed (`MITStore.delete(dn)`, public wrapper added in
PR-9 around the pre-existing `_remove_subtree` helper `upsert` already used).
**`status` attribute.** cisco.aci's `payload()` never sets a `status` key —
delete goes through the dedicated DELETE verb above, not a POST body with
`status:"deleted"`. That POST-body form is a Fabric Build / aci-py
convention this sim already supported (§3) and continues to: `status:
"deleted"` on any planned `(class, attrs)` node removes that DN's entire
subtree (verified: `MITStore.upsert` already special-cased this pre-PR-9;
PR-9 added regression coverage). Any other `status` value (including the
literal string `"created,modified"`, which real APIC/aci-py tooling can
emit) is ignored and the write proceeds as a normal upsert — no special
handling, no error.
**`annotation`/`ownerKey`/`ownerTag`/`nameAlias` passthrough.** cisco.aci's
`aci_annotation_spec()` defaults `annotation` to the literal string
`"orchestrator:ansible"` and `payload()` injects it (plus `ownerKey`/
`ownerTag` when set) into `proposed["attributes"]` for every managed
object, unconditionally. `name_alias` is a separate per-module spec, also
folded into the same attributes dict when set. None of these are special
sim attributes — they round-trip because `MO.attrs` is a generic
`dict[str,str]` and the write path never allowlists known attribute names;
this section just documents and regression-tests that this passthrough
survives POST → `mo` GET → class query unchanged.
**Certificate signature auth (accept-mode).** cisco.aci supports
password-less auth via `private_key`; `cert_auth()` sets a single `Cookie`
header per request (never touches `APIC-cookie` / never calls aaaLogin):
```python
sig_dn = "uni/userext/user-{username}/usercert-{certificate_name}"
self.headers["Cookie"] = (
"APIC-Certificate-Algorithm=v1.0; "
f"APIC-Certificate-DN={sig_dn}; "
"APIC-Certificate-Fingerprint=fingerprint; "
f"APIC-Request-Signature={base64(RSA-SHA256(method+path+body))}"
)
```
This sim implements **accept-mode**: `require_session()` treats a request
carrying all four `APIC-Certificate-*`/`APIC-Request-Signature` cookies as
authenticated, for the user named in `APIC-Certificate-DN`, PROVIDED that
user matches `SIM_USERNAME` — WITHOUT verifying the RSA-SHA256 signature
bytes at all. A missing cookie, an unparsable `APIC-Certificate-DN`, or a
DN naming a different user all fall through to the normal cookie-session
check (which also fails for a request that never called aaaLogin),
yielding the standard 403 envelope. This is a deliberate simulator trust
model — see docs/DESIGN.md's "Certificate accept-mode trust model" note.
`SIM_CERT_STRICT` env var is a documented placeholder for a future
signature-verifying mode; it currently has no effect.
**firmwareCtrlrRunning.** Seeded per controller (`build/fabric.py`,
alongside the existing controller `topSystem`) at
`{ctrl_dn}/ctrlrfwstatuscont/ctrlrrunning`, `version` = `site.apic_version`
— the same value `topSystem.version` already carried, so the two can never
drift apart. Small, trivially-derived addition; not previously built.
+829
View File
@@ -0,0 +1,829 @@
# aci-sim — DESIGN
A software simulator that faithfully impersonates the REST APIs of a **2-site Cisco ACI
fabric** (two per-site APICs + one ND/NDO), driven by an editable `topology.yaml`, so that
pointing local **autoACI** at it makes Topology, NDO views, all query plugins, and Fabric
Build render internally-consistent, believable control- + data-plane data.
The contract we emulate is in `docs/CONTRACT.md` (authoritative — trust it, don't re-explore autoACI).
## Locked decisions (owner-approved)
1. **Generic MIT engine** — model each APIC as a real Managed Information Tree (DN-keyed
objects, parent/child) with a query resolver implementing APIC's class/mo/node-scoped +
`rsp-subtree`/`target-subtree-class` + `eq/wcard/and/or` + pagination semantics. One YAML →
full object graph; every current AND future autoACI query works with no per-class code.
2. **Accept-reflect writes + reset/snapshot**`POST /api/mo` and NDO schema/deploy upsert
into the live MIT so read-back + Topology show the change (enables push→verify playbook
testing). In-memory, with `reset` (to baseline) and `snapshot`/restore.
3. **Static deterministic snapshot** — stable, internally-consistent, reproducible data (no
liveness engine in v1). Deterministic so views are easy to verify.
4. **Configurable topology** — YAML parametrizes spine/leaf counts; leaves can be added/removed
(statically via YAML+reload, dynamically via the `add_leaf_switch_pair` playbook's
`fabricNodeIdentP` write reaction). Border-leaf vPC → CSW, 2 sites + ISN, full L3Out subtree.
5. **Scope** — ACI + NDO REST surface (~22 REST playbooks fully testable + the ACI half of
migration). **NX-OS SSH is OUT of v1** (extension seam left, no auth stub).
6. **Stack** — Python 3.11+, FastAPI + uvicorn + Pydantic v2, PyYAML, httpx (tests). Self-signed
TLS on each host (autoACI uses `verify=False`). Matches autoACI's stack.
## Process / host layout
One process, three HTTPS apps on three localhost ports (self-signed certs in `certs/`):
`APIC Site-A :8443`, `APIC Site-B :8444`, `NDO :8445`. Each site APIC serves its own MIT;
both sites are built from one global topology so stretched tenants + ISN peering are consistent.
The NDO app derives its schema model from the same topology (same tenant/VRF/BD names).
Point autoACI: Site-A→`127.0.0.1:8443`, Site-B→`127.0.0.1:8444`, NDO→`127.0.0.1:8445`.
## Module map + interfaces (the contracts subagents implement against)
### `mit/` — object store (foundation; no deps)
- `mo.py`: `MO` = `{class_name:str, attrs:dict[str,str], children:list[MO]}`; `attrs["dn"]` is the key.
Helpers: `MO(cls, **attrs)`, `.dn`, `.add_child(mo)`, `.flat()` (self+descendants).
- `dn.py`: bracket-aware DN utils — `dn_split(dn)->list[str]` (never split inside `[]`),
`parent_dn(dn)`, `rn_of(dn)`, `dn_class_hint`, `pod_from_dn`, `name_from_dn`.
- `store.py`: `MITStore``add(mo)`, `upsert(mo)` (merge attrs + recurse children; honor
`status=deleted`), `get(dn)->MO|None`, `children(dn, classes=None)`, `subtree(dn, classes=None)`,
`by_class(cls)->list[MO]`, `all()`; maintains a `dn→MO` map and a `parent→[child dn]` index.
Deterministic iteration order (insertion order).
### `query/` — ACI query engine (deps: mit)
- `filters.py`: parse `eq/wcard/and/or(...)` strings → predicate `fn(MO)->bool`. Bracket/quote aware.
- `engine.py`: `run_class_query(store, cls, params)`, `run_mo_query(store, dn, params)`,
`run_node_scoped(store, node_dn, cls, params)`. Params object carries filter, subtree mode
(`rsp-subtree` children|full or legacy `query-target=subtree`), subtree classes, page, page_size.
Returns `(imdata:list[dict], total:int)` in the exact `{cls:{attributes,children}}` shape.
**Two subtree shapes (must match real APIC + autoACI):** (a) legacy `query-target=subtree` (autoACI's
`query_dn(subtree=True)`) → descendants FLAT as top-level imdata, filtered by `target-subtree-class` (root
included only if it matches); consumers read the target class at the top level, never recurse into `children`.
(b) modern `rsp-subtree=children|full` (autoACI's `query_class(subtree=…)`) → matched objects with NESTED
`children` (children=one level, full=recursive). Pagination slices the result set; `total` = pre-pagination count.
### `topology/` — the YAML contract (deps: none)
- `schema.py`: Pydantic v2 models for the whole topology (see `topology.yaml` + §Topology schema below).
Validates counts, references (bd→vrf, epg→bd, l3out→vrf), site/ISN config. Fail loudly on bad refs.
Also validates (PR-8, findings #27/#28):
- `fabric.loopback_pool`/`fabric.oob_pool` fall within the fixed `10.0.0.0/8` /
`192.168.0.0/16` ranges that `build/fabric.py`'s `loopback_ip()`/`oob_ip()`
actually generate (those functions are not parameterized by an arbitrary CIDR
— 6+ call sites across `build/*.py` call them with just `(pod, node_id)` — so
the pool fields describe, and are checked against, the builder's real output
rather than being purely cosmetic).
- CSW peering params duplicated between `site.border_leaves[].csw` (authoritative
— carries `local_ip`/`vlan` too, used by `l3out.py` for the physical
sub-interface/`l3extMember`) and `tenant.l3outs[].csw_peer` (used for the
L3Out's `bgpPeerP`/static-route nexthop) must agree on `name`/`asn`/`peer_ip`
for any border-leaf the l3out references, or loading fails with a clear error.
- `loader.py`: `load_topology(path)->Topology`.
### `build/` — generators (deps: topology, mit)
Each sub-builder is `build(topo, site, store)` and only ADDS MOs. `orchestrator.build_site(topo, site)
-> MITStore` runs them in dependency order and returns a populated store; `build_all(topo)->{site:store}`.
Sub-builders (one concern each, <~200 lines):
- `fabric.py` — fabricNode (roles/ids/model/serial/version from YAML), topSystem, fabricHealthTotal, vpcDom/vpcIf (border-leaf pair), infraWiNode.
- `cabling.py` — fabricLink (with `/lnk-…/` DN), lldpAdjEp + cdpAdjEp derived STRICTLY from the cabling graph, isisAdjEp.
- `underlay.py` — bgpInst (local ASN), ospfAdjEp/**ospfIf**✅ (spine↔leaf), IS-IS dom.
- `overlay.py` — intra-site BGP-EVPN (spines as RR, leaves as clients): bgpPeer/bgpPeerEntry(established)/bgpPeerAfEntry;
**ISN**: on each spine's `sys/bgp/inst`, add inter-site bgpPeer with peer addr /32 + remote (other-site) ASN.
- `endpoints.py` — fvCEp/fvIp distributed across leaves+EPGs (encap/fabricPathDn consistent), epmMacEp/epmIpEp, fvIfConn.
Runs BEFORE `tenants.py` (PR-3b-batch1 reorder) so `tenants.py` can read back real endpoint paths for `fvRsPathAtt`.
- `tenants.py` — fvTenant/fvCtx/fvBD/fvSubnet/fvAp/fvAEPg + fvRs* rels + vzBrCP/vzSubj/vzFilter/vzEntry from YAML,
plus **fvRsPathAtt**✅ (static binding, reusing the EPG's own endpoint path) and **vzRsSubjGraphAtt**✅.
- `access.py` — infra AAEP/IPG/port-profiles/selectors/blocks + physDomP/l3extDomP + fvnsVlanInstP/fvnsEncapBlk + rels,
plus the **leaf-interface-policy cluster**✅ (infraAccPortP/infraHPortS/infraPortBlk/infraRsAccBaseGrp/infraAccBndlGrp)
and an `infraInfra` root MO at `uni/infra`.
- `l3out.py` — full l3extOut subtree (LNodeP/LIfP/InstP/subnets/RsEctx/RsL3DomAtt/RsPathAtt) + bgpPeerP/bgpAsP to CSW,
plus the operational bgpPeer/bgpPeerEntry(established) representing the CSW eBGP session; actrlRule for contracts;
plus the **route-control cluster**✅ (rtctrlProfile/rtctrlCtxP/rtctrlSubjP/rtctrlMatchRtDest/rtctrlSetComm/rtctrlSetPref),
**static routes**✅ (ipRouteP/ipNexthopP), **l3extMember**✅, and **ospfExtP**✅.
- `routing.py` (new, PR-3b-batch1) — per-(tenant,vrf) **uribv4Route/uribv4Nexthop**✅ RIB entries on every leaf/border-leaf
(BD subnets as connected routes + a 0.0.0.0/0 default via the L3Out CSW peer on border leaves), and **arpAdjEp**✅ —
one per locally-learned endpoint, reusing that endpoint's real MAC/IP. Runs after `endpoints`.
- `interfaces.py` — l1PhysIf + ethpmPhysIf (per node, ports incl. fabric uplinks + host ports), ethpmFcot, eqptcapacityPolUsage5min.
- `health_faults.py` — healthInst per node/tenant (`…/sys/health`, `/tn-X/health`), a seeded believable faultInst set
(node-local DNs so node-scoped queries work), configExportP/configJob, coopPol/coopInst.
- `userprofile.py` — static `uni/userprofile-admin` (aaaUserEp) + `aaaUserDomain` child (name="all"), so the
CONTRACT §1 login probe (`query-target=subtree&target-subtree-class=aaaUserDomain`) resolves instead of 404ing.
### `rest_aci/` — APIC FastAPI app (deps: query, mit, build)
- `auth.py` — aaaLogin/aaaRefresh/aaaLogout, token+cookie, the topSystem/userprofile login probes.
- `app.py``make_apic_app(site_state)`: routes `/api/aaaLogin.json`, `/api/aaaRefresh.json`,
`/api/aaaLogout.json`, `/api/class/{cls}.json`, `/api/mo/{dn:path}.json` (GET+POST),
`/api/node/class/{dn:path}/{cls}.json` (node-scoped) and `/api/node/class/{cls}.json`
(fabric-wide — equivalent to `/api/class/{cls}.json`; PR-6 FIX 3). Parses query params →
query engine → envelope. Error mapping (§5 of CONTRACT).
- `writes.py` — POST body → upsert into MIT (recurse children, honor status); `fabricNodeIdentP`
reaction materializes a full node-registration set (fabricNode + topSystem + healthInst +
fabricLink/lldpAdjEp/cdpAdjEp cabling to every spine + l1PhysIf inventory), fires for a
`fabricNodeIdentP` anywhere in the planned write (not just top-level), and derives pod from the
target site instead of hardcoding it (PR-6 FIX 2; shared with `/_sim/add-leaf`). Returns
created/modified imdata.
### `ndo/` — NDO (deps: topology, mit optional)
- `model.py``build_ndo_model(topo)`: sites, tenants, schemas/templates (vrfs/bds/anps/epgs/contracts/filters/externalEpgs
with `/vrfs/…` `/bds/…` refs), template summaries + tenantPolicy DHCP, fabric-connectivity, policy-states, audit records.
Names MUST match the APIC MIT (cross-plane consistency).
- `app.py``make_ndo_app(ndo_state)`: login/version + all §7 GETs + schema/deploy POST (store + return id/status).
### `runtime/` — supervisor (deps: all)
- `config.py` — ports, cert paths, topology path, host→site map (env-overridable).
- `supervisor.py` — load topology → `build_all` + `build_ndo_model` → start 3 uvicorn servers (asyncio) with TLS.
Keeps a `baseline` deep-copy of each store for `reset`.
### `control/` — admin API (mounted on each app under `/_sim/…`, deps: runtime state)
- `reset` (restore baseline), `snapshot`/`restore` (named), `reload` (re-read YAML + rebuild against
the FRESH matching `Site` object from the just-loaded topology — matched by `Site.id`, falling back
to `Site.name` — not the stale pre-reload one, so edits to a site's nodes/leaves actually take
effect; returns a 400 APIC error envelope if the site no longer exists in the reloaded topology
instead of crashing or silently keeping stale state; PR-6 FIX 1), `add-leaf`/`remove-leaf` (runtime
mutate + rebuild affected derived data — `add-leaf` shares `writes.py`'s enriched node-registration
reaction, not a bare `fabricNode`). Not part of the APIC/NDO contract — a side control plane for
testing.
## Internal-consistency rules (the "尽可能真实" core — enforce in build/)
- **Nothing free-floating.** Every operational MO derives from a topology fact.
- Cabling graph is the single source for `fabricLink`, `lldpAdjEp`, `cdpAdjEp` (neighbors ↔ actual links).
- IS-IS/OSPF underlay adjacencies only between physically-cabled spine↔leaf pairs.
- BGP-EVPN overlay: spines = route-reflectors, leaves = clients (sessions to spine loopbacks), operSt=established.
- ISN: for each spine, an inter-site eBGP-EVPN peer to the remote site's spines — addr /32, remote ASN =
the OTHER site's fabric ASN (this is what makes autoACI draw the ISN cloud). EVPN AF up.
- L3Out: border leaves peer eBGP to CSW (asn from YAML); the l3extOut config subtree AND the operational
bgpPeer(Entry) to the CSW must both exist and agree (addr/asn).
- Endpoints: each fvCEp lives on a real leaf port, in a real EPG/BD, with a plausible IP in the BD subnet;
epmMacEp/epmIpEp mirror them on the owning node. Endpoint host ports never overlap the APIC-controller
port range on the same leaf (see PR-5 FIX 3 below). A stretched-tenant endpoint is local (front-panel port)
on exactly ONE site and tunnel/vxlan-learned on the other — never front-panel-local on both (PR-5 FIX 4).
- Health/faults: healthInst per node/tenant/controller; a small seeded faultInst set with node-local DNs so
counts render and node-scoped fault queries return.
- Two sites share stretched tenants (per YAML `stretch:true`) → same tenant/VRF/BD names present in both MITs
+ NDO schema references them; site-local tenants present only in their site.
## PR-5 topology-consistency fixes (findings #13-#18)
Six internal-consistency findings, all fixed in `build/`. Every port an operational MO references
(`l3extRsPathL3OutAtt.tDn`, `ospfAdjEp` interface, `fvCEp.fabricPathDn`) must resolve against a real
`l1PhysIf` built by `interfaces.py` for that exact node — this is now a checked invariant
(`tests/test_pr5_topology_consistency.py`).
**Full port inventory per node role** (see `interfaces.py` module docstring for the authoritative version):
- Spines: `eth1/1..4` fabric uplinks (one per downlink leaf/BL) + `eth1/49[,50,...]` dedicated ISN/IPN
uplink ports, **multi-site topologies only**, one per spine (`eth1/{49+spine_index}` — matches the port
`underlay.py`'s `ospfAdjEp` already referenced; previously these ports didn't exist at all → FIX 2).
- Leaves + border-leaves: `eth1/1..8` host-facing access ports, `eth1/49[,50,...]` fabric uplinks (one per
spine). Border-leaves ONLY also get a dedicated `eth1/48` routed L3Out sub-interface port (previously
`l3out.py` referenced this port but nothing ever built it → FIX 1).
**FIX 1 (#13) — L3Out port.** Chose to extend the border-leaf port inventory with a dedicated `eth1/48`
routed port (`mode="routed"`, `usage="l3out"`) rather than reassign the L3Out onto an existing host port
(e.g. `eth1/8`), because real ACI border-leaf L3Out sub-interfaces conventionally live on a dedicated
front-panel port, not one shared with regular EPG/host traffic. `l3out.py` was unchanged — it already
referenced `eth1/48`; the fix makes that port real.
**FIX 2 (#14) — spine OSPF/ISN port.** Chose to extend the spine port inventory with a dedicated
`eth1/{49+i}` ISN uplink (one per spine, multi-site only) rather than reuse an existing fabric-uplink port,
because every spine port in `eth1/1-4` is already consumed by the intra-fabric spine↔leaf mesh (full mesh
= 2 leaves + 2 border-leaves = 4 downlinks per spine in the default topology) — there is no free fabric port
to reuse. This also matches real ACI: multi-site spines use a dedicated ISN-facing port, distinct from the
leaf-facing fabric ports. `underlay.py` was unchanged — its `ospfAdjEp` already used `eth1/{49+si}`; the fix
makes that port real via the same index arithmetic.
**FIX 3 (#15) — endpoint/APIC port separation.** `fabric.py` wires each `APIC-{cid}` (cid 1..
`site.controllers`, default 3) to `eth1/{cid}` on the first two leaves (`site.leaf_nodes()[:2]`) — the same
leaves `endpoints.py` places host endpoints on. `endpoints.py` now reserves `eth1/1..{controllers}` on those
leaves and starts endpoint port allocation at `eth1/{controllers+1}`, wrapping within the remaining
`_HOST_PORTS - controllers` budget. Non-APIC-attached leaves are unaffected (still start at `eth1/1`).
**FIX 4 (#16) — multi-site endpoint semantics.** See below.
**FIX 5 (#17) — BD unicastRoute.** `unicastRoute` now reflects whether the BD actually has `fvSubnet`
children (routed) rather than `l2stretch` — a stretched BD can still carry routed subnets, and this
topology's stretched BDs (`bd-BD3_LAB0`, `bd-BD4_LAB0`) do. This agrees with `ndo/model.py`, which
unconditionally reports `unicastRouting: True` for every BD (all BDs in `topology.yaml` have subnets).
**FIX 6 (#18) — controller health.** `health_faults.py` now emits a `healthInst` at
`topology/pod-{pod}/node-{cid}/sys/health` for every controller node id (1..`site.controllers`), matching
the `topSystem` `fabric.py` already builds there. Real APIC exposes controller health the same way as
switch health.
### FIX 4 — multi-site endpoint representation (design)
Real ACI never shows the identical endpoint as locally front-panel-attached at both sites of a stretched
tenant simultaneously. One site has it physically attached; the other only knows about it via the
multi-site overlay (ISN) tunnel. `endpoints.py` now models this:
- **HOME site** (physically attached): `fvCEp.lcC="learned"`, `fabricPathDn` = the real front-panel leaf
path (`topology/pod-{pod}/paths-{leaf_id}/pathep-[eth1/{port}]`), epm mirrors use `flags="local"`, and a
`fvIfConn` is emitted (real front-panel port).
- **REMOTE/peer site** (learned via the ISN): same MAC/IP/DN, but `fvCEp.lcC="learned,vxlan"` and
`fabricPathDn` points at a `tunnelIf` on one of the local spines
(`topology/pod-{pod}/paths-{spine_id}/pathep-[tunnel{remote_spine_id}]`), epm mirrors use
`flags="learned,vxlan"`, and no `fvIfConn` is emitted (there is no front-panel port to attach one to).
- A minimal `tunnelIf` MO is created per (local spine, remote spine) pair used
(`topology/pod-{pod}/node-{spine_id}/sys/tunnel-[tunnel{remote_spine_id}]`), `dest` = the remote spine's
loopback (representing its spine-proxy anycast TEP), `src` = the local spine's own loopback. `overlay.py`
does not yet build tunnel/TEP objects, so this is a new minimal object, not a reuse.
- **HOME-site assignment is deterministic and stateless across the two independent per-site `build()`
calls**: for the i-th endpoint in a stretched tenant's sequence, home site =
`tenant.sites[i % len(tenant.sites)]`. Since both sites iterate the exact same tenant/AP/EPG/per-EPG
sequence in the same order, they agree on which site is home for a given MAC without sharing any state —
each site's independent build() computes the same `i` for the same endpoint and reaches the same
conclusion.
- Non-stretched (site-local) tenants are unaffected — every endpoint is simply local, exactly as before.
## Topology schema (topology.yaml) — shape
```
fabric: { name, evpn_rr: spine }
sites:
- name, id, apic_host, asn, pod
spines: N # count OR explicit list [{id,name,model,serial}]
leaves: M # count OR explicit list
border_leaves: [{id, name, vpc_domain, csw:{name,asn,peer_ip,local_ip,vlan}}]
cabling: auto|explicit # auto = full spine×leaf mesh + border-leaf uplinks
isn: { enabled: true, transit_asn?, peer_mesh: full } # drives inter-site EVPN
tenants:
- name, stretch: bool, sites: [..]
vrfs: [{name}]
bds: [{name, vrf, subnets:[cidr], l2stretch?}]
aps: [{name, epgs:[{name, bd, contracts:{provide:[],consume:[]}}]}]
contracts: [{name, scope, filters:[{name, entries:[{proto,from,to,ether}]}]}]
l3outs: [{name, vrf, border_leaves:[..], csw_peer:{...}, ext_epgs:[{name, subnets:[cidr]}]}]
endpoints: { per_epg: K, ip_pool_from_subnet: true } # generation knobs
faults: { seed: [{severity,code,node,descr}] , health_defaults:{node:95,tenant:98} }
access: { vlan_pools:[{name,from,to}], phys_domains:[..], aaeps:[..] } # sensible defaults if omitted
```
Default `topology.yaml` ships the spec baseline: 2 sites × (2 spine + 2 leaf + 1 border-leaf vPC pair to CSW),
ISN between them, a couple stretched + site-local tenants, one L3Out per site to its CSW, seeded endpoints/faults.
## Phase plan (each phase: implementer subagent(s) → independent reviewer per 第十律)
- **P1 foundation** — `mit/` (mo, dn, store) + `query/` (filters, engine). Unit-tested in isolation.
- **P2 topology** — `topology/schema.py` + `loader.py` + default `topology.yaml`.
- **P3 fabric builders** — fabric, cabling, underlay, overlay(+ISN), health_faults, interfaces → Topology view renders.
- **P4 tenant builders** — tenants, access, l3out, endpoints → config + data-plane plugins render.
- **P5 ACI REST app** — auth + GET routes + POST writes + reactions + runtime supervisor + TLS + control API.
- **P6 NDO** — model + app.
- **P7 verify** — verification harness that mimics autoACI's queries for every view/plugin + write→readback +
add_leaf; README + run scripts.
## Verification (第二律 evidence-first)
`tests/verify_autoaci.py` replays the exact query patterns from `docs/CONTRACT.md` §89 against the running
sim and asserts non-empty, shape-correct, internally-consistent results for: Topology (2 sites + ISN cloud +
vpc pairs + faults/health), every plugin's class set, L3Out BGP-to-CSW, NDO sites/tenants/schemas, and a
write→read-back cycle (POST a tenant → GET returns it) + an add_leaf reaction. Real command output is the
acceptance evidence.
## PR-3b-batch1 — 21 previously-catalogued-but-unbuilt classes seeded
CONTRACT.md §6 catalogued 21 classes across 6 concern groups that had no builder ever emitting them, so
their class queries returned `totalCount: 0` and the depending autoACI plugins (access_policy.py,
epg_detail.py, contract_detail/contract_map/contract_debug.py, l3out_detail.py, l3out_status.py,
route_control.py, routing_state.py, route_table.py, fabric_ospf.py) rendered nothing. All 21 are now seeded,
derived from the existing topology/tenant objects rather than invented data.
**Access-policy cluster (`build/access.py`)** — one leaf-interface-policy tree per border-leaf vPC domain
(`infraAccBndlGrp` lagT="node", consistent with the vpcDom/vpcIf pair `fabric.py` already builds for the same
border leaves) plus one for the regular-leaf pair (lagT="link", a PC bundle). Each `infraAccPortP` gets one
`infraHPortS` (type="range") selector spanning the real host-port inventory (`eth1/1..eth1/_HOST_PORTS`) via
one `infraPortBlk` + one `infraRsAccBaseGrp` pointing at the bundle group. Also seeds an `infraInfra` MO at
`uni/infra` itself — previously a structural-only DN with no MO, which 404'd `GET /api/mo/uni/infra.json`
even under `rsp-subtree=full` (fixed as part of this batch since the TESTS requirement needed it; the generic
subtree engine already worked correctly once a root MO existed).
**Route-control cluster (`build/l3out.py`)** — one `rtctrlProfile` (export) per L3Out, with a nested
`rtctrlCtxP` (+ `rtctrlSetComm`/`rtctrlSetPref` children using real ACI's fixed short-form RNs `scomm`/`spref`).
One tenant-level `rtctrlSubjP` match rule per L3Out (not nested under the L3Out — autoACI's route_control.py
queries `rtctrlSubjP` with `wcard(dn,"tn-{tenant}")`, tenant-scoped, verified read-only against the plugin
source) with an `rtctrlMatchRtDest` child reusing a real tenant BD subnet. A static default route (`ipRouteP`
ip="0.0.0.0/0") nested under each L3Out's `l3extRsNodeL3OutAtt`, with an `ipNexthopP` child pointing at the
same CSW peer IP the eBGP session builder (`_upsert_ebgp_sessions`) already peers with. `l3extMember`
(side="A") on the existing `l3extRsPathL3OutAtt` — this topology's L3Out path is a single port per
border-leaf (not vPC-bundled), so only one member is emitted, per the batch-1 placement note. `ospfExtP`
added to the SAME L3Out that already carries `bgpExtP` (real ACI supports OSPF+BGP coexistence on one
L3Out) rather than fabricating a second OSPF-only L3Out in topology.yaml — the simpler faithful choice since
this topology has no existing OSPF-only L3Out to attach to.
**EPG/contract classes (`build/tenants.py`)**`fvRsPathAtt` (static binding) per EPG, pointing at the SAME
front-panel path that EPG's own endpoints already use. This required reordering the orchestrator's builder
list (`endpoints` now runs BEFORE `tenants`, moved up from after `l3out`) so `tenants.py` can read back the
real `fvCEp.fabricPathDn`/`encap` endpoints.py already wrote for that EPG, instead of independently
re-deriving the port-allocation algorithm (which risked silently drifting out of sync). Every other builder
remains pure add-only; `endpoints.py` itself never reads the store, so the reorder is safe. A stretched EPG
whose only endpoints at this site are REMOTE (tunnel-learned) gets no `fvRsPathAtt` — a static binding must
reference a real local port. `vzRsSubjGraphAtt` attached to exactly one contract's subject per tenant (the
tenant's first contract), with a plausible `tnVnsAbsGraphName`; `vnsAbsGraph` itself stays out of scope.
**Operational/routing classes (new `build/routing.py` + `build/underlay.py`)**`uribv4Route`/`uribv4Nexthop`
seeded per (tenant, vrf) on every leaf/border-leaf: BD subnets as directly-connected routes (nexthop = the
node's own loopback) plus a 0.0.0.0/0 default via the L3Out's CSW peer on border leaves, matching the
`ipRouteP` static route already seeded on the same node. VRF "dom" name reuses the exact `{tenant}:{vrf}`
convention `l3out.py`'s BGP dom already established. `uribv4Nexthop`'s RN uses the 4-bracket
`nh-[proto]-[addr]-[ifname]-[vrf]` shape autoACI's `_RE_NH_PARENT` regex (`route_table.py`/`routing_state.py`,
identical in both files) requires — verified read-only against the plugin source, not re-derived from
guesswork. `arpAdjEp` — one per locally-learned (front-panel) endpoint, reusing that endpoint's real MAC/IP
from the store `endpoints.py` already populated (routing.py runs after `endpoints` in the orchestrator
order); REMOTE/tunnel-learned endpoints get no ARP entry (real ACI resolves those via the overlay, not local
ARP). `ospfIf` nested between the existing `ospfDom` and `ospfAdjEp` in `underlay.py`, on the same
`eth1/{49+i}` ISN uplink port `interfaces.py` already builds a dedicated `l1PhysIf` for — so the pre-existing
`ospfAdjEp` now sits on a real, matching `ospfIf`.
**Writes**`rest_aci/writes.py` `_RN_TEMPLATES` gained entries for all POST-able batch-1 classes so a
child posted without an explicit `dn` lands on the same canonical DN a GET or builder-created MO would use.
## PR-3b-batch2/3 — remaining previously-catalogued-but-unbuilt classes seeded
Batches 2+3 of the seeding effort closed the rest of CONTRACT.md §6's gap: 9 more classes across
`build/fabric.py`, `build/interfaces.py`, `build/tenants.py`, a new `build/zoning.py`, and `build/overlay.py`,
plus a deliberate decision NOT to build `infraNodeIdentP`. DN/attribute shapes were verified read-only
against the specific autoACI consumer plugins (a targeted grep of autoACI's `backend/plugins/*.py`)
before writing any builder code, per the same discipline batch-1 established.
**Port-channel/vPC (`build/fabric.py`)**`pcAggrIf` + `pcRsMbrIfs`, one pair per vPC leg (border-leaf),
consistent with the `vpcDom`/`vpcIf` this module already builds for the same pairs. `vpc_status.py` matches
`pcAggrIf` to `vpcIf` by `(node, name)` — so `pcAggrIf.name` intentionally reuses the exact same `"po{bl.id}"`
string `vpcIf.name` already carries (not `access.py`'s differently-named `infraAccBndlGrp`, a different IPG
identity autoACI's plugin does not join on). **Naming correction**: CONTRACT.md catalogued this relation
class as `vpcRsMbrIfs`; a read-only grep of `vpc_status.py` showed it actually queries `pcRsMbrIfs` (the real
ACI class, child of `pcAggrIf`, RN `rsmbrIfs-[{ifDn}]`) — built as `pcRsMbrIfs`, with a CONTRACT.md §6
changelog note recording the correction. Member ports are two real host-facing `l1PhysIf` per leg
(`eth1/1`, `eth1/2` — always present on every leaf/border-leaf) so `pcRsMbrIfs.tDn` always resolves.
**Optics (`build/interfaces.py`)**`ethpmFcot`, sibling of `ethpmPhysIf` under the same port DN (own
`fcot` RN). Emitted only for fabric-uplink, ISN-uplink, and L3Out routed ports — real optics-bearing
front-panel ports — not plain host-access ports, mirroring real labs where copper/DAC host ports often
report no transceiver inventory while fabric/WAN-facing optical ports do (a deliberate realism choice,
documented in the module docstring, not an omission).
**Cluster health (`build/fabric.py`)**`infraWiNode`, the APIC appliance-vector: every controller carries
its own view of every cluster member (an N×N matrix for N=`site.controllers`), all seeded `health="fully-fit"`
to match this sim's other all-healthy defaults (`fabricHealthTotal.cur=100`).
**Zoning-rule / pcTag plumbing (`build/tenants.py` + new `build/zoning.py`)** — real ACI zoning rules key off
per-EPG/VRF `pcTag` ("sclass") + `scopeId`, neither of which existed anywhere in this sim before batch-2
(verified: zero prior `pcTag` references). `tenants.py` gained `pctag_for`/`scope_id_for` — small
CRC32-derived helpers (same technique `endpoints.py`'s `_vnid` already uses) offset clear of autoACI's 5
hardcoded system pcTags (1/13/14/15/16384) — assigning a stable `pcTag` to every `fvAEPg` and `fvCtx`
(+ `scope` on `fvCtx`), and seeding one `fvEpP` (`uni/epp/fv-[{epg_dn}]`) per EPG carrying the matching
`pcTag`, the EPG's VRF `scopeId`, and `epgPKey`=the EPG's own dn (`zoning_rules.py` extracts the friendly
EPG name back out of `epgPKey`'s trailing `epg-` segment). New `build/zoning.py` derives `actrlRule` — one
permit rule per (contract subject filter, provider EPG, consumer EPG) triple, emitted on every leaf where
either side has a locally-learned endpoint (read back from `endpoints.py`'s `fvCEp`, same technique
`tenants.py`'s `fvRsPathAtt` already established), with `sPcTag`/`dPcTag`/`scopeId` reusing the exact
`pctag_for`/`scope_id_for` values the owning EPGs/VRF already carry. Also seeds real MOs at the
`.../sys/actrl` and `.../sys/actrl/scope-[...]` structural containers — the same class of gap batch-1's
`infraInfra` fix addressed for `uni/infra`: without a real MO at the exact DN a subtree query targets,
`GET /api/mo/{dn}.json` 404s even though the generic subtree engine itself works fine once a root MO exists.
`zoning` runs right after `tenants` in the orchestrator (both already after `endpoints`).
**EVPN routes (`build/overlay.py`)**`bgpVpnRoute` + `bgpPath`, the per-spine EVPN VPN route family
`fabric_bgp_evpn.py`'s "vpn_routes" view two-pass parses (pass 1 collects routes by dn; pass 2 recovers
the parent route dn from a path dn via `path_dn.rsplit("/path-", 1)[0]` — so `bgpPath`'s RN must start with
`path-` directly under the route's own dn, verified read-only against the plugin). Seeded per spine
(the EVPN route-reflector role) for each BD subnet deployed at the site, with one `bgpPath` per leaf that
actually hosts a locally-learned endpoint for that BD (read back from `fvRsBd`+`fvCEp`). This needs the
same post-`tenants`/`endpoints` data `zoning.py` needs, but `overlay.build()` itself must stay in its
original EARLY orchestrator slot (BGP peer setup only needs fabric/topology data) — so the new logic lives
in a second function, `overlay.build_vpn_routes()`, called by the orchestrator as an explicit extra step
after the main `_BUILDERS` loop completes, rather than moving/splitting `overlay.build()`'s existing peer
logic.
**Node registration (`build/fabric.py` + `rest_aci/writes.py`)**`fabricNodeIdentP`, boot-seeded one per
real switch `fabricNode` (spines/leaves/border-leaves — NOT controllers; real ACI's Fabric Membership
registers switches joining the fabric, not the APICs themselves) at `uni/controller/nodeidentpol/nodep-{id}`,
so a GET returns already-registered nodes without requiring a prior POST. Keys on the node **id**, not
serial, to stay consistent with `writes.py`'s pre-existing `_materialize_fabric_node` reaction (which parses
the id back out of the same `nodep-(\d+)$` pattern) — a boot-seeded and a later-POSTed registration must
agree on the identifier scheme, or they'd silently diverge into two different DNs for the same node.
`_RN_TEMPLATES` gained a `fabricNodeIdentP -> nodep-{nodeId}` entry (the one batch-2/3 class an actual
Fabric Build playbook POSTs, per CONTRACT.md §3). Regression-tested: `tests/test_pr3b_batch2.py` re-verifies
`test_rest_aci.py::test_add_leaf_reaction`'s exact scenario (POSTing a NEW `nodep-950` must still
materialize a matching `fabricNode`) to confirm boot-seeding existing nodes doesn't interfere with the
write-path reaction for a genuinely new one.
**`infraNodeIdentP` — deliberately NOT built.** No autoACI plugin references it (verified via a fabric-wide
grep across autoACI's `backend/plugins/*.py`). Real ACI's `infraNodeIdentP` is a node *provisioning-policy*
object, a different concept from `fabricNodeIdentP`'s Fabric Membership registration despite the similar
name — building it with `fabricNodeIdentP`-like semantics would be a wrong object satisfying no real
consumer. Removed from CONTRACT.md's catalog entirely rather than built incorrectly; see that file's
batch-2/3 changelog note for the full reasoning.
## PR-9 — Ansible cisco.aci fidelity
The owner's primary public use case for this simulator is running Ansible `cisco.aci` playbooks against
it directly. PR-9 hardens the write/delete/auth surface to match what `cisco.aci`'s
`plugins/module_utils/aci.py` actually sends, verified read-only against the upstream collection source
(https://github.com/CiscoDevNet/ansible-aci). Full behavioral contract: `docs/CONTRACT.md` §11. Highlights:
- **DELETE `/api/mo/{dn}.json`** added (state=absent), server-side idempotent (200-empty on both
existing-then-removed and already-missing DNs) — see CONTRACT.md §11 for why idempotent-on-missing
was chosen over 404 despite the module never actually exercising that exact case itself.
- **GET-on-missing-DN is 200-empty, not 404** (FEATURE 6, a live blocker found running an actual playbook
against the sim: `fatal: APIC Error 103` on the very first task of a `state=present` create). This
superseded the pre-PR-9 design (404-on-missing), which had been justified as matching a DIFFERENT
consumer's (autoACI's `ACINotFoundError`) expectation — cisco.aci is now this simulator's primary
target per the owner's stated use case, and its `api_call()` treats any non-200 GET as fatal with no
"not found is fine" exception. `/api/class/*` and node-scoped queries already had no such 404 case
(they return `200` + empty on a no-match query); this brings `/api/mo/*` in line for "the DN itself
doesn't exist" too.
- **`status="deleted"` cascade** — already correctly removed the full subtree pre-PR-9
(`MITStore.upsert`'s `status="deleted"` branch predates this PR); PR-9 added regression coverage
proving it (not just "skips planning children", which is a separate, narrower behavior the
write-planning layer also has). `status="created,modified"` (and any status other than `"deleted"`)
is a plain upsert — no special-casing needed since the store only ever special-cases the literal
`"deleted"` string.
- **annotation/nameAlias/descr passthrough** — no fix needed; `MO.attrs` is a generic dict and the
write path never allowlists attribute names. PR-9 added the regression test proving the round-trip
survives POST → mo GET → class query, since this was previously undocumented/unverified rather than
actually broken.
- **firmwareCtrlrRunning** — new, small: one per controller in `build/fabric.py`, `version` reusing the
same `site.apic_version` the controller's `topSystem` already carries.
### Certificate accept-mode trust model (cert-auth, PR-9)
cisco.aci supports password-less certificate-signature auth: instead of calling `aaaLogin`, every request
carries a single `Cookie` header with four fields (`APIC-Certificate-Algorithm`, `APIC-Certificate-DN`,
`APIC-Certificate-Fingerprint`, `APIC-Request-Signature` — an RSA-SHA256 signature over
`METHOD + path + body`, base64-encoded). Real APIC verifies that signature against a public key
previously uploaded for the named user (`aaaUserCert`).
**This simulator does NOT verify the signature.** `require_session()` (`rest_aci/auth.py`) implements
**accept-mode**: if all four cookies are present and `APIC-Certificate-DN` parses to
`uni/userext/user-{user}/usercert-{name}` with `user == SIM_USERNAME`, the request is treated as
authenticated for that user — full stop. The `APIC-Request-Signature` bytes are read (cookie must be
non-empty) but never decoded or checked against any key. This is a deliberate, documented trust
shortcut, not an oversight:
- The simulator has no real user-certificate registry (`aaaUserCert` upload flow is out of scope,
same tier as NX-OS SSH being out of v1 per the "Locked decisions" above) — implementing real
verification would require building that entire subsystem for a security property the simulator
doesn't need (there is no real secret being protected; it's a local test fixture). Trusting the
claimed identity once the DN's user matches `SIM_USERNAME` is sufficient to unblock cert-auth
playbooks running unmodified against the sim, but is insufficient as a real security boundary.
- **Do not point this accept-mode design at anything internet-reachable or multi-tenant.** Anyone who
can reach the sim's port can authenticate as `SIM_USERNAME` by sending four cookies with no
cryptographic proof of possession.
- `SIM_CERT_STRICT` env var is reserved (parsed but currently a no-op) for a **future** mode that would
register a real public key per user and verify `APIC-Request-Signature` against it. Not implemented in
PR-9 — out of scope; noted here so a future PR doesn't have to rediscover the gap.
## PR-18 — Tier-1 fabric configuration parameters
Exposes real-ACI-lab-variable Tier-1 fabric knobs that Ansible/`cisco.aci`/aci-py fidelity depends on,
all optional with defaults matching the pre-PR-18 hardcodes so the existing `topology.yaml` (and any
hand-authored one) validates and builds unchanged.
- **`site.controllers: int` (default 3)** — already existed pre-PR-18 (introduced with the APIC-cluster
work) and was already fully wired: `build/fabric.py`'s controller `fabricNode`/`topSystem`/
`firmwareCtrlrRunning` loop, the `infraWiNode` appliance-vector double loop (both viewer and member
ranges), `build/health_faults.py`, and `build/endpoints.py`'s APIC-reserved-port accounting all already
read `site.controllers` — no hardcoded `3` remained to replace. PR-18 only added `--controllers` to
`aci-sim new` and surfaced the value in `aci-sim show`.
- **`fabric.tep_pool: str` (default `"10.0.0.0/16"`)** — the infra TEP address space. Stored + CIDR-
validated + surfaced (`aci-sim show`/`new`). **No builder derives a per-node address from this field**
`build/fabric.py`'s `loopback_ip()`/`oob_ip()` are the actual address source (see the pre-existing
`loopback_pool`/`oob_pool` note in `topology/schema.py`'s `Fabric` docstring, which documents the same
"hardcoded family, pool is descriptive metadata" pattern this field follows). Treat `tep_pool` as
store-only/informational until a future PR threads it into an actual infra-address builder.
- **`fabric.infra_vlan: int` (default 3967, range 1-4094)** — the fabric infrastructure VLAN. Stored +
range-validated + surfaced. No MIT class in this sim currently carries a distinct infra-VLAN attribute
to wire it into (real ACI's `infraSetPol.fabricInfraVlan` classes are not part of the current CONTRACT.md
catalog) — store-only, same caveat as `tep_pool`.
- **`fabric.gipo_pool: str` (default `"225.0.0.0/15"`)** — the multicast GIPo pool. Stored + CIDR-validated
+ surfaced. Store-only, same caveat.
- **Deterministic node serials** — `build/fabric.py`'s `default_serial(site_id, node_id)` replaces the
previous inline `f"SIM{node.id:08d}"` fallback used whenever an auto-generated `Node.serial` is empty
(`fabricNode.serial` and `fabricNodeIdentP.serial`). New format: `f"SAL{site_id}{node_id:04d}"` (e.g.
site "1" node 101 → `"SAL10101"`) — encodes the site id so serials stay visibly distinct per site.
Explicit `serial:` values in a YAML `Node` entry are unaffected (still override — this is only the
`serial or default_serial(...)` fallback path). No test asserted the old `"SIM..."` format, so this is
a safe behavior change; `aci-sim show` now prints the resolved serial per node.
- **`site.pod: int` (already existed, default 1)** — verified honored end-to-end: every builder that
emits a `topology/pod-{N}/...` DN reads `site.pod` (grepped all of `build/*.py`, `rest_aci/writes.py`,
`control/admin.py` — zero remaining `pod-1`/`pod=1` literals). A site with `pod: 2` builds all its
`fabricNode`/`topSystem`/etc. DNs under `pod-2` (verified: `topology/pod-2/node-201`, ...).
### N-site limit (verified, not fixed by PR-18)
`aci-sim new --sites N` (N>2) and the builders that only depend on `site.pod`/`site.controllers`/node-ID
offsets work for arbitrary N with zero node-ID collisions (verified with a 3-site scaffold: sites at
offsets 0/200/400, `infraWiNode` count = `controllers²` per site, all DNs correct).
**However, `Topology.other_site(name)` (topology/schema.py) is hard-coded to 2-site semantics**: it
returns "the first site in `self.sites` that is not `name`", not "every other site". Two builders call it
to resolve the ISN/multi-site remote peer:
- `build/overlay.py`'s ISN inter-site eBGP-EVPN peering (`build()`, "ISN: each local spine → each
remote-site spine loopback").
- `build/endpoints.py`'s stretched-tenant HOME/REMOTE endpoint placement (`is_stretched` branch).
Verified with a live 3-site topology (`generate_topology(sites=3, ...)`, ISN `peer_mesh: full`): site C
(3rd site, asn 65003) peers ISN eBGP ONLY with site A (asn 65001) — `other_site("LABC")` picks the first
non-C site in list order (`LABA`) and never reaches `LABB`. This means with 3+ sites:
- ISN is NOT a true full mesh across all sites, despite `isn.peer_mesh: full` — each site peers with
exactly one other (the first site in the YAML's `sites:` list order that isn't itself), not N-1 others.
- A tenant with `stretch: true` and 3+ `sites:` entries passes schema validation (the validator only
requires `len(tenant.sites) >= 2`, not `== 2`) but `endpoints.py`'s remote-peer logic will treat only
one of the non-home sites as the ISN-reachable remote for tunnel/REMOTE-endpoint purposes.
**This is a pre-existing limitation, not introduced by PR-18** (confirmed via `git diff main --
aci_sim/topology/schema.py`: `other_site()` is byte-identical to what PR-18 branched from).
Fixing it to a real N-site full mesh is a bigger architectural change (both call sites would need to
iterate `[s for s in topo.sites if s.name != site.name]` instead of a single `other_site()` call, and
`endpoints.py`'s single `remote_site`/`tunnel_spine`/`home_spine` variables would need to become
per-remote-site collections) — out of scope for a Tier-1-parameters PR. **Practical limit: ISN/stretch
correctness is verified for exactly 2 sites; 3+ site topologies build without collision but do not get
a correct N-site ISN mesh or N-site stretched-endpoint placement.**
## PR-19 — Tier-2 topology-elasticity parameters
Exposes ISN underlay knobs (`ospf_area`, `mtu`) and a VMM domain vCenter-connection model, plus fixes a
real leaf/spine-count-elasticity bug in `aci-sim new`'s node-ID generator. All new fields optional with
defaults matching pre-PR-19 hardcoded behavior, so the existing `topology.yaml` validates and builds
byte-identical output.
### `isn.ospf_area: str` (default `"0.0.0.0"`) and `isn.mtu: int` (default 9150)
Both were previously hardcoded literals in the builders:
- `build/underlay.py`'s `ospfIf`/`ospfAdjEp` `area` attribute was hardcoded to `"0.0.0.0"` — now reads
`topo.isn.ospf_area`. Accepts either dotted-decimal (`"0.0.0.0"`) or the decimal-integer shorthand ACI
also renders (`"0"`) — validated in `Topology.normalize_and_validate` via `IPv4Address` parse or
`str.isdigit()`.
- `build/interfaces.py`'s dedicated ISN/IPN spine uplink port (`eth1/{49+i}`, multi-site only) was built
via the same `_add_port()` helper as every other port, which hardcoded `mtu="9216"` (the intra-fabric
default) for every port including this one. `_add_port()` now takes an `mtu` keyword (default `"9216"`,
unchanged for all ordinary ports); the ISN uplink call site alone passes `str(topo.isn.mtu)`. Verified:
a leaf's `eth1/49` fabric-uplink port and a spine's `eth1/49` **ISN** uplink port are different physical
interfaces on different node roles (interfaces.py's port-numbering docstring), so this change touches
only the true ISN port — ordinary fabric ports (including any leaf port that happens to share the same
port *number*) are unaffected. `isn.mtu` range-validated 576-9216 (real ACI inter-pod/inter-site MTU
range, same ceiling as the fabric's own 9216).
- `isn.peer_mesh: partial` remains a **documented validate-and-reject** (unchanged from PR-17/18's
decision in `build/overlay.py`) — re-examined for PR-19 and kept: a partial mesh has no sensible default
partition without additional per-site config this schema doesn't carry (which spine pairs with which
isn't inferable from spine count alone). Silently building full-mesh (ignoring the setting) or an
arbitrary subset (guessing a partition no author asked for) would both be worse than failing fast with a
clear message, so PR-19 keeps the existing rejection rather than inventing a partition scheme.
### VMM domain vCenter connection model — pure addition, not a rewire
`access.vmm_domains: list[VMMDomain]` (default: **empty list**, unlike `phys_domains`/`l3_domains`/
`aaeps`, which each default to one seeded entry). Each `VMMDomain` carries `name` (required) plus
optional `vcenter_ip`, `vcenter_dvs`, `vlan_pool`, `datacenter`.
**Why this is additive, not a wire-up of an existing gap** (verified read-only against a real
aci-py + cisco.mso production chain on a lab test host):
- `~/aci-py/aci_py/playbooks.py`'s `bind_epg_to_vmm_domain` playbook (MSO section) drives
`mso_schema_site_anp_epg_domain` (shimmed in `~/aci-py/aci_py/shims/mso_mso_epg.py`), which appends a
JSON-Patch op `add /sites/{site}-{tpl}/anps/{anp}/epgs/{epg}/domainAssociations/-` with
`value.dn = "uni/vmmp-VMware/dom-{domain_profile}"` — a **DN string**, not a live object reference. The
sim's NDO layer (`aci_sim/ndo/patch.py`) stores this JSON-Patch document verbatim; nothing
cross-validates the referenced `dn` against a real `vmmDomP` in any APIC-side MIT store.
`~/aci-py/environments/lab-multisite-sim/vars/bind_epg_to_vmm_domain_MS-TN1.yml` confirms the domain
names actually exercised in the live MS-TN1 chain (`vmm-vmw-lab1`, `vmm-vmw-lab2`) are just opaque
strings from this var file's perspective.
- The single-fabric branch of the same playbook template (`bind_epg_to_vmm_domain.j2.yml`'s `APIC`
section) DOES drive `cisco.aci.aci_domain` (shimmed in `~/aci-py/aci_py/shims/access.py`'s
`_domain_dn_class`, which maps `vmmDomain``vmmDomP` at `uni/vmmp-{provider}/dom-{name}`) — but this
branch is gated on `single_fabric|default(false)` and is not what the MS-TN1 multi-site chain
(`scripts/live_chain.py environments/lab-multisite-sim MS-TN1`) exercises.
- Conclusion: **no consumer in either verified production chain (autoACI sandbox e2e, aci-py MS-TN1)
reads a `vmmDomP`/`vmmCtrlrP` object from this sim today.** Adding one is genuinely new surface, not a
fix for an existing silently-broken path — hence the empty-list default (true backward-compat baseline:
omitting `access.vmm_domains` produces zero `vmmDomP`/`vmmCtrlrP`/`vmmUsrAccP` MOs, byte-identical to
pre-PR-19) rather than seeding a default domain the way `phys_domains`/`l3_domains`/`aaeps` do.
**MO shape** (`build/access.py::_build_vmm_domain`), verified against the real DN scheme in
`~/aci-py/aci_py/shims/access.py`'s `_domain_dn_class` (`"vmmDomP", "uni/vmmp-{provider}/dom-{name}"`) and
`~/aci-py/aci_py/shims/mso_mso_epg.py`'s `dn_map["vmmDomain"] = "uni/vmmp-VMware/dom-{0}"`:
- `vmmDomP` at `uni/vmmp-VMware/dom-{name}` — always built (only "VMware" is modeled; the sole provider
either verified call site actually exercises).
- `vmmCtrlrP` at `{vmmDomP dn}/ctrlr-{name}` — built **only when `vcenter_ip` is set**: `hostOrIp` =
`vcenter_ip`, `rootContName` = `datacenter` (falls back to the domain's own `name` if `datacenter` is
unset). `dvsName` is additionally set when `vcenter_dvs` is provided.
- `vmmUsrAccP` at `{vmmCtrlrP dn}/usracc-{name}` — built alongside `vmmCtrlrP` (placeholder
credential-profile reference; no real credentials in a sim, matching the no-secret pattern phys/l3
domains already use for their own child objects).
- `infraRsVlanNs` at `{vmmDomP dn}/rsvlanNs` — built only when `vlan_pool` is set, pointing at the first
`access.vlan_pools[]` entry's DN (same convention `_build_phys_dom`/`_build_l3_dom` already use).
`vlan_pool` is cross-referenced against `access.vlan_pools[].name` in
`Topology.normalize_and_validate` — an unknown pool name is a validation error, same discipline as
`tenant.bds[].vrf`/`epg.bd`.
`_build_epg()` in `build/tenants.py` (the actual EPG-domain binding builder) is **untouched** — EPGs still
bind only to the first `phys_domains[]` entry via `fvRsDomAtt`, exactly as before PR-19. This is
intentional: the real production chain's `bind_epg_to_vmm_domain` never touches this MIT tree (see
above), so there was nothing to rewire.
### Leaf/spine count elasticity — a real generator bug found and fixed
`aci-sim new`'s node-ID scheme (`topology/schema.py`'s docstring: leaves base 101, spines base 201,
`site_offset=200`) has a documented ceiling — **"max ~40 leaves + ~40 spines per site before the ranges
touch"** — but prior to PR-19, `generate_topology()` (`cli.py`) had **no guard enforcing that ceiling**.
Stress-testing the exact boundary (required by this PR's acceptance bar) found three distinct ways a
large `--leaves-per-site`/`--spines-per-site`/`--border-pairs` value silently produced an invalid
`topology.yaml` dict that only failed deep inside `Topology.normalize_and_validate`, with a generic "node
ids collide" message that didn't name the offending flag:
1. **`--leaves-per-site > 100`** — the leaf block itself (`101..100+N`) runs into the spine block
(`201+`). Reproduced: `--leaves-per-site 150 --spines-per-site 4` → leaf ids run 101-250, directly
overlapping spine ids 201-204.
2. **`--spines-per-site > 100`** — the spine block itself (`201..200+N`) runs into the NEXT site's leaf
block (`301+`). Reproduced: `--leaves-per-site 2 --spines-per-site 101` → site 0's spine ids run
201-301, colliding with site 1's leaf ids starting at 301.
3. **Large `--border-pairs`** — the border-leaf block (already pushed past whichever of leaves/spines
extends further, per the pre-existing PR-17 border-ID fix) can itself run past the next site's leaf
block if `border_pairs` is large enough relative to a small leaf/spine count. Reproduced:
`--leaves-per-site 2 --spines-per-site 2 --border-pairs 50` → border ids run up to 300+, colliding with
the next site's leaf block at 301.
**Fix:** `generate_topology()` now computes the same leaf/spine/border-top arithmetic the ID-assignment
loop uses and raises a `ValueError` naming the specific offending flag *before* building any node dicts,
for all three cases above. Verified boundaries: `--leaves-per-site 100`/`--spines-per-site 100` (exact
max) still validate and build cleanly; `101` of either is rejected with a clear message;
`--border-pairs 49` at the default leaf/spine count is the max safe value, `50` is rejected. The
acceptance-bar case from this PR's scope (`--leaves-per-site 8 --spines-per-site 4`, well within the
safe range) validates clean and builds a full `MITStore` with the expected node count, verified via
`orchestrator.build_site()` (not just schema validation) in `tests/test_pr19_tier2.py`'s
`TestLeafSpineElasticity.test_full_build_succeeds_for_8_leaves_4_spines`.
This is a genuine bug fix (not a new feature) — the ID scheme's stated ceiling was never actually
enforced anywhere before this PR, meaning any topology author who requested more than ~100
leaves/spines-per-site via the CLI would previously get a confusing Pydantic stack trace instead of an
actionable error.
## PR-20 — Tier-3 fine-tuning parameters (final param tier)
The last planned parameter tier. Exposes commonly-tuned mgmt-subnet/BD-MAC/routing-timer/APIC-version
knobs as optional `topology.yaml` fields, wired into a real MO attribute wherever this sim already builds
one — and honestly marked store-only where it doesn't (BFD timers, `inb_subnet`). All new fields optional
with defaults matching pre-existing hardcoded behavior (or the real ACI factory default where there was
no prior hardcoded value at all), so the existing `topology.yaml` validates and builds byte-identical
output — verified via the full `tests/test_pr20_tier3.py::TestBackwardCompat` class plus the unchanged
672-test suite passing alongside 41 new tests (713 total).
### `bd.mac` / `fabric.default_bd_mac` — the critical backward-compat case
`build/tenants.py`'s `_build_bd` hardcoded `fvBD.mac = "00:22:BD:F8:19:FF"` (the real ACI default gateway
MAC) for **every** BD before this PR. `fabric.default_bd_mac` (fabric-wide default) reproduces that exact
literal as its own schema default, and `bd.mac` (per-BD, optional) overrides it when set — resolution
order in `_build_bd` is `bd.mac or topo.fabric.default_bd_mac`. Because the fabric-wide default equals
the pre-existing hardcoded literal exactly, a topology with neither field set produces **byte-identical**
`fvBD.mac` output to pre-PR-20 — this is the single highest-risk field in this PR (any drift from the
existing literal would visibly change every BD's gateway MAC that a consumer like autoACI might read),
so it got its own dedicated backward-compat test (`test_repo_fvBD_mac_is_real_aci_default`) asserting the
literal value directly against the real repo `topology.yaml`, not just "no schema-level default drift."
### BGP keepalive/hold — `l3out.csw_peer.keepalive`/`.hold` → `bgpPeerP`
Real ACI defaults 60s/180s (the standard 1:3 keepalive:hold ratio). `build/l3out.py`'s
`_build_node_profile` already builds a `bgpPeerP` MO (control-plane BGP peer policy, distinct from the
operational `bgpPeer`/`bgpPeerEntry` session MOs `_upsert_ebgp_sessions` builds) with only `addr`/
`peerCtrl` attrs pre-PR-20; this PR adds `keepAliveIntvl`/`holdIntvl` to the same object. Validated:
`hold` must exceed `keepalive` (a hold at or below the keepalive interval can never survive a single
missed keepalive) — same sanity-check pattern as PR-19's `isn.mtu` range check. The operational
`bgpPeer`/`bgpPeerEntry` MOs are **not** touched — real ACI's operational BGP session table doesn't carry
configured-timer attrs (those live on the policy object), so there's nothing to wire there.
### OSPF hello/dead — `isn.ospf_hello`/`.ospf_dead` → `ospfIf`
Real ACI defaults 10s/40s. `build/underlay.py`'s `ospfIf` MO (already built for `isn.ospf_area`, PR-19)
gains `helloIntvl`/`deadIntvl` attrs. Validated: `ospf_dead` must exceed `ospf_hello`, matching real ACI's
own OSPF sanity check (a dead timer at or below the hello interval can never detect a live neighbor
before declaring it down).
### BFD timers — `isn.bfd_min_rx`/`.bfd_min_tx`/`.bfd_multiplier` — explicitly STORE-ONLY
Real ACI defaults 50ms/50ms/3. Grepped `build/*.py` for "bfd"/"Bfd"/"BFD" — **zero matches**. This sim
builds no `bfdIfP`/`bfdRsIfPol`-style MO anywhere, and adding a faithful one would additionally require a
believable `bfdIfStats`/oper-state child tree with no verified consumer (autoACI plugin or aci-py
playbook) to anchor its shape against — inventing an MO purely to hold a timer, with no real read path
exercising it, would be worse than being explicit that these three fields are schema-level only. Store +
range-validate (`min_rx`/`min_tx` positive; `multiplier` 1-50, real ACI's UI range) + surface in
`aci-sim show`/`new`.
### OOB/INB mgmt subnets — `fabric.oob_subnet` (validate-only) / `fabric.inb_subnet` (store-only)
`fabric.oob_subnet` sits alongside the existing `fabric.oob_pool` (PR-early/#27) — `build/fabric.py`'s
`oob_ip()` derives every node's OOB address from `(pod, node_id)` only and does not consume an arbitrary
subnet/pool CIDR (documented limitation, unchanged by this PR). `oob_subnet` is stored + validated (must
fall within `192.168.0.0/16`, same family as `oob_pool`) + surfaced, letting a topology author record the
*intended* OOB subnet distinctly from the pool the builder actually draws from.
`fabric.inb_subnet` is genuinely new surface, not a wire-up of an existing gap: grepped `build/*.py` for
"mgmtInB"/"inbMgmt" — zero matches. Real ACI's in-band management is a distinct `mgmtInB` bridge-domain
living in the special `mgmt` tenant, requiring its own tenant/BD/EPG-style builder — a materially bigger
addition than "wire an existing attr," and out of scope for a fine-tuning-parameters PR. Stored +
validated (must fall within a private RFC1918 range: `10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16`)
+ surfaced, explicitly marked store-only in `aci-sim show`'s own output (`(store-only)` label) so a user
never mistakes it for something the sim actually builds.
### `fabric.default_apic_version` — fabric-wide override, on top of already-independent versioning
**Important finding while implementing this PR:** the task ask ("make apic_version drive the APIC
controller topSystem/firmwareCtrlrRunning version consistently, independent of switch-node version") was
**already fully satisfied** before this PR — `site.apic_version` (added in the PR-9/PR-18 lineage) has
driven `build/fabric.py`'s controller `fabricNode.version`/`topSystem.version`/`firmwareCtrlrRunning.version`
since PR-9, completely independently of the per-node switch `Node.version` field (PR-18). Verified: grepped
`build/*.py` for the literal `"4.2(7f)"` and found it only as `Site.apic_version`'s own schema default in
`topology/schema.py` — no builder hardcodes an APIC version literal that bypasses `site.apic_version`.
What was missing: (1) no fabric-wide override existed for topologies with many sites that all want the
same APIC version, and (2) `aci-sim show`/`new` never surfaced `apic_version` at all (verified: zero
matches for "apic_version" in pre-PR-20 `cli.py`). This PR adds `fabric.default_apic_version` (optional,
default `None` = unchanged behavior — each site keeps using its own `site.apic_version`) with resolution
`topo.fabric.default_apic_version or site.apic_version` in `build/fabric.py`, and surfaces the *effective*
resolved value per-site in `aci-sim show`'s new `apic_version=` column — closing the surfacing gap without
touching the wiring that already worked.
### Backward-compat verification performed vs. NOT performed (be precise)
**Performed in this sandboxed development environment:** `aci-sim validate` on the repo `topology.yaml`
(OK), the full unit suite (713 passed: 672 pre-existing + 41 new, zero regressions), and `aci-sim show`/
`new` manual smoke checks confirming every new field surfaces correctly.
**NOT performed — no access from this environment:** the real production-chain regression gate this
project's acceptance bar calls for (autoACI sandbox e2e 11/11, aci-py MS-TN1 live_chain 0-fail) requires
an SSH-reachable hardware test host, which does not exist in
this sandboxed clone. The `fvBD.mac` default was double-checked against the pre-PR-20 hardcoded literal
in `build/tenants.py` specifically because it is the one Tier-3 change with plausible autoACI-regression
risk (autoACI reads `fvBD` attrs) — but this is static-code verification, not a live e2e run. The main
thread's independent re-run of the real Ansible/aci-py chains is the authoritative backward-compat gate
for this PR, per this project's established process (several prior PRs passed unit tests but broke the
real production chain).
## PR-21 — single-APIC default + per-controller OOB IPs
**Design intent (owner directive):** APIC cluster size is irrelevant to Ansible playbook *deployment*
testing — a playbook connects to and pushes config through exactly **one** APIC endpoint, regardless of
how many controllers back it in real ACI. Cluster size only shows up in cluster-health/appliance-vector
tooling, e.g. autoACI's `health_score.py`, which reads `infraWiNode` across every controller. This PR
therefore flips `Site.controllers`'s default from `3` (pre-PR-21) to **`1`** (single-APIC-per-site), while
keeping the field fully configurable (validated range **1-5**) for anyone who does need to exercise
multi-controller/cluster-health semantics.
### `Site.controllers` default 3 → 1, validated range 1-5
This is a **breaking change to generated node counts**, not merely a config default: every site's
`fabricNode`/`topSystem` count drops by (old `controllers` new `controllers`) = 2 by default (e.g. a
2-spine/2-leaf/2-border-leaf/3-controller site was 9 `fabricNode`s; the same shape at the new
1-controller default is 7). `fabricLink`/`lldpAdjEp`/`cdpAdjEp` counts for the APIC↔leaf attachment drop
proportionally (2 leaves × N controllers, so N=3→6 links becomes N=1→2 links), and `infraWiNode` count
(N²) drops from 9 to 1. Every hardcoded count in the test suite and e2e scripts that assumed the old
default was audited and updated to the new default — see the PR-21 commit message / CHANGELOG for the
full list of touched assertions (the pattern used throughout: prefer a test that derives the expected
count from `site.controllers` dynamically, as `test_pr3b_batch2.py`'s `infraWiNode` test already did
pre-PR-21, over one that hardcodes a literal that will need updating again if the default ever changes).
Range validation (1-5) lives in `Topology.normalize_and_validate` alongside the other per-site checks —
`6` (or `0`, or negative) is rejected with a clear error naming the site.
### `Site.controller_ips` — per-controller OOB mgmt IP (optional)
Pre-PR-21, every controller's `topSystem.oobMgmtAddr` was derived from `(pod, controller_id)` via the
same `oob_ip()` helper switch nodes use — **never** from the site's own `mgmt_ip`, which is the address
the sim's REST server actually binds to (controller-1 / the cluster VIP in sandbox mode). That's fine
for a single controller (there's only one address to assign, and CI/tests never asserted a specific
controller OOB IP), but becomes a believability gap for `controllers > 1`: real multi-controller clusters
assign each APIC its own OOB IP typically adjacent to the cluster's primary address, not derived from an
arbitrary per-node formula.
`controller_ips: list[str]` (optional, default unset) fixes this: when set, `controller_ips[i]` is
controller `i+1`'s OOB IP verbatim (length must equal `controllers`, each entry a valid IP — validated in
`Topology.normalize_and_validate`, same place as the `controllers` range check). When unset, `Site.
controller_oob_ips()` (new helper, `topology/schema.py`) auto-derives sequentially from `mgmt_ip`:
controller 1 = `mgmt_ip`, controller 2 = `mgmt_ip + 1`, controller 3 = `mgmt_ip + 2`, etc. (plain
`ipaddress` integer arithmetic, so it correctly rolls over octets). If `mgmt_ip` is *also* unset (empty
string — some hand-authored topologies never set it), `controller_oob_ips()` returns `[]` and
`build/fabric.py` falls back to the legacy `oob_ip(pod, cid)` scheme, so a topology with neither field
set keeps its pre-PR-21 controller OOB addresses unchanged.
Wired into `build/fabric.py`'s controller loop for both `topSystem.oobMgmtAddr` and the leaf-side
`lldpAdjEp`/`cdpAdjEp.mgmtIp` (the neighbor-reported address must match the controller's own OOB IP, or
autoACI's LLDP-based topology rendering would show a controller "announcing" an IP that doesn't match its
own `topSystem`) — both read the same per-call `controller_oob_ips` list so they can never drift apart.
### Shipped `topology.yaml` — dropped to the new default, not left at 3
Both `LAB1`/`LAB2` already omitted an explicit `controllers:` field pre-PR-21 (it was commented out at
its then-default of 3), so they automatically pick up the new default of 1 with no YAML edit required.
The commented example line was updated to describe the *current* default (1) rather than the old one
(3), and a new commented block was added showing the `controllers: 3` + `controller_ips: [...]` opt-in
shape for anyone who copies this file as a starting point and wants multi-controller semantics.
### Backward-compat verification performed vs. NOT performed (be precise)
**Performed in this sandboxed development environment:** `aci-sim validate` on the repo `topology.yaml`
(OK, now reporting 1 controller/site), the full unit suite (counts below), and manual `aci-sim show`
checks confirming `controller_ips` surfaces correctly for both the auto-derived and explicit cases.
**NOT performed from this sandboxed environment** (no SSH/network access here): the real production-chain
regression gate this project's acceptance bar calls for (autoACI sandbox e2e, aci-py MS-TN1
`live_chain.py`) — this PR explicitly changes generated node counts, which is exactly the kind of change
that gate exists to catch (a hardcoded node-count assertion inside autoACI itself, if one exists, would
only surface there). The main thread's independent re-run of the real Ansible/aci-py/autoACI chains on
ACI hardware is the authoritative backward-compat gate for this PR.
+317
View File
@@ -0,0 +1,317 @@
# aci-sim — Operation Manual / 操作手册
A stateful REST simulator of a Cisco ACI/NDO **management plane** (per-site APIC + Nexus
Dashboard/NDO). It answers `moquery`/class/DN queries computed from an in-memory MIT, applies
`POST`/`DELETE` mutations atomically, and models `aaaLogin` + NDO orchestration — enough fidelity
to test Ansible playbooks, aci-py, autoACI, or any tool that speaks the APIC/NDO REST API, with
**no hardware, no data plane, fully deterministic and resettable.**
> 一个有状态的 Cisco ACI/NDO **管理平面** REST 模拟器(每站点一个 APIC + Nexus Dashboard/NDO)。
> 它从内存中的 MIT 计算 `moquery`/class/DN 查询的响应、原子地应用 `POST`/`DELETE` 变更、模拟
> `aaaLogin` 与 NDO 编排 —— 保真度足以测试 Ansible playbook、aci-py、autoACI,或任何讲 APIC/NDO
> REST API 的工具,**无需硬件、无数据平面、完全确定且可重置。**
---
## 1. Overview / 概述
- **What it does**: serves the APIC REST plane (GET class/mo queries, POST/DELETE mutations,
aaaLogin) and the NDO/ND REST plane (schema/template CRUD + deploy). Both are backed by one
in-memory Management Information Tree (MIT) built from `topology.yaml`.
- **What it does NOT do**: no data plane, no real NX-OS/APIC software stack, no live telemetry.
It models the object model + REST behavior, not packet forwarding.
> **能做什么**:提供 APIC REST 平面(GET class/mo 查询、POST/DELETE 变更、aaaLogin)和 NDO/ND
> REST 平面(schema/template 增删改 + deploy)。两者都由一棵从 `topology.yaml` 构建的内存 MIT 支撑。
> **不做什么**:没有数据平面、没有真实 NX-OS/APIC 软件栈、没有实时遥测。它模拟对象模型 + REST
> 行为,不做报文转发。
**Primary use cases / 主要场景**: Ansible playbook testing · CI gate before real hardware ·
SDK/provider testing · AI-agent sandbox · contract/regression tests · teaching/demo.
---
## 2. Installation / 安装
Requires Python >= 3.11.
```bash
# from a checkout (or pip install "aci-sim @ git+<repo-url>")
python3 -m venv .venv
.venv/bin/pip install -e . # runtime deps: fastapi, uvicorn[standard], pydantic, pyyaml
.venv/bin/pip install -e '.[dev]' # + test deps: pytest, pytest-asyncio, httpx
aci-sim --help
```
> 需要 Python >= 3.11。`pip install -e .` 安装运行依赖(fastapi/uvicorn/pydantic/pyyaml);
> `.[dev]` 额外装测试依赖。装完 `aci-sim --help` 应列出子命令,无 traceback。
---
## 3. Topology & the init wizard / 拓扑与 init 向导
Everything is driven by `topology.yaml` (Pydantic-validated). Author it three ways:
```bash
aci-sim init # interactive wizard (APIC setup-dialog style) — RECOMMENDED
aci-sim new --sites 2 # non-interactive scaffold with flags
# or hand-edit topology.yaml
aci-sim validate topology.yaml # validate before running (CI gate)
aci-sim show topology.yaml # print the derived inventory (IDs/IPs/ports)
```
The **`aci-sim init` wizard** walks: Step 0 Admin account -> Step 1 Deployment type (single
fabric / multi-site) -> Step 2 per-site (name/id/asn/pod/controllers/IPs) -> Step 3 multi-site
(NDO IP + ISN). Press ENTER to accept the `[bracketed]` default.
> 一切由 `topology.yaml` 驱动(Pydantic 校验)。三种方式创建:`aci-sim init`(交互式向导,
> 仿 APIC setup dialog,**推荐**)、`aci-sim new`(带 flag 非交互脚手架)、手改。运行前用
> `aci-sim validate` 校验。向导顺序:Step 0 管理员账号 -> 1 部署类型 -> 2 每站点参数 -> 3 多站点
> (NDO IP + ISN)。回车接受 `[方括号]` 里的默认值。
---
## 4. Admin account / 管理员账号
The wizard's **Step 0** sets the fabric admin credentials, written to a `topology.yaml` `auth:`
section:
```yaml
auth:
username: admin
password: cisco
# ndo_username / ndo_password only if you gave NDO a separate account
```
- **APIC plane ENFORCES it** — `aaaLogin` returns 401 on a username/password mismatch.
- **NDO plane is lenient by design** (accepts any credential, for cisco.mso/aci-py compat);
the same account drives it by default.
- **Credential precedence** at `aci-sim run`: an explicit `SIM_USERNAME`/`SIM_PASSWORD` in the
environment wins (atomic — set either and you own both) -> else the topology `auth:` section ->
else the built-in `admin`/`cisco` default. `aci-sim run` prints which source is live (never the
password).
> 向导 Step 0 设置管理员凭据,写进 `topology.yaml` 的 `auth:` 段。**APIC 平面强制校验**(密码错
> 返回 401);**NDO 平面故意宽松**(接受任意凭据,为兼容 cisco.mso/aci-py),默认用同一账号。
> `aci-sim run` 的凭据优先级:环境里显式的 `SIM_USERNAME`/`SIM_PASSWORD` 最高(原子:设了任一就
> 两个都归你)-> 其次 topology 的 `auth:` -> 再次内建默认 `admin`/`cisco`。run 会打印当前凭据来源
> (不打印密码)。
---
## 5. Running — two modes / 运行的两种模式
### 5a. Port mode (default) / 端口模式(默认)
Binds `127.0.0.1` on distinct ports. Simplest; good for local dev + CI.
```bash
aci-sim run topology.yaml
# APIC LAB1 -> https://127.0.0.1:8443 APIC LAB2 -> :8444 NDO -> :8445
```
### 5b. Sandbox mode (real per-device IPs on :443) / sandbox 模式(真实 per-device IP,:443)
Each APIC/NDO gets its **own loopback-alias IP on :443** (like real gear). Needed for tools that
assume one IP per controller (e.g. autoACI multi-site discovery). Requires root (adds `lo`
aliases + binds :443).
```bash
sudo bash scripts/sandbox-up.sh # adds lo aliases from topology mgmt IPs, binds :443, detaches
# APIC LAB1 -> https://10.192.0.11:443 APIC LAB2 -> https://10.192.128.11:443
# NDO -> https://10.192.0.10:443
sudo bash scripts/sandbox-down.sh # stop + remove aliases
```
The sandbox IPs come from `topology.yaml`'s `fabric.ndo_mgmt_ip` + each `site.mgmt_ip`. On Linux
they are `lo` aliases -> reachable **only on that host** (run clients on the same box). PID in
`/tmp/aci-sim-sandbox.pid`, log in `/tmp/aci-sim-sandbox.log`.
> **端口模式**(默认):绑 `127.0.0.1` 不同端口,最简单,适合本地开发/CI。
> **sandbox 模式**:每个 APIC/NDO 拿到自己的 loopback 别名 IP、都在 :443(像真机),适合假设"每个
> 控制器一个 IP"的工具(如 autoACI 多站点发现)。需 root(加 `lo` 别名 + 绑 :443)。sandbox IP 来自
> `topology.yaml` 的 `ndo_mgmt_ip` 和各 `site.mgmt_ip`;Linux 上是 `lo` 别名,**只在本机可达**(客户端
> 要在同一台跑)。PID/日志见 `/tmp/aci-sim-sandbox.{pid,log}`。
---
## 6. Connecting clients / 连接客户端
Credentials = your admin account (default `admin`/`cisco`). TLS is self-signed -> disable cert
validation.
**Ansible — cisco.aci (single fabric) / cisco.mso (multi-site):**
```yaml
# inventory: point apic_host / MSO ansible_host at the sim's IP:port
apic_host: "10.192.0.11:443" # sandbox, or 127.0.0.1:8443 in port mode
apic_username: admin
apic_password: cisco
apic_validate_certs: no
```
**aci-py** (Python pusher): point `--apic` at the APIC IP; for multi-site pass an NDO connector
(`Ndo(host, username, password)`) so tenants become NDO-managed (visible in autoACI).
**autoACI**: log in to the **NDO** IP (`10.192.0.10`) -> Discover -> Connect All Sites. It reads
NDO, so multi-site tenants (created via cisco.mso or aci-py's NDO path) appear there.
> 凭据 = 你的管理员账号(默认 `admin`/`cisco`),TLS 自签 -> 关掉证书校验。Ansible 用 cisco.aci
> (单站点)/cisco.mso(多站点),inventory 里把 `apic_host`/MSO `ansible_host` 指向 sim 的 IP:端口。
> aci-py 把 `--apic` 指向 APIC;多站点要传 NDO 连接器,tenant 才会被 NDO 管理(autoACI 才看得到)。
> autoACI 登录 **NDO** IP(`10.192.0.10`)-> Discover -> Connect All Sites。
> **Multi-site invariant / 多站点不变量**: a tenant is "multi-site" only if it exists in **NDO**.
> An APIC-only push (e.g. aci-py without an NDO connector) creates an APIC-local tenant that
> autoACI's NDO view will NOT show. 名字含多站点语义的 tenant 必须出现在 NDO 上。
---
## 7. Querying the MIT / 查询 MIT
Speaks the plain APIC REST API — works with `curl`, httpx, cisco.aci, or a real APIC.
```bash
# login -> cookie
curl -sk -c j.txt -X POST https://<apic>/api/aaaLogin.json \
-d '{"aaaUser":{"attributes":{"name":"admin","pwd":"cisco"}}}'
# class query (== moquery -c fvTenant)
curl -sk -b j.txt https://<apic>/api/class/fvTenant.json
# mo query + subtree
curl -sk -b j.txt "https://<apic>/api/mo/uni/tn-MS-TN1.json?query-target=subtree&target-subtree-class=fvBD"
# LLDP / CDP neighbors
curl -sk -b j.txt https://<apic>/api/class/lldpAdjEp.json
```
`aci-sim lldp topology.yaml [--site N --node N --cdp --json]` prints a `show lldp neighbors`-style
table without hand-writing URLs. Query subscriptions + websocket push-on-change are supported
(`?subscription=yes` + `/socket<token>`).
> 讲标准 APIC REST API,`curl`/httpx/cisco.aci/真机 APIC 都能用。class 查询等价 `moquery -c`;
> mo 查询支持 `query-target=subtree`。`aci-sim lldp` 直接打印邻居表。支持查询订阅 + websocket
> 变更推送(`?subscription=yes`)。
---
## 8. CLI reference / CLI 参考
| Command | Purpose / 用途 |
|---|---|
| `aci-sim init` | interactive wizard -> topology.yaml / 交互向导 |
| `aci-sim new --sites N ...` | non-interactive scaffold / 非交互脚手架 |
| `aci-sim validate FILE` | validate topology (CI gate) / 校验拓扑 |
| `aci-sim show FILE [--json]` | print derived inventory / 打印推导出的清单 |
| `aci-sim lldp FILE [--site --node --cdp --json]` | LLDP/CDP neighbor table / 邻居表 |
| `aci-sim graph FILE -o out.svg\|.html` | render topology diagram / 渲染拓扑图 |
| `aci-sim run FILE` | run the supervisor (port mode) / 起 supervisor(端口模式) |
| `scripts/sandbox-up.sh` / `sandbox-down.sh` | sandbox mode (real IPs :443) / sandbox 模式 |
| `scripts/sim-state.sh {save\|restore} NAME` | save/restore whole-fabric state (§10) / 保存恢复整个 fabric 状态(见 §10) |
---
## 9. `/_sim` control API / 控制 API
For AI-agent sandboxes and test isolation: snapshot / restore / reset the MIT, and seed faults —
so each test starts from a known state.
> 面向 AI agent 沙箱和测试隔离:对 MIT 做 snapshot / restore / reset、注入 fault —— 让每个测试从
> 已知状态开始。
---
## 10. State persistence / 状态保存与恢复
In sandbox/port mode the sim's state is **in-memory only** — restarting the process wipes every
tenant/schema/template on every plane. To survive a restart, save state to disk first and restore
it after.
**Per-plane endpoints** (mirrors the `/_sim` control API in §9):
| Plane | Save | Load |
|---|---|---|
| APIC (each site) | `POST /_sim/save/{name}` | `POST /_sim/load/{name}` |
| NDO | `POST /_sim/save/{name}` | `POST /_sim/load/{name}` |
- APIC's save writes `{name}.{site.id}.apic.json` (keyed by site id, so multiple sites in the same
`SIM_STATE_DIR` never collide); NDO's save writes `{name}.ndo.json`.
- A missing `load` name returns 404 — an APIC-envelope error (`imdata[0].error`) on the APIC plane,
a plain `{"detail": ...}` on the NDO plane (matching each plane's existing error shape).
- Files are plain JSON under `SIM_STATE_DIR` (default `~/.aci-sim/state`) — human-readable,
diffable, safe to check into a fixtures directory for a known-good baseline.
**Wrapper script — save/restore the WHOLE fabric (all APIC sites + NDO) in one command:**
```bash
scripts/sim-state.sh save mybaseline # snapshot every plane
scripts/sim-state.sh restore mybaseline # restore every plane (maps to /_sim/load)
```
It derives each plane's address the same way `scripts/sandbox-up.sh` does (straight from
`topology.yaml`'s `fabric.ndo_mgmt_ip` / each `site.mgmt_ip`, sandbox mode's real per-device
`:443` IPs), and falls back to port mode (`127.0.0.1:8443`/`:8444`/`:8445`) if those IPs aren't
reachable — so it works unmodified in either running mode.
**Restart workflow:**
```bash
scripts/sim-state.sh save mybaseline
sudo bash scripts/sandbox-down.sh && sudo bash scripts/sandbox-up.sh # or Ctrl-C + aci-sim run
scripts/sim-state.sh restore mybaseline
```
**`SIM_STATE_DIR`** — override the base directory for all saved state (default
`~/.aci-sim/state`; created automatically). Tests set this to an isolated `tmp_path` so
nothing is ever written under the real home directory.
> sandbox/端口模式下 sim 状态**只在内存里**——重启进程会清空所有平面上的 tenant/schema/template。
> 要跨重启保留状态,重启前先保存、重启后再恢复。
>
> **各平面端点**(与 §9 的 `/_sim` 控制 API 呼应):APIC(每个站点)和 NDO 都提供
> `POST /_sim/save/{name}` / `POST /_sim/load/{name}`。APIC 的保存文件名为
> `{name}.{site.id}.apic.json`(按 site id 区分,同一个 `SIM_STATE_DIR` 里多个站点不会冲突);
> NDO 的保存文件名为 `{name}.ndo.json`。`load` 遇到不存在的 name 返回 404 ——APIC 平面是
> APIC 信封错误(`imdata[0].error`),NDO 平面是普通 `{"detail": ...}`(各自匹配平面已有的
> 错误格式)。文件是 `SIM_STATE_DIR`(默认 `~/.aci-sim/state`)下的纯 JSON——可读、可
> diff,也可以存进 fixtures 目录当已知良好基线用。
>
> **一键保存/恢复整个 fabric**(所有 APIC 站点 + NDO):`scripts/sim-state.sh save <name>` /
> `scripts/sim-state.sh restore <name>`(`restore` 对应 `/_sim/load`)。地址推导方式与
> `scripts/sandbox-up.sh` 一致(从 `topology.yaml` 读 `ndo_mgmt_ip`/各 `site.mgmt_ip`),这些 IP
> 不可达时自动退回端口模式(`127.0.0.1:8443`/`:8444`/`:8445`),两种运行模式下都能直接用。
>
> **重启工作流**:先 `sim-state.sh save` → 重启 sim(`sandbox-down.sh && sandbox-up.sh`,或
> Ctrl-C 后 `aci-sim run`)→ 再 `sim-state.sh restore`。
>
> **`SIM_STATE_DIR`**:覆盖所有保存状态的根目录(默认 `~/.aci-sim/state`,自动创建)。
> 测试会把它设成隔离的 `tmp_path`,绝不会写到真实 home 目录下。
---
## 11. Multi-site vs single-fabric / 多站点 vs 单站点
- **Single-fabric**: one APIC. Tenants pushed directly via cisco.aci — APIC-local, no NDO.
- **Multi-site**: NDO + per-site APICs. Tenants are created as NDO schemas/templates and
**deployed** to sites; only then do they appear on the site APICs (and in autoACI).
- **multi-pod ~= single-fabric** for Ansible tenant testing — the IPN lives on external Nexus (NX-OS),
not cisco.aci; the only per-pod nuance is the pod id in static binding paths, covered by `site.pod`.
> 单站点:一个 APIC,tenant 经 cisco.aci 直推,APIC 本地,无 NDO。多站点:NDO + 各站点 APIC,tenant
> 建成 NDO schema/template 再 **deploy** 到站点,之后才落到站点 APIC(和 autoACI)。对 Ansible
> tenant 测试,**multi-pod ~= single-fabric** —— IPN 在外部 Nexus 上(NX-OS),不经 cisco.aci。
---
## 12. Troubleshooting / 排错
| Symptom / 现象 | Cause & fix / 原因与解决 |
|---|---|
| `aaaLogin` 401 | password != the enforced account; check `auth:` / `SIM_USERNAME`+`SIM_PASSWORD` / the `run` banner. |
| sandbox: `Needs root` | `sudo bash scripts/sandbox-up.sh` (needs `lo` alias + :443 bind). |
| sandbox: nothing on :443 after start | old listener still holds the socket; the script waits for it to free — check `/tmp/aci-sim-sandbox.log`. |
| deep-root subtree query returns 0 | query a DN that has a materialized MO (e.g. `.../sys`, not an un-materialized container). |
| client can't reach sandbox IP from another host | sandbox IPs are `lo` aliases -> same-host only; use port mode + the host LAN IP for remote clients, or run the client on the sim host. |
| `pip install` CLI crashes on import | ensure v0.13.1+ (earlier packaging didn't declare deps / omitted subpackages). |
---
*Generated as part of the sim playbook-E2E effort. For the REST contract details see
`docs/CONTRACT.md`; for design rationale see `docs/DESIGN.md`; for the changelog see `CHANGELOG.md`.*