Object-valued analysis I/O in the OpenMDAO bridge (design)¶
Status: adopted 2026-08-27. OpenMDAO’s discrete-variable machinery is the native object pipe. Longeron does not fork it. All empirical claims were verified against longeron 0.10.0 and OpenMDAO 3.45.0, and each claim says which. The two load-bearing decisions (optionality, result lifecycle) are stated in the Decisions section below.
Goal: let objects – not just scalars – cross the OpenMDAO bridge. Two motivating cases:
Discrete entities. Analyses often vary discrete entities, not scalars: a case selects a motor with a specific mass, Kt, and Kv, and the bridge must carry that selection as one object, not as loose parameters.
Object handoff. Components hand whole artifacts to one another – a geometry generator feeds an RCS or CFD/FEA analysis – so the bridge needs a convention for file-backed payloads.
And the integration question behind both: how does this compose with longeron’s SysML v2 constructs, so the wiring is derivable from the model rather than hand-assembled in Python?
The central identity: a case is an interpretation¶
The design’s organizing idea comes from longeron’s own M0 layer
(M0 interpretations): the M1 model is the
space of possibilities; an OpenMDAO evaluation is always a point in
that space; and a point in that space already has a name in longeron –
an M0 interpretation (longeron.m0.interpret(), m0.py
l. 227). A DOE or covering-array run is then a population of
interpretations. Everything below is that identity, applied:
A discrete case (motor A × prop B) is an interpretation with its variation points pinned –
m0.interpret(model, part, selection={"motor": "at2814"}). A tradesArchitectureis already a proto-interpretation (from_architecture(),m0.pyl. 453), and the verify spike’s counterexample configurations are the same currency, oneinterpret(bindings=...)away. One currency across trades, verify, and mdao.Entity binding (tier 1) passes the individual – the
m0.Individualwith its resolved attribute values – not the raw M1 usage. The M1 usage is the whole space; the individual is the point.Geometry flowing between components (tier 2) is M0-keyed:
drone_scene()(grand.pyl. 131) already stamps every rendered part with its individual id (Drone::QuadCopter#0.motors#2), and that id is the join key that says which sub-mesh belongs to which configured individual in a downstream RCS/CFD component.Results land on the interpretation (below): OM outputs become attribute values on the case’s individuals – stable ids for recording, traceability from an output back to the exact configured individual that produced it, roll-ups over populations, and a direct
values=feed into the scoreboard.
The tension this creates – continuous-sweep users must not pay M0 ceremony – is the design’s first decision (D1 below); the resolution is that the light path materializes one implicit anonymous interpretation and never mentions it.
The native boundary: what OpenMDAO provides (verified)¶
OpenMDAO has carried object-valued variables since 2.5:
add_discrete_input / add_discrete_output declare them, connect()
and promotion wire them, and compute() receives them in separate
discrete_inputs / discrete_outputs dicts. Every behavior below was
verified against the installed 3.45.0 with throwaway single-process
Problems (dict-valued motor and mesh payloads); line numbers are
from the installed package.
behavior |
verdict |
evidence |
|---|---|---|
declare + promote + explicit |
works |
probes A1/A3 |
|
works |
probes A2/A4 |
|
works – FD partials simply skip discretes |
probe B1 |
|
fails with an opaque |
probe B2 |
discrete design var under |
rejected cleanly at |
probe B3 |
SLSQP over continuous desvars with discretes in the model |
converges normally – the two-loop pattern (discrete case outside, gradient opt inside) works today |
probe J1 |
|
works; each case records and reads back the full dict ( |
probe D1 |
discrete↔continuous connection |
rejected with a clear |
probe E1 |
discrete↔discrete value compatibility |
checked at setup by |
probes E2/I1 |
|
works – the bridge’s existing |
probe K2 |
|
works – the external-binding contract validation ( |
probe K1 |
serial discrete transfer |
by reference, not copy – the source comment is explicit that a downstream mutation is visible upstream ( |
source |
MPI discrete transfer |
values move through |
source; MPI run not exercised |
recorder + discretes |
recorded, but see the recorder section: JSON-text storage, silent lossy degradation for non-JSON objects |
probes C/F/G/K3 |
The OM documentation’s own claims (discrete variables page) match what
we verified; the reference-aliasing and class-name-degradation
behaviors below are not documented and were established from source
and probes. ExternalCodeComp (external_code_comp.py l. 234) is
file-based by contract: command, external_input_files,
external_output_files options (l. 41-56) – the file boundary in
tier 3 is what it already expects.
What longeron supports today (gap analysis)¶
Every row was established empirically: spec-shaped samples parsed with
longeron.loads(), instantiated, pushed through
build_problem(), and validated.
construct |
grammar / model layer |
|
interpreter |
mdao bridge |
|
|---|---|---|---|---|---|
|
full ( |
fixpoint |
nested |
scalar-shredded: |
clean |
|
full |
fixpoint |
first variant only; unpinned slots |
fails opaquely: |
no diagnostic |
variant with inline redefinitions ( |
full |
fixpoint |
– |
trades |
no diagnostic |
|
full (direction + types survive) |
fixpoint |
ignored |
silently ignored – not even a |
clean |
|
full: |
fixpoint |
ignored |
silently ignored |
zero diagnostics even when both endpoints and the payload type dangle |
|
– |
– |
works: pins variants, resolves values, |
not consumed |
– |
|
– |
– |
works: entity override evaluates derived attributes correctly |
no |
– |
Flow endpoints deserve emphasis because tier 4 stands on them: the
model layer stores source='build.mesh', target_end='rcs.mesh',
payload='MeshModel' as verbatim strings – parsed, exported at a
fixpoint, never resolved by the Resolver, never validated. This
sample produces zero diagnostics today:
part def Sys {
action a { out mesh : Mesh; }
action b { in mesh : Mesh; }
flow of Mesh from a.mesh to nonexistent.pin; // dangles silently
}
The five most consequential findings:
The pipe already exists and is sufficient. Discrete declaration, connection, promotion, setup-time type checking, DOE enumeration over dict-valued design variables, clean rejection by gradient drivers, and case recording all work in stock OpenMDAO. No fork, no subclassing of OM internals – tiers 1-3 are conventions over an existing mechanism.
The recorder is the trap, not the pipe.
SqliteRecorderstores iteration data as JSON text (sqlite_recorder.pyl. 476): a 2.4 MB mesh became ~6 MB of JSON per iteration (30 MB for fiverun_modelcalls), and any non-JSON-native object silently degrades throughmake_serializable(general_utils.pyl. 772) too.to_json()if it exists, else its class name as a string – a recordedRecipe()reads back as'Recipe', no warning. Theto_jsonhook is the lossless seam tiers 1 and 3 exploit.Aliasing and pickling are the two transfer regimes. In serial, OM passes discrete values by reference (
group.pyl. 2066 – deliberate, per the source comment), so a downstream component mutating a received mesh corrupts its upstream producer. Under MPI, values cross ranks throughcomm.gather– pickled. The recipes-not-solids rule and the frozen-payload convention below are load-bearing, not stylistic.Longeron already parses everything tier 4 needs, and all of it evaporates before analysis. Item defs, typed action parameters, and flows survive to the model layer at a
to_sysmlfixpoint – then the bridge scalar-shreds item members, silently drops actions and flows, fails opaquely on variation points, and validation says nothing about dangling flow endpoints.M0 is the missing currency, and it already works.
m0.interpret(selection=...)materializes exactly the casebuild_problemchokes on (verified:gaps == [], correct derived values).Individuals pickle (3-22 KB – they drag a copy of the reachable M1 graph; acceptable per-case, wasteful per-mesh),rollup()aggregates over populations,Instance.set()writes results back,to_dict()is JSON-clean, anddrone_scenealready keys meshes by individual id.
Tier 1: entity binding¶
A variation-typed (or designated item-typed) member becomes one discrete input carrying the whole entity, instead of today’s scalar shred. The bound value is the M0 individual – resolved attribute values, stable id, definition backlink – not the M1 usage:
# mdao.py additions (sketch; signatures illustrative)
def build_problem(
model: M.Model,
part: str | M.Definition | M.Usage,
requirements: tuple[str, ...] = (),
setup: bool = True,
fidelity: Mapping[str, str] | None = None,
interpretation: Interpretation | None = None, # the case being evaluated
) -> ProblemBuild: ...
def bind_entity(build: ProblemBuild, feature: str, entity: str | Instance) -> None:
"""Rebind a variation point to an individual (qname resolved via the interpreter)."""
def entity_cases(study: TradeStudy, *points: str) -> list[list[tuple[str, Any]]]:
"""DOE cases over the catalog: one case per mix, values are individuals."""
Mechanics, reusing the bridge’s existing patterns:
build_problem(..., interpretation=itp)builds the Problem around a point: free scalars seed from the interpretation’s slots exactly as they seed frominstantiate()today (mdao.pyl. 589), and each entity member becomes anadd_discrete_outputon the group’s existingconstsIndepVarComp(probe K2) with the individual as its value.ProblemBuildgainsentities: dict[str, str](promoted name -> item/part def qname) andinterpretation.An expression component whose value references
motor.massdeclaresadd_discrete_input("motor")and reads.get("mass")incompute()– the floats-only invariant of the units design holds, because slot leaves stay floats.Rebinding between cases is
problem.set_val("motor", individual)(probe A4);bind_entityadds qname resolution and a conformance check of the individual’s definition against the variation point’s base type.The trades machinery is the discrete-case source:
entity_cases(study)walksTradeStudy.points(trades.pyl. 376 already collects bothpart- anditem-typed variation points) and yieldsom.ListGenerator-shaped cases whose values arefrom_architecture-style individuals.DOEDriveraccepts them as discrete design variables and records each mix (probe D1). The verify design’s covering arrays slot in later as another generator of the same currency: a population of interpretations.Gradient safety needs no work: discrete desvars are rejected by gradient drivers with a clear error (probe B3), and the supported pattern – discrete case outside, SLSQP over continuous variables inside – runs today (probe J1).
Tier 2: object flow¶
Geometry (and any other structured payload) moves between components as discrete values. Two conventions, both forced by finding 3:
Pass recipes, not kernel objects. A cadquery/OCC solid is a live CFFI handle – unpicklable, unrecordable, meaningless on another rank. The payload is the recipe (the parameter dict
to_cadquery()already rebuilds from) or the baked mesh dict ({"unit", "parts", "bounds"}, plain lists –geometry.py’s existing currency). Workers rebuild solids locally from recipes.Payloads are frozen by convention. Serial OM aliases discrete values across components; a consumer must never mutate a received payload. The bridge documents this and the mesh convention keeps producer output and consumer input distinguishable by construction (producers always emit a fresh dict).
Payload parts carry their M0 individual id in the existing key
slot (tag_parts()), so a downstream
RCS/CFD component can attribute per-part results – and the results
tier can land them – on the exact individuals that produced the
geometry. Analysis components consuming entity + emitting payload
compose with tier 1:
class BuildGeometry(om.ExplicitComponent):
def setup(self) -> None:
self.add_discrete_input("airframe", val=None) # an m0 Individual
self.add_discrete_output("mesh", val={})
def compute(self, inputs, outputs, discrete_inputs, discrete_outputs) -> None:
airframe = discrete_inputs["airframe"]
mesh = drone_geometry(motor_mass=airframe.get("motor.mass"), split_instances=True)
discrete_outputs["mesh"] = tag_parts(mesh, {"frame": airframe.id})
(One OM sharp edge, verified: declare typed defaults on both ends of a
discrete connection – val={}, not val=None – because setup-time
compatibility is isinstance on the declared defaults and None
matches nothing; probe I1.)
Tier 3: the file boundary¶
External tools (RCS codes, CFD, FEA) want files, and
ExternalCodeComp already expects them. The convention is a tiny
frozen dataclass that flows as a discrete value while the bytes stay
on disk:
@dataclass(frozen=True)
class FileArtifact:
"""A file crossing the analysis boundary: a path plus content identity."""
path: str
sha256: str
media_type: str = "application/octet-stream"
def to_json(self) -> dict[str, str]:
return {"path": self.path, "sha256": self.sha256, "media_type": self.media_type}
A boundary component (e.g. a STEP/STL writer over a tier-2 recipe) writes the file and emits
FileArtifact(path, sha256(content)). Consumers handartifact.pathtoExternalCodeComp’sexternal_input_filesor their own subprocess.The hash is the point: it is the caching identity (same recipe, same hash – skip the external run) and the recorder-bloat fix. The recorder sees ~200 bytes of JSON instead of megabytes of mesh:
to_jsonis exactly the hookmake_serializabletries first, so a recorded case reads back the full artifact record losslessly (verified, probe K3 – against finding 2’s silent'Recipe'degradation).Paths are per-case working directories keyed by interpretation id (
Uav::Drone#0+ case counter), so concurrent DOE cases never collide and a recorded case can locate its files afterwards.
Tier 4: SysML integration¶
The handoff is modeled as flows between analysis actions – the
constructs longeron already parses (ItemDefinition, FlowUsage,
typed action parameters; vendored ecore nsURI 20250201) – so OM
wiring becomes derivable from the model:
item def MeshModel;
action def BuildGeometry { in span : Real; out mesh : MeshModel; }
action def RcsAnalysis { in mesh : MeshModel; out rcs : Real; }
part def Uav {
attribute span = 2.5;
action build : BuildGeometry { in span = span; }
action rcs : RcsAnalysis;
flow of MeshModel from build.mesh to rcs.mesh;
}
def derive_flows(
model: M.Model, part: str | M.Definition | M.Usage
) -> list[tuple[str, str, str | None]]:
"""Resolved (source, target, payload qname) triples from the part's flow usages."""
derive_flowsresolves eachFlowUsage’s endpoint strings against the part’s action members and returns the connection listbuild_problemturns intoconnect()calls – discrete for item-typed parameters, continuous forReal-typed ones. The declared payload type is validated against both ends’ parameter types, the same declared-contract stance the@ExternalAnalysisbinding already takes for calc defs.Analysis actions bind to components the same way calcs do: an
@ExternalAnalysis { component = "module:attr"; }annotation on anaction defnames the ExplicitComponent; the existing contract validation extends becauselist_inputs/list_outputsalready report discrete variables (probe K1).Two validation diagnostics come first, independent of everything else:
dangling-flow(an endpoint that does not resolve; warning severity, mirroringdangling-exposefrom the view-persistence design) andflow-payload-mismatch(payload type vs endpoint parameter types). Today both dangle silently.
Results land on the interpretation¶
Closing the loop is what makes cases auditable: after a run, OM outputs become attribute values on the case’s individuals.
def record_case(build: ProblemBuild, outputs: Mapping[str, Any] | None = None) -> Interpretation:
"""A new interpretation: the case's individuals with the problem's outputs as slots."""
record_casedeep-copies the build’s interpretation and writes each promoted output onto the matching individual’s slot (Instance.set()– verified write-back mechanics). The result is an immutable interpretation snapshot: input point + output values, one object.Individual ids give case recording stable, position-independent keys (
Uav::Drone#0.motorstaysUav::Drone#0.motoracross mixes); traceability runs from any output back to the configured individuals that produced it; population roll-ups (Interpretation.rollup,m0.pyl. 155) and the scoreboard’svalues=seam consume the snapshot directly.The snapshot’s
to_dict()is JSON-clean (verified: 274 chars for the probe model) – the natural recorder payload and the futureapplication/vnd.longeron.m0+jsonsidecar shape, per the M0 design’s API stance.
Picklability and MPI¶
Serial: discretes pass by reference (finding 3); the frozen- payload convention is the only defense. No copies means no cost.
MPI: every discrete crossing a rank boundary is pickled by mpi4py.
Individuals pickle (verified: 3-22 KB, because thedefinitionbacklink drags a copy of the reachable M1 graph – fine per-case, unacceptable inside a large mesh). Rule: entities cross ranks as individuals; bulk payloads cross as plain dicts/recipes keyed by individual id strings, never embeddingIndividualobjects.cadquery/OCC solids never enter a discrete slot (tier 2); rebuild from recipes on the consuming rank.
Gradient machinery is unaffected: discretes are invisible to the derivative system except through the documented driver gate (probes B1/B3).
Case recording¶
The recorder findings (2, and probes C/F/G/K3) fix the conventions:
Dict/list/scalar payloads record losslessly as JSON – at full size, per iteration. Anything above ~100 KB flows as a
FileArtifact(tier 3), never as an inline discrete, or is excluded viarecording_options['excludes'].Custom classes crossing the recorder must implement
to_json(themake_serializablehook); otherwise they silently record as a class-name string.FileArtifact.to_jsonexists for exactly this, andIndividualgains ato_jsonalias forto_dictso a recorded entity case reads back as its full bundle rather than'Individual'.The interpretation snapshot (previous section) is the durable record; the OM sqlite file is a per-run artifact. Snapshot ids link the two.
What we deliberately do not build¶
No OpenMDAO fork, no patched internals. Everything rides
add_discrete_input/add_discrete_output, stock drivers, stock recorders.No auto-CFD/RCS/FEA adapters. The design ships the pipe and the conventions (entity bundles, mesh dicts, recipes,
FileArtifact); physics components remain user code behind the existing@ExternalAnalysiscontract.No custom serialization format. Pickle in memory and across MPI,
to_jsonat rest, real files behindFileArtifact. Nothing new to version.No object units. Payloads are structures; their scalar leaves keep the units design’s story unchanged.
No M1 mutation. Interpretation snapshots never write back into the model;
Interpreter.snapshotremains a separate, explicit tool (M0 design, non-goals).No dataflow engine.
derive_flowsproduces OM connections; OM owns execution order, convergence, and parallelism.
Decisions¶
Adopted 2026-08-27. Two sequencing decisions define the implementation order: the trades variant-bundle fix lands first (D5), and the flow diagnostics ship independently, also first (D6).
Optionality: the continuous-sweep user pays nothing. The light path materializes one implicit anonymous interpretation lazily – without
interpretation=,build_problembehaves exactly as today (verified unchanged for scalar-only models), and the implicit point is created only when something asks for it (record_case, entity binding, M0-keyed payloads). Zero ceremony, zero cost until used; the identity stays true because the implicit point is an interpretation.Result lifecycle: a new immutable interpretation snapshot per case.
record_casereturns a fresh object and the input interpretation stays pristine, matching the trades machinery’s interpreter-exact re-evaluation honesty. A mutable case history is un-auditable: re-running a case overwrites the evidence. In-placeset()remains for interactive notebook use, documented as outside the recorded lifecycle.The discrete value is the
Individualin-process (identity,get(), definition backlink for conformance checks), and itsto_dict()bundle at process/file/recorder boundaries, converted automatically by theto_jsonhook and the MPI bulk-payload rule.FileArtifactlives in both layers: the dataclass inlongeron.analysis, plus an examples-shippeditem def FileArtifactconvention matching the@ExternalAnalysisprecedent (convention packages over stdlib additions until the shape settles).The empty variant bundles are fixed first, in trades. Variants declaring inline redefinitions (
variant item x : Motor { :>> mass = ... }) yield emptyVariationPointbundles (trades.pyl. 397 instantiates the variant’s type, dropping body redefinitions). Entity binding inherits the fix for free; the bug silently zeroes catalogs today.The flow diagnostics ship independently, first.
dangling-flowandflow-payload-mismatchneed no OM work at all; they close finding 4’s silent half at validation time and make tier 4’s inputs trustworthy.Heterogeneous per-index selection is deferred to the existing trades phase-2 item. Entity binding naturally extends (
motors : MotorChoice[4]becomes one discrete input per index, idsmotors#0..3), but trades enumerates homogeneously today; nothing in this design blocks it.
References¶
OpenMDAO 3.45.0 (installed):
core/driver.pyl. 530,drivers/doe_driver.pyl. 63,core/conn_graph.pyl. 85 (are_compatible_values),core/group.pyl. 2066 (_discrete_transfer),utils/general_utils.pyl. 772 (make_serializable),recorders/sqlite_recorder.pyl. 476,components/external_code_comp.pyl. 41-56, 234; the discrete variables feature docs (openmdao.org, “Discrete Variables”).Longeron surfaces:
longeron.analysis.mdao(l. 555, 589, 655),longeron.analysis.trades(l. 376, 397),longeron.m0(l. 60, 155, 227, 453),longeron.analysis.geometry,longeron.analysis.grand(l. 131),model.pyl. 606 (FlowUsage),interpreter.pyl. 770 (instantiate).SysML v2 constructs:
ItemDefinition/ItemUsage,Flow/FlowUsage/FlowEnd/PayloadFeature(vendoredSysML.ecore, nsURI 20250201); grammar rulesitemDefinition(SysML.g4l. 1171) andflowUsage/flowDeclaration(l. 1553, 1573).Sibling designs: M0 interpretations (the identity this design stands on), units (floats-only invariant, OM
units=boundary), view persistence (the dangling-reference diagnostic precedent).