Spenso parsing flow

Lifecycle

current

Owner

Ownership decision required

Review

2026-08-17 · c9f4e32acd2c

Freshness

reviewed

Evidence

Not an evidence record

Verified scopes

• spenso::network::parsing::ShorthandParsing
  crates/spenso/src/network/parsing/mod.rs · digest aaa5b8535c39

This document describes how symbolic atoms are parsed into tensor networks in crates/spenso/src/network/parsing. It focuses on the control flow, shorthand expansion, opaque leaves, structure inference, and edge cases that affect Schoonschip-style notation.

Audit status: reviewed 2026-08-17 against c9f4e32acd2c. Lifecycle: current implementation architecture.

Entry Points

Parsing starts from NetworkParse methods such as parse_to_atom_net. The parser creates a fresh ParseState and calls try_from_view_impl on the input AtomView.

Settings

Option Examples

Main Dispatch

try_from_view_impl dispatches by atom kind:

Mul  -> try_from_mul
Fun  -> try_from_fun
Add  -> try_from_add
Pow  -> try_from_pow
else -> scalar fallback

Products, sums, and powers may hit the depth leaf boundary before they recurse. Function parsing validates dot arity, applies is_tensorial(strict_tensor_filter), handles broadcast wrappers, and uses the opaque tensor-expression boundary only for recognized shorthand roots.

crates/spenso/src/network/parsing/mod.rs:362

match value {
    AtomView::Mul(m) => Self::try_from_mul(...),
    AtomView::Fun(f) => Self::try_from_fun(...),
    AtomView::Add(a) => Self::try_from_add(...),
    AtomView::Pow(p) => Self::try_from_pow(...),
    a => Ok(Network::from_scalar(a.try_into()?)),
}

Function Flow

Function parsing has the most important ordering rules.

1. Reject malformed dot syntax first. Dot has exactly two arguments in parser syntax.
2. Apply is_tensorial(settings.strict_tensor_filter). If it rejects the root, parse it as a scalar; pure_scalar(x) is unwrapped in that scalar path.
3. If the head has the broadcast tag, require one argument, parse that argument recursively, and apply the head to the result.
4. If shorthand_parsing is Opaque and the root is recognized shorthand (chain, trace, dot, or compact Schoonschip syntax), parse the whole function as one inferred tensor leaf. Ordinary tensor heads do not take this branch.
5. Otherwise enter expanded-function parsing. bracket parses its arguments and multiplies them; other roots enter the shorthand materializer.
6. The Expand fields independently decide whether chain, trace, and compact Schoonschip syntax materialize. A disabled shorthand root is retained as a fast-inferred leaf. An unchanged ordinary head uses regular leaf parsing.
7. Try S::parse. If it succeeds, use a library tensor when possible, otherwise concretize a tensor shell.
8. If structure parsing reports StructureError::EmptyStructure, parse the function as a scalar expression. Propagate other structure errors.

crates/spenso/src/network/parsing/mod.rs:606

if symbol == SPENSO_TAG.dot && value.get_nargs() != 2 {
    return Err(TensorNetworkError::InvalidDotFunction(...));
}

if !value.as_view().is_tensorial(settings.strict_tensor_filter) {
    return Self::parse_scalar_function(value);
}

crates/spenso/src/network/parsing/structure_inference.rs:50

pub trait AtomStructureExt {
    fn infer_structure<S: StructureFromAtom>(...);
    fn is_tensorial(&self, filter: StrictTensorFilter) -> bool;
}

crates/spenso/src/network/parsing/mod.rs:626

if let Some(inference) = settings.shorthand_parsing.opaque_inference()
    && Self::is_shorthand_function(value)
{
    return Self::as_inferred_leaf(...);
}

crates/spenso/src/network/parsing/mod.rs:729

Self::materialize_shorthand(
    value,
    state,
    library,
    function_library,
    settings,
)

Important: materialization must run before the generic leaf parser. Otherwise F(p(rep), slot) can be accepted as a lower-rank tensor with p(rep) stored as metadata, losing the required p(dummy) factor.

Expand vs Opaque

Expand

Selected shorthand families are rewritten into explicit parser input. The schoonschip, trace, and chain fields can be configured independently. Expanded chain and trace roots allocate links; compact vector arguments allocate fresh slots.

Opaque

A recognized shorthand root that survives strict tensor filtering and broadcast handling becomes one leaf. Structure is inferred first, then TensorFromExpression constructs the target tensor from the original expression and inferred structure. Ordinary tensor heads still use regular leaf parsing.

Opaque mode is checked before chain or trace expansion, but after strict tensor classification and broadcast handling. Non-tensor functions stay scalar in opaque mode; depth-limited composite expressions use the separate leaf boundary described below.

Opaque Leaf Boundary

Opaque shorthand parsing is split into two steps.

1. StructureFromAtom::structure_from_atom determines the exposed slots. Fast uses a syntactic walk. Expanded builds an expanded shorthand network and reads dangling slots.
2. TensorFromExpression::tensor_from_expression turns the original atom plus inferred structure into a tensor. Symbolic tensors keep the original expression on a tensor with the inferred structure. Concrete and parametric tensor classes parse the same atom again with ShorthandParsing::expand_all(), execute that nested network, and return the resulting tensor or tensor scalar.

crates/spenso/src/network/parsing/structure_inference.rs:162

match mode {
    StructureInferenceMode::Fast => Self::leaf_structure_from_atom(value),
    StructureInferenceMode::Expanded =>
        Self::expanded_shorthand_structure_from_atom(value),
}

crates/spenso/src/network/parsing/tensor_from_expression.rs:25

/// Builds a tensor leaf from an opaque expression once its structure is known.
pub trait TensorFromExpression<S, Sc, K, FK, Aind, Lib, FunLib> {
    fn tensor_from_expression(...);
}

crates/spenso/src/network/parsing/tensor_from_expression.rs:120

let mut expanded_settings = settings.clone();
expanded_settings.shorthand_parsing = ShorthandParsing::expand_all();

let mut network = Network::try_from_view_with_function_library(
    expression,
    tensor_library,
    function_library,
    &expanded_settings,
)?;
network.execute::<Sequential, SmallestDegree, ...>(...)?;

crates/idenso/src/tensor/mod.rs:118

let mut tensor = structure.structure;
...
tensor.expression = expression.to_owned();
Ok(tensor)

Structure Inference

Fast structure inference is syntactic and intentionally non-semantic.

Materialization

The parser's materialize_shorthand method is the configurable shorthand boundary. It always returns a network. Enabled chain and trace roots materialize topology directly. Enabled compact dot/Schoonschip syntax rewrites to ordinary parser input and recurses. A disabled shorthand root becomes a fast-inferred leaf; an unchanged root containing disabled compact shorthand also remains a leaf. An unchanged ordinary function uses regular leaf parsing.

SchoonschipMaterializer is the narrower atom rewriter used by that network boundary. It stores a current atom plus additional_factors. Additional factors are multiplied beside the current atom and are not inspected by the Schoonschip materializer during that same pass; they are parsed when the parser recurses on the complete expression.

crates/spenso/src/network/parsing/materialization.rs:457

fn materialize_shorthand(...)
    -> Result<Self, TensorNetworkError<K, Symbol>>
{
    if symbol == SPENSO_TAG.chain && !root_chain_disabled {
        return Self::materialize_chain_shorthand(...);
    }
    if symbol == SPENSO_TAG.trace && !root_trace_disabled {
        return Self::materialize_trace_shorthand(...);
    }

    let materialized = SchoonschipMaterializer::with_mode(&state, mode)
        .materialize_shorthand(value.as_view());
    if materialized == value.as_view().to_owned() {
        if root_chain_disabled || root_trace_disabled || has_schoonschip_shorthand {
            return Self::as_inferred_leaf(..., StructureInferenceMode::Fast, ...);
        }
        return Self::parse_regular_function_leaf(...);
    }
    Self::try_from_view_impl(materialized.as_view(), ...)
}

crates/spenso/src/network/parsing/materialization.rs:114

pub(super) fn materialize_shorthand(&self, value: AtomView<'_>) -> Atom {
    self.materialize_shorthand_root(value)
        .map(SchoonschipMaterialization::into_expression)
        .unwrap_or_else(|| value.to_owned())
}

crates/spenso/src/network/parsing/materialization.rs:171

let rep = Self::compact_vector_rep(*lhs)?;
if rep != Self::compact_vector_rep(*rhs)? || !rep.rep.is_self_dual() {
    return None;
}

let slot = self.state.slot(&rep).to_atom();

crates/spenso/src/network/parsing/materialization.rs:20

// Chain and trace materialization chooses the symbolic in/out
// replacements, then lets this Schoonschip helper expand compact
// arguments inside each factor.

Compact Vector Argument

F(..., p(rep), ...)
-> F(..., rep(dummy), ...) * p(rep(dummy))

The dummy slot is allocated from the active ParseState, so it shares the namespace of the surrounding network parse.

Compact Scalar Product

g(p(rep), q(rep))
-> p(rep(dummy)) * q(rep(dummy))

This currently requires both compact vectors to have the same self-dual representation. dot(p(rep), q(rep)) follows the same rule.

The stripped representation in p(rep) means that the vector replaces an omitted tensor slot. It is valid only where such a slot exists: inside another tensor argument, or inside a recognized compact scalar product head such as g or dot. For example, p!(q!(mink!(4))) is valid shorthand and materializes like p(mink(4, d)) * q(mink(4, d)). A bare product such as p!(mink!(4)) * q!(mink!(4)) is malformed input, because the stripped vectors are not replacing slots.

Compact Vector Detection

A compact vector is a function that:

• is not metric or dot;
• is not itself a representation;
• has no explicit slot argument;
• has exactly one direct argument matching the representation wildcard convention.

Sums are accepted only when every summand is a compact vector with the same representation. Products and powers are not compact vector syntax.

Chain Expansion

When chain expansion is enabled, chain(start, end, factors...) first tries to parse each endpoint as a concrete slot. If Schoonschip expansion is enabled, a compact vector endpoint can instead materialize into a fresh endpoint slot plus an additional rank-one factor. The network-level materializer then chooses internal links, rewrites each factor's in/out placeholders, and optionally expands compact arguments inside the factor.

1. Materialize compact endpoints when configured and retain their additional factors.
2. With no chain factors, add g(start, end).
3. Each factor gets a left and right link slot.
4. in and out placeholders are replaced syntactically.
5. The factor is passed through shorthand materialization.
6. Each materialized factor is parsed into one or more factor networks.
7. The factor networks are multiplied in chain order.

If a materialized chain factor is a product with scalar factors and exactly one tensor factor, scalar factors are split out so they can be handled separately. Otherwise the whole factor product is parsed as one expression.

chain(s, e, F(in, out, p(rep)))
-> replace placeholders: F(s, e, p(rep))
-> materialize factor:   F(s, e, d) * p(d)
-> parse product into the chain network

crates/spenso/src/network/parsing/materialization.rs:540

let start = Self::materialize_chain_endpoint(args[0], "start", ...)?;
let end = Self::materialize_chain_endpoint(args[1], "end", ...)?;
...
let factor = ChainExpansion::replace_placeholders(...);
let factor = SchoonschipMaterializer::with_mode(&state, factor_mode)
    .materialize_shorthand(factor.as_view());

Trace Expansion

When trace expansion is enabled, trace(rep, factors...) starts from a representation, not endpoint slots. The network-level shorthand materializer returns the representation dimension for an empty trace and otherwise assigns cyclic links uniformly across all factors. Depending on the Schoonschip expansion settings, it can materialize compact arguments inside each factor after placeholder replacement and before parsing.

1. No factors: parse the representation dimension.
2. For a non-empty trace, allocate one cyclic link per factor.
3. Replace each factor's placeholders with its link and the dual of the next cyclic link.
4. Materialize shorthand inside each factor when configured.
5. Multiply the factor networks.

trace(rep, F(in, out, p(rep)))
-> allocate one cyclic link
-> replace placeholders: F(link, link*, p(rep))
-> materialize factor:   F(link, link*, d) * p(d)

crates/spenso/src/network/parsing/materialization.rs:701

let rep = Representation::try_from(*rep_view)?;
...
let links = (0..factors.len())
    .map(|_| state.slot(&rep))
    .collect::<Vec<_>>();
let left = links[position].to_atom();
let right = links[(position + 1) % factors.len()].dual().to_atom();
let factor = ChainExpansion::replace_placeholders(factor, &left, &right);

Products, Sums, Powers

Edge Cases

Short Algorithm

parse(atom):
  dispatch by atom kind

parse_fun(fun):
  if fun is dot with arg count != 2:
    return InvalidDotFunction

  if !fun.is_tensorial(strict_tensor_filter):
    return scalar(fun)

  if fun is a broadcast wrapper:
    parse its single argument with broadcast handling

  if opaque mode and fun is recognized shorthand:
    infer structure
    return TensorFromExpression(original expression, inferred structure)

  return materialize_shorthand(fun)

materialize_shorthand(fun):
  if chain expansion is enabled and fun is chain:
    materialize compact endpoints when configured
    allocate links
    replace in/out placeholders
    materialize configured shorthand inside each factor
    parse factor expressions
    return chain network

  if trace expansion is enabled and fun is trace:
    return the dimension for zero factors
    allocate one cyclic link per factor
    replace in/out placeholders
    materialize configured shorthand inside each factor
    parse factor expressions
    return trace network

  select the effective Schoonschip expansion mode
  rewrite the function if that mode is enabled
  if an enabled shorthand remains unchanged:
    return an inferred tensor leaf
  if an ordinary function remains unchanged:
    return a regular tensor leaf
  parse the rebuilt expression recursively