Graph

Graph is the main application assembly API in SiMa Neat.

If you come from ML, think of a Graph like a small model graph or a function:

• it has inputs;
• it has outputs;
• it contains work in the middle, such as decode, resize, preprocess, inference, postprocess, and custom logic;
• it can be reused inside a larger graph.

If you come from embedded, think of a Graph like a named data-flow wiring diagram:

• frames or tensors enter through named doors;
• each processing step runs in order or through explicit branches;
• outputs leave through named doors;
• Neat decides the efficient runtime wiring underneath.

Most users should not need to know about GStreamer, appsrc, appsink, queues, or internal runtime ports. Those are implementation details. The public API is:

simaai::neat::Graph g;
g.add(...);      // continue the same linear chain
g.connect(...);  // add explicit graph topology

auto run = g.build();

The shortest mental model

Use add() for the common single-path case:

simaai::neat::Model model("resnet50.tar.gz");

simaai::neat::Graph g("classifier");
g.add(simaai::neat::nodes::Input("image"));
g.add(model);
g.add(simaai::neat::nodes::Output("classes"));

auto run = g.build();
run.push("image", simaai::neat::TensorList{image_tensor});
auto classes = run.pull("classes");

For a graph with exactly one public input and one public output, the names are optional at runtime:

run.push(simaai::neat::TensorList{image_tensor});
auto classes = run.pull();

Use connect() when the graph is not just one chain:

simaai::neat::Graph camera("camera");
camera.add(simaai::neat::nodes::Input("image"));

simaai::neat::Graph route("model_route");
route.add(simaai::neat::nodes::Input("image"));
route.add(model);
route.add(simaai::neat::nodes::Output("classes"));

simaai::neat::Graph app("app");
app.connect(camera, route);

auto run = app.build();
run.push("image", simaai::neat::TensorList{image_tensor});
auto classes = run.pull("classes");

Input("name") and Output("name") are named doors

The most important rule is:

A named Input or Output describes a public boundary of a Graph fragment. It is a door, not always a physical runtime source or sink.

That sentence matters because reusable graph fragments are meant to be composed.

Example reusable fragment:

simaai::neat::Graph route("route");
route.add(simaai::neat::nodes::Input("image"));
route.add(model);
route.add(simaai::neat::nodes::Output("classes"));

This fragment says:

• "I accept something called image."
• "I produce something called classes."
• "Between those names, run model."

It does not necessarily mean Neat should create a separate physical input queue and a separate physical output queue every time the fragment is used. If this fragment is placed inside a larger graph, its Input("image") and Output("classes") are often just connection points.

Plain-language analogy

A Graph fragment is like a function:

classes = route(image)

The function has a parameter named image and a return value named classes. When you call the function from another function, you do not create a new keyboard and monitor inside the function. You just connect the caller's value to the parameter and connect the return value to the next step.

Neat treats Graph boundaries the same way.

Boundary materialization: when doors become real runtime I/O

During build(), Neat performs a boundary materialization pass. This pass decides which named doors become real runtime I/O and which named doors are just internal wiring labels.

The rules are intentionally simple:

Boundary node	If it is on the outside of the final app	If it is connected inside a larger graph
Input("name")	Becomes a public run.push("name", ...) entry point.	Is removed from the executable pipeline and replaced by a direct internal connection.
Output("name")	Becomes a public run.pull("name") exit point.	Is removed from the executable pipeline and replaced by a direct internal connection.

So this reusable fragment:

simaai::neat::Graph route("route");
route.add(simaai::neat::nodes::Input("image"));
route.add(model);
route.add(simaai::neat::nodes::Output("classes"));

becomes one of two things depending on where it is used.

Used as a complete app

auto run = route.build();
run.push("image", simaai::neat::TensorList{image_tensor});
auto classes = run.pull("classes");

Input("image") is materialized as a public input. Output("classes") is materialized as a public output.

Used inside a larger app

simaai::neat::Graph app("app");
app.connect(camera, route);
app.connect(route, display);

The route fragment's Input("image") and Output("classes") are internal connection points. Neat removes those boundary declarations from the executable pipeline and connects the real work directly:

camera -> model -> display

The logical names are still kept for diagnostics, named Run APIs, and visualization. They are just not extra runtime sinks/sources.

Why Neat does this

This design avoids several common problems.

1. Reusable fragments behave like ML modules

ML users expect modules to compose:

image -> preprocess -> model -> postprocess -> classes

You should be able to package preprocess -> model -> postprocess as a reusable Graph and insert it into a larger app without changing the app's mental model.

2. Internal boundaries do not create fake outputs

Without boundary materialization, a fragment like this:

route.add(Input("image"));
route.add(model);
route.add(Output("classes"));

could accidentally create a real output sink inside the middle of a larger graph. Then the real final output would be a second sink. Many media runtimes do not allow that shape in a single linear segment, and even when they do, it usually means the wrong thing.

Boundary materialization prevents that. Internal Input/Output nodes are treated as declarations and are removed before executable pipeline construction.

3. It avoids hidden copies and bridge overhead

A boundary that becomes a real runtime input or output may require a queue, a memory handoff, or a device/CPU boundary.

If a boundary is only there to name a reusable fragment's input or output, creating that physical handoff would be wasteful. It could add latency, reduce throughput, or accidentally move data out of device-visible memory.

The compiler therefore keeps data on the most direct path it can.

4. Error messages can stay customer-friendly

Even though Neat removes internal boundary nodes from the executable pipeline, it keeps a mapping from the customer-facing Graph to the lowered runtime graph.

That means diagnostics can still say things like:

Graph endpoint "classes" is connected to multiple producers.
Use graphs::Combine(..., CombinePolicy::ByFrame) or split the output names.

instead of exposing low-level runtime element names.

How Neat lowers a Graph at build time

When you call:

auto run = graph.build();

Neat roughly does this:

1. Collects the public Graph: all Nodes, Models, reusable Graph fragments, names, and connections.
2. Resolves named boundaries: finds Input("name") and Output("name") declarations.
3. Classifies boundaries:
   • external inputs become run.push(...) entry points;
   • external outputs become run.pull(...) exit points;
   • internal boundaries become graph wiring only.
4. Builds an executable graph: removes internal boundary declarations and keeps only real executable work.
5. Preserves a map back to your Graph: used by describe(), diagnostics, metrics, and visualization.
6. Runs one unified Run: the same Run API is used for linear and connected graphs.

You do not need to choose pipeline mode vs graph mode. Graph::build() builds the whole application.

add() vs connect()

Use add() when the next thing is simply after the previous thing:

g.add(nodes::Input("image"));
g.add(nodes::VideoConvert());
g.add(model);
g.add(nodes::Output("classes"));

This means:

image -> VideoConvert -> model -> classes

Use connect() when you are wiring fragments explicitly:

app.connect(camera, route);
app.connect(route, display);

This means:

camera -> route -> display

When a graph has branched and there is no longer one obvious "last node", add() fails with a diagnostic and asks you to use connect() instead. This is deliberate. It prevents Neat from guessing wrong.

Multiple inputs and multiple outputs

For multi-input or multi-output graphs, use names.

run.push("image", simaai::neat::TensorList{image_tensor});
run.push("metadata", simaai::neat::TensorList{metadata_tensor});

auto classes = run.pull("classes");
auto preview = run.pull("preview");

Plain push(...) and pull() are still available when there is exactly one public input or output. If there is more than one, Neat fails closed and lists the available names.

Branch: one input, multiple outputs

A branch means one stream is intentionally split into named outputs, for example one copy to preview and one copy to a model route.

auto branch = simaai::neat::graphs::Branch("image", {"preview", "model"});

Conceptually:

image -> preview
      -> model

Branching can introduce backpressure: if one branch stops consuming data, it can slow or block the producer depending on the selected runtime policy. Neat's branch helper makes that behavior explicit instead of hiding it behind accidental duplicate outputs.

Combine: multiple inputs, one output

A combine means several named inputs must be packaged into one named output.

auto combine = simaai::neat::graphs::Combine(
    {"left", "right"},
    "pair",
    simaai::neat::graphs::CombinePolicy::ByFrame);

Conceptually:

left  ----\
          -> pair
right ----/

CombinePolicy tells Neat how to match items:

• ByFrame: combine samples that have the same frame_id.
• ByPts: combine samples that have the same presentation timestamp (pts_ns).
• None: do not synchronize; only use when order alone is correct for your application.

There is no hidden fallback. If you select ByFrame, missing frame IDs are an error. If you select ByPts, missing timestamps are an error. This makes bugs visible early instead of silently mixing the wrong frames.

What to name endpoints

Use names that describe application meaning, not implementation details:

Good:

nodes::Input("image")
nodes::Input("left_camera")
nodes::Output("classes")
nodes::Output("detections")
nodes::Output("preview")

Avoid:

nodes::Input("appsrc0")
nodes::Output("sink1")
nodes::Output("out")  // okay for tiny tests, not great in apps

If a fragment contains several unnamed outputs, Neat gives them deterministic suffixes such as classes_0, classes_1, and classes_2. Prefer explicit names in production code.

Practical examples

Reusable model route

simaai::neat::Graph make_classifier(simaai::neat::Model& model) {
  simaai::neat::Graph route("classifier");
  route.add(simaai::neat::nodes::Input("image"));
  route.add(model);
  route.add(simaai::neat::nodes::Output("classes"));
  return route;
}

Use by itself:

auto route = make_classifier(model);
auto run = route.build();
run.push("image", simaai::neat::TensorList{image});
auto classes = run.pull("classes");

Use inside a larger app:

simaai::neat::Graph app("app");
app.connect(camera, route);
app.connect(route, telemetry);

In the larger app, the route's boundary nodes are internal declarations. They do not become extra public push/pull endpoints unless they are still on the outside of the final graph.

Pass-through adapter

Sometimes a fragment only renames a boundary:

simaai::neat::Graph adapter("adapter");
adapter.add(simaai::neat::nodes::Input("raw"));
adapter.add(simaai::neat::nodes::Output("image"));
adapter.connect("raw", "image");

When used inside another graph, this can compile down to a direct wire. Neat keeps the names for readability and diagnostics, but it does not create useless runtime work.

Rules of thumb

• Use Graph for applications and reusable fragments.
• Use Model directly in Graph::add(model) when you want the model's normal route.
• Use named Input and Output nodes to declare the public contract of a fragment.
• Use add() for a straight chain.
• Use connect() for topology.
• Use named run.push("name", ...) and run.pull("name") for multi-input or multi-output apps.
• Let Neat own the low-level runtime details.

See also

• Node
• Model
• Input and Output
• Pipeline
• Public Graph and graph helpers

Tutorials

• Build a Custom Data Graph
• Embed a Model Inside a Graph
• Run Multiple Streams in One Graph