Shows the graph of a loaded model with what Transmuter does to it: where Transmuter.compile/3 breaks the graph and why, which subgraphs Transmuter.Fusion replaced, and which values are static.
For the graph as the exporter wrote it, load the model with fuse: false and call as_graph/2 or to_html/3 without :inputs:
model = Transmuter.load!({:hf, "Xenova/all-MiniLM-L6-v2", "onnx/model.onnx"}, fuse: false)
Transmuter.Graph.to_html(model, "minilm.html")Transmuter.load!/2 makes the large Constant nodes params, so these values show as inputs without a producer node.
In Livebook, as_graph/2 shows an interactive view. It needs the optional kino dependency:
model = Transmuter.load!({:hf, "Xenova/all-MiniLM-L6-v2", "onnx/model.onnx"}, backend: EXLA.Backend)
inputs = %{"input_ids" => ids, "attention_mask" => mask, "token_type_ids" => types}
Transmuter.Graph.as_graph(model, inputs: inputs, compile: [compiler: EXLA])Outside Livebook, to_html/3 writes the same view to one HTML file, which a browser opens without a network connection:
Transmuter.Graph.to_html(model, "minilm.html", inputs: inputs, compile: [compiler: EXLA])The view puts the nodes in groups by their name scope, such as /encoder/layer.0. Click a group to open it, and click the frame of an open group to close it. Click a node to see its details. The fill of a node gives its segment. A node with static outputs has a dashed border, a node that replaces fused nodes has a thick border, and a break node is red. Open /encoder/layer.0/attention/self to see the Shape, Gather, Unsqueeze and Concat nodes that make the targets of the Reshape nodes. They are static: they run on the host when the model compiles, and the compiled function holds only their results.
Without :inputs, the view is static only: it shows the fused nodes and the nodes that fold on the host from the static values of the model, of its subgraphs and of the :static_inputs. It cannot show the graph breaks, and it does not follow static values from node to node, because some of them depend on the shapes of the inputs. Thus it does not mark the Shape chains. With :inputs, overlay/2 compiles the model and calls it once, and it shows what that call did.
Summary
Functions
Shows the graph of the model in Livebook, as a Kino.JS view.
Returns the nodes of the model with what Transmuter does to them.
Writes the view of as_graph/2 to one HTML file at path, with the scripts and the data in the file. It needs no Kino and no network connection.
Returns the groups of overlay, from overlay/2, as a tree.
Functions
Shows the graph of the model in Livebook, as a Kino.JS view.
It needs the optional kino dependency. The options of tree/2 go to it, and the other options go to overlay/2, which raises on an unknown option. The view loads no file from the network.
Returns the nodes of the model with what Transmuter does to them.
Each node is a map with:
:id- the index of the node in the order of the graph. The nodes of a subgraph come after their parent node:position- the index of the node inmodel.nodes, which telemetry events give, ornilfor a node of a subgraph:name,:op_type,:inputsand:outputs- those of the ONNX node:path- the parts of the name. The nodes of a subgraph have the path of their parent node and the name of the subgraph attribute first, such as["optimum::if", "then_branch", ...]:reads- the ids of the nodes whose outputs the node reads, also the outputs of its subgraphs. The inputs and the initializers of a subgraph hide the values of the same names outside it:fused- the nodes thatTransmuter.Fusionreplaced with this node, as"OpType name":attributes- the ONNX attributes of the node by name, as plain values: a number, a string or a list of them. A tensor of at most 16 elements gives its values, and a larger tensor gives%{shape: dims, type: type}. A subgraph gives its name:placement- where the node runs, seeTransmuter.placement/2, ornilfor a node that did not run:literals- the static inputs of a node that runs on the device. A traced segment takes them as literals:static- whether the outputs of the node are static values, such as the output ofShape. Without:inputs, it istruefor the nodes that fold on the host. It isnilfor a node that did not run:shapes- the shape and the type of each output, as%{shape: [1, 8, 384], type: "f32"}, ornilfor an output that is not a tensor. It isnilwithout:inputsand for a node that did not run:segment- the index of the segment of the node in:segments, ornil:break-%{reason: String.t(), after: integer}for a node where the graph breaks, after the segment of index:after, ornil
The result also has :segments, a list of %{start: integer, stop: integer | :end, compiled: boolean, reason: String.t() | nil}. It is empty without :inputs.
Options
:inputs- the inputs of one call. Then the function compiles the model, calls it once and records what the call did. The default isnil, which gives the static view.:static_inputs- the static inputs, as forTransmuter.compile/3. The default is%{}.:compile- the options ofTransmuter.compile/3for the call, such as[compiler: EXLA]. The default is[].
An unknown option raises ArgumentError.
Writes the view of as_graph/2 to one HTML file at path, with the scripts and the data in the file. It needs no Kino and no network connection.
The options are those of as_graph/2.
Returns the groups of overlay, from overlay/2, as a tree.
The groups follow the name scopes of the nodes. A group is a map with :id, :kind ("group"), :name, :scope (such as "/encoder/layer.0"), :children, :segments (the segments of its nodes) and :counts: the numbers of its :nodes, of its :static and :fused nodes, of its :breaks and of its hidden :constants.
A node is a map with the fields of overlay/2, an :id such as "n12", :kind ("node") and a :label: its name relative to the scope of its group, or nil when the exporter generated the name from the op type, such as Shape_6488. Its :reads are the ids of the nodes that it reads, so a view can draw each edge between the shown groups that hold its ends.
Options
:max_members- a group with more nodes directly in its scope than this splits them, in the order of the graph, into groups of equal size, such as"ops 1-21"and"ops 22-41". Exports without name scopes and large flat scopes, such as the encoder of Kokoro, then still give small groups. The default is40.:constants- whether to keep theConstantnodes. Else each group counts theConstantnodes in its scope. The default isfalse.