Runs ONNX models on any Nx backend.
Transmuter reads an ONNX file, such as an export from the Hugging Face Hub, and turns its graph into Nx operations. It does not port model architectures. The graph that the exporter traced is the model.
model = Transmuter.load!({:hf, "Xenova/all-MiniLM-L6-v2", "onnx/model.onnx"}, backend: EMLX.Backend)
run = Transmuter.compile(model, inputs, compiler: EMLX)
%{"last_hidden_state" => hidden} = run.(inputs)Shapes are resolved when the function is compiled, so compile/3 returns a
function for the shapes of the inputs it was given. Compile again for other
shapes. run/4 is the uncompiled function, for Nx.Defn.jit/2, gradients or
eager execution.
Summary
Functions
Compiles the model for the shapes and types of inputs.
Loads an ONNX model.
Runs the model graph. Call it inside Nx.Defn.jit/2 or Nx.Defn.compile/3.
Functions
Compiles the model for the shapes and types of inputs.
Returns a function that takes the inputs and returns the outputs. The params
are the model's own, so load it with :backend first.
Options
:static_inputs- inputs known when the model compiles, seerun/4
Other options go to Nx.Defn.compile/3, for example compiler: EMLX.
Loads an ONNX model.
The source is a local path or {:hf, repo, file}. Hub files are downloaded
once into ~/.cache/transmuter, together with their external data files.
Options
:cache_dir- the directory for Hub downloads:backend- moves the params to this backend, for exampleEMLX.Backendor{EXLA.Backend, client: :cuda}. On EMLX, 4-bit and 8-bitMatMulNBitsweights stay packed, seeTransmuter.EMLX. By default the params stay onNx.BinaryBackend.:compute_type- runs f16 and bf16 graphs in this type, for example:f32. ONNX Runtime on the CPU computes half-precision graphs in f32, so use:f32to match it. By default the graph keeps its own types.:trainable- whentrue, every float initializer is a param, so small weights such as a 1-element bias get gradients. By default float initializers with up to 16 elements are folded as constants.
Returns a map with:
:params- the weights, on the:backendwhen it is given:inputsand:outputs- the graph input and output names
Runs the model graph. Call it inside Nx.Defn.jit/2 or Nx.Defn.compile/3.
static_inputs are graph inputs whose values are known when the function is
traced. Give them as Nx.BinaryBackend tensors. Use them for flags that select
a branch, such as use_cache_branch, and for inputs that decide an output
shape, such as the input_ids that feed NonZero in Grounding DINO. Each new
value compiles again.
Outputs that are empty are left out of the result.