Offchain Model Execution Node

• What is it?

  • A server that takes data and model execution instructions as inputs, and outputs modeled results

  • Built using specialized hardware

  • For Reppo protocol v1, the execution node must run a Ritual container that is Infernet-compatible

• Why is it important?

  • Handles model execution interfaces and execution for model customers

• What it needs to do:

  • import necessary libraries for model execution

  • store model weights/model files in memory

  • maintain an http server that can:

    • provide model customers an offchain interface to request inference

    • provide model customers an onchain interface via the model-specific onchain contract

    • define acceptable data and model execution instruction input formats

    • provide model outputs to the customer—either onchain via the model-specific onchain contract or to an offchain location defined by the model customer

  • execute model inference requests

  • maintain model and data privacy requirements

  • generate model execution proofs

• Who starts the server and who is responsible for uptime?

  • For Reppo v1, the model developer

  • Reppo labs is currently architecting a trusted model execution service for hosting model execution nodes on behalf of model developers

Latest demo can be accessed here