Open Model Interface Spec
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.
|2021-07-26||1.0||Initial version of OMI Spec|
|2022-03-24||1.1||Updated OMI Spec to include new requirements|
To qualify as an Open Model Interface (OMI) compliant container, a container MUST pass the following tests.
The container MUST be tagged (e.g.
registry/org/image:tag) with a unique ID which corresponds to the ID of the model in the upstream model management system.
For example, the container image tag MAY be set to the MLflow Run ID which generated the model.
This is to enable provenance tracking from a given model which generated certain inference results to metadata about how the model was trained. The metadata referred to by the model tag SHOULD record the versions of the code, data and environment that were used to train the model.
Declares Openmodel Annotations
The container MUST have OCI annotations of the following form:
ml.openmodel.interfaces=["kfserving", "modzy"] ml.openmodel.protocols=[["v1","v2"],["v2"]] ml.openmodel.port="8080" ml.openml.model_name="modelname"
ml.openmodel.protocol is list of lists such that the inner list's index corresponds to its interface index e.g. a model with the above configuration would support kfserving v1 and v2 protocols as well as the modzy interface with v2 protocol.
This is so that the serving system at runtime can determine whether it supports a given model without trying to run it so that it can give the user fast feedback.
When started, the container image MUST listen on the TCP port given as the
When a container starts listening on the prescribed port, it MUST adhere to the interface described below corresponding to its configuration.
OCI annotations are not available in a running Docker container. As a result, an OMI container MUST be configurable with the following environment variables to provide a copy of the values associated with the OMI labels:
env: - name: INTERFACE value: <ml.openmodel.interfaces> # e.g. kfserving - name: HTTP_PORT value: <ml.openmodel.port> # e.g. 8080 - name: PROTOCOL value: <ml.openmodel.protocols> # e.g. v2 - name: MODEL_NAME value: <ml.openml.model_name> # e.g. mdodel name
INTERFACEMUST be the string interface supported, as defined in the interfaces section below.
PROTOCOLMAY be the optional sub-protocol of the given interface
MODEL_NAMEMUST be some human-readable name of the model that is to be run inside the container.
HTTP_PORTMUST be the TCP port that the container should listen on when started
If a conflict arises between the OCI annotations and the environment variables, The OMI specification considers OCI annotations to be authoritative.
Interfaces and Protocols
The following interfaces are permitted in the spec:
- v1 REST API with
ml.openmodel.interfaces=["kfserving"] and ml.openmodel.protocols="v1"
- v2 gRPC API with
ml.openmodel.interfaces=["kfserving"] and ml.openmodel.protocols="v2"
- v1 REST API with
ml.openmodel.interfaces=["modzy"] and ml.openmodel.protocols="v1"
- v2 gRPC API with
ml.openmodel.interfaces=["modzy"] and ml.openmodel.protocols="v2"
OMI compliant container images MUST:
* implement at least one of the
modzy API interfaces
* be runnable and responsive on both KServe and Modzy platforms
* be runnable independently in isolation
The Chassis reference implementation of the OMI specification produces containers
* that implement the
* that implement a KServe compatible architecture that follows the
kfserving.v1 specification with one exception. Chassis-generated model containers expect binary input data instead of the json input data the
kfserving.v1 spec designates. Therefore, for KFServing users must base64-encode their input bytes to be included within their json input submission. In this way, the model containers generated by Chassis will run on KFServing as well as Modzy.
* that can run in isolation
Updates to this spec
If an OMI image declares or supports any strings other than the ones defined above, this document MUST be updated with the new interfaces (with links to their upstream protocol specifications), otherwise the container MUST NOT be described as Open Model Interface compliant.
This document can be updated via Pull Request on GitHub.
Any proposed PRs must be accompanied by a public comment period of no less than 2 weeks (14 days) before a merge can be approved.