Skip to content
Community

logo

Kubeflow KServe#

k0rdent facilitates the deployment and lifecycle management of KServe within Kubernetes clusters by leveraging its multi-cluster orchestration capabilities. KServe provides a Kubernetes-native platform for serving and managing machine learning models with high performance and scalability.

KServe builds on Kubernetes to provide a standardized, scalable, and reliable solution for deploying, monitoring, and managing machine learning models in production. With features like traffic management, canary deployments, and autoscaling, KServe enables seamless deployment of models across various frameworks including TensorFlow, PyTorch, scikit-learn, XGBoost, and ONNX.

KServe consists of two main components:

  1. KServe CRDs - Custom Resource Definitions that provide the foundation for KServe's functionality
  2. KServe Controller - The controller that implements serverless model serving capabilities

Through k0rdent, users can easily install and manage KServe across multiple clusters, providing a centralized control plane to monitor health, enforce policies, and enable enterprise-grade machine learning serving. This application provides the ability to deploy the KServe Controller in Serverless mode without ModelMesh functionality enabled, which is optimized for environments where autoscaling and scale-to-zero capabilities are prioritized.

By deploying KServe from the catalog, k0rdent enables users to build and scale ML inference services while leveraging Kubernetes-native infrastructure.

Prerequisites#

Deploy k0rdent v1.8.0: QuickStart

Install template to k0rdent#

helm upgrade --install kserve-crd oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=kserve-crd:v0.15.0" -n kcm-system
helm upgrade --install kserve oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=kserve:v0.15.0" -n kcm-system

Verify service template#

kubectl get servicetemplates -A
# NAMESPACE    NAME                            VALID
# kcm-system   kserve-crd-v0-15-0              true
# kcm-system   kserve-v0-15-0                  true

Deploy service template#

apiVersion: k0rdent.mirantis.com/v1beta1
kind: MultiClusterService
metadata:
  name: kserve
spec:
  clusterSelector:
    matchLabels:
      group: demo
  serviceSpec:
    services:
    - template: cert-manager-1-18-2
      name: cert-manager
      namespace: kserve
      values: |
        cert-manager:
          crds:
            enabled: true
    - template: kserve-crd-v0-15-0
      name: kserve-crd
      namespace: kserve
      dependsOn:
      - name: cert-manager
        namespace: kserve
    - template: kserve-v0-15-0
      name: kserve
      namespace: kserve
      dependsOn:
      - name: kserve-crd
        namespace: kserve
      values: |
        kserve:
          controller:
            deploymentMode: RawDeployment # Serverless
          modelmesh:
            enabled: false

Prerequisites#

Deploy k0rdent v1.8.0: QuickStart

Install template to k0rdent#

helm upgrade --install istio-base oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=istio-base:1.27.1" -n kcm-system
helm upgrade --install istiod oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=istiod:1.27.1" -n kcm-system
helm upgrade --install istio-gateway oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=istio-gateway:1.27.1" -n kcm-system
helm upgrade --install cert-manager oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=cert-manager:1.18.2" -n kcm-system
helm upgrade --install knative-operator oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=knative-operator:1.17.4" -n kcm-system
helm upgrade --install kserve-crd oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=kserve-crd:v0.15.0" -n kcm-system
helm upgrade --install kserve oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=kserve:v0.15.0" -n kcm-system

Verify service template#

kubectl get servicetemplates -A
# NAMESPACE    NAME                            VALID
# kcm-system   istio-base-1-27-1               true
# kcm-system   istiod-1-27-1                   true
# kcm-system   istio-gateway-1-27-1            true
# kcm-system   cert-manager-1-18-2             true
# kcm-system   knative-operator-1-17-4         true
# kcm-system   kserve-crd-v0-15-0              true
# kcm-system   kserve-v0-15-0                  true

Deploy service template#

apiVersion: k0rdent.mirantis.com/v1beta1
kind: MultiClusterService
metadata:
  name: kserve
spec:
  clusterSelector:
    matchLabels:
      group: demo
  serviceSpec:
    services:
    - template: istio-base-1-27-1
      name: istio-base
      namespace: kserve
    - template: istiod-1-27-1
      name: istiod
      namespace: kserve
    - template: istio-gateway-1-27-1
      name: istio-gateway
      namespace: kserve
    - template: cert-manager-1-18-2
      name: cert-manager
      namespace: kserve
      values: |
        cert-manager:
          crds:
            enabled: true
    - template: knative-operator-1-17-4
      name: knative-operator
      namespace: kserve
    - template: kserve-crd-v0-15-0
      name: kserve-crd
      namespace: kserve
    - template: kserve-v0-15-0
      name: kserve
      namespace: kserve
      values: |
        kserve:
          controller:
            deploymentMode: RawDeployment # Serverless
          modelmesh:
            enabled: false
BACK TO CATALOG