Skip to content
Community

logo

KubeRay#

Chart Name Version App version
kuberay-operator 1.5.1
ray-cluster 1.5.1

KubeRay is an open-source Kubernetes operator for deploying and managing Ray applications. It provides Kubernetes-native custom resources to support distributed workloads efficiently and reliably.

Key Resources#

  • RayCluster
    Manages the full lifecycle of a Ray cluster, including autoscaling, fault tolerance, and cluster upgrades.

  • RayJob
    Creates a temporary RayCluster, submits a job, and optionally deletes the cluster after completion. Useful for batch and scheduled workloads.

  • RayService
    Combines a RayCluster with a Ray Serve deployment graph. Enables zero-downtime upgrades and high availability for serving APIs or models.

k0rdent Integration#

KubeRay is compatible with k0rdent's multi-cluster model. You can deploy it using the MultiClusterService and manage Ray workloads declaratively across clusters.

Prerequisites#

Deploy k0rdent v1.8.0: QuickStart

Install template to k0rdent#

helm upgrade --install kuberay-operator oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=kuberay-operator:1.5.1" -n kcm-system
helm upgrade --install ray-cluster oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=ray-cluster:1.5.1" -n kcm-system

Verify service template#

kubectl get servicetemplates -A
# NAMESPACE    NAME                            VALID
# kcm-system   kuberay-operator-1-5-1          true
# kcm-system   ray-cluster-1-5-1               true

Deploy service template#

apiVersion: k0rdent.mirantis.com/v1beta1
kind: MultiClusterService
metadata:
  name: kuberay
spec:
  clusterSelector:
    matchLabels:
      group: demo
  serviceSpec:
    services:
    - template: kuberay-operator-1-5-1
      name: kuberay-operator
      namespace: kuberay
    - template: ray-cluster-1-5-1
      name: ray-cluster
      namespace: kuberay

Install template to k0rdent#

helm upgrade --install traefik oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=traefik:39.0.5" -n kcm-system
helm upgrade --install kuberay-operator oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=kuberay-operator:1.3.2" -n kcm-system
helm upgrade --install ray-cluster oci://ghcr.io/k0rdent/catalog/charts/kgst --set "chart=ray-cluster:1.3.2" -n kcm-system

Verify service template#

kubectl get servicetemplates -A
# NAMESPACE    NAME                            VALID
# kcm-system   traefik-39-0-5                  true
# kcm-system   kuberay-operator-1-3-2          true
# kcm-system   ray-cluster-1-3-2               true

Deploy service template#

apiVersion: k0rdent.mirantis.com/v1beta1
kind: MultiClusterService
metadata:
  name: kuberay
spec:
  clusterSelector:
    matchLabels:
      group: demo
  serviceSpec:
    services:
    - template: traefik-39-0-5
      name: traefik
      namespace: kuberay
      values: |
        traefik:
          deployment:
            kind: DaemonSet
          ports:
            web:
              port: 8000
              hostPort: 80
            websecure:
              port: 8443
              hostPort: 443
    - template: kuberay-operator-1-3-2
      name: kuberay-operator
      namespace: kuberay
    - template: ray-cluster-1-3-2
      name: ray-cluster
      namespace: kuberay
      values: |
        ray-cluster:
          head:
            service:
              type: ClusterIP
              ports:
                - name: dashboard
                  port: 8265
                  targetPort: 8265

Now you need to create an Ingress rule manually in the child cluster(s): 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kuberay-dashboard
namespace: kuberay
spec:
rules:
    - host: kuberay.example.com
    http:
        paths:
        - path: /
            pathType: Prefix
            backend:
            service:
                name: ray-cluster-kuberay-head-svc
                port:
                number: 8265

You can now access your KubeRay web UI at kuberay.example.com.

BACK TO CATALOG