Large Language Model (LLM) Serving workshop with Red Hat OpenShift AI

Introduction

Welcome. In this workshop, you will learn how to deploy models as API endpoints using OpenShift AI and make requests.

You use an example LLM to complete the following tasks in Red Hat OpenShift AI without the need to install anything on your computer:

  • Deploy a containerized LLM model.

  • Create a Jupyter notebook.

  • Test the endpoint with a simple query.

  • Deploy an LLM model from object storage.

Before you begin

  • You should have access to an OpenShift cluster where Red Hat OpenShift AI is installed.

  • If possible, you should have access to a cluster that has GPUs available, if not we’ve included instructions for deploying on a CPU-only cluster.

If don’t have access to a cluster that includes an instance of OpenShift AI, check out demo.redhat.com.

If you’re ready, start the workshop.