Skip to content

Commit

Permalink
ci(ai): add AI pipelines document structure
Browse files Browse the repository at this point in the history
  • Loading branch information
rickstaa committed Apr 18, 2024
1 parent 2b2f204 commit 7b1d280
Show file tree
Hide file tree
Showing 10 changed files with 107 additions and 5 deletions.
3 changes: 3 additions & 0 deletions ai/api-reference/health.mdx
@@ -0,0 +1,3 @@
---
openapi: get /health
---
3 changes: 3 additions & 0 deletions ai/api-reference/image-to-image.mdx
@@ -0,0 +1,3 @@
---
openapi: post /image-to-image
---
3 changes: 3 additions & 0 deletions ai/api-reference/image-to-video.mdx
@@ -0,0 +1,3 @@
---
openapi: post /image-to-video
---
3 changes: 3 additions & 0 deletions ai/api-reference/text-to-image.mdx
@@ -0,0 +1,3 @@
---
openapi: post /text-to-image
---
58 changes: 54 additions & 4 deletions ai/introduction.mdx
@@ -1,20 +1,70 @@
---
title: Exploring the AI Subnet
title: What is the AI Subnet?
icon: microchip-ai
iconType: regular
---

<Warning>**Notice:** The Livepeer AI Video Subnet is currently in its Alpha phase. During this early stage, you may encounter various bugs or issues. Should you discover any problems, please report them via our [issue tracker](https://github.com/livepeer/go-livepeer/issues/new/choose) on GitHub. We appreciate your patience and feedback as we work to improve the AI Subnet. Your contributions are vital to our progress, and we look forward to seeing what you'll create!</Warning>

### Introduction

The **AI Subnet** is the newest addition to the Livepeer ecosystem, first outlined in [this SPE treasury proposal](https://explorer.livepeer.org/treasury/82843445347363563575858115586375001878287509193479217286690041153234635982713). This cutting-edge platform provides a **decentralized**, **open-source** framework for a broad array of Generative AI inference tasks, including image and video generation and upscaling, all within the Livepeer network. Leveraging Livepeer's decentralized protocol, the AI Subnet offers accessible, cost-effective generative AI capabilities, empowering developers and creators worldwide to incorporate advanced AI tools into their projects.

### Key Components
### Currently available Pipelines

The AI Subnet currently supports the following generative AI pipelines:

- **text-to-image**: Convert text descriptions into images.
- **image-to-image**: Transform images from one style to another.
- **image-super-resolution**: Enhance image quality by upscaling resolution.

For more information on these pipelines, and the Diffusion models suported in these pipelines, refer to the [AI Subnet Pipelines](/ai/pipelines) page.

### Advantages of Livepeer's AI Subnet

- **Decentralization**: The AI Subnet operates on a decentralized network, ensuring that AI tasks are processed securely and efficiently.
- **Cost-Effective**: By leveraging the Livepeer network, the AI Subnet offers cost-effective AI inference capabilities compared to traditional cloud-based solutions.
- **Scalability**: The AI Subnet is designed to scale with demand, allowing for the addition of new Orchestrators and Gateways as needed.
- **Open-Source**: The AI Subnet is built on open-source technology, enabling developers to contribute to its ongoing development and improvement.

### Terminology

- **AI Orchestrator**: This is a specialized node tasked with carrying out AI inference operations within the subnet.
- **AI Gateway**: A designated node that routes AI tasks to the correct AI Orchestrators for processing.
- **Mainnet Transcoding Network**: This network comprises Orchestrators and Gateways that coordinate transcoding tasks over the mainnet.
- **AI Subnet**: Represents the interconnected framework of AI Orchestrators and AI Gateways, designed to streamline the processing of AI inference tasks on the Livepeer network.

### How does it Work?

The AI Subnet is built on top of the existing Livepeer network, leveraging the same core infrastructure to facilitate AI inference tasks. The AI Orchestrator nodes are responsible for executing AI tasks, while the AI Gateway nodes manage the routing of these tasks to the appropriate Orchestrators. The AI Subnet is designed to be scalable, enabling the addition of new Orchestrators and Gateways as needed to accommodate growing demand.

### Present Constraints

- **Alpha Phase**: The AI Subnet is currently in its Alpha phase, and users may encounter bugs or issues during this early stage.
- **Supports Lited Set of Models**: The AI Subnet currently supports a limited set of AI models. This range is however slowely expanding with the aim to support any custom model in the future.
- **Only Higher VRAM GPUs Supported**: Currently the AI Subnet requires GPUs with at least 16GB of VRAM to run AI inference tasks effectively. We are working to expand this support to lower VRAM GPUs in the future.

### Kickstart Your Journey


<CardGroup cols={2}>
<Card
href="/ai/orchestrator/get_started"
title="Setup AI Orchestrator"
icon="robot"
>
Setup your AI Orchestrator and earn fees from AI tasks.
</Card>
<Card
href="/ai/setup-gateway"
title="Setup AI Gateway"
icon="play"
>
Establish an AI Gateway node to serve a broad customer base with AI tasks.
</Card>
<Card
href="/ai/builders"
title="Build your dApp"
icon="screwdriver-wrench"
>
Develop your innovative dApp on the AI Subnet efficiently and at scale.
</Card>
</CardGroup>
Empty file added ai/pipelines.mdx
Empty file.
9 changes: 9 additions & 0 deletions ai/pipelines/image-to-image.mdx
@@ -0,0 +1,9 @@
---
title: Image-to-Image
---

<Accordion title="Tested and Verified Diffusion Models">
- [sd-turbo](https://huggingface.co/stabilityai/sd-turbo): A robust diffusion model by Stability AI, designed for efficient and high-quality image generation.
- [sdxl-turbo](https://huggingface.co/stabilityai/sdxl-turbo): An extended version of the sd-turbo model, offering enhanced performance for larger and more complex tasks.
- [ByteDance/SDXL-Lightning](https://huggingface.co/ByteDance/SDXL-Lightning): A lightning-fast diffusion model by ByteDance, optimized for high-speed image-to-image transformations.
</Accordion>
8 changes: 8 additions & 0 deletions ai/pipelines/image-to-video.mdx
@@ -0,0 +1,8 @@
---
title: Image-to-video
---

<Accordion title="Tested and Verified Diffusion Models">
- [stable-video-diffusion-img2vid-xt](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt): A model by Stability AI designed for stable video diffusion from images.
- [stabilityai/stable-video-diffusion-img2vid-xt-1-1](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1): An updated version of the stable-video-diffusion-img2vid-xt model with enhanced performance.
</Accordion>
12 changes: 12 additions & 0 deletions ai/pipelines/text-to-image.mdx
@@ -0,0 +1,12 @@
---
title: Text-to-Image
---

<Accordion title="Tested and Verified Diffusion Models">
- [sd-turbo](https://huggingface.co/stabilityai/sd-turbo): A high-performance diffusion model by Stability AI.
- [sdxl-turbo](https://huggingface.co/stabilityai/sdxl-turbo): An extended version of sd-turbo with enhanced capabilities.
- [stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5): A stable diffusion model by Runway ML.
- [stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0): A base model for stable diffusion by Stability AI.
- [openjourney-v4](https://huggingface.co/prompthero/openjourney-v4): A model by Prompthero for open-ended journey generation.
- [ByteDance/SDXL-Lightning](https://huggingface.co/ByteDance/SDXL-Lightning): A lightning-fast diffusion model by ByteDance.
</Accordion>
13 changes: 12 additions & 1 deletion mint.json
Expand Up @@ -323,7 +323,8 @@
"thumbsRating": true
},
"openapi": [
"https://raw.githubusercontent.com/livepeer/studio/master/packages/api/src/schema/api-schema.yaml"
"https://raw.githubusercontent.com/livepeer/studio/master/packages/api/src/schema/api-schema.yaml",
"https://raw.githubusercontent.com/livepeer/ai-worker/main/runner/openapi.json"
],
"api": {
"baseUrl": "https://livepeer.studio/api"
Expand Down Expand Up @@ -514,6 +515,16 @@
"group": "AI Video",
"pages": [
"ai/introduction",
{
"group": "AI Pipelines",
"icon": "wand-magic-sparkles",
"iconType": "solid",
"pages": [
"ai/pipelines/text-to-image",
"ai/pipelines/image-to-image",
"ai/pipelines/image-to-video"
]
},
{
"group": "Setup an AI Orchestrator",
"icon": "robot",
Expand Down

0 comments on commit 7b1d280

Please sign in to comment.