Guide: Training a LoRA Model for SD Turbo

This guide will walk you through the process of training a LoRA (Low-Rank Adaptation) model for use with Stable Diffusion Turbo (SD Turbo). We'll cover dataset preparation, environment setup, training, and integration with TouchDesigner for real-time video stylization.

Step 1: Prepare Your Dataset

Collect 20 high-quality images of the specific mural style you want to train on
Resize images to 512x512 pixels (standard input size for SD models)
Ensure consistent image quality and style representation

Tip: Use a tool like ImageMagick for batch resizing:
magick mogrify -resize 512x512^ -gravity center -extent 512x512 *.jpg

Step 2: Set Up the Training Environment

We'll use the Automatic1111 WebUI for its user-friendly interface and compatibility with SD Turbo.

Clone the repo:
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
Install dependencies:
cd stable-diffusion-webui && pip install -r requirements.txt
Download the SD Turbo model and place it in models/Stable-diffusion/
Launch the WebUI (use webui-user.bat on Windows or webui.sh on Linux)

Note: Ensure you have a CUDA-capable GPU with sufficient VRAM (8GB minimum, 16GB+ recommended).

Step 3: Train the LoRA Model

In the WebUI, navigate to the DreamBooth or LoRA training tab
Configure training settings:
- Instance prompt: "houston mural style"
- Class prompt: "mural art"
- Training steps: 2000-4000
- Batch size: 1-2
- Learning rate: 5e-6
- Enable FP16 mixed precision
Start training and monitor progress

Tip: If training is slow or you're running out of VRAM, try reducing the resolution to 448x448 or even 384x384.

Step 4: Use the Trained LoRA with SD Turbo

Save the best-performing checkpoint
In the WebUI, select SD Turbo as the base model
Load your trained LoRA and adjust its strength (typically 0.5-1.0)
Test with prompts like: "A cityscape in houston mural style, vibrant colors"

Step 5: Integrate with TouchDesigner

For real-time video stylization:

Set up a Python CHOP in TouchDesigner to communicate with the Automatic1111 API
Create a pipeline to feed video frames to SD Turbo + LoRA
Optimize for real-time processing (lower resolution, frame skipping, etc.)

Note: Real-time processing is computationally intensive. A high-end GPU like the NVIDIA RTX 4090 is recommended for best results.

Step 6: Fine-Tuning and Post-Processing

Experiment with different prompts and LoRA strengths
Consider training multiple LoRA models for different aspects of the mural style
Use video editing software for final touches and speed adjustments