Python / Python Modern Generative AI and Agents Interview Questions
How do you generate and manipulate images using Hugging Face's Diffusers library?
The diffusers library provides a unified API for diffusion models including Stable Diffusion, SDXL, Flux, and ControlNet. Diffusion models generate images by progressively denoising random Gaussian noise, guided by a text prompt encoded by a text encoder (typically CLIP or T5). The DiffusionPipeline wraps the full pipeline — scheduler, UNet/DiT, VAE, and text encoder — into a single callable.
# pip install diffusers accelerate from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler import torch # ââ Text-to-image with Stable Diffusion 2.1 pipe = StableDiffusionPipeline.from_pretrained( 'stabilityai/stable-diffusion-2-1', torch_dtype=torch.float16, ) # Faster scheduler (20 steps instead of default 50) pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) pipe = pipe.to('cuda') pipe.enable_attention_slicing() # reduce VRAM usage image = pipe( prompt='A serene mountain lake at sunset, photorealistic, 8k', negative_prompt='blurry, low quality, distorted, ugly', # what to avoid num_inference_steps=20, guidance_scale=7.5, # higher = more prompt-adherent, less diverse height=768, width=768, generator=torch.Generator('cuda').manual_seed(42), # reproducible ).images[0] image.save('landscape.png') # ââ Image-to-image (modify an existing image) from diffusers import StableDiffusionImg2ImgPipeline from PIL import Image img2img_pipe = StableDiffusionImg2ImgPipeline(**pipe.components) input_image = Image.open('sketch.png').resize((512, 512)) output = img2img_pipe( prompt='oil painting style, masterpiece', image=input_image, strength=0.75, # 0=no change, 1=ignore input entirely ).images[0] # ââ FLUX.1 (2024 state-of-the-art) from diffusers import FluxPipeline flux_pipe = FluxPipeline.from_pretrained( 'black-forest-labs/FLUX.1-schnell', torch_dtype=torch.bfloat16 ).to('cuda') img = flux_pipe('A futuristic city at night', num_inference_steps=4).images[0]
More Related questions...