Skip to content

Latest commit

 

History

History
66 lines (41 loc) · 1.42 KB

README.md

File metadata and controls

66 lines (41 loc) · 1.42 KB

GUI application to create animations with stable diffusion model

This script creates pointcloud from a single image applies different transformations to create animation controlled by camera movement.

Showcase

WIP interface preview:

StablePointsShowcaseOpt1.mov

This program uses Stable Diffusion Inpainting model for filling gaps caused by rotation camera. Depthmap is estimated by MiDaS model or LeReS.

For pointcloud rendering using pure pytorch render.

Installation

  1. Install dependencies

    pip install -r requirements.txt

  2. Download weights

    TODO: add stable diffusion weights link TODO: add depth model weights link TODO: add FILM weights

  3. Install repositories

    • LeRes
    • MiDaS
    • FILM-pytorch

Using

  1. Prepare init image
  2. Choose depthscale
  3. Rotate camera
  4. Select inpainting mask
  5. Set inpainting options
  6. Save Keyframe
  7. Repeat points 2-6
  8. Create animation with frame interpolation and optional upscaling

Dependencies

  1. PyTorch
  2. Stable Diffusion
  3. MiDaS, LeReS
  4. DearPyGUI
  5. OpenCV
  6. Automatic1111 API

Examples

  1. 3D camera movement
test_1.mov
  1. Img2Img with zoom example
test_2.mov