diffusers0.25.0で実装された aMUSEd を試してみる
npakaさんのnoteで0.25.0のリリースを知る。
aMUSEd というアルゴリズムによる画像生成もあるようなので実験してみる。
Hugging Faceのリリースより
Google Colab T4環境で実行してみる
!pip install diffusers
これで 0.25.0がインストールできる。パッケージの依存関係から他のパッケージのバージョンも見ておこう
Requirement already satisfied: diffusers in /usr/local/lib/python3.10/dist-packages (0.25.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.10/dist-packages (from diffusers) (7.0.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from diffusers) (3.13.1)
Requirement already satisfied: huggingface-hub>=0.19.4 in /usr/local/lib/python3.10/dist-packages (from diffusers) (0.19.4)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from diffusers) (1.23.5)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from diffusers) (2023.6.3)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from diffusers) (2.31.0)
Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from diffusers) (0.4.1)
Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from diffusers) (9.4.0)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.19.4->diffusers) (2023.6.0)
Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.19.4->diffusers) (4.66.1)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.19.4->diffusers) (6.0.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.19.4->diffusers) (4.5.0)
Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.19.4->diffusers) (23.2)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata->diffusers) (3.17.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers) (2023.11.17)
!pip install diffusers
!pip install accelerate
一度に9枚生成してみた
import torch
from diffusers import AmusedPipeline
# Initialize the pipeline
pipe = AmusedPipeline.from_pretrained(
"amused/amused-512", variant="fp16", torch_dtype=torch.float16
)
pipe.vqvae.to(torch.float32) # Fix for vqvae producing NaNs in fp16
pipe = pipe.to("cuda")
# Define your prompt
num_images = 9
prompt = "miku in dancing"
for i in range(num_images):
image = pipe(prompt, generator=torch.manual_seed(i)).images[0]
image.save(f"miku_in_dancing{i}.png")
ちょっと結果は酷かったので公開するのはやめておきます
image2imageも試してみた。プロンプトは「apple watercolor」。
import torch
from diffusers import AmusedImg2ImgPipeline
from diffusers.utils import load_image
pipe = AmusedImg2ImgPipeline.from_pretrained(
"amused/amused-512", variant="fp16", torch_dtype=torch.float16
)
pipe.vqvae.to(torch.float32) # vqvae is producing nans in fp16
pipe = pipe.to("cuda")
prompt = "apple watercolor"
input_image = (
load_image(
"miku_39.png"
)
.resize((512, 512))
.convert("RGB")
)
image = pipe(prompt, input_image, strength=0.7, generator=torch.manual_seed(39)).images[0]
image.save("image2image_512.png")
速くても美しいとは限らない・・・?
aMUSEdのドキュメントもあるのでしっかり読もう
https://huggingface.co/docs/diffusers/main/en/api/pipelines/amused
ソースはこちらに置いておきました
https://github.com/aicuai/GenAI-Steam/blob/main/20231228_diffusers0_25_0.ipynb