Model Introduction (英文)

I Introduction

AnimeIllustDiffusion is a pre-trained, non-commercial and multi-styled anime illustration model. It DOES NOT generate "AI face". You can use some trigger words (see Appendix A) to generate specific styles of images. Due to plenty of contents, AID needs a lot of negative prompts to work properly. If you get noisy images (most case will be noisy) when generating, you need to use it with my negative text embedding [1] to cancel noise, which is crucial. Otherwise, you will get bad results. For VAE, I recommend sd-vae-ft-mse-original [5]. Part II of this introduction describes how the model was made; part III presents my proposed negative text embeddings; and Appendix A provides a partial list of keywords.

Please carefully browse the version information before downloading!!!

The model has over 100 stable anime illustration styles and 100 anime characters. See Appendix A for specific style trigger words. To generate a specific character, just use the character's name as prompt directly. The AID model is like a palette, and you can create new styles by combining different prompts.

1 Suggested Parameters

Sampler: Euler a

Steps: 32

Resolutions: 512x768, 640x690, 768x1152, etc.

CLIP skip: 1

Prompts format: best quality, masterpiece, highres, by {xxx}, best lighting and shadow, stunning color, radiant tones, ultra-detailed, amazing illustration, an extremely delicate and beautiful, {other prompts}

, where by {xxx} is the name of the style (trigger words in appendix A).

Negative prompts format：aid210, {other negative prompts}

, where aid210 is the special negative embedding which you can download and learn to use it from [1].

2 Version Comparisons

Each version of AID has its own strengths. The newer version is not absolutely better.

For beginners: v2.8, v2.91 - Weak, v2.10beta1

Great creativity: v2.6, v2.7, v2.91 - Weak, v2.91 - Strong

Relatively stable: v2.5, v2.6, v2.8, v2.91 - Weak

Various styles: v2.91 - Weak, v2.91 - Strong, v2.10beta1

If you'd like to upload and share your own images, or would like to contribute training images for future AID models, please move to:

anime-illust-diffusion-gallery - a Hugging Face Space by Eugeoter

II Model

This model is a fusion of three different models, two of which I trained and one is the Pretty 2.5D model fused by GoldSun [2].

1 Model Training

I use 4300+ artificially cropped, tagged, 512x512 size anime illustration images as the training set, and use dreambooth to fine-tune the Naifu 7G model. I trained for 100 epochs per training set image with a high learning rate. I didn't use regularized images. I also trained its text encoder. If interested, you can find detailed parameter information at [3].

2 Model Merging

I merged 3 models using Merge Block Weighted to create this AnimeIllustDiffusion model. Among the three models, one model is used to provide style and text encoder (base alpha and all OUT layers), one model is used to optimize hand details (IN layers 00 - 05), and another model (Pretty 2.5D [3]) are used to provide better composition (IN layers 06 - 11 and M00 layers).

III Negative Text Embedding

The model recommends using badv3 - a text embedding file of negative cue words. It not only simplifies the writing of prompt words, but also stimulates the potential of the model and improves the quality of generated images. Usually, the effect of badv3 is enough, and you don't need to fill in additional quality prompt words. But it doesn't solve 100% of the picture problems.

1 How to Use It

You should place the downloaded negative text embedding file, the badv3.pt file, in the embeddings folder of your stable diffusion directory. After that, you just need to enter badv3 in the negative prompt word field.

2 Ideas on It

My idea is to train a concept of bad images and put it into negative prompt to avoid generating such bad images. I trained a negative text embedding, badv3, using a few hundred bad images generated by the model, which works in a similar way to EasyNegative [4]. I tried training it to overfit to mitigate the effect of traditional negative text embeddings on the style of the model, and it seemed to work. Badv3 works better for this model than EasyNegative. I haven't compared other negative text embeddings yet. badv3 is the nth negative text embedding I trained after deformityv6. It's pretty easy to make, but the results are pretty random. I have tried removing weights from another model trained with bad images by adding differencing, but so far with no promising results. My next plan is to train Negative Lora instead of Negative Text Embeddings to directly "remove" some of the weights from the model rather than "avoid" them.

IV Declarations

This model is used to test multi-style model training, non-profit or commercial, all interest. If there is any infringement, it will be deleted immediately.

All cover images were generated by text2image without using any Lora, using the negative text embedding at [1] in the negative prompts.

Users are only authorized to use this model to generate pictures, and unauthorized reproduction is not allowed.

Any commercial use of this model is strictly prohibited!

The display picture in Appendix A is a large classification prompt word for the special label of this model, and it is for reference only.

V 引用网页 / Referenced Pages

[1] Useful Quality Embeddings - AnimeIllustDiffusion - aid210 | Stable Diffusion Textual Inversion | Civitai

[2] Pretty 2.5D | Stable Diffusion Checkpoint | Civitai

[3] 多风格模型 - 赛璐璐风格科幻插画 - AI加速器社区 (acceleratori.com)

[4] EasyNegative | Stable Diffusion TextualInversion | Civitai

[5] vae-ft-mse-840000-ema-pruned.ckpt · stabilityai/sd-vae-ft-mse-original at main (huggingface.com)

附录 A / Appendix A

截止至 AIDV2.5 / Until AIDV2.5：by 35s00, by agm, by ajimita, by akizero, by ask, by chicken utk, by demizu posuka, by dino, by fadingz, by fuzichico, by hamukukka, by hitomio16, by ichigo ame, by key999, by kooork55, by matcha, by mika pikazo, by modare, by myung yi, by naji yanagida, by nezukonezu32, by nico tine, by nikuzume, by ninev, by oda non, by palow, by qooo003, by rolua, by samip, by serie niai, by shirentutu, by sho, by silver, by sonomura00, by void, by wlop, by xilmo, by yoneyama mai, by yosk6000, by zumizumi

Anime Illust Diffusion

AIDV2.6 新增 / AIDV2.6 adds：by caaaarrot, by hinaki, by homutan, by kazari tayu, by kitada mo, by roitz, by teffish, by ukiatsuya, by yejji, by ziyun

AIDV2.7 新增 / AIDV2.7 adds：by poharo, by jnthed, by 7thknights, by some1else45, by yohan, by yomu, by tsvbvra

AIDV2.9 新增 / AIDV2.9 adds: by kkuni, by starshadowmagic, by star furu, by rella, by tukumi bis, by yumenouchi, by chon, by eku uekura, by tira27, by kuroume, by hachisan, by nounoknown, by kurige horse, by konya karasue, by noyu, by ame929, by muryou tada, by yun216, by nekojira, by nanmo, by wait ar, by Anime Illust Diffusionakasaai, by momoco, by sushi0831, by taiki, by siki, by kinta, by hata, by anteiru, by lemoneco, by umaiyo puyoman, by freng, by rin7914, by shimanun, by hidulme, by whoisshe, by 5eyo, by cutesexyrobutts, by shiren, by omutatsu, by gesoking, by 3meiji, brushstrokes

AIDV2.9 更新 / AIDV2.9 Update: (i) by demizu posuka; (ii) by fuzichico -> by fuzichoco; (iii) 提高了训练图像的分辨率 / Increased resolution of training dataset; (iv) 在 skip clip = 1 上训练 / Trained on "skip clip = 1".

AIDV2.91 新增 / AIDV2.91 adds: impasto, pseudo-impasto, semi-realistic, concept art, flat color, celluloid

直到 AIDV2.10beta1 / Until AIDV2.10beta1: by 35s00, by 3meiji, by 5eyo, by 7nu, by 7thknights, by adenim, by agm, by ajimita, by akizero, by ame929, by anmi, by anteiru, by arutera, by ask, by atelier irrlicht, by bunbun, by caaaaarrot, by camu, by canking, by ccroquette, by chi4, by chicken utk, by chon, by cola, by cutesexyrobutts, by darumakarei, by dino, by dora, by dsmile9, by ei maestrl, by ekita kuro, by ekita xuan, by eku uekura, by fadingz, by fajyobore, by foomidori, by freng, by fuzichoco, by gesoking, by gomzi, by hachisan, by hakuhiru oeoe, by hamukukka, by haru, by hata, by hidulme, by hikinito0902, by hinaki, by hitoimim, by hitomio16, by hizumi, by homutan, by hotatenshi, by houk1se1, by hyatsu, by icecenya, by ichigo ame, by inoriac, by iromishiro, by iwzry, by jnthed, by joezunzun, by junsui0906, by karohroka, by kaya7hara, by kazari tayu, by killow, by kin, by kinta, by kishiyo, by kitada mo, by kkuni, by konya karasue, by kooork55, by kot rou020, by krenz, by kurige horse, by kuroume, by lalalalack, by lemoneco, by lm7, by lovelymelm, by lpmya, by mar takagi, by matcha, by matsukenmanga, by melowh, by menou, by midori xu, by mika pikazo, by misumigumi, by miv4t, by mochizukikei, by mogumo, by momoco, by momoku, by morikuraen, by mqkyrie, by muina, by munashichi, by muryou tada, by myaru, by myc0t0xin, by myung yi, by nack, by naji yanagida, by nanmo, by nardack, by narue, by nekojira, by netural, by nezukonezu32, by nico tine, by nikuzume, by nine, by nineo, by ninev, by niwa uxx, by nixeu, by noco, by noodle4cool, by nounoknown, by noyu, by oda non, by omutatsu, by onineko, by palow, by panp, by pikuson, by poharo, by poire, by potg, by pro-p, by qooo003, by rai hito, by rattan, by reiko, by rella, by rhtkd, by rin7914, by roitz, by ryuseilan, by saberiii, by sais, by sakiika, by samip, by sanosomeha, by say hana, by scottie0073, by senryoko, by serie niai, by seuhyo99, by shal-e, by shimanun, by shirabii, by shiraishi kanoya, by shiren, by shirentutu, by sho, by sia, by siki, by silver, by solipsist, by some1else45, by sonomura00, by sooon, by star furu, by starshadowmagic, by starzin07, by sui 0z0, by sul, by sushi0831, by suzukasuraimu, by taiki, by takumi bis, by teffish, by tidsean, by tira27, by tsukiho tsukioka, by tsvbvra, by ttosom, by tukumi bis, by uiiv, by ukiatsuya, by umaiyo puyoman, by void, by wait ar, by walzrj, by wanke, by whoisshe, by wlop, by xilmo, by yejji, by yogisya, by yohan, by yomu, by yoneyama mai, by yosk6000, by yumenouchi, by yun216, by yunikon147, by yunsang, by ziyun, by zumoti4