Jarvislabs.ai

March 23, 2024

How to create a dataset for finetuning using Fooocus's face swap feature.

Praveen

Praveen

R&D @JarvisLabs.ai

Fooocus

Fooocus is a GUI for stable diffusion that provides a similar user experience to Midjourney. Fooocus focuses (I had to 😅) on making its users not have a tough time working with complex parameters and have fun with prompting.

In this blog we will explore how to prepare a dataset easily with Fooocus's face swap feature.

Contents

Choosing our star ⭐️

To create a dataset we must choose a main character. It can be anyone who is real or AI generated. For this blog I am going to use Fooocus's generating capabilities to generate the star of our dataset.

photoOfAWoman
A portrait of a beautiful western asian woman

This is the photo that we are going to work with in this blog. I am going to create a dataset to finetune a SDXL model so, I chose images of size 1024x1024. You can choose different image sizes by enabling the advanced option.

Face swap

Now, the part you all have been waiting for... "FACE SWAP!". To use the face swap function we have to enable the input image and advanced option.

advanedOptions

After enabling the options we have to drop our image in one of the four image input boxes and choose the option face swap. Now, we have to give a prompt similar to the one from our base image and any extra features that we want.

I go with a weight value of 1.2 but feel free to mess around and find what suits best fo your face swap.

faceSwapResults

You can see that we got very good face swaps with just a few clicks

Face swap + Posture

What if you wanted to generate a image in your dataset in the same posture of another image? Fooocus has you covered!

describeTab

First navigate to the describe tab. There upload the image with the pose you want and hit describe this image into prompt and copy the result prompt.

Now copy the description made by fooocus. Switch to the image prompt tab and upload the face photo and the pose photo. For the pose input image we will choose the option pyracanny.

pyracannyOption

Pyracanny is an algorithm that is a combination of pyramid and canny. Canny is an algorithm that is used to create a line art of a provided image.

Pyramiding (Image pyramid) is a process in which the image is represented in multi scale aspect ratios. This helps greatly with object detection.

pyrcannyOption

Now paste our copied description of our pose image and combine it with the original description of our face swap image and hit generate.

pyracannyResults

We can see that the result images has maintained the pose of the image we chose for pyracanny.

Wrap up

Fooocus makes it incredibly fun to create a dataset for your own fine-tuned Stable Diffusion models. With face swapping and the ability to mimic poses, you can experiment with endless combinations. Thanks for reading 😃.