Fizgig LoRA Isolation: train 2 identities into one LoRA and keep everyone else looking like themselves.
A normal LoRA of a person spreads their face onto everyone else in the picture: strangers in the background, a friend beside them, a crowd. LoRA Isolation stops that. Your subjects can stand together in one picture, or next to someone the model already knows, and every face stays its own.
The numbers (Krea 2, same pictures, same settings): with a normal LoRA, 48 of 50 strangers came out looking like the trained person. With LoRA Isolation, 0 of 50. Likeness of the trained person stayed the same (ArcFace 0.78 vs 0.77).
Demo video: (same as above) https://youtu.be/CNh4sN5vN_8
A second person in a LoRA is usually the problem. Here it's part of the fix: learning two identities together is what teaches the LoRA that each face belongs to its own trigger. They don't need to be the same gender, and yo uegt a great multicharacter LoRA. In the demo the second person was a woman, and every stranger was a man, demonstrating it did not have to be the same gender as me to preserve the isolation (that bit was a surprise to me tbh).
How it works: DOP at the base, with Fizgig's own layers on top.
- DOP (Differential Output Preservation), from Ostris's AI-Toolkit
- Multi-subject identity anchoring
- Fused-token class decoupling (
gwelt, a man trains as gweltman)
- In-frame negative-pair compositing with causal-order captions (stranger first)
- Per-sample LR scaling, strided DOP and balanced sampling
- Class-aware training paths for people, animals and things, and styles
Tips:
- Train two people even if you only want yourself. The second face is part of what keeps strangers clean.
- Two trained people in one picture, plus someone the model already knows, works well. Three or more trained people in one picture is less reliable.
- Prompt with the joined word (
gweltman man at a cafe) and describe the other people.
The side-by-side training pictures use 32 generated strangers with natural skin texture, so they don't pull your LoRA towards poor skin and no real person's photo is used. They download with your models.
It's all set up on one card on the Training tab, and works on every model in Fizgig except ACE-Step (the music model). Tested on Krea 2, with more model testing to follow.
*And before anyone points it out, its not perfect, I know but I think its progress and its fully documented in my github, and opensource. One thing I'd like to improve is less lip similarities - but maybe some ideas will come from people using it *
How to set it up (video): https://youtu.be/8uW5NdL5b20
Release notes: https://github.com/shootthesound/Fizgig/releases/tag/v8.4.0
Fizgig on GitHub: https://github.com/shootthesound/Fizgig