Struggling to maintain character consistency in Midjourney v6 face references

I have been wrestling with a specific problem in Midjourney v6 for the last two days. I am trying to generate a series of five different portraits for a freelance client who needs consistent character designs for a web novel. The issue is that while the style and lighting remain relatively stable, the facial structure drifts significantly between generations, even when I use the same character reference image. I am currently using the --cref parameter with a weight of 1.0, but the results are still hit or miss. Sometimes the nose shape changes, other times the eye distance shifts, making it impossible to present them as the same person. I have tried upscaling the reference image and removing background clutter to see if that helps the model focus, but I haven't noticed a meaningful improvement. Has anyone found a reliable workflow for locking down facial features in v6 without having to manually retouch every single image in Photoshop? I am considering switching to Stable Diffusion with ControlNet for more precise control, but I want to avoid that complexity if there is a simpler prompt engineering trick I am missing. Any advice on adjusting the cref weight or combining it with other parameters would be greatly appreciated. I am running out of time on this project and need a repeatable method that produces consistent results across multiple prompts.