* Initial support for SDXL refiner
* Cleanup
* Add arg for converting Unet in float32 precision
* Setup scale factor with pipeline in CLI
* Update cli arg and future warning
* Bundle refiner unet if specified
* Update script for bundled refiner
- Also skip loading model if check_output_correctness is missing, since the model does not require inferencing at conversion time
* Flip skip_model_load bool
* Cleanup
* Support bundled UnetRefiner
* Add seperate refiner config value
- Includes unloading base unet when swapping to refiner
* Update readme for SDXL refiner
* Add condition for new SDXL coreml input features
* Revert pipeline interface change, add extra logging on pipe load
* Reset model_version after refiner conversion
* Reset model_version before refiner conversion but after pipe init
* Add refiner chunking
* Ensure unets are unloaded for reduceMemory true
* Handle missing UnetRefiner.mlmodelc on pipeline load
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Prewarm refiner on load, unload on complete
* Force cpu_and_gpu for VAE until it can be fixed
* Include output dtype of np.float32 for all conversions
* Allow a custom VAE to be converted.
* Revert hardcoded reduceMemory
* Fix merge
* Default chunking arg for --merge-chunks-in-pipeline-model when called from torch2coreml
---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Extend ControlNet image conversion with hole punching for InPaint
The inpainting ControlNet model expects the masked areas to be filled with -1. This never occured with the original conversion, as the pixel data was converted to values between [0;1], thus the inpainting CN model never had any effect (apart from maybe generating artifacts).
This commit uses the _previously discarded_ alpha channel of the source image to mark masked pixels: any non-opaque pixel is cleared out to -1.
For an end-user, this allows creating masks simply by erasing areas with Preview to transparent, saving the result as PNG, and feeding it to the ControlNet.
* Mask only fully transparent pixels
* add controlnet tentatively
* add controlnet in python code
* implement swift part
* support 8-bit quantization
* add controlnet unload when reduce memory
* remove irrelevant changes
* add more description about controlnet option in swift
* fix some for pr and update README
* pre-allocate zero shapedArray + make multi-controlnet faster
* Image2Image Encoder
Encoder
* Scheduler and pipeline
* fix scheduler
* cli
* remove CLI comment
* disable dpm multistep solver with image2image
* clamp initial timestamp
* Store timesteps in reverse order for consistency.
* Report actual number of steps.
* uint32
* PRComments
* remove old initializer
* pr comments
* change name and add error handling
also fix 512 hard coded
* Add fix for Jpegs
---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Stuart Moore