You might have to use the gpu_memory_limit and/or lora_on_cpu config choices to prevent functioning from memory. If you continue to operate outside of CUDA memory, you may make an effort to merge in method RAM https://www.decaty.com/