feat: Implement unified memory management for ONNX model inference

- Introduced a new memory management policy for ONNX Runtime to release temporary GPU memory after inference completion.
- Established a shared inference entry point for all ONNX models, differentiating between long-lived sessions and temporary memory arenas.
- Updated CUDA sessions to utilize a shrinkable memory arena, reclaiming GPU memory post-inference.
- Ensured consistent memory release semantics across BEN2, DA3, MoGe-2, and Upscale models while reusing existing sessions.
- Added lifecycle tests to validate temporary memory release and session reuse.
- Modified model adapters to integrate the new memory management features, ensuring backward compatibility with existing inference behavior.
- Enhanced unit tests to cover new memory release scenarios and verify correct behavior across different execution providers.

src/anyimage/server/jobs/debug_predictions.py

@@ -34,6 +34,7 @@ def run_moge2(context, parameters):
                 context.resource("model_manager"),
                 parameters,
                 image,
+                release_memory=index == len(images) - 1,
             )
             suffix = "" if len(images) == 1 else f".{index + 1:04d}"
             write_moge2_prediction(
@@ -60,6 +61,7 @@ def run_ben2(context, parameters):
                 parameters["device"],
                 model_cache=model_manager,
                 cancel_check=context.check_cancelled,
+                release_memory=index == len(images) - 1,
             )
             suffix = "" if len(images) == 1 else f".{index + 1:04d}"
             foreground.save(context.directory / f"foreground{suffix}.png")
@@ -80,6 +82,7 @@ def run_upscale(context, parameters):
                 parameters,
                 image,
                 cancel_check=context.check_cancelled,
+                release_memory=index == len(images) - 1,
             )
             suffix = "" if len(images) == 1 else f".{index + 1:04d}"
             prediction.save(

src/anyimage/server/jobs/upscale.py

@@ -30,6 +30,7 @@ def run(context, parameters):
                 image,
                 cancel_check=context.check_cancelled,
                 preserve_alpha=True,
+                release_memory=index == total - 1,
             )
             prediction.save(
                 context.directory / f"upscale.{index + 1:04d}.png",