Implement AI Host functionality with model caching and job management
- Added ai_host.py to handle AI tasks with FastAPI integration. - Introduced HostState and ModelCache classes for managing client connections and model loading. - Implemented job submission and cancellation endpoints in the AI host. - Enhanced tests for AI host functionality, including client tracking and job cancellation. - Updated existing tests to accommodate changes in the operator and command structure. - Refactored runtime commands to use request-based architecture instead of command-line arguments. - Improved model loading logic to handle CUDA memory management and reuse models efficiently.
This commit is contained in:
@@ -12,7 +12,7 @@ platforms = ["windows-x64"]
|
||||
license = ["SPDX:GPL-3.0-or-later"]
|
||||
|
||||
[permissions]
|
||||
network = "Download DA3 runtime dependencies and model weights"
|
||||
network = "Access the local AI Host and download runtime dependencies and model weights"
|
||||
files = "Read source images and store runtime data"
|
||||
|
||||
[build]
|
||||
|
||||
Reference in New Issue
Block a user