Feature suggestion: Common Request/Response for Providers and a ProviderManager factory

[raffits]

I am not sure if this fits AIProxySwift's vision/goal but it would make it much more flexible if there is an abstraction layer on top of the current concrete providers response/request implementation. A "generic provider" request/response model such as ProviderStreamingResponse/Request, ProviderUsageResponse/Request, ProviderCompletionResponse/Request... coupled with a provider manager factory would make working with the library super simple.

Simplified Example: Creating a new provider from the factory by specifying the type (.openAI, .anthropic, .gemini, .deepseek, ...)

The usage could be as simple as

let providerService = try await ProviderManager.createProviderService(type: .openAI)

var streamingRequest = ProviderStreamingRequest(model: modelName, messages: messagesHistory)

let streamingResponse = await providerService.fetchStreamingResponse(request: streamingRequest)

[richarddas]

Just to chime in here: there are some nice ergonomics on Apple's Foundation Frameworks API like having a session, and Generable in particular. While it'd be great to work something like this, I do think that the abstraction layer you're suggesting @raffits sits above/outside what AIProxySwift provides. Can't speak for the owners of this product, but I personally came to the conclusion that I might try out creating such an abstraction layer for myself first, and then seeing whether that actually intersects usefully in practice with the AIProxySwift package. Like you, I suspect there is some nice overlap, but I don't think it's necessarily obvious or trivial. If you're interested in developing your idea further, gimme a shout. Maybe this is where it starts!

[richarddas]

Take a look at this new project, could be relevant/useful: https://github.com/mattt/AnyLanguageModel

[raffits]

I agree with your assessment that the abstraction layer sits above AIProxy. The AIProxy team have done a great job so far! I have started with such a layer for my own use but it is still work in progress and much narrower in scope for now. I will share and perhaps reach out to you @richarddas once I have a solid hold. Thanks for your suggestions.