Spring AI Provider Portability: Running One App on Gemini and OpenRouter

Spring AI's whole pitch for provider portability is that your application code talks to a ChatClient abstraction, not to any specific model vendor, so you can swap providers without rewriting anything. This recipe puts that pitch to the test. I took the structured-output triager from recipe 02, ran it against Gemini and then against OpenRouter, and watched where the promise held and where it quietly leaked.

This is the fourth recipe in the Spring AI Recipes series. Recipe 03 gave the model tools to query a database. This one keeps the application code frozen and changes only the provider underneath it.

Quick Answer

To run the same Spring AI code against two providers:

1. Keep your Java code provider-agnostic. Inject ChatClient.Builder, never a vendor-specific class. The triager's service, controller, and records don't change at all between providers.
2. Put each provider's starter in a Maven profile, so only one chat-model starter is on the classpath per build. Two native starters at once cause an ambiguous-bean startup failure.
3. Put each provider's config in a Spring profile (application-gemini.yml, application-openrouter.yml).
4. Switch with flags: ./mvnw spring-boot:run for Gemini, ./mvnw spring-boot:run -Popenrouter -Dspring-boot.run.profiles=openrouter for OpenRouter.

There's a subtlety worth knowing upfront: swapping between two native providers (Gemini's starter and OpenAI's) takes more than one line, because they need different starters. But once you route through OpenRouter, switching between the hundreds of models it hosts genuinely is one config line, because they all share the OpenAI protocol.

Working code: github.com/TheRavi/spring-ai-recipes.

What provider portability actually means in Spring AI

The portability promise rests on one design choice: your code depends on the ChatClient interface, not on GoogleGenAiChatModel or OpenAiChatModel. The triager service from recipe 02 calls chatClient.prompt().user(report).call().entity(TriagedReport.class), and nothing in that line knows or cares which provider answers.

So "portability" really means: the abstraction sits at the ChatClient level, and everything below it (which HTTP client, which auth scheme, which request format, which model) is configuration and dependency wiring, not code.

That's a real and valuable guarantee. But it's worth being precise about what it does and doesn't cover, because the marketing version ("swap providers with one line") is only true under specific conditions.

Two kinds of portability, and they don't behave the same

This is the heart of the recipe. There are two different portability layers in play, and conflating them is where people get confused.

Spring AI's abstraction keeps your code identical across providers. What it does not do is let two native providers coexist effortlessly. Gemini's starter and the OpenAI starter each auto-configure a ChatModel bean, and if both are on the classpath, Spring can't decide which one to inject. The app fails to start with an ambiguous-bean error. So switching native providers means switching which starter is compiled in, which is a build concern, not a one-line config edit.

OpenRouter's abstraction is different. OpenRouter exposes hundreds of models from many vendors behind a single OpenAI-compatible endpoint. Spring AI talks to it through the ordinary OpenAI starter. Once you're pointed at OpenRouter, switching from Claude to Llama to DeepSeek to Gemini really is one config line, because every one of those models is reached through the same protocol and the same starter. No dependency change, no rebuild.

So the recipe demonstrates both: the build-level swap between native Gemini and OpenRouter, and the config-level swap between models within OpenRouter. The first shows where Spring AI's abstraction stops. The second shows what frictionless portability actually looks like, and what it costs.

Why this matters before you pick a provider

Most teams choose one model provider early and wire their whole app to it. Then a better or cheaper model ships somewhere else, and switching turns out to be a rewrite. The point of building on the ChatClient abstraction from day one is that your business logic never hard-codes a vendor, so the switch is a configuration decision later instead of a migration project.

This recipe is the proof that the abstraction holds up under an actual swap, and an honest look at the seams where it doesn't.

Prerequisites

• A working Spring AI setup (covered in recipe 01)
• Java 21+
• Maven 3.9+ (the project ships with the Maven wrapper)
• A free Gemini API key from aistudio.google.com/apikey
• A free OpenRouter API key from openrouter.ai/keys (OpenRouter hosts free models you can test with)

Step 1: Keep the application code provider-agnostic

This is the part that does not change. The triager service is the same one from recipe 02, with no provider-specific imports anywhere:

@Service
public class TriageService {
 
    private final ChatClient chatClient;
 
    public TriageService(ChatClient.Builder builder) {
        this.chatClient = builder
            .defaultSystem("You are a bug report triager ...")
            .build();
    }
 
    public TriagedReport triage(String bugReport) {
        return chatClient.prompt()
            .user(bugReport)
            .call()
            .entity(TriagedReport.class);
    }
}

ChatClient.Builder is the generic, auto-configured builder. Whichever provider's starter is on the classpath supplies the underlying ChatModel, and Spring wires it in. The service never names a vendor. That's the portability contract, expressed in code.

Step 2: Put the provider in a Maven profile

Because two native chat starters cannot coexist, the provider dependency lives in a Maven profile rather than the main dependencies block. Only one is ever compiled in.

<profiles>
    <profile>
        <id>gemini</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-starter-model-google-genai</artifactId>
            </dependency>
        </dependencies>
    </profile>
 
    <profile>
        <id>openrouter</id>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-starter-model-openai</artifactId>
            </dependency>
        </dependencies>
    </profile>
</profiles>

The OpenRouter profile uses the OpenAI starter. That is not a mistake. OpenRouter speaks the OpenAI protocol, so the OpenAI starter is exactly the right client for talking to it.

Step 3: Put each provider's config in a Spring profile

Two profile-specific config files hold the per-provider settings. The base application.yml holds only provider-agnostic values.

application-gemini.yml:

spring:
  ai:
    google:
      genai:
        api-key: ${GEMINI_API_KEY}
        chat:
          options:
            model: gemini-2.5-flash
            temperature: 0.2

application-openrouter.yml:

spring:
  ai:
    openai:
      api-key: ${OPENROUTER_API_KEY}
      base-url: https://openrouter.ai/api
      chat:
        options:
          model: nvidia/nemotron-3-ultra-550b-a55b:free
          temperature: 0.2

The OpenRouter config is just the OpenAI config with a different base-url and an OpenRouter model id. That model line is the one you change to swap models within OpenRouter.

Step 4: Run against each provider

Gemini is the default, so a bare run uses it:

set -a && source .env && set +a
./mvnw spring-boot:run

OpenRouter needs two flags, and you need both:

./mvnw spring-boot:run -Popenrouter -Dspring-boot.run.profiles=openrouter

The -Popenrouter is the Maven profile: it decides which starter is compiled in. The -Dspring-boot.run.profiles=openrouter is the Spring profile: it decides which config is read. They are separate mechanisms, and the most common mistake is switching one but not the other.

Confirm the active provider before comparing anything:

curl http://localhost:8080/provider

The comparison: same code, two providers

Here is the test that matters. The identical request, run under each provider:

curl -X POST http://localhost:8080/triage \
  -H 'Content-Type: application/json' \
  -d '{"bugReport": "The CSV export on the reports page returns an empty file when the date range is over 90 days. Customers on the enterprise plan are blocked from their monthly reporting."}'

Gemini (gemini-2.5-flash) returned:

{
  "summary": "CSV export on reports page returns an empty file for date ranges over 90 days.",
  "severity": "HIGH",
  "component": "reports",
  "suggestedLabels": [
    { "label": "csv-export", "confidence": 0.95 },
    { "label": "reporting", "confidence": 0.9 },
    { "label": "date-range", "confidence": 0.8 },
    { "label": "enterprise", "confidence": 0.75 }
  ]
}

The same code, switched to OpenRouter running nvidia/nemotron-3-ultra-550b-a55b:free, returned:

{
  "summary": "CSV export returns empty file for date ranges over 90 days, blocking enterprise customers from monthly reporting.",
  "severity": "HIGH",
  "component": "reports",
  "suggestedLabels": [
    { "label": "csv-export", "confidence": 0.95 },
    { "label": "date-range", "confidence": 0.9 },
    { "label": "enterprise", "confidence": 0.85 },
    { "label": "blocker", "confidence": 0.8 }
  ]
}

Two different vendors, two completely different models, one unchanged line of Java, and they agreed on everything that matters. Same severity (HIGH), same component (reports), same top label (csv-export at 0.95). The structured output came back clean and schema-valid from both. That is the portability promise actually delivering: the .entity() call worked identically whether Google or NVIDIA was behind it.

The differences are in the soft edges, and they are instructive. Gemini's summary is tighter and sticks to the mechanical fault. Nemotron folded the business impact into its summary, noting that enterprise customers are blocked. On labels, both produced csv-export, date-range, and enterprise, but Gemini added "reporting" while Nemotron added "blocker." Nemotron's "blocker" is arguably the sharper call, since the report says outright that customers are blocked.

None of those differences are bugs. They are two capable models exercising slightly different editorial judgment on the parts of the schema that are open to interpretation. The structured fields, the ones a downstream system would branch on, matched exactly. That is the result you want from a portability test: the rigid parts are stable, and the only variation is in the genuinely subjective parts.

Where the leak would show up: structured output on weaker models

The .entity() call depends on the model returning clean, schema-valid JSON. The test above used two capable models, Gemini 2.5 Flash and a 550B-parameter Nemotron, and both produced valid TriagedReport JSON without complaint. The structured-output instructions Spring AI injects into the prompt were robust enough that neither model tripped.

That is worth saying plainly, because it is the good news: across two strong models from different vendors, structured output was portable, not just the code. It held.

But this is exactly where the abstraction gets fragile as you go down the model tier. Structured output is not a hard API guarantee. It is a prompt instruction plus a parser, and weaker or smaller models are less disciplined about following the instruction. The same .entity() call that worked here can throw a deserialization error against a smaller model that wraps its JSON in prose, adds a markdown fence, or omits a required field.

So the honest rule is: code portability does not imply behavior portability, and the gap widens as the model gets weaker. When you move structured-output code to a new model, especially a cheaper one chosen to save cost, test the structured path specifically before trusting it. The compiler will not warn you. The first sign of trouble is a 500 in production when a model returns JSON your record can't parse.

To see the edge for yourself, point the OpenRouter config at a deliberately small model and run the same request:

# in application-openrouter.yml
model: a-small-or-budget-model-id

The smaller the model, the more likely you are to watch the exact same code that just worked start failing to deserialize. That failure is not a Spring AI bug. It is the boundary of what a prompt-based structured-output contract can promise when the model on the other end is weak.

What can go wrong with provider portability

1. Ambiguous ChatModel bean at startup

If you put both native starters on the classpath at once, Spring can't decide which ChatModel to inject and the app fails to start. This recipe avoids it by keeping exactly one chat starter compiled in per Maven profile. If you genuinely need two providers available at runtime (a router that sends different requests to different models), you have to define qualified ChatClient beans by hand and inject them with @Qualifier. Spring AI does not auto-disambiguate this for you.

2. Maven profile and Spring profile out of sync

The two-switch setup is the most error-prone part. Run -Popenrouter without -Dspring-boot.run.profiles=openrouter and you compile the OpenAI starter but load Gemini config, or the reverse. The startup error is confusing because it looks like a config problem when it's really a profile-alignment problem. Switch both together, every time.

3. Structured output reliability varies by model

Covered above. .entity() is only as reliable as the model's JSON discipline. Test structured output specifically when you move to a weaker model, rather than assuming the code's portability implies the behavior's portability.

4. Tool calling support varies by model

If you take recipe 03's tools to a random OpenRouter model, do not assume parity. Not every hosted model supports tool calling, and some support it inconsistently. Check the model's capabilities on its OpenRouter page before relying on it.

5. OpenRouter is a dependency, not free portability

Routing through OpenRouter adds a network hop and a dependency on OpenRouter's uptime, rate limits, and pricing margin. You also give up provider-native features that haven't been normalized into the OpenAI-compatible surface. Portability has a cost, and the cost is a middleman.

Frequently asked questions

Can Spring AI use OpenRouter?

Yes. OpenRouter exposes an OpenAI-compatible API, so you use the standard spring-ai-starter-model-openai starter, set spring.ai.openai.base-url to https://openrouter.ai/api, and use your OpenRouter API key. No special OpenRouter starter is needed.

How do I switch LLM providers in Spring AI without changing code?

Keep your application code on the ChatClient abstraction (inject ChatClient.Builder, never a vendor-specific model class). Then change only the starter dependency and the configuration. This recipe uses Maven profiles for the dependency swap and Spring profiles for the config swap.

Why does my Spring AI app fail to start with two providers?

Each provider's starter auto-configures a ChatModel bean. With two native starters on the classpath, Spring finds multiple ChatModel candidates and can't choose, so it fails with an ambiguous-bean error. Either keep one starter per build (Maven profiles, as here) or define qualified beans manually.

Is switching models on OpenRouter really one line?

Within OpenRouter, yes. Because every model is reached through the same OpenAI-compatible endpoint and the same starter, changing the model property to a different OpenRouter model id is the whole change. Switching between native providers (not through OpenRouter) is more involved, because it needs a different starter.

Does the same code give the same results across providers?

No, and that's expected. The code is portable; model behavior is not. The same prompt and schema can produce different severities, summaries, and confidence scores on different models, and weaker models may not return schema-valid JSON at all. Portability of integration is not portability of output.

What's next

Recipe 05 is RAG basics: retrieval-augmented generation. So far the model has only ever known what's in the prompt and what it learned in training. RAG changes that by letting it pull in your own documents at query time, embedding them into a vector store, retrieving the relevant chunks for a given question, and feeding those chunks to the model as grounding. It's how you get a model to answer questions about your data instead of just its training data, and Spring AI has the vector-store and retrieval pieces built in. That's the next recipe.

Why provider portability matters for Java teams

The reason this works cleanly in Spring is the same reason the rest of the framework works cleanly: the abstraction is at the right level, and configuration is a first-class citizen. Profiles, dependency injection, and conditional beans are not AI features. They are the same Spring mechanisms Java teams have used to swap databases, message brokers, and cloud providers for years. Spring AI just put model providers behind the same kind of seam.

That's the quiet advantage. A Java team adopting AI doesn't need a new mental model for keeping vendors swappable. It's the same profile-and-config discipline they already practice, applied to one more dependency. The model becomes just another external service behind an interface, which is exactly where you want it.

Working code

The complete project for this tutorial is on GitHub:

github.com/TheRavi/spring-ai-recipes (the 04-provider-portability/ folder)

Clone it, copy .env.example to .env, add your keys, and run it against each provider.

Further reading

• Spring AI ChatClient reference
• Spring AI OpenAI chat reference (used for OpenRouter)
• OpenRouter documentation
• OpenRouter model catalog
• Spring Boot profiles reference

Related posts on this blog:

• Spring AI Tool Calling: Let Gemini Query Your Database in Java. Recipe 03, the tool-calling app this portability test could also run on.
• Spring AI Structured Output: Typed Java Records from Gemini in One Line. Recipe 02, the triager this recipe runs across providers.

Run this against a model combination I didn't cover? Open an issue on the repo or reach out on LinkedIn. I'm especially interested in which models hold up on structured output and which ones quietly break it.