Spring AI + Ollama: Call a Local LLM from Spring Boot

To call a local Ollama model from Spring Boot, add Spring AI's Ollama starter, configure the server URL and model name, and inject ChatClient.Builder. Your Java code sends the prompt through Spring AI; Ollama runs the model on your Mac and sends the answer back.

In the previous post, we installed Ollama, downloaded llama3.2:3b, and called its API with curl. This time we'll put a small Spring Boot application in front of that same model.

We'll build one endpoint: GET /chat?message=.... It returns your prompt and the model's reply as JSON. You don't need a cloud account or an API key for this local setup.

Quick Answer

To call Ollama from a Spring Boot application using Spring AI:

1. Start Ollama and download the model: ollama pull llama3.2:3b.
2. Generate a project in Spring Initializr with Spring Web and Ollama. The Ollama dependency adds spring-ai-starter-model-ollama.
3. Configure the connection: set spring.ai.ollama.base-url to http://localhost:11434 and spring.ai.ollama.chat.model to llama3.2:3b.
4. Inject ChatClient.Builder and call .prompt().user(message).call().content() to get the answer text.

What Spring AI adds

The request now takes this path:

curl → Spring Boot on port 8080 → Spring AI ChatClient
     → Ollama on port 11434 → Llama 3.2 → response

Ollama still runs as a separate process. Adding a Maven dependency doesn't put the model inside your JVM or start the Ollama server.

Spring AI handles the HTTP request to Ollama and reads the response for you. Your controller works with a ChatClient, using the same dependency injection and configuration patterns as the rest of your Spring application.

Prerequisites and versions

You'll need Ollama installed, the llama3.2:3b model downloaded, Java 21, and Maven. This example uses:

Spring AI 2.0 supports Spring Boot 4.0 and 4.1, according to the getting-started documentation. The earlier Gemini posts used Spring Boot 3.5 and Spring AI 1.1. Some configuration names have changed since then, so follow the versions and settings shown here.

Check your Java and Maven installations:

java -version
mvn -version

If you have several JDKs installed on macOS, you can select Java 21 for your current terminal:

export JAVA_HOME=$(/usr/libexec/java_home -v 21)

That command selects an installed JDK; it doesn't install one.

Step 1: Check Ollama before adding Spring

If Ollama isn't already serving, start it in a separate terminal:

ollama serve

Leave that terminal open. If the Ollama app already started the server, you can use that instance instead.

In your working terminal, download the model if needed and check the installed models:

ollama pull llama3.2:3b
ollama ls

Then make a direct request:

curl http://localhost:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "llama3.2:3b",
    "messages": [
      {"role": "user", "content": "Say hello in one short sentence."}
    ],
    "stream": false
  }'

Look for an answer in message.content. The Ollama chat API documentation describes this request format.

If this request works, you know Ollama can answer a prompt before you add Spring to the picture. If it fails, check that the server is running and the model is downloaded first.

Step 2: Generate the project with Spring Initializr

Open Spring Initializr and choose:

• Project: Maven
• Language: Java
• Spring Boot: 4.1.1
• Group: dev.ravibuilds
• Artifact: hello-ollama
• Name: Ollama
• Package name: dev.ravibuilds.ollama
• Packaging: Jar
• Java: 21
• Dependencies: Spring Web and Ollama

Click Generate, extract the ZIP, and open the hello-ollama project folder in your IDE. Initializr creates the application class, Maven wrapper, and dependency setup for you. We'll add the controller and YAML configuration next.

Here are the files we'll work with; the generated project also includes wrapper and test files:

hello-ollama/
├── pom.xml
└── src/main/
    ├── java/dev/ravibuilds/ollama/
    │   ├── OllamaApplication.java
    │   └── ChatController.java
    └── resources/
        └── application.yml

Check the generated pom.xml: set the Spring Boot parent version to 4.1.1 and spring-ai.version to 2.0.1 to match the tested example. Keep the generated test dependencies and other project files.

The minimal pom.xml below shows the tested application dependencies and build plugin. Compare it with your generated file; keep the test dependencies that Initializr added:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>4.1.1</version>
        <relativePath/>
    </parent>
    <groupId>dev.ravibuilds</groupId>
    <artifactId>hello-ollama</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <properties>
        <java.version>21</java.version>
        <spring-ai.version>2.0.1</spring-ai.version>
    </properties>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-bom</artifactId>
                <version>${spring-ai.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webmvc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-ollama</artifactId>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

The Boot parent manages the Spring Boot dependencies and plugins. The Spring AI BOM, or Bill of Materials, manages compatible versions of the Spring AI modules. The spring-ai-starter-model-ollama dependency adds the Ollama integration itself.

spring-boot-starter-webmvc adds the web support needed for our endpoint. The Boot Maven plugin lets us run the application and package an executable JAR. Spring's dependency-management guide explains the BOM setup.

Step 3: Configure the local model

Initializr creates src/main/resources/application.properties. Delete that file and create application.yml in the same directory so all the example's settings are in one place:

spring:
  application:
    name: hello-ollama
  ai:
    ollama:
      base-url: http://localhost:11434
      init:
        pull-model-strategy: never
      chat:
        model: llama3.2:3b
        temperature: 0.2
        num-predict: 256

The server address is Ollama's base URL. Don't append /api/chat; Spring AI supplies the endpoint path.

The model name must match the model you downloaded. We've set it explicitly so the application uses the same llama3.2:3b model as the previous post.

pull-model-strategy: never tells Spring AI not to download models at startup. We already downloaded ours in Step 1.

temperature controls randomness when generating an answer; a lower value reduces variation. num-predict limits the response to 256 tokens, the pieces of text the model generates. These settings help control the output, but they don't guarantee accuracy or an exact sentence count.

These are the Spring AI 2.0 Ollama properties. Notice that model, temperature, and num-predict sit directly under chat. Older 1.x examples put them under chat.options; the upgrade notes explain the flattened configuration. Use the property layout for the version you're running.

Step 4: Add the application and controller

Initializr has already created src/main/java/dev/ravibuilds/ollama/OllamaApplication.java. It should look like this:

package dev.ravibuilds.ollama;
 
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
 
@SpringBootApplication
public class OllamaApplication {
    public static void main(String[] args) {
        SpringApplication.run(OllamaApplication.class, args);
    }
}

Then create ChatController.java in the same directory:

package dev.ravibuilds.ollama;
 
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
 
@RestController
public class ChatController {
    private final ChatClient chatClient;
 
    public ChatController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }
 
    @GetMapping("/chat")
    public ChatReply chat(@RequestParam("message") String message) {
        String reply = chatClient.prompt()
                .user(message)
                .call()
                .content();
        return new ChatReply(message, reply);
    }
 
    public record ChatReply(String prompt, String reply) {}
}

The Ollama starter supplies the model integration, and Spring AI auto-configures a ChatClient.Builder that we inject into the controller. The ChatClient documentation shows this pattern.

Read the method chain from top to bottom: create a prompt, add the user's message, send the request, and get the answer text. This is a synchronous call, so the endpoint waits for the model's answer before returning.

ChatReply holds the original prompt and the answer. Spring converts that Java record into JSON. The model supplies the text inside reply; Spring handles the surrounding JSON structure.

Step 5: Run and test

From the directory containing pom.xml, run:

mvn spring-boot:run

Keep Ollama running too. In another terminal, call your Spring Boot endpoint:

curl --get http://localhost:8080/chat \
  --data-urlencode 'message=Explain what an HTTP request is in two short sentences.'

--data-urlencode handles spaces and punctuation in the query parameter. This request goes to your application on port 8080, which calls Ollama on port 11434.

Here is the actual response from testing, formatted for readability:

{
  "prompt": "Explain what an HTTP request is in two short sentences.",
  "reply": "An HTTP request is a message sent from a client (such as a web browser) to a server, requesting access to a specific resource, such as a webpage or data. The request includes information like the requested URL, HTTP method (e.g. GET, POST, PUT), headers, and data, which the server then processes to fulfill the request."
}

The answer explains the basic request flow in the two sentences we asked for. You may get different wording when you run the same prompt.

This example was compiled and tested with Java 21.0.10, Maven 3.9.16, Spring Boot 4.1.1, Spring AI 2.0.1, and Ollama 0.40.3 on the M1 Pro Mac with 16 GB of memory. Both the packaged JAR and the mvn spring-boot:run command were verified with live requests.

What just happened?

When the GET request reached /chat:

1. Spring MVC passed the message query parameter to the controller.
2. ChatClient built a user message, and the Ollama integration sent it to the local server with the configured model and generation settings.
3. Ollama ran llama3.2:3b and returned the generated answer.
4. .content() extracted the answer text, and Spring serialized our ChatReply record into the JSON response.

The controller follows the same pattern as the Gemini integration. The starter and configuration select the provider; the Java call still goes through ChatClient.

A running application can still have a missing model

To test a missing model, the same application was run on a second port with its model set to portfolio-missing-model:test, a name that wasn't installed. Spring Boot started successfully, but the first /chat request returned HTTP 500.

The server log showed the underlying problem:

HTTP 404 - {"error":"model 'portfolio-missing-model:test' not found"}

There are two HTTP calls here. Ollama returned 404 to Spring AI because it couldn't find the model. Our controller didn't handle that exception, so Spring Boot returned 500 to the caller.

In this case, the fix is to configure the installed model, llama3.2:3b. If you intended to use a different model, download it with ollama pull and make sure its name matches your configuration.

With automatic downloads disabled, a “Started” message in the Spring Boot log doesn't tell you whether the model is available. Check ollama ls, then send a request through your application to verify the full path.

Other problems to check

Connection refused

Check which server is unreachable. A failure to connect to port 8080 means your Spring Boot app isn't reachable. An exception in the app log about port 11434 points to the Ollama connection. Start the missing process and retry.

The model setting seems to be ignored

Check the property name against your Spring AI version. For this 2.0.1 example, it is spring.ai.ollama.chat.model, with no options level. Also check for an environment variable or command-line argument overriding the YAML value.

The endpoint returns 400

The message query parameter is required. Calling /chat without it returned HTTP 400 during testing. Include it using the curl command above.

Frequently asked questions

Do I need an API key?

No. This example calls the Ollama server on your Mac and runs a downloaded local model. No API key is configured in the application.

Does ChatClient remember previous requests?

Each call to this controller sends a single user message. Previous questions and answers aren't included, even though we reuse the same ChatClient. To support follow-up questions, you'll need to send the conversation history or add a chat-memory advisor.

Why can the first answer take longer?

Local inference depends on the model and your available hardware. A request may also need to load the model first. Use ollama ps to inspect the loaded model. Because this endpoint uses a synchronous call, the caller waits until generation completes.

What to build next

You can send a question to a Spring Boot endpoint and get a response from a model running on your own Mac. The controller uses Spring AI's ChatClient, while the server address and model choice live in configuration.

The example keeps each request independent and lets model errors become HTTP 500 responses. As you build on it, decide how to handle an unavailable Ollama server and whether your feature needs conversation history.

For now, replace the HTTP question with a short function and ask the model to explain it. Compare the answer with what the code actually does. That gives you a small, concrete task for judging whether the integration's answers are useful.

Further reading

• Spring AI getting started and version compatibility
• Spring AI Ollama integration
• Spring AI ChatClient
• Spring Boot system requirements

Related posts on this blog

• Run an LLM Locally on Your Mac with Ollama
• Spring AI + Gemini: Your First API Call in Java
• Spring AI Provider Portability: Running One App on Gemini and OpenRouter