Retrieval-Augmented Generation (RAG) with Spring AI, Oracle Database 23ai, and OpenAI
Introduction
As you may have heard, Oracle released Oracle Database 23ai, an accessible, free, yet feature-complete offering of the industry-leading Oracle Database.
Oracle Database 23ai now includes semantic search capabilities using AI vectors. The collection of features, called Oracle AI Vector Search, includes a new vector data type, vector indexes, and vector search SQL operators that enable the Oracle Database to store the semantic content of documents, images, and other unstructured data as vectors, and use these to build Generative AI applications with fast similarity queries.
These new capabilities also support Retrieval Augmented Generation (RAG), a breakthrough generative AI technique that combines Large Language Models (LLMs) and business data to deliver more informed answers to natural language questions.
Please have a look at this blog post to further understand why Oracle AI Vector Search is a powerful feature — What Is Vector Search? The Ultimate Guide.
Spring AI is an exciting addition to the Spring Framework ecosystem. It provides a robust API, an abstraction layer that encapsulates the complexities of interacting with various GenAI providers, such as OpenAI, Cohere, and others, to address typical scenarios such as chat, text-to-image, and embedding models.
With Spring AI, the same Spring ecosystem design principles of portability and modular design are maintained, and POJOs are promoted as the building blocks of an application in the AI domain.
Last but not least, I assume OpenAI needs no introduction at all. Pretty much everyone has heard about it, and its most popular creation — ChatGPT. I will show you how you can use Spring AI to interact with GenAI models provided by OpenAI — a chat model (gpt-4o) and an embedding model (text-embedding-3-small), as such models will be used to create the vector embeddings as well as to support the chat completions.
This blog post provides a quick guide to building an RAG application with Spring AI, the Oracle Database 23ai, its JDBC Driver, and OpenAI. So, without further ado, let’s get started!
Prerequisites
- JDK — Java Development Kit 17 or newer
- Oracle Database Free Release 23ai — Container Image
- Oracle JDBC Driver 23ai (23.4.0.24.05) — Maven Central
- Your preferred Java IDE — Eclipse, IntelliJ, VS Code
- Apache Maven
Oracle Database 23ai
If you check the download link, you will see that it is available as a Linux x64 RPM installation file (OL8 or RHEL8). So, if you’re a Java developer using Windows or macOS, you may wonder how you can kick the tires on it.
The good news is that a container image is also available on Oracle Container Registry (OCIR), which you can use to install it and test its amazing features. Let’s try it quickly and easily and test it on Windows. To do so, first, run the docker pull command below:
docker pull container-registry.oracle.com/database/free:latestNow, you can run the docker images command below to confirm that it was pulled properly:
Then, run the command below from the Windows Command Prompt (cmd):
docker run -d -p 1521:1521 -e ORACLE_PWD=<your_password> -v oracle-volume:/opt/oracle/oradata container-registry.oracle.com/database/free:latestReplace <your password> above with your chosen password as required. The command above will run a new persistent database container (data is kept throughout container lifecycles).
If everything goes well, you can run the command docker ps -al to confirm that your container instance is starting as expected.
Please note that the Oracle Database 23ai will be ready to use when the STATUS field shows (healthy), as below.
There are some additional options if you want to configure environment variables to support your connection details, such as username, password, and other variables. You can also check more details about the specific database user (SYS, SYSTEM, PDBADMIN) you may want to use.
If you want to explore such options, please check the official page for the related container image on the Oracle Container Registry (OCIR).
Now your database is installed! Let’s proceed to access it from both SQL Developer and Java!
Connect to Oracle Database 23ai
The next step is to connect to Oracle Database 23ai to run the required DDL script. Please follow the instructions in this blog post, which covers how to do so with the IntelliJ IDEA tool.
Alternatively, you can also use Oracle SQL Developer, a free tool that simplifies the development and management of Oracle Databases in traditional and cloud deployments, or any other tool you might prefer.
SQL DDL script
Provided that everything was configured properly, now you can connect to your Oracle Database 23ai instance, and execute the SQL DDL script below as required. Note the placeholder <YOUR_DB_PASSWORD>, and replace it with our database password.
-- ADMIN
CREATE USER VECTOR_USER IDENTIFIED BY <YOUR_DB_PASSWORD> QUOTA UNLIMITED ON USERS;
GRANT DB_DEVELOPER_ROLE TO VECTOR_USER;
GRANT CREATE SESSION TO VECTOR_USER;
GRANT SELECT ANY TABLE ON SCHEMA VECTOR_USER TO VECTOR_USER;
ALTER SESSION SET CURRENT_SCHEMA = VECTOR_USER;
-- AS VECTOR_USER
CREATE TABLE VECTOR_USER.VECTOR_STORE (ID VARCHAR2(64) PRIMARY KEY, METADATA VARCHAR(256), CONTENT CLOB, VECTOR_DATA VECTOR(1536, FLOAT64));
You may you can run a describe command to check the table details as shown below.
The Spring AI application
Now, I will introduce basic concepts related to GenAI to allow Java developers to use the Oracle Database, Oracle AI Vector Search, Spring AI, and the OpenAI models to implement a GenAI-related use case, as explained in the introduction section of this blog post.
You will learn how to implement a RAG (Retrieval Augmented Generation) scenario, to augment the context of an existing Large Language Model (LLM) and add business-related context to your application.
It’s beyond the scope of this blog post to teach you everything about Spring AI, as the official documentation is quite extensive, and Javadoc documentation is available as well.
Nevertheless, we’ll explore the basic components of our application, providing the most important details for each of them, and the respective code sample.
Such details will allow you to understand what you should expect the RAG application to do, along with other details about configuring a Spring AI application, including both Oracle Database 23ai and OpenAI connection details.
First, let’s have a look at our application.properties file, used to configure a Spring application.
The first section highlighted above has the OpenAI API key, which will allow Spring AI to interact with the OpenAI models. Instructions on how to get an API key are here. Note that I configured it as an environment variable on Windows.
The second section has the usual Oracle Database connection details, including a JDBC URL, credentials (again, configured as environment variables), and additional properties to use UCP (Universal Connection Pool) and replace HikariCP. Check my previous blog post for more details.
The third section is a property that enables the use of Java Virtual Threads, which is an interesting option as it allows you to optimize your application and its resource utilization.
Again, check this blog post to learn more about Virtual Threads with JDBC and the couple of blog posts listed below by the Spring team at Broadcom to learn about lightweight Virtual Threads with the Spring Framework.
Using virtual threads might be a low-hanging fruit opportunity, but depending on the characteristics of your target application, you must analyze it and decide if it’s feasible.
- Embracing Virtual Threads
- All together now: Spring Boot 3.2, GraalVM native images, Java 21, and virtual threads with Project Loom
No tricks here, as this class has the main entry point used to bootstrap and launch a Spring application from a Java main method, as usual.
OracleDatabaseVectorStoreConfig.java
Again, this is a simple class with another typical Spring Configuration annotation. However, note that it instantiates a custom vector store implementation, that is, OracleDatabaseVectorStore.java.
Such custom implementation follows the contract defined by the org.springframework.ai.vectorstore.VectorStore interface in Spring AI, which defines the operations for managing and querying documents in a vector database. More about such implementation in the next sections.
A controller class that exposes a REST endpoint with a URI such as (/ragpage) that will present a simple page to collect the input prompt and then call the respective vector search URI method, which in turn will delegate the call to its related service method.
OracleDatabaseVectorService.java
A service class with the RAG interaction implementation, comprising methods to start the similarity search, and the template for the input prompt.
OracleDatabaseVectorStore.java
This class has a custom implementation of Spring AI’s VectorStore interface with the actual vector store implementation that uses the Oracle AI Vector Search functionality provided by the Oracle Database 23ai, and it provides the core functionality that supports the vector/similarity search mechanisms.
Utility classes
Last, there are two helper classes that support the custom vector store implementation: the RAG ingestion steps and the vector search functionality. They are OracleDistanceType.java and PdfDataExtractor.java. The former specifies the Oracle distance types (metrics) that support the similarity search, and the latter helps with the unstructured data extraction from the sample PDF file.
Run the Spring AI application
Change to the root directory of your project, and run the Maven commands below as usual. An example is below:
cd <YOUR_WORKSPACE_ROOT_DIRECTORY>\spring-ai-oracle-rag-oai
mvn clean package
mvn spring-boot:runYou will see some messages logged by the Spring AI application that indicate the PDF document ingestion steps, and a final confirmation message that indicates that your application started successfully.
To confirm that, you can run a SQL query below to see the actual vector embeddings as expected.
Now you can proceed to the final step, and test the application!
Test the application — input prompts
First, we’ll test our application with a direct call to OpenAI, that is, we’ll interact directly with the target LLM model (chat) without involving RAG and the augmented information context that is usually provided by a RAG pipeline. Open your browser, and access the application with this URL http://localhost/direct
You will see the page below, provide Oracle as the Database and What is the Oracle Database 23ai? as the Question, and click the Ask Now! button.
You’ll be redirected to a page with the answer from the LLM; however, note that it refers to Oracle Database 23c instead of 23ai. That’s because LLMs usually do not have a way to follow the latest happenings in real-time, so effectively, the GPT-4o model does not know yet about the Oracle Database 23ai release and provided an answer about 23c then, and that’s somehow inaccurate.
Now, let’s have an interaction that involves RAG and uses the PDF document that we ingested to augment the context of our interaction and the related chat completion.
Again, open your web browser and access the application with this URL http://localhost/ragpage. The application’s main page will appear.
Again, submit your question about the content in the provided sample PDF file. The Question to be used now is What’s in Oracle Database 23ai for Java Developers? - note that this is now very specific to Oracle Database 23ai. Again, click the Ask Now! button.
Now we have an answer that’s really meaningful and related to Oracle Database 23ai, as shown below.
To confirm it, let’s pick one of the returned topics, such as OJVM Support for FIPS, as highlighted above, open the PDF file located at /src/main/resources/pdf/ Whats-in-OracleDB23ai-for-Java-Developers.pdf, and look for such a technical topic.
You will see that it is indeed one of the topics included in the PDF file.
Finally, note that the same answers you get when using the pages are also presented as logging messages on the console.
Wrapping it up
That’s it. You can now proceed to create your applications with Spring AI, OpenAI, JDBC, and the Oracle Database 23ai!
We hope you liked this blog post. I will soon post more examples with other GenAI libraries and frameworks, such as LangChain4J and others. Stay tuned!
References
Oracle Database Free Release 23ai — Container Image
Oracle® Database JDBC Java API Reference, Release 23ai
Oracle JDBC Driver 23ai (23.4.0.24.05) — Maven Central
Developers Guide For Oracle JDBC on Maven Central
Develop Java applications with Oracle Database
Oracle Developers and Oracle OCI Free Tier
Join our Oracle Developers channel on Slack to discuss Java, JDK, JDBC, GraalVM, Microservices with Spring Boot, Helidon, Quarkus, Micronaut, Reactive Streams, Cloud, DevOps, IaC, and other topics!
Build, test, and deploy your applications on Oracle Cloud — for free! Get access to OCI Cloud Free Tier!
