From Prototype to Production - Local Kubernetes: Our AI Resume Matcher's Journey

Hey everyone, Tim and Juri here! 👋

Remember that awesome AI Resume Matcher we built together in our last post? (If you missed it or need a refresher, you can check it out right here). We got our Java Spring Boot application, with Google Vertex AI (Gemini), to scan CVs and match them with job descriptions, all neatly packaged with Docker Compose. 🐳

It was a fantastic first step, turning an idea into a working prototype. But what comes next on its journey from prototype to production? How do we make it more robust, scalable, and ready for bigger things? That's where Local Kubernetes enters the scene!

📝 What You'll Learn

🐳 Containerizing our Java Spring Boot AI application with Docker by writing a Dockerfile.
⚙️ Setting up a local Kubernetes environment right on your machine (we'll look at options like Minikube and Docker Desktop).
📜 Crafting Kubernetes deployment manifests – the YAML files that tell Kubernetes how to run our AI Resume Matcher app and its PostgreSQL (pgvector) database.
🔑 Securely configuring Google Cloud authentication for Vertex AI so our application can access it from within the Kubernetes cluster.
🌐 Exposing your application to be accessible from your browser using a Kubernetes Service.

🤔 Our Motivation: Why Go Local K8s?

So, why Local Kubernetes if Docker Compose worked? Great question! While Docker Compose is fantastic, leveling up to K8s locally offers key benefits for our "journey to production":

• Mimics Production: Get a feel for how apps run in real-world, production-like clusters.
• Handles Growth: Better equipped to manage more complex applications and future microservices.
• Cloud-Ready Skills: Smooths your transition to cloud platforms like Google Cloud and SAP BTP.
• Standardized Deployments: Promotes more consistent and reliable application rollouts.

Essentially, using local K8s helps us build production-ready skills and prepares our AI Resume Matcher for its future in the cloud! ☁️

⚙️ Prerequisites

• Our AI Resume Matcher Project: You'll need the application code from our previous blog post. If you haven't got it yet, you can grab it from our repository.
• Docker Desktop: includes one click local Kubernetes.
• Google Cloud Account & Project: Since our application relies on Google Vertex AI:
  • A Google Cloud Platform (GCP) account.
  • A GCP Project where you've enabled the Vertex AI API (as covered in the previous post).

📦 Step 1: Containerizing Our AI Resume Matcher

First things first, we need to package our Spring Boot application into a Docker container. This makes it portable and ensures it runs the same way everywhere, from our local machine to a Kubernetes cluster.

Building the Application with Gradle

Since our project uses Gradle, the first step is to build our application to produce an executable JAR file. Open your terminal in the root directory of the smarthire-blog project and run:

./gradlew build

This command will compile your code, run tests and package everything into a JAR file. You'll find this JAR in the build/libs/ directory.

Crafting the Dockerfile

Next, we need to create a file named Dockerfile (no extension) in the root of our project. This file contains the instructions Docker uses to build our image. Here’s the content for our Dockerfile:

FROM eclipse-temurin:21-jre-jammy
RUN groupadd --system spring && useradd --system --gid spring spring
USER spring:spring
COPY build/libs/*.jar app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]
EXPOSE 8080

Let's break down what each line does:

• FROM eclipse-temurin:21-jre-jammy: This tells Docker to use the official Eclipse Temurin Java 21 JRE (Java Runtime Environment) image based on Ubuntu Jammy as our base. Using a JRE image is great because it's smaller than a full JDK image, making our container more lightweight as it only contains what's needed to run the compiled Java code.
• RUN groupadd --system spring && useradd --system --gid spring spring: Here, we create a system group named spring and then a system user also named spring, assigning it to that group. This dedicated, unprivileged user will run our application, enhancing security.
• USER spring:spring: This instruction switches the active user inside the Docker image to our newly created spring user. All subsequent commands, including our ENTRYPOINT, will run as this user.
• COPY build/libs/*.jar app.jar: This line copies the JAR file created by the Gradle build (from build/libs/ – the *.jar helps grab the versioned JAR file without needing to specify the exact name) into the container's filesystem and names it app.jar.
• ENTRYPOINT ["java", "-jar", "/app.jar"]: This specifies the command that will be run when the container starts. It executes our Spring Boot application using java -jar. The /app.jar path refers to the JAR we copied in the previous step (it's placed in the root of the container's filesystem by default if no WORKDIR is specified).
• EXPOSE 8080: This line informs Docker that the application inside the container will listen on port 8080 at runtime. This doesn't actually publish the port; it's more of a documentation for users and a hint for tools.

Building the Docker Image

With our Dockerfile in place and the application JAR built, we can now build the Docker image. In your terminal (still in the root directory of your project), run:

docker build -t smarthire-app:v1 .

Let's dissect this command:

• docker build: The command to build an image from a Dockerfile.
• -t smarthire-app:v1: The -t flag tags our image. Here, smarthire-app is the name of the image, and :v1 is the tag. Use a specific tag like v1, not :latest. Kubernetes default imagePullPolicy for :latest is Always (tries pulling remotely), while for v1 it's IfNotPresent (uses your local image if available).
• .: This dot at the end tells Docker to look for the Dockerfile in the current directory. After this command finishes, you'll have a Docker image ready to be run locally! You can check your list of images with docker images.

⚙️ Step 2: Launching Kubernetes with Docker Desktop

Now that we have our application containerized, it's time to set up our local Kubernetes environment. For this guide, we'll use the Kubernetes cluster that comes built into Docker Desktop.

Enabling Kubernetes in Docker Desktop

If you haven't enabled Kubernetes in Docker Desktop yet, it's just a few clicks away:

1. Go to Settings (usually the gear icon ⚙️).
2. Navigate to the Kubernetes section in the sidebar.
3. Make sure the Enable Kubernetes checkbox is ticked.
4. Click Apply & Restart. Docker Desktop will then download the necessary Kubernetes components and start your single-node cluster. This might take a few minutes.

Verifying Your Kubernetes Cluster

To interact with Kubernetes, we use a command-line tool called kubectl (Gets installed with enabling Kubernetes in Docker Desktop).

Let's verify that your cluster is up and running. Open your terminal and type:

kubectl get nodes

Since Docker Desktop runs a single-node cluster, you should see one node listed, typically named docker-desktop, with a status of Ready.

Congratulations, you have a local Kubernetes cluster running and ready for our AI Resume Matcher. 🎉

✨ Step 3: Orchestrating with Kubernetes - Writing Our Manifests

With our Kubernetes cluster ready and image built, it's time to define how our application and database will run using Kubernetes manifest files. These YAML blueprints describe our desired setup. We'll create two main files for this.

The Database Blueprint (PostgreSQL + pgvector)

First, let's define our PostgreSQL database, including data persistence and internal access. Create k8s/postgres-k8s.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: pgvector/pgvector:pg17
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_DB
              value: "smarthire"
            - name: POSTGRES_USER
              value: "YOUR_USER"
            - name: POSTGRES_PASSWORD
              value: "YOUR_PASSWORD" # ❗ Use K8s Secrets in production!
          volumeMounts:
            - name: postgres-data
              mountPath: /var/lib/postgresql/data
      volumes:
        - name: postgres-data
          persistentVolumeClaim:
            claimName: postgres-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  type: ClusterIP
  ports:
    - port: 5432
      targetPort: 5432
  selector:
    app: postgres

Key parts of postgres-k8s.yaml:

• Deployment: Runs the pgvector/pgvector:pg17 image. We've set crucial environment variables like POSTGRES_DB, POSTGRES_USER, and POSTGRES_PASSWORD directly for simplicity (though for passwords, Kubernetes Secrets are recommended in production). It also mounts a persistent volume for data.
• PersistentVolumeClaim (PVC): Named postgres-pvc, this requests 1Gi of storage, ensuring our database data isn't lost if the pod restarts.
• Service: Named postgres and of type: ClusterIP, this gives our database an internal, stable IP address and DNS name (postgres:5432) so our application can reach it from within the cluster.

The AI Resume Matcher App Blueprint (app-k8s.yaml)

Next, the manifest for our Spring Boot AI Resume Matcher application. Create k8s/app-k8s.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: smarthire-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: smarthire-app
  template:
    metadata:
      labels:
        app: smarthire-app
    spec:
      containers:
        - name: smarthire-app
          image: smarthire-app:v1 # Our image from Step 1
          imagePullPolicy: IfNotPresent # Use local image if present
          ports:
            - containerPort: 8080
          env:
            - name: SPRING_DATASOURCE_URL
              value: "jdbc:postgresql://postgres:5432/smarthire"
            - name: SPRING_DATASOURCE_USERNAME
              value: "YOUR_USER"
            - name: SPRING_DATASOURCE_PASSWORD
              value: "YOUR_PASSWORD"
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: /etc/gcp-auth/key.json
          volumeMounts:
            - name: gcp-sa-key-volume
              mountPath: /etc/gcp-auth
              readOnly: true
      volumes:
        - name: gcp-sa-key-volume
          secret:
            secretName: gcp-sa-key # GCP Secret (created in Step 4)
---
apiVersion: v1
kind: Service
metadata:
  name: smarthire-app
spec:
  type: LoadBalancer
  ports:
    - port: 8090 # External port
      targetPort: 8080 # App's internal port
  selector:
    app: smarthire-app

Key parts of app-k8s.yaml:

• Deployment: Runs our smarthire-app:v1 image (with imagePullPolicy: IfNotPresent to favor local images). It sets environment variables for the database connection (SPRING_DATASOURCE_URL, username, password) and for Google Cloud credentials (GOOGLE_APPLICATION_CREDENTIALS). The Google Cloud credentials will be mounted from a Secret named gcp-sa-key (which we'll create in Step 4).
• Service: Named smarthire-app and of type: LoadBalancer, this makes our application accessible externally (e.g., via localhost:8090 in Docker Desktop). It maps the external port 8090 to the application's internal port 8080.

With these manifest files defining our application's structure in Kubernetes, we're ready to move on to the crucial step of handling Google Cloud authentication.

🔑 Step 4: Google Cloud Authentication from Kubernetes

Our AI Resume Matcher application needs to communicate with Google Vertex AI services. When running locally (not in a container), it picks up your user credentials from the gcloud CLI. However, inside a Kubernetes pod, it's a different environment and needs its own explicit way to authenticate securely.

For this setup, we'll use a Google Cloud Service Account and its JSON key. We'll then store this key as a Kubernetes Secret and mount it into our application's pod.

Prerequisites for GCP Authentication

• Google Cloud Service Account with appropriate permissions to use Vertex AI (e.g., the "Vertex AI User" role).
• JSON key file for this service account downloaded to your local machine. Keep this file secure!

Here is the documentation on how to create a service account.

Creating the Kubernetes Secret

Now, let's take that downloaded JSON key file and create a Kubernetes Secret from it. This secret will securely store the key within your Kubernetes cluster.

Open your terminal and run the following kubectl command. Make sure to replace /path/to/your-downloaded-service-account-key.json with the actual path to your key file. The name of the key file itself doesn't matter as much as its content and the path you provide in the command.

kubectl create secret generic gcp-sa-key \
  --from-file=key.json=/path/to/your-downloaded-service-account-key.json

Let's break this command down:

• kubectl create secret generic gcp-sa-key: This tells Kubernetes to create a generic secret named gcp-sa-key. This is the exact name (gcp-sa-key) our application's deployment manifest (app-k8s.yaml) expects for the secret.
• --from-file=: This flag tells kubectl to create the secret from the contents of one or more files.
• key.json: This will be the filename inside the secret data. Our application's deployment is configured to look for this specific filename (key.json) when the secret is mounted as a volume.
• /path/to/your-downloaded-service-account-key.json: The actual path to the JSON key file on your local system.

After running this, Kubernetes will store the contents of your JSON key in the gcp-sa-key secret.

🚀 Step 5: Deploy! Applying Our Manifests

We'll use kubectl apply to deploy our application using the YAML files from Step 3. Make sure your terminal is in your project's root directory, where your k8s folder is located.

Applying the Manifests

Apply the database manifest first, then the application manifest:

kubectl apply -f k8s/postgres-k8s.yaml
kubectl apply -f k8s/app-k8s.yaml

These commands instruct Kubernetes to create the deployments, services, and other resources we defined.

Verifying Your Deployment

Let's quickly check if everything is running correctly:

Check Pod Status:

To see if our application and database pods are up and running:

kubectl get pods

You should see pods for postgres and smarthire-app eventually reach a STATUS of Running. This might take a moment.

Check the Application Service:

To find out how to access your deployed application:

kubectl get services

Look for the smarthire-app service in the list. If you're using Docker Desktop, its TYPE will be LoadBalancer, and the EXTERNAL-IP should be localhost. Note the PORT(S) column – it will show something like 8090:XXXXX/TCP. This means your application should be accessible at http://localhost:8090.

Inspect Application Logs:

If the smarthire-app pod isn't behaving as expected (or you just want to see its output, such as the Spring Boot startup messages), you can check its logs.
First, get the exact pod name (it will start with smarthire-app-):

kubectl get pods

Then, display the logs for that specific pod (replace with the actual name from the command above):

kubectl logs <smarthire-app-pod-name>

To follow the logs in real-time (useful for seeing live requests or errors), use the -f flag:

kubectl logs -f <smarthire-app-pod-name>

If your smarthire-app pod is Running and the logs show a successful startup (like the Spring Boot ASCII art and "Tomcat started on port(s): 8080" message), then congratulations! Your AI Resume Matcher should now be running on your local Kubernetes cluster! 🎉

🧪 Step 6: Test Drive Time! Checking Our K8s-Powered App

It's time to make sure it's working as expected by sending a test request.

First, remember that our application should be accessible at http://localhost:8090 (based on the smarthire-app Service of type LoadBalancer we checked with kubectl get services in Step 5). The endpoint for uploading a resume is still /api/candidates/upload.

Now, using Bruno (or your preferred API client), send a POST request to http://localhost:8090/api/candidates/upload. Make sure the request body contains the plain text of a sample resume from our previous post.

You should receive a successful JSON response. This response will list the job offers that best match the CV you provided, just like the example response we saw in our initial Docker Compose setup.

🎯 Wrapping Up: Our AI Resume Matcher on Kubernetes!

We've successfully taken our AI Resume Matcher application from a Docker Compose setup and deployed it onto a local Kubernetes cluster.

Together, we've walked through:

• 🐳 Containerizing our Java Spring Boot application with Docker.
• 📜 Crafting the necessary Kubernetes manifest files for both our application and its PostgreSQL database.
• 🔑 Configuring Google Cloud authentication from within Kubernetes using a Service Account and Secrets.
• 🚀 Deploying all components to the cluster.
• 🧪 Testing our application to ensure it's running correctly.

By working through these steps, you've not only got our AI Resume Matcher running in a more robust, orchestrated environment but also gained hands-on experience with key DevOps practices and cloud-native technologies.

Did you run into any challenges, or do you have suggestions for improvements? Let us know in the comments below!

👉 Check out the project on GitHub

👨‍💻 Happy coding!