DEV Community

Cover image for Adding an AI chat to your Ruby on Rails application
reinteractive
reinteractive

Posted on

Adding an AI chat to your Ruby on Rails application

#webdev #programming #ai #tutorial

Credited to: Todd Price

Unless you've been living under a rock for the last couple of years, you've heard about AI and how one day it will do everything for you. Well, we aren't quite at AGI yet but we are certainly on the way. So to better understand our future computer overlords I've spent a lot of time using them and have recently been experimenting with the RubyLLM Gem. It's a great gem which makes it very easy to integrate the major LLM providers into your rails app (at the time of writing only Anthropic, DeepSeek, Gemini and OpenAI are supported).

To demonstrate, I'm going to add an AI chat to a new rails 8 application but you can just as easily apply most of this to your existing rails application. We'll go beyond the most basic setup and allow each user to have their own personal chats with the AI.

Let's start by setting up the a new app:

rails new ai_chat --database postgresql
Enter fullscreen mode Exit fullscreen mode

and then follow Suman's post to use the new built-in rails user auth. Alternatively, use your preferred user & auth setup.

Now we're ready to add in ruby_llm:

# Gemfile gem "dotenv" # for managing API keys, you may want to handle them differently gem "ruby_llm"
Enter fullscreen mode Exit fullscreen mode 
bundle install
Enter fullscreen mode Exit fullscreen mode

Add in an initializer to set the API key for your provider(s) of your choice

# config/initializers/ruby_llm.rb RubyLLM.configure do |config| config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"] config.deepseek_api_key = ENV["DEEPSEEK_API_KEY"] config.gemini_api_key = ENV["GEMINI_API_KEY"] config.openai_api_key = ENV["OPENAI_API_KEY"] end
Enter fullscreen mode Exit fullscreen mode

Set up your .env file if using dotenv (however you choose to save these keys, keep them secure, don't commit to version control)

OPENAI_API_KEY=sk-proj-
Enter fullscreen mode Exit fullscreen mode

Now we create the new models.
First, we create our Chat model which will handle the conversation:

# app/models/chat.rb class Chat < ApplicationRecord acts_as_chat belongs_to :user broadcasts_to ->(chat) { "chat_#{chat.id}" } end
Enter fullscreen mode Exit fullscreen mode

The acts_as_chat method comes from RubyLLM and provides:

  • Message management
  • LLM provider integration
  • Token tracking
  • History management

Next, we create our Message model to handle individual messages in the chat. Each message represents either user input or AI responses:

# app/models/message.rb class Message < ApplicationRecord acts_as_message end
Enter fullscreen mode Exit fullscreen mode

The acts_as_message method from RubyLLM provides:

  • Role management (user/assistant/system)
  • Token counting for both input and output
  • Content formatting and sanitization
  • Integration with the parent Chat model
  • Tool call handling capabilities

Finally, the ToolCall model. I'll cover this in another post, but you need to add it here for RubyLLM to work.

# app/models/tool_call.rb class ToolCall < ApplicationRecord acts_as_tool_call end
Enter fullscreen mode Exit fullscreen mode

Next we link the chats to users:

# app/models/user.rb class User < ApplicationRecord # ...existing code has_many :chats, dependent: :destroy # ...existing code end
Enter fullscreen mode Exit fullscreen mode

Create the migrations:

# db/migrate/YYYYMMDDHHMMSS_create_chats.rb class CreateChats < ActiveRecord::Migration[8.0] def change create_table :chats do |t| t.references :user, null: false, foreign_key: true t.string :model_id t.timestamps end end end # db/migrate/YYYYMMDDHHMMSS_create_messages.rb class CreateMessages < ActiveRecord::Migration[8.0] def change create_table :messages do |t| t.references :chat, null: false, foreign_key: true t.string :role t.text :content t.string :model_id t.integer :input_tokens t.integer :output_tokens t.references :tool_call t.timestamps end end end # db/migrate/YYYYMMDDHHMMSS_create_tool_calls.rb class CreateToolCalls < ActiveRecord::Migration[8.0] def change create_table :tool_calls do |t| t.references :message, null: false, foreign_key: true t.string :tool_call_id, null: false t.string :name, null: false t.jsonb :arguments, default: {} t.timestamps end add_index :tool_calls, :tool_call_id end end
Enter fullscreen mode Exit fullscreen mode

Run the migrations:

rails db:migrate
Enter fullscreen mode Exit fullscreen mode

Then we'll set up ActionCable so we can stream the chat and make it appear as though the AI is typing out the response.
For further details on this, see the Rails Guides

# app/channels/application_cable/connection.rb # This file was created by rails g authentication so if you are using a different auth setup you'll need to adapt this module ApplicationCable class Connection < ActionCable::Connection::Base identified_by :current_user def connect set_current_user || reject_unauthorized_connection end private def set_current_user if session = Session.find_by(id: cookies.signed[:session_id]) self.current_user = session.user end end end end # app/channels/application_cable/channel.rb module ApplicationCable class Channel < ActionCable::Channel::Base end end # app/channels/chat_channel.rb class ChatChannel < ApplicationCable::Channel def subscribed chat = Chat.find(params[:id]) stream_for chat end end
Enter fullscreen mode Exit fullscreen mode 
// app/javascipt/channels/consumer.js import { createConsumer } from "@rails/actioncable" export default createConsumer() // app/javascipt/channels/chat_channel.js import consumer from "./consumer" consumer.subscriptions.create( { channel: "ChatChannel", id: this.element.dataset.chatId } )
Enter fullscreen mode Exit fullscreen mode

Now we set up our controllers.

First, our ChatsController which will handle the overall conversation. It provides:

  • Index action for listing all user's chats
  • Create action for starting new conversations for a user
  • Show action for viewing a user's individual chats
  • Scoped queries to ensure users can only access their own chats 
# app/controllers/chats_controller.rb class ChatsController < ApplicationController def index @chats = chat_scope end def create @chat = chat_scope.new if @chat.save redirect_to @chat else render :index, status: :unprocessable_entity end end def show @chat = chat_scope.find(params[:id]) end private def chat_scope Current.user.chats end end
Enter fullscreen mode Exit fullscreen mode

Next, we create our MessagesController to handle individual message creation and the AI response.

# app/controllers/messages_controller.rb class MessagesController < ApplicationController def create @chat = find_chat GenerateAiResponseJob.perform_later(@chat.id, params[:message][:content]) redirect_to @chat end private def find_chat Current.user.chats.find(params[:chat_id]) end def message_params params.require(:message).permit(:content) end end
Enter fullscreen mode Exit fullscreen mode

Add the necessary routes:

# add to config/routes.rb resources :chats, only: [ :index, :new, :create, :show ] do resources :messages, only: [ :create ] end
Enter fullscreen mode Exit fullscreen mode

Considering AIs can take a bit of time to "think", we're making the call in a background job:

class GenerateAiResponseJob < ApplicationJob queue_as :default def perform(chat_id, user_message) chat = Chat.find(chat_id) thinking = true chat.ask(user_message) do |chunk| if thinking && chunk.content.present? thinking = false Turbo::StreamsChannel.broadcast_append_to( "chat_#{chat.id}", target: "conversation-log", partial: "messages/message", locals: { message: chat.messages.last } ) end Turbo::StreamsChannel.broadcast_append_to( "chat_#{chat.id}", target: "message_#{chat.messages.last.id}_content", html: chunk.content ) end end end
Enter fullscreen mode Exit fullscreen mode

The ask method from RubyLLM will add 2 new messages to the chat. The first one is the message from the user and the second is for the AI's response. The response from the LLM comes back from the provider in chunks and each chunk is passed to the block provided. We wait for the first non-empty chunk before appending the chat's last message (the one created for the AI) to the conversation log. After that we can stream the content of subsequent chunks and append them to the message.

Tip: You can customize the AI's behavior by adding system prompts to the chat instance, see the RubyLLM docs

Finally, we create the views:

<%# app/views/chats/index.html.erb %> <div> <h1>Chats</h1> <% if @chats.empty? %> <p>No chats found. Create a new chat.</p> <% else %> <div> <% @chats.each do |chat| %> <div> <span>ID: <%= chat.id %></span> <span><%= chat.created_at.strftime('%Y-%m-%d %H:%M:%S') %></span> <span> <%= link_to 'View', chat_path(chat) %> </span> </div> <% end %> </div> <% end %> <div> <%= link_to 'New Chat', chats_path, data: { turbo_method: :post } %> </div> </div> <%# app/views/chats/show.html.erb %> <div data-controller="chat" data-chat-id="<%= @chat.id %>"> <%= turbo_stream_from "chat_#{@chat.id}" %> <div> <h1>Chat #<%= @chat.id %></h1> <div>Created: <%= @chat.created_at.strftime('%Y-%m-%d %H:%M:%S') %></div> </div> <div id="conversation-log"> <% if @chat.messages.present? %> <%= render @chat.messages %> <% else %> <p>No messages yet.</p> <% end %> </div> <div> <%= render "messages/form", chat: @chat, message: @chat.messages.new %> </div> <div> <%= link_to 'Back to Chats', chats_path %> </div> </div> <%# app/views/messages/_message.html.erb %> <div id="message_<%= message.id %>"> <div> <span><%= message.role %>:</span> <span><%= message.created_at.strftime('%H:%M:%S') %></span> </div> <div id="message_<%= message.id %>_content"> <%= message.content %> </div> </div> <%# app/views/messages/_form.html.erb %> <%= form_with(model: [chat, message], url: chat_messages_path(chat), id: "new_message") do |form| %> <div> <%= form.text_area :content, placeholder: "Enter message", rows: 2 %> </div> <%= form.submit "Send" %> <% end %>
Enter fullscreen mode Exit fullscreen mode

And that's it!

What's Next?

Now you should have a working AI chat that allows users to have persistent conversations with AI models. In terms of usefulness to your app, this is only the beginning. The real power comes when we let the AI interact with our application's data and functionality through Tools.
If you were to set this up in an e-commerce app, you could use tools to allow an AI to check inventory, calculate shipping costs or search for a specific order. We'll dive into this and explain tools in the next post.

For now, try adding this to your own Rails app and don't forget to add some proper error handling and security measures before deploying to production.

Top comments (0)