DEV Community

Cover image for Ruby class pattern to work with API requests with built-in async approach
Mykola Zozuliak
Mykola Zozuliak

Posted on

Ruby class pattern to work with API requests with built-in async approach

#ruby #concurrency #oop #async

Intro

I have been working with dozens of different Web APIs (mostly JSON) as a Ruby developer. Some of them can provide an official Ruby gem that you can install and use in order to interact with the API in your Ruby code. Others don't have anything like It at all and you need to implement the interaction with the API yourself. In this case, there are a lot of helpful API ruby gems that are easier to use than the default Ruby URI module https://github.com/ruby/uri. Let's list a few of them:

Those tools are just tools that you can use in order to implement the connection and interaction with any Web API. But the problem is that every API has Its interface, authorization rules, rate limiter, the way It handles errors, etc. So if in your project you need to connect 2 totally different APIs - you need to implement the interaction to each one yourself (assuming that those APIs have no gems available). If that API interaction code will not be properly encapsulated in Ruby classes, every developer in your team will need to carefully read API documentation in order to use the code that triggers those API calls. With every added API to your project, this issue will be a bigger and bigger problem with time. In this article, I will provide my own idea of how to create a simple, but powerful Ruby class pattern to hide HTTP interaction for any API in a Ruby class. This idea is based on the work that I did with my colleague years ago. Every such class will have the same external interface. As a bonus - my solution includes a way to do async HTTP requests.

Pattern

We should start with the list of Ruby gems that I am gonna use In my example:

gem 'concurrent-ruby' gem 'faraday' gem 'faraday_curl'
Enter fullscreen mode Exit fullscreen mode
  • concurrent-ruby - to add async requests ability. https://github.com/ruby-concurrency/concurrent-ruby
  • faraday - to do HTTP requests (you can use any approach to do HTTP requests as I mentioned in the Intro section)
  • faraday_curl - to get nice Curl requests in logs (totally optional)

The next step is to create a module that I am gonna call FaradayConnector - this module will have all the needed logic in order to make any kind of HTTP requests to Web API (JSON by default). This module will be used to mix in other modules (API-specific ones). I will add more clarity on that later in the article:

require 'faraday' require 'faraday_curl' require 'concurrent' module FaradayConnector class ServerError < RuntimeError attr_reader :response def initialize(response) @response = response super end end class TimeoutError < RuntimeError; end class ClientError < RuntimeError; end def request return @_request if defined?(@_request) # creating a Promise for async approach @_request = Concurrent::Promises.future { do_request } end def process return @_process if defined?(@_process) request @_process = do_process end def as_json(_options = {}) process end protected def do_request # implement the real request in Child end def do_process # implement additional response decorations in Child request.value! end def url # must be added in Child raise 'Undefined url' end def auth # must be added in Child or use nil, if API has no Authorization raise 'Undefined auth' end def additional_headers {} end def content_type 'application/json' end def request_type :url_encoded end def get(path, params = {}) handle_request { connection.get(path, params) } end def post(path, body = {}) formatted_body = json_content? ? body.to_json : body handle_request { connection.post(path, formatted_body) } end def delete(path, params = {}) handle_request { connection.delete(path, params) } end def put(path, body = {}) formatted_body = json_content? ? body.to_json : body handle_request { connection.put(path, formatted_body) } end def timeout 45 end def connection @connection ||= Faraday.new(url: url) do |faraday| faraday.request request_type faraday.headers['Authorization'] = auth if auth faraday.headers['Content-Type'] = content_type faraday.headers = faraday.headers.merge(additional_headers) if additional_headers faraday.options.timeout = timeout faraday.response(:logger) faraday.request :curl, Logger.new($stdout), :info faraday.adapter Faraday.default_adapter end end def handle_request response = handle_errors { yield } parse_response(response) end # just an easier way to handle HTTP errors def handle_errors response = yield e = if [502, 504].include?(response.status) TimeoutError.new(response) elsif [500, 503].include?(response.status) ServerError.new(response) elsif [400, 401, 404, 422].include?(response.status) ClientError.new(response) end return response unless e raise e end def parse_response(response) return {} unless response.body json_content? ? JSON.parse(response.body) : response rescue JSON::ParserError {} end def json_content? content_type == 'application/json' end end
Enter fullscreen mode Exit fullscreen mode

Now, assume we need to connect 2 different APIs (I just googled 2 public ones to show as an example):

Every API has Its own Authorization rules, OpenMeteo has none, and SportMonks uses an Authorization token. We need to create 2 folders with modules called ApiConnector:

  • open_meteo/api_connector.rb 
require './faraday_connector' module OpenMeteo module ApiConnector include FaradayConnector def url 'https://api.open-meteo.com' end def auth # no auth for this example end end end
Enter fullscreen mode Exit fullscreen mode
  • sport_monks/api_connector.rb 
require './faraday_connector' module SportMonks module ApiConnector include FaradayConnector def url 'https://api.sportmonks.com' end def auth # a token goes here based on the API documentation # https://docs.sportmonks.com/football/welcome/authentication 'some-token' end end end
Enter fullscreen mode Exit fullscreen mode

With these 2 easy steps, we created a codebase that can successfully authorize those 2 APIs we are using.
Time to create some HTTP requests. Let's start with OpenMeteo. On their home page, there are 2 examples of 2 different endpoints - forecast and historical. The next step is to create a separate class for every endpoint using the already created ApiConnector module.

  • forecast 
module OpenMeteo class Forecast include ApiConnector attr_reader :latitude, :longitude def initialize(latitude, longitude) @latitude = latitude @longitude = longitude end def do_request get('v1/forecast', latitude: latitude, longitude: longitude ) end def do_process request.value! # additional data manipulations goes here end end end
Enter fullscreen mode Exit fullscreen mode
  • historical 
module OpenMeteo class Historical include ApiConnector attr_reader :latitude, :longitude, :start_date, :end_date def initialize(latitude, longitude, start_date, end_date) @latitude = latitude @longitude = longitude @start_date = start_date @end_date = end_date end def do_request get('v1/era5', latitude: latitude, longitude: longitude, start_date: start_date, end_date: end_date ) end def do_process request.value! # additional data manipulations goes here end private def url # you can change url when needed easily 'https://archive-api.open-meteo.com/' end end end
Enter fullscreen mode Exit fullscreen mode

There are 3 essential methods here:

  • initialize - a class constructor is only needed when an API request has input parameters
  • do_request - a place where you are triggering an HTTP request (available methods are get, post, put, delete). Remember this method returns Concurrent::Promise class instances.
  • do_process - calling value! method on Concurrent::Promise a class instance is essential to get the request-response.

Note: You probably noticed the url method override in the second file, we need this because a second API call has a different URL, and with this pattern, it's very easy to do)

This is the way to use those classes to do requests:

forecast = OpenMeteo::Forecast.new(42.361145, -71.057083) historical = OpenMeteo::Historical.new(42.361145, -71.057083, '2024-01-01', '2024-01-30') # Async part forecast.request historical.request puts 'historical', historical.process puts 'forecast', forecast.process
Enter fullscreen mode Exit fullscreen mode

With the async approach, it means that we are doing both requests in 2 different threads, so they will work faster. When you call process method on any of It, promise will wait until all requests will be finished before returning any results, this is the way to synchronize code back.

The output will look like this:

I, [2024-05-09T14:34:30.756703 #74264] INFO -- request: GET https://api.open-meteo.com/v1/forecast?latitude=42.361145&longitude=-71.057083 I, [2024-05-09T14:34:30.756857 #74264] INFO -- request: Content-Type: "application/json" User-Agent: "Faraday v2.9.0" I, [2024-05-09T14:34:30.756822 #74264] INFO -- request: GET https://archive-api.open-meteo.com/v1/era5?end_date=2024-01-30&latitude=42.361145&longitude=-71.057083&start_date=2024-01-01 I, [2024-05-09T14:34:30.761810 #74264] INFO -- request: Content-Type: "application/json" User-Agent: "Faraday v2.9.0" I, [2024-05-09T14:34:31.084956 #74264] INFO -- response: Status 200 I, [2024-05-09T14:34:31.085185 #74264] INFO -- response: date: "Thu, 09 May 2024 18:34:31 GMT" content-type: "application/json; charset=utf-8" transfer-encoding: "chunked" connection: "keep-alive" content-encoding: "gzip" I, [2024-05-09T14:34:31.090995 #74264] INFO -- response: Status 200 I, [2024-05-09T14:34:31.091058 #74264] INFO -- response: date: "Thu, 09 May 2024 18:34:31 GMT" content-type: "application/json; charset=utf-8" transfer-encoding: "chunked" connection: "keep-alive" content-encoding: "gzip" historical {"latitude"=>42.355007, "longitude"=>-71.12906, "generationtime_ms"=>0.0050067901611328125, "utc_offset_seconds"=>0, "timezone"=>"GMT", "timezone_abbreviation"=>"GMT", "elevation"=>9.0} forecast {"latitude"=>42.36515, "longitude"=>-71.0618, "generationtime_ms"=>0.00095367431640625, "utc_offset_seconds"=>0, "timezone"=>"GMT", "timezone_abbreviation"=>"GMT", "elevation"=>9.0}
Enter fullscreen mode Exit fullscreen mode

As we can see, both requests are running at the same time, so we are sure the async approach is working. If we just run process methods for both instances one by one, we will have a synchronized approach, that will look like this:

forecast = OpenMeteo::Forecast.new(42.361145, -71.057083) historical = OpenMeteo::Historical.new(42.361145, -71.057083, '2024-01-01', '2024-01-30') puts 'historical', historical.process puts 'forecast', forecast.process
Enter fullscreen mode Exit fullscreen mode 
I, [2024-05-09T14:37:15.830733 #74386] INFO -- request: GET https://archive-api.open-meteo.com/v1/era5?end_date=2024-01-30&latitude=42.361145&longitude=-71.057083&start_date=2024-01-01 I, [2024-05-09T14:37:15.830765 #74386] INFO -- request: Content-Type: "application/json" User-Agent: "Faraday v2.9.0" I, [2024-05-09T14:37:16.154027 #74386] INFO -- response: Status 200 I, [2024-05-09T14:37:16.154220 #74386] INFO -- response: date: "Thu, 09 May 2024 18:37:16 GMT" content-type: "application/json; charset=utf-8" transfer-encoding: "chunked" connection: "keep-alive" content-encoding: "gzip" historical {"latitude"=>42.355007, "longitude"=>-71.12906, "generationtime_ms"=>0.0050067901611328125, "utc_offset_seconds"=>0, "timezone"=>"GMT", "timezone_abbreviation"=>"GMT", "elevation"=>9.0} I, [2024-05-09T14:37:16.157443 #74386] INFO -- request: GET https://api.open-meteo.com/v1/forecast?latitude=42.361145&longitude=-71.057083 I, [2024-05-09T14:37:16.157511 #74386] INFO -- request: Content-Type: "application/json" User-Agent: "Faraday v2.9.0" I, [2024-05-09T14:37:16.495799 #74386] INFO -- response: Status 200 I, [2024-05-09T14:37:16.496003 #74386] INFO -- response: date: "Thu, 09 May 2024 18:37:16 GMT" content-type: "application/json; charset=utf-8" transfer-encoding: "chunked" connection: "keep-alive" content-encoding: "gzip" forecast {"latitude"=>42.36515, "longitude"=>-71.0618, "generationtime_ms"=>0.00095367431640625, "utc_offset_seconds"=>0, "timezone"=>"GMT", "timezone_abbreviation"=>"GMT", "elevation"=>9.0}
Enter fullscreen mode Exit fullscreen mode

Now let's get back to the second API that we planned to connect. It has a lot more examples that you can check out here https://docs.sportmonks.com/football/api/demo-response-files. To be very simple, let's say we want to get data about some team with an id 8.

module SportMonks class Team include ApiConnector attr_reader :team_id def initialize(team_id) @team_id = team_id end def do_request get("v3/football/teams/#{team_id}") end def do_process request.value! # additional data manipulations goes here end end end team = SportMonks::Team.new(8) team.request team.process
Enter fullscreen mode Exit fullscreen mode

Same interface, but a different API.

Summary

Feel free to use this pattern for your API needs. I have been using this for years for a wide variety of services. I prefer using this, then connecting API-specific gems. From my perspective, this approach has a few benefits:

  • integrated async way of doing requests
  • same external interface for every API you will use in your codebase, which makes It very easy to use by any developer in your team
  • once ApiConnector is created with configured authorization, you will need to think about It ever again. (always use a secure way to store tokens or other secrets). If you don't know what to use, I can suggest dotenv gem - https://github.com/bkeepers/dotenv
  • flexible request configuration that is encapsulated in one class with a method override approach

PS: It's a copy of my original post on Medium - https://medium.com/@zozulyak.nick/ruby-class-pattern-to-work-with-api-requests-with-built-in-async-approach-bf0713a7dc96

Top comments (3)

Collapse
 
chuggingcoffee profile image
Mike H •

Great write-up.

Collapse
 
davidteren profile image
David Teren •

Nicely done. Thanks for sharing.

Collapse
 
softwaredeveloping profile image
FrontEndWebDeveloping •

Good post!