Consume external API and performance - ruby-on-rails

Question

I am working a Ruby on Rails project that consumes data through some external API.
This API allows me to get a list of cars and display them on my single webpage.
I created a model that holds all methods related to this API.
The controller uses list_cars method from the model to forward the data to the view.
This is the model dedicated to the API calls:
class CarsApi
#base_uri = 'https://api.greatcars.com/v1/'
def self.list_cars
cars = Array.new
response = HTTParty.get(#base_uri + 'cars',
headers: {
'Authorization' => 'Token token=' + ENV['GREATCARS_API_TOKEN'],
'X-Api-Version' => ENV["GREATCARS_API_VERSION"]
})
response["data"].each_with_index do |(key, value), index|
id = response["data"][index]["id"]
make = response["data"][index]["attributes"]["make"]
store = get_store(id)
location = get_location(id)
model = response["data"][index]["attributes"]["model"]
if response["data"][index]["attributes"]["status"] == "on sale"
cars << Job.new(id, make, store, location, model)
end
end
cars
end
def self.get_store(job_id)
store = ''
response_related_store = HTTParty.get(#base_uri + 'cars/' + job_id + "/relationships/store",
headers: {
'Authorization' => 'Token token=' + ENV['GREATCARS_API_TOKEN'],
'X-Api-Version' => ENV["GREATCARS_API_VERSION"]
})
if response_related_store["data"]
store_id = response_related_store["data"]["id"]
response_store = HTTParty.get(#base_uri + 'stores/' + store_id,
headers: {
'Authorization' => 'Token token=' + ENV['GREATCARS_API_TOKEN'],
'X-Api-Version' => ENV["GREATCARS_API_VERSION"]
})
store = response_store["data"]["attributes"]["name"]
end
store
end
def self.get_location(job_id)
address, city, country, zip, lat, long = ''
response_related_location = HTTParty.get(#base_uri + 'cars/' + job_id + "/relationships/location",
headers: {
'Authorization' => 'Token token=' + ENV['GREATCARS_API_TOKEN'],
'X-Api-Version' => ENV["GREATCARS_API_VERSION"]
})
if response_related_location["data"]
location_id = response_related_location["data"]["id"]
response_location = HTTParty.get(#base_uri + 'locations/' + location_id,
headers: {
'Authorization' => 'Token token=' + ENV['GREATCARS_API_TOKEN'],
'X-Api-Version' => ENV["GREATCARS_API_VERSION"]
})
if response_location["data"]["attributes"]["address"]
address = response_location["data"]["attributes"]["address"]
end
if response_location["data"]["attributes"]["city"]
city = response_location["data"]["attributes"]["city"]
end
if response_location["data"]["attributes"]["country"]
country = response_location["data"]["attributes"]["country"]
end
if response_location["data"]["attributes"]["zip"]
zip = response_location["data"]["attributes"]["zip"]
end
if response_location["data"]["attributes"]["lat"]
lat = response_location["data"]["attributes"]["lat"]
end
if response_location["data"]["attributes"]["long"]
long = response_location["data"]["attributes"]["long"]
end
end
Location.new(address, city, country, zip, lat, long)
end
end
It takes... 1 minute and 10 secondes to load my home page!
I wonder if there is a better way to do this and improve performances.

Answer 1

It takes... 1 minute and 10 seconds to load my home page! I wonder if there is a better way to do this and improve performances.
If you want to improve the performance of some piece of code, the first thing you should always do is add some instrumentation. It seems you already have some metric how long loading the whole page takes but you need to figure out now WHAT is actually taking long. There are many great gems and services out there which can help you.
Services:
https://newrelic.com/
https://www.skylight.io/
https://www.datadoghq.com/
Gems
https://github.com/MiniProfiler/rack-mini-profiler
https://github.com/influxdata/influxdb-rails
Or just add some plain old logging
N+1 request
One assumption why this is slow could be that you do for every car FOUR additional requests to fetch the store and location. This means if you display 10 jobs on your homepage you would need to do 50 API requests. Even if each request just takes ~1 second it's almost one minute.
One simple idea is to cache the additional resources in a lookup table. I'm not sure how many jobs would share the same store and location though so not sure how much it would actually safe.
This could be a simple lookup table like:
class Store
cattr_reader :table do
{}
end
class << self
def self.find(id)
self.class.table[id] ||= fetch_store(id)
end
private
def fetch_store(id)
# copy code here to fetch store from API
end
end
end
This way, if several jobs have the same location / store you only do one request.
Lazy load
This depends on the design of the page but you could lazy load additional information like location and store.
One thing many pages do is to display a placeholder or dominant colour and lazy load further content with Java Script.
Another idea could be load store and location when scrolling or hovering but this depends a bit on the design of your page.
Pagination / Limit
Maybe you're also requesting too many items from the API. See if the API has some options to limit the number of items you request e.g. https://api.greatcars.com/v1/cars/limit=10&page=1
But as said, even if this is limited to 10 items you would end up with 50 requests in total. So until you fix the first issue, this won't have much impact.
General Caching
Generally I think it's not a good idea to always send an API request for each request you page gets. You could introduce some caching to e.g. only do an API request once every x minutes / hour / day.
This could be as simple as just storing in a class variable or using memcached / Redis / Database.