So I have this plan to use some raspberry PI Zero's to send data to a Rails backend API i'll be designing. This is to get more experience with Rails and just for a fun side project.
Im going to be reading sensor data on the Raspberry Pi's and sending them to my REST API (the data will be stored and then I can go to each "sensors" page on a dashboard and see historical values/timeline of temperature or whatever sensor I decide to use).
Im mostly familiar with HTTP requests...but usually only on the same server. And this made me realize I don't really understand "HOW" they work. (I just know that with Forms I need to use POST/GET for normal pages/etc....,and just general routing involving that).
That being said, I know on the Rails side i'll need to receive the data Probably via a POST request (Sort of like a form being submitted)...but im not sure how I should be sending it? As in what format? How should it be constructed?
I thought this was simple, but then the more I thought about it the more I realized I didn't know. I know that I need to send the data every x intervals (Via a CRON job running a python script) so i'd be sending a POST request to the API right? I guess I understand that a Form on a site in sending a POST request but what exactly are we requesting? And what would be returned from a Form POST request? Just a 200 status ok?
What about in the case of these devices: Sending a POST request to send their sensor data to be saved on the backend, should the server respond back to them?
Thanks, and sorry for the sorta scattered question. Im realizing I know a lot less than when I started learning rails. Just trying to clear up my understanding on http requests.
Send the data as JSON. Building the rails app is not really different from building any other JSON API.
Lets say you set your models up like so:
# app/models/measurement.rb
class Measurement
belongs_to :unit
end
# app/units/unit.rb
class Unit
has_many :measurements
end
You can use a token based authentication scheme to authenticate each PI unit. Or you can use shared secret such as a MAC address of the unit.
You can register each PI unit in the rails app via the rails console or create the routes and controllers for it in the rails app if you want to be thorough.
If you don't want to manually enter the rails db id for each PI unit you can create a "discovery route" where the PI sends its MAC address and gets an id.
To register measurements you just have send a POST request:
# /config/routes.rb
resources :units do
resources :measurements, module: :units, only: [:create, :index]
end
# /app/controllers/units/measurements_controller.rb
module Units
class MeasurementsController
before_action :set_unit
before_action :authenticate_unit!
# POST /units/:unit_id/measurements
def create
#measurement = #unit.measurements.new(measurement_params)
if #measurement.save
head :created, location: #measurement
else
head :unprocessable_entity
end
end
# GET /units/:unit_id/measurements
def index
render json: #unit.measurements
end
private
def authenticate_unit!
# #todo check auth token in header.
# should raise an exception and return 401 unauthorized if not authenticated
end
def set_unit
#unit = Unit.find(params[:id])
end
def measurement_params
params.permit(:temperature, :foo, :bar)
end
end
end
The Pi would send the data to POST /units/:unit_id/measurements and get a 201 - Created response or 422 - Bad entity. Sending a Location header is really optional since the PI is probally not going to do anything with the response.
How exactly the payload is formatted is up to you as an author. This example just uses a "flat" JSON object:
{
temperature: 3,
foo: 2,
bar: 3
}
Related
As i mentioned on the question title, I want to know the best approach to get this to work and i'll need and example ( very simple one as the follow: )
Let's say i have an API which has 1 controller and 1 action for example simplicity
root 'main#index'
and inside the index action i have
def index
#date = Data.today
end
Then i have another rails app which will work for front-end rendering
How can i pass this #date as JSON from the API to the other app to render it ?
Should i have same controller on the other app ?
How can i connect and send http request and receive response ?
Thanks in advance
For such a simple example, you can do something as simple as:
def index
#date = Date.today
respond_to do |format|
format.json #date
end
end
However, you're most likely going to want to deal with more complicated JSON responses, so before long you'll probably want to use something like the Jbuilder gem or ActiveModel Serializers (my preferred approach).
On the other end, your front-end will need to make an HTTP GET request. Lots of ways (and gems) to do this, but one common approach is just to use the built in Net::HTTP class.
require 'net/http'
url = URI.parse('http://backend.dev/main/index')
request = Net::HTTP::Get.new(url.to_s)
response = Net::HTTP.start(url.host, url.port) do |http|
http.request(request)
end
raise response.body.inspect
In your situation, a better approach might be to use the Active Resource gem. This gem allows you to create models that are backed by a REST API rather than a database. For example, if your API app provides basic Create-Read-Update-Destroy actions for a particular model (let's call it Widget) at the following URLs:
GET http://backend.dev/widget # listing of widgets
GET http://backend.dev/widget/1 # Read for widget id: 1
POST http://backend.dev/widget # Create new widget
UPDATE http://backend.dev/widget/1 # Update widget id: 1
DELETE http://backend.dev/widget/1 # Destroy widget id: 1
then in your front-end app you could declare an Active Resource like this:
class Widget < ActiveResource::Base
self.site = "http://backend.dev"
end
which will auto-magically access all of those methods in your API, and behave much like a regular Active Record model. That way, you basically design your front-end app like a "normal" rails app, but using ActiveResource-based models in place of ActiveRecord.
I would note, however, that a more common thing to do these days would be to build your API in Rails, and build your front-end with client-side Javascript, using something like JQuery or Angular to make requests from the API. I'm not sure what you're gaining by splitting API and front-end, where both of them are Rails apps - unless you've got a compelling reason, I'd just build one Rails app that handles both API and front-end, or build a Rails API + Angular (or similar) front-end.
Disclaimer: I'm doing something which may qualify for Code Smell of 2015 Award. Using rails 4.2, no javascript or anything like that.
I have a form into which users input their data. With this data I call a third-party API which will remain nameless. :)
I have no model, I'm not persisting anything. (Part of a larger app, not a one-pager.) Thus when faced with presenting the user with the response, I find myself stuck on how to render the data properly into a view. The response contains an array of hashes which I obviously intend to present the user.
I render the form into widgets/new, etc, create and process the request, etc, but then what?
I thought maybe I could make use of decorators to do my dirty work but not sure how to actually get the user off to the respective view. I don't care which view. Call it a widget_path.
WidgetsController < ApplicationController
def new
render :new
end
def create
# preparing request
...
# data = response, each_serializer, WidgetSerializer, root: false
# data = WidgetDecorator.new(render_serialized(response, WidgetSerializer))
# #data = WidgetDecorator.new(JSON.parse(response))
# redirect_to ??_path ... and take your #data with you
end
end
What do I do?
Your idea of Model is unfortunately corrupted by Rails itself (sorry).
A model is business logic not an ActiveRecord::Base (not necessarily). Controller methods shouldn't be big, ~5 lines long is probably the maximum with a ~100 lines max per controller file. Try to stick with this and it will automatically correct good chunck of code smells.
Anyway, you may handle this with a Model, as a PORO (plain old ruby object).
class MyApiResponse
attr_reader :myapikey
attr_reader :whatever
def initialize(myapikey, whatever)
#myapikey = myapikey
#whatever = whatever
end
def get
#_response ||= JSON.parse(run_api_stuff(myapikey))
end
end
So in controller you would do something like
def create
myapiresponse = MyApiResonse.new(myapikey, whatever)
#response = myapiresponse.get
end
Last but not least, you can't pass what you obtained through the API in the redirect. You are subject to HTTP limits so, you have a limit on GET params size, a limit on session and you can't redirect to a POST. You have 3 options
Best is store last api request for given user in the database and fetch it back through an ID (which will travel through the redirect)
Store it in session if request is really small (and you must ensure it is small!)
Perform the API request again after the redirect, horrible. Otherwise perform the API request only after redirect, not sure if this is an option though
I have the most basic of questions, but the more I think about it, the more complex it gets.
I've been using rails and it follows the MVC paradigm in that db and api calls are abstracted through calls generated through the controller. This seems way too heavy for what I want.
1) I want a simple (basic) web server that sits in front of my datastore. (The contents of which happen to be stored in a directory structure that follows: /LOCATIONS/LOCATION/PRESENTERS/PRESENTER/YEAR/MN/)
2) I want to be able to host json files within that directory structure and GET them as needed.
3) I want to be able to PUT/POST append to those json files.
Seems like all I'd need is nginx with my datastore as a doc root and index.html files at critical places within the structure (e.g. site.com/Locations/index.html , site.com/locations/SF/presenters/solomon/index.html)?
How would I begin to solve this problem, (without the use of controllers of coarse)?
MVC Frameworks
without the use of controllers
You must be aware that there are many more frameworks than Rails out there, so when you ask about using a system to "sit in front of your datastore", you're really looking for different frameworks to handle requests, of which there are many.
The problem you have is how do you keep data consistency, whilst ensuring you can handle the relevant API requests (through JSON). The answer is to look at how the systems you're examining work.
I can only really vouch for Rails (it's the only framework I've got production apps for) -
--
Rails
Creating an API in Rails is so simple - I don't know why you'd think about doing anything else
Using the MVC principle might seem bloated to you, but the security, structure and extensibility it provides is unmatched.
Here is how to create an API in Rails:
#config/routes.rb
namespace :api do
resources :controller, only: [:update, :create] #-> only PUT / POST
end
#app/controllers/api/your_controller.rb
class API::YourController < ApplicationController
respond_to :json
def update
# handle PUT request
end
def create
# handle POST request
end
end
#app/models/model.rb
Class Model < ActiveRecord::Base
end
This is all you need - you'll be able to access domain.com/api/controller.json POST to create data, and domain.com/api/controller/4.json PUT to update it :)
This post is more about a coding approach to a problem rather than a problem itself (for a change!).
I have a number of projects that i'm working on the moment that require me to gather sales data from a number of disperate sources.
The data from each vendor is almost always accessed and structured in different ways; best case a nice valid JSON reponse, worst case I screen scrape the data.
As each vendor's source data is so different i've decided that a dedicated Rails-api App that feeds json data to a master sales-data App via it's API is the way forward.
I did look at using Sinatra for each vendor app but my knowledge is with Rails so I can get things done much quicker. I feel a dedicated app for each vendor app is the right approach as these can be maintained independently and should a vendor decide to start feeding their data themselves, I (or another developer) can easily swap things over without having to delve into one massive monothlic sales data gathering app, however, do say if you think dedicated apps doesn't make much sense.
So, as a simple stripped down example, each vendor app is structured around a class like this. Currently i'm just calling the methods via console, but will automate via rake tasks and background workers eventually.
class VendorA < ActiveRecord::Base
def self.get_report
# Uses Mechanize Gem to fetch a report in CSV format
# returns report
end
def self.save_report(report)
# Takes the report from the get_report method and saves it, currently to the app root but eventually this will be S3
# returns the local_report
end
def self.convert_report_to_json(local_report)
# Reads the local report, iterates through it grabbing the fields required for the master sales-data app and constructs a JSON repsonse
# returns a valid JSON repsonse called json_data
end
def self.send_to_master_sales_api(json_data)
# As you can see here I take the repsonse from convert_report_to_json and post it to my sales data App
require 'rest-client'
RestClient.post('http://localhost:3000/api/v1/sales/batch', json_data)
end
end
This works fine with the send_to_master_sales_api method doing what is expected. I haven't yet tested this beyond approx 1000 data objects/lines though.
On the receiving end, at the master sales-data App, things look like this:
module Api
module V1
class SalesController < ApplicationController
before_filter :restrict_access
skip_before_filter :verify_authenticity_token
respond_to :json
def batch
i = 0
clientid = params[:clientid]
objArray = JSON.parse(params[:sales])
objArray.each do |sale|
sale = Sale.new(sale)
sale.clientid = clientid #Required / Not null
sale.save
i +=1
end
render json: {message: "#{i} Sales lines recevied"}, status: 202
end
private
def restrict_access
api_key = ApiKey.find_by_api_key_and_api_secret(params[:api_key],params[:api_secret])
render json: { message: 'Invalid Key - Access Restricted' }, status: :unauthorized unless api_key
end
end
end
end
So, my main question is really about the volume of JSON data you think this approach can handle. As I mentioned above, i've tested with around 1000 lines/objects and it worked fine. But should I start having to process data with one of the vendor apps where i'm approaching say, 10,000, 100,000 or even 1,000,000 lines/objects per day from each source what should I be thinking about in terms of the above two apps being able to cope. Is there something like find_in_batches I could be using to ease the load of receiving the data? Currently I plan to write the master sales-data records to a Postgres DB. While i've limited experience of NoSQL, would writing to MongoDB or similar speed things up at the receiving end?
I realise this isn't a particularly direct question, but I would really appreicate input and thoughts from those with experience of this kind of thing.
Thanks in advance!
I need to pass a parameter from one method in a controller to another.
From my understanding I have to pass the parameters as a GET exposing it in the url string.
What is the best way to encrypt the data so no one can see what is actually getting passed in the string? Also, is there a way to pass it via POST or is my original understanding correct?
I haven't used RoR, but in the web world, this problem is solved with sessions. Using sessions you can store the parameters on the server and avoid sending sensitive data with GET or POST (both are insecure).
The Ruby on Rails Security Guide looks like a great read related to this.
I suggest you abstract your code into lib/ so that you don't have to call additional methods. Instead of making a new HTTP request, just put the code in a central place and call it from there.
class MyController < ApplicationController
def index
MyLibrary::Thing.do_stuff
end
def show
MyLibrary::Thing.do_stuff
end
end
# lib/my_library/thing.rb
module MyLibrary
module Thing
def self.do_stuff
# do stuff!
end
end
end
That way you can access the same code in multiple actions, without doing extra HTTP requests.