I have the code that I need to do all the scraping and then printing the results to the console, but, I am confused about how to use it in an app.
The way it's supposed to work is through the list#new action I take user input for one parameter, :url. This URL is then passed to the scraping code, which obtains all the additional parameters and adds everything to Postgres tables. Using all of this newly acquired data, a new list is rendered.
The questions that I have:
the lists controller:
class UsersController < ApplicationController
.
.
.
def create
#list = List.new ( #what goes in here?
#only one param comes from the user
if #list.save
#how to set it up so that the save is successful
#only if the extra params have been scraped?
.
.
.
I assume this will go into the models/list.rb:
class List < ActiveRecord::Base
require 'open-uri'
url = #assuming that the url is proper and for something this code is supposed to scrape
#is it better to add the url to db first or send it straight from the input
#and how is that defined here
doc = Nokogiri::HTML(open(url))
.
.
.
Could you give me some guidance here, please?
The services file:
class ScrapingService
require 'open-uri'
require 'nokogiri'
def initialize(list)
#list = list
end
url = :url
doc = Nokogiri::HTML(open(url))
name = doc.at_css(".currentverylong").text
author = doc.at_css(".user").text
def scraped_successfully?
if name != nil && author != nil
true
else
false
end
end
private
attr_reader :list
end
Some questions that I have are:
How do I properly introduce :url into HTML(open...? The way I have it now throws no implicit conversion of Symbol into String error.
The part where :url along with :name and :author are supposed to be saved into one db entry is really murky.
Any article suggestions on this stuff are always welcome.
app/controllers/lists_controller.rb
class UsersController < ApplicationController
def create
#list = List.new(list_params)
if #list.save
redirect_to #list
else
render :new
end
private
#Assuming that you are using Rails 4 or the strong_params gem
def list_params
params.require(:list).permit(:url)
end
end
app/models/list.rb
class List < ActiveRecord::Base
# This runs only when you try to create a list. If you want to run this
# validation when the user updates it, the remove the on: :create
before_validation :ensure_website_is_scrapable, on: :create
private
def ensure_website_is_scrapable
if ScrapingService.new(self).scraped_successfully?
true
else
errors.add(:url, 'The website is not scrapable')
end
end
end
app/services/scraping_service.rb
class ScrapingService
def initialize(list)
#list = list
end
def scraped_successfully?
# Do the scraping logic here and return true if it was successful or false otherwise
# Of course split the implementation to smaller methods
end
private
attr_reader :list
end
Related
I have a model Person and based on a condition I'd like to add a field to it based on a condition.
That is, I'd need an applied field if the condition in the controller is met.
I did #person.applied = true before the method returns, and if I do puts #person.applied, true is logged, as expected. However, if I do puts #person, the applied field is not listed, while all the others (stored in the database) are.
I also have attr_accessor :applied in the model.
What am I doing wrong?
EDIT (snippets)
# people_controller.rb
def show
if #application
#person.applied = true
end
puts #person.applied # logs true
puts #person # applied is not included still
render json: #person
# person.rb
class Person < ApplicationRecord
attr_accessor :applied
end
make this:
# person.rb
class Person < ApplicationRecord
attribute :applied
end
# people_controller.rb
def show
if #application
#person.applied = true
end
p #person.applied # logs true
p #person # applied is not included still
render json: #person
# rest of your code
this works fine with rails 6 and sqlite3
Please note that this is a virtual attribute. you have to set it in the controller actions that you want to use it because it does not persist in the database.
I have a ruby class in a rails helper and inside that class I cannot access the ActiveController cookies method. I have looked at this question and it doesn't solve my problem and I'm not doing what that guy is since my ruby class is in the scope of the controller and is only ever called by it. What's the best way to get rid of the
NameError (undefined local variable or method `cookies` for #<CartHelper::CartObject:...>):
app/helpers/cart_helper.rb:##:in `save`
app/helpers/cart_helper.rb:##:in `add_product`
app/controllers/cart_controller.rb:##:in `add`
in my helper? (I'm also open to suggestions on how to make this more Ruby-y approach wise)
module CartHelper
class CartObject
def load (cart)
# converts from json
end
private def save
cookies.signed.permanent[:cart] = dump
end
private def dump
# converts to json
end
def add_product (product)
# ...
save
end
def remove_product (product)
# ...
save
end
end
def get_cart
return CartObject.new if cookies.signed.permanent[:cart].nil?
CartObject.load(cookies.signed.permanent[:cart])
end
end
and CartHelper is included in the application controller of course
class CartController < ApplicationController\
skip_before_action :verify_authenticity_token, only: [:add]
def view
cart = get_cart
#products = cart.products
end
def add # only by ajax
cart = get_cart
cart.add_product(Oj.strict_load(params[:product]))
head 200
end
def remove
cart = get_cart
cart.remove_product(params['rem'])
flash['success'] = 'Product removed.'
redirect_to '/cart'
end
end
So i'm working on a sort of custom-rolled history tracking for a RoR application. The part i'm hung up on is getting the logged in users information to tie to the record. I've figured out getting the user, its by a submodule which is attached to the ActionController::Base class. The problem is, I'm having trouble retrieving it from the submodule.
Here is my code:
module Trackable
# This is the submodule
module TrackableExtension
extend ActiveSupport::Concern
attr_accessor :user
included do
before_filter :get_user
end
def get_user
#user ||= current_user # if I log this, it is indeed a User object
end
end
# Automatically call track changes when
# a model is saved
extend ActiveSupport::Concern
included do
after_update :track_changes
after_destroy :track_destroy
after_create :track_create
has_many :lead_histories, :as => :historical
end
### ---------------------------------------------------------------
### Tracking Methods
def track_changes
self.changes.keys.each do |key|
next if %w(created_at updated_at id).include?(key)
history = LeadHistory.new
history.changed_column_name = key
history.previous_value = self.changes[key][0]
history.new_value = self.changes[key][1]
history.historical_type = self.class.to_s
history.historical_id = self.id
history.task_committed = change_task_committed(history)
history.lead = self.lead
# Here is where are trying to access that user.
# #user is nil, how can I fix that??
history.user = #user
history.save
end
end
In my models then its as simple as:
class Lead < ActiveRecord::Base
include Trackable
# other stuff
end
I got this to work by setting a Trackable module variable.
In my TrackableExtension::get_user method I do the following:
def get_user
::Trackable._user = current_user #current_user is the ActionController::Base method I have implemented
end
Then for the Trackable module I added:
class << self
def _user
#_user
end
def _user=(user)
#_user = user
end
end
Then, in any Trackable method I can do a Trackable::_user and it gets the value properly.
Given the following:
class WebsitesController < ApplicationController
# POST /websites/save
# POST /websites/save.json
def save
Website.exists?(name: params[:website][:name]) ? update : create
end
# POST /websites
# POST /websites.json
def create
#server = Server.find_or_create_by_name(params[:server_id])
#website = #server.websites.new(params[:website])
#etc... #website.save
end
# PUT /websites/1
# PUT /websites/1.json
def update
#website = Website.find_by_name(params[:website][:name])
#etc... #website.update_attributes
end
end
The client does not have any IDs of these models
The request that gets sent only has the names, but not the ids.
And the following models
class Website < ActiveRecord::Base
serialize :website_errors
attr_accessible :plugins_attributes
has_many :plugins
accepts_nested_attributes_for :plugins
end
class Plugin < ActiveRecord::Base
belongs_to :website
end
When I make a POST request to /websites/save.json, the Website gets updated correctly if it exists, but the Plugins that belong to it always get recreated causing duplicate content in the Database. Why does this happen? I redirect to the update action which calls update_attributes so how can it be that it does not update it? I take it that it's because no ID is given with the request.
Can I make the Controller listen to plugin_name instead of plugin_id?
Modify your controller to have this:
def update
#website = Website.find_by_name(params[:website][:name])
if #website.update(params)
redirect_to website_path(#website)
else
render :edit
end
end
Also, if you're using strong_parameters, you'll need this at the bottom of your controller:
params.require(:website).
permit(
:name,
...,
plugins_attributes: [
:name,
...,
]
)
end
Within Authlogic, is there a way that I can add conditions to the authentication method? I know by using the find_by_login_method I can specify another method to use, but when I use this I need to pass another parameter since the find_by_login_method method only passes the parameter that is deemed the 'login_field'.
What I need to do is check something that is an association of the authentic model.. Here is the method I want to use
# make sure that the user has access to the subdomain that they are
# attempting to login to, subdomains are company names
def self.find_by_email_and_company(email, company)
user = User.find_by_email(email)
companies = []
user.brands.each do |b|
companies << b.company.id
end
user && companies.include?(company)
end
But this fails due to the fact that only one parameter is sent to the find_by_email_and_company method.
The company is actually the subdomain, so in order to get it here I am just placing it in a hidden field in the form (only way I could think to get it to the model)
Is there a method I can override somehow..?
Using the answer below I came up with the following that worked:
User Model (User.rb)
def self.find_by_email_within_company(email)
# find the user
user = self.find_by_email(email)
# no need to continue if the email address is invalid
return false if user.nil?
# collect the subdomains the provided user has access to
company_subdomains = user.brands.map(&:company).map(&:subdomain)
# verify that the user has access to the current subdomain
company_subdomains.include?(Thread.current[:current_subdomain]) && user
end
Application Controller
before_filter :set_subdomain
private
def set_subdomain
# helper that retreives the current subdomain
get_company
Thread.current[:current_subdomain] = #company.subdomain
end
User Session Model (UserSession.rb)
find_by_login_method :find_by_email_within_company
I have read a few things about using Thread.current, and conflicting namespaces.. This is a great solution that worked for me but would love to hear any other suggestions before the bounty expires, otherwise, +100 to Jens Fahnenbruck :)
Authlogic provides API for dealing with sub domain based authentication.
class User < ActiveRecord::Base
has_many :brands
has_many :companies, :through => :brands
acts_as_authentic
end
class Brand < ActiveRecord::Base
belongs_to :user
belongs_to :company
end
class Company < ActiveRecord::Base
has_many :brands
has_many :users, :through => :brands
authenticates_many :user_sessions, :scope_cookies => true
end
Session controller:
class UserSessionsController < ApplicationController
def create
#company = Company.find(params[:user_session][:company])
#user_session = #company.user_sessions.new(params[:user_session])
if #user_session.save
else
end
end
end
On the other hand
Here is a way to solve the problem using your current approach(I would use the first approach):
Set custom data - to the key email of the hash used to create the UserSession object.
AuthLogic will pass this value to find_by_login method. In the find_by_login method access the needed values.
Assumption:
The sub domain id is set in a field called company in the form.
class UserSessionsController < ApplicationController
def create
attrs = params[:user_session].dup #make a copy
attrs[:email] = params[:user_session] # set custom data to :email key
#user_session = UserSession.new(attrs)
if #user_session.save
else
end
end
end
Model code
Your code for finding the user with the given email and subdomain can be simplified and optimized as follows:
class User < ActiveRecord::Base
def find_by_email params={}
# If invoked in the normal fashion then ..
return User.first(:conditions => {:email => params}) unless params.is_a?(Hash)
User.first(:joins => [:brands => :company}],
:conditions => ["users.email = ? AND companies.id = ?",
params[:email], params[:company]])
end
end
Edit 1
Once the user is authenticated, system should provide access to authorized data.
If you maintain data for all the domains in the same table, then you have to scope the data by subdomain and authenticated user.
Lets say you have Post model with company_id and user_id columns. When a user logs in you want to show user's posts for the sub domain. This is one way to scope user's data for the subdomain:
Posts.find_by_company_id_and_user_id(current_company, current_user)
Posts.for_company_and_user(current_company, current_user) # named scope
If you do not scope the data, you will have potential security holes in your system.
In your lib folder add a file with the follwing content:
class Class
def thread_local_accessor name, options = {}
m = Module.new
m.module_eval do
class_variable_set :"###{name}", Hash.new {|h,k| h[k] = options[:default] }
end
m.module_eval %{
FINALIZER = lambda {|id| ###{name}.delete id }
def #{name}
###{name}[Thread.current.object_id]
end
def #{name}=(val)
ObjectSpace.define_finalizer Thread.current, FINALIZER unless ###{name}.has_key? Thread.current.object_id
###{name}[Thread.current.object_id] = val
end
}
class_eval do
include m
extend m
end
end
end
I found this here
Then add code in the controller like this:
class ApplicationController < ActionController
before_filter :set_subdomain
private
def set_subdomain
User.subdomain = request.subdomains[0]
end
end
And now you can do the following in your user model (assuming your company model has a method called subdomain:
class User < ActiveRecord::Base
thread_local_accessor :subdomain, :default => nil
def self.find_by_email_within_company(email)
self.find_by_email(email)
company_subdomains = user.brands.map(&:company).map(&:subdomain)
company_subdomains.include?(self.subdomain) && user
end
end
And FYI:
companies = user.brands.map(&:company).map(&:subdomain)
is the same as
companies = []
user.brands.each do |b|
companies << b.company.subdomain
end
With rails 3 you can use this workaround:
class UserSessionsController < ApplicationController
...
def create
#company = <# YourMethodToGetIt #>
session_hash = params[:user_session].dup
session_hash[:username] = { :login => params[:user_session][:username], :company => #company }
#user_session = UserSession.new(session_hash)
if #user_session.save
flash[:notice] = "Login successful!"
redirect_back_or_default dashboard_url
else
#user_session.username = params[:user_session][:username]
render :action => :new
end
...
end
Then
class UserSession < Authlogic::Session::Base
find_by_login_method :find_by_custom_login
end
and
class User < ActiveRecord::Base
...
def self.find_by_custom_login(hash)
if hash.is_a? Hash
return find_by_username_and_company_id(hash[:login], hash[:company].id) ||
find_by_email_and_company_id(hash[:login], hash[:company].id)
else
raise Exception.new "Error. find_by_custom_login MUST be called with {:login => 'username', :company => <Company.object>}"
end
end
...
end
Which is quite plain and "correct". I take me a lot of time to find out, but it works fine!