I have a Rails app on a Postgres database and I need to have a search field for the user to enter a string and look up in the database for possible address matches (within a city). In the database I have a column with full addresses.
I cannot make assumptions on the input, so I am thinking that I should first try to directly look up the address on the database somehow (using a LIKE query maybe?), and if that fails, request to a Geocoding API (i.e. Google) to return a well formatted addresses list matching the query and search those in my database.
I would appreciate any guidance on how to do this.
I don't think FTS (full text search) is what you want. You'll have to use an address API that can match addresses.
I've successfully and easily used SmartyStreets for something like this. They have a free account you can use.
http://smartystreets.com
Also if you did want to try going down the FTS route here is a Gist that explains how to do it.
https://gist.github.com/4365593
You may know it already, but postresql has a fulltext search engine integrated so it's a great time to take advantage of it. I suggest watching thats excellent railscast.
Then once implemented :
class Place < AR
def search_db_or_geokit(query)
res = db_search()
if res.empty?
res = geokit_search(query)
else
res
end
end
def geokit_search(query)
# ...
end
def db_search(query)
# ...
end
end
For the geocoding google search api there's probably a good gem out there like geokit
Related
I am writing a AWS-Federation proxy in Rails. This means I grab for some groups using net-ldap on our local ActiveDirectory and want to compare those to a list and look for matches. My NetLDAP-searchresult is this hash:
[#<Net::LDAP::Entry:0x000000048cfdd0 #myhash={:dn=>["CN=Username,OU=Support,OU=mycompany ,OU=Organisation,DC=mycompany,DC=com"], :memberof=>["CN=My AWS Groupname,CN=Receiver,CN=Users,DC=mycompany,DC=com"]}>]
Now I want to parse this hash and look for matches in a local "groups" table. It looks like that:
Name AWS-Role
My AWS-Groupname Some Group
AWS-Othergroup Some Other-Group
I have a group-model.
What is a best practices approach? I've never done something like this before. Would I use a Regex here? Do I loop the groups through all tables? What's the rails way to do this?
edited for more information
I'm going to assume a few things here, since I don't know where you get the LDAP search results from, but assuming your hash looks like this:
EDIT:
based on the additional information:
// Example
require 'net-ldap'
entry = Net::LDAP::Entry.new
entry.dn = ["CN=Username,OU=Support,OU=mycompany ,OU=Organisation,DC=mycompany,DC=com"]
entry[:memberof] =["CN=My AWS Groupname,CN=Receiver,CN=Users,DC=mycompany,DC=com"]
name = entry.memberof.first.split(',').first.gsub('CN=', '')
And assuming you have a model called Group that is mapped to this "groups" table, you can do something like this:
Group.where(name: name).any?
If you find any results, it means you have a match in the table.
But this completely depends on the table structure and hash. To properly answer your question, I'd need to see what Objects you have in Rails, and what the structure of your Hash looks like.
EDIT:
Updated my answer based on the received feedback. Use code at own risk.
I am using Ruby on Rails 4.1, Ransack and attr_encrypted. I have sensitive data being stored in my database and I want to protect it using the gem attr_encrypted.
As I expected, I got zero results when searching encrypted test data with Ransack.
I tried the following solution to but it didn't seem to work for me. I was under the impression that the load function was used to return the decrypted value.
ReportsController
def index
#report_list = Report.all.load
#q = #report_list.search(params[:q])
#reports = #q.result(distinct: true).order('created_at DESC')
end
Has anyone had any experience searching across encrypted data and could help me generate a working solution?
load will cause the Active Record Collection to execute a query and retrieve the results matching your query (in addition to running after_create call backs which I believe is where the decrypt you were expecting is happening).
def index
#returns all records in DB
#report_list = Report.all.load
#I'm surprised these aren't throwing undefined method search of Array (or something similar)
#q = #report_list.search(params[:q])
#reports = #q.result(distinct: true).order('created_at DESC')
end
I would like to precede this with, I normally do this thing manually and am not familiar with attr_encrypted or Ransack, but I believe these concepts are general enough they could be applied to any setup. So, as to your question, 2 possibilities.
If your ok searching for exact values:
Model.where(encrypted_field: encrypt(params[:value])).first
where encrypt is a method that encrypts and returns the passed string.
Secondly (and painfully)
Model.all.delete_if{|m| !m.encrypted_field.include?(params[:value]) }
This will literally pull, decrypt, and scan every entry in your database.
I would highly recommend not doing this, but you need to do what you need to do.
If you absolutely need to have the information encrypted but still need to be able to do searches like this. I would highly recommend adding tags of some sort to your model. This would allow you to remove sensitive information but still search by some attributes.
Im trying to search my User model for all Users that start with any integer, I have code for individual letters and it works, but Im having trouble getting it working with a wild card. Right now I have this code:
in my view:
<%= link_to '#', users_charlist_path(:char => '[0123456789]' %>
and in my controller I have:
def charlist
#a = User.where('goal like ?', "#{params[:char]}%").to_a
end
how ever, '[0123456789]', doesnt seem to work as it does not return anythign to me even though I have users whose names begin with an integer. how do i do this?
The where method is a part of ActiveRecord which maps the objects to the database. So how you can query the database with a regex depends on which db you are using not on ruby. You need to look up the regex functions of the database your using. For mysql you can find them here:
http://dev.mysql.com/doc/refman/5.1/de/regexp.html
An alternative is to select all objects an use the select method to filter the results that match your needs. That method is documented here:
http://www.ruby-doc.org/core-2.1.0/Array.html#method-i-select
On big amounts of data I whould suggest to use the database even if that means your application isnt 100% portable between different database systems.
I need to implement some search functionality within a Rails application. Most of the stuff I have found is generally aimed at simple plain-text search. I am trying to implement something much more specific. The sort of functionality I am looking to create is this (from a C application):
http://andyc.ac/query.gif
The form just submits the data entered by the user. So I need to translate strings like "3..7" into SQL conditions for the where method e.g.
TestLine.where( "test_int >= ? and test_int <= ?", MinInt, MaxInt )
It seems like this is something that already exists somewhere. The exact format expected is not too important, as the users are not shared between the Rails and C applications. How would this be done?
FWIW the specific functionality you describe is actually supported directly. Well.. almost. From the docs:
A range may be used in the hash to use the SQL BETWEEN operator:
Student.where(:grade => 9..12)
Of course then it's a matter of translating the user's string input to a Range, which isn't very complex, e.g.:
def str_to_range str
str =~ /(\d+)\.\.(\d+)/
Range.new *$~.captures.map(&:to_i)
end
It would probably make the most sense in a scope on your model. (Of course a shortcut would be to simply eval '9..12' but evaling input from the end user is a really, really bad idea.)
Give a look at thinking sphinx(http://freelancing-god.github.com/ts/en/). It might make your task a lot easier. You can search in that:
http://freelancing-god.github.com/ts/en/searching.html#basic
I have the following:
#users = User.all
User has several fields including email.
What I would like to be able to do is get a list of all the #users emails.
I tried:
#users.email.all but that errors w undefined
Ideas? Thanks
(by popular demand, posting as a real answer)
What I don't like about fl00r's solution is that it instantiates a new User object per record in the DB; which just doesn't scale. It's great for a table with just 10 emails in it, but once you start getting into the thousands you're going to run into problems, mostly with the memory consumption of Ruby.
One can get around this little problem by using connection.select_values on a model, and a little bit of ARel goodness:
User.connection.select_values(User.select("email").to_sql)
This will give you the straight strings of the email addresses from the database. No faffing about with user objects and will scale better than a straight User.select("email") query, but I wouldn't say it's the "best scale". There's probably better ways to do this that I am not aware of yet.
The point is: a String object will use way less memory than a User object and so you can have more of them. It's also a quicker query and doesn't go the long way about it (running the query, then mapping the values). Oh, and map would also take longer too.
If you're using Rails 2.3...
Then you'll have to construct the SQL manually, I'm sorry to say.
User.connection.select_values("SELECT email FROM users")
Just provides another example of the helpers that Rails 3 provides.
I still find the connection.select_values to be a valid way to go about this, but I recently found a default AR method that's built into Rails that will do this for you: pluck.
In your example, all that you would need to do is run:
User.pluck(:email)
The select_values approach can be faster on extremely large datasets, but that's because it doesn't typecast the returned values. E.g., boolean values will be returned how they are stored in the database (as 1's and 0's) and not as true | false.
The pluck method works with ARel, so you can daisy chain things:
User.order('created_at desc').limit(5).pluck(:email)
User.select(:email).map(&:email)
Just use:
User.select("email")
While I visit SO frequently, I only registered today. Unfortunately that means that I don't have enough of a reputation to leave comments on other people's answers.
Piggybacking on Ryan's answer above, you can extend ActiveRecord::Base to create a method that will allow you to use this throughout your code in a cleaner way.
Create a file in config/initializers (e.g., config/initializers/active_record.rb):
class ActiveRecord::Base
def self.selected_to_array
connection.select_values(self.scoped)
end
end
You can then chain this method at the end of your ARel declarations:
User.select('email').selected_to_array
User.select('email').where('id > ?', 5).limit(4).selected_to_array
Use this to get an array of all the e-mails:
#users.collect { |user| user.email }
# => ["test#example.com", "test2#example.com", ...]
Or a shorthand version:
#users.collect(&:email)
You should avoid using User.all.map(&:email) as it will create a lot of ActiveRecord objects which consume large amounts of memory, a good chunk of which will not be collected by Ruby's garbage collector. It's also CPU intensive.
If you simply want to collect only a few attributes from your database without sacrificing performance, high memory usage and cpu cycles, consider using Valium.
https://github.com/ernie/valium
Here's an example for getting all the emails from all the users in your database.
User.all[:email]
Or only for users that subscribed or whatever.
User.where(:subscribed => true)[:email].each do |email|
puts "Do something with #{email}"
end
Using User.all.map(&:email) is considered bad practice for the reasons mentioned above.