I have a Rails web application. I want to create a class that takes an email address, say "matt#trucksandstuff.com," parses out the domain, and then checks if the domain is found in the Spamhaus DBL. I am having no luck with the dig or host commands as described on their website and the Charon gem doesn't seem to work with their sample URL either. Any ideas?
EDIT: Here is what is on the website:
In response to "How can I test the DBL?" they said:
First, the DBL follows RFC5782 for determining whether a URI zone is operational with an entry for TEST. Second, the DBL has a specific domain for testing DBL applications: dbltest.com. To test functionality of the DBL use the host or dig command to do a manual query. (If you need to look up a domain in the DBL via the web, use the domain lookup form at our Blocklist Removal Center. Do not query our website with automated tools.).
I have tried using the Charon gem, which I think should be as simple as running
Charon.query('dbltest.com')
with variations that remove the parentheses, add a space, etc.
Also tried
resolver = Resolv::DNS.new
name = 'dbltest.com'
resolver.getresources("#{name}.zen.spamhaus.org", Resolv::DNS::Resource::IN::A)
in the Rails console.
The Zen database is only for IP addresses. The DBL list is for hostnames. Therefore Charon (Zen query) only works with IP addresses. To test hostnames, query them with Resolv and dbl.spamhouse.org:
def is_spammer?(host)
!Resolv::DNS.new.getresources("#{host}.dbl.spamhaus.org",
Resolv::DNS::Resource::IN::A).empty?
end
is_spammer?('dbltest.com')
=> true
is_spammer?('google.com')
=> false
Related
I'm a little bit stuck with implementing ipsets for iptables with Chef using data bags. I know you may say that this solution is not elegant and ideal, but believe me I have my own reasons why. What I'm trying to achieve; I need to create the ip set "allowed_subnet" for future using with iptables for whitelisting some ip addresses. The "allowed" ip addresses are in the data bag. Unfortunately I could not find that Chef supports ipset resource so I have to use execute. Please correct me if I'm wrong.
Right, I have data bag with the IP list:
{
"id": "ipset_for_iptables",
"ip_list": [
"1.1.1.1",
"1.1.1.2",
"1.1.1.3",
"1.1.1.4"
]
}
Data bag name is equal to the "id".
And I have my default recipe file default.rb where I've added the following code:
package 'ipset'
execute 'create timeout ipset' do
command 'ipset create allow_selected hash:ip timeout 120'
not_if 'ipset -L allow_selected'
end
execute 'create ipset' do
command 'ipset create allowed_subnet hash:ip hashsize 8192'
not_if 'ipset -L allowed_subnet'
end
servers = data_bag('ipset_for_iptables' , 'ipset_for_iptables')
template "/opt/data/data_hosts.txt" do
source 'ipset.erb'
owner 'ipset'
group 'ipset'
action :create
variables :properties => servers['ip_list']
end
And now, my question is: How to add the IP addresses from the data bag to the ip set "allowed_subnet" using "execute" and "ipset" linux command.
Here is the template "ipset.erb" content:
<% #properties.each do |host|%>
<%= host['ipaddress'] %>
<% end %>
BTW, I'm not sure that this template is correct, this is legacy from a previous admin.
I would really appreciate if somebody can help me and also point me to the right documentation which can help me in a future as I have a lot of inherited stuff like this in my zoo. I have tried to find how to do that reading Chef official documentation, but I guess it is something beyond the Chef itself and more Ruby stuff.
I have a Rails app on a Postgres database and I need to have a search field for the user to enter a string and look up in the database for possible address matches (within a city). In the database I have a column with full addresses.
I cannot make assumptions on the input, so I am thinking that I should first try to directly look up the address on the database somehow (using a LIKE query maybe?), and if that fails, request to a Geocoding API (i.e. Google) to return a well formatted addresses list matching the query and search those in my database.
I would appreciate any guidance on how to do this.
I don't think FTS (full text search) is what you want. You'll have to use an address API that can match addresses.
I've successfully and easily used SmartyStreets for something like this. They have a free account you can use.
http://smartystreets.com
Also if you did want to try going down the FTS route here is a Gist that explains how to do it.
https://gist.github.com/4365593
You may know it already, but postresql has a fulltext search engine integrated so it's a great time to take advantage of it. I suggest watching thats excellent railscast.
Then once implemented :
class Place < AR
def search_db_or_geokit(query)
res = db_search()
if res.empty?
res = geokit_search(query)
else
res
end
end
def geokit_search(query)
# ...
end
def db_search(query)
# ...
end
end
For the geocoding google search api there's probably a good gem out there like geokit
I am using django multiple DB router concepts, having multiple sites with different db's. Base database user will login with all other sub sites.
When i try syncdb in base site its worked properly(at any time), but trying syncdb with other sites works first time only, if we try next time on-wards it throws integiry error like below
django.db.utils.IntegrityError: (1062, "Duplicate entry
'22-add_somesame' for key 'content_type_id'")
Once i removed multiple DB router settings in that project means syncdb works properly(at any time).
So is this relates to multiple db router? or what else?
Please anyone advise on this, thanks.
The problem here is with the db router and django system objects. I've experienced the same issue with multiple DBs and routers. As I remember the problem here is with the auth.permission content types, which get mixed in between databases. The syncdb script otherwise tries to create these in all databases, and theb it creates permission content type for some object, which id is already reserved for a local model.
I have the following
BASE_DB_TYPES = (
'auth.user',
'auth.group',
'auth.permission',
'sessions.session',
)
and then in the db router:
def db_for_read(self, model, **hints):
if hasattr(model, '_meta') and str(model._meta) in BASE_DB_TYPES:
return 'base_db' # the alias of base db that will store users
return None # the default database, or some custom mapping
EDIT:
Also, the exception might say that you're declaring a permission 'add_somesame' for your model 'somesame', while Django automatically creates add_, delete_, edit_ permissions for all objects.
I have a website which is www.abrisud.com. This website has 7 domain names (one for each language): abrisud.com, abrisud.it, abrisud.de, etc... and abrisud-enclosure.co.uk.
The problem is on the last one: I am losing my session on every single request. Each Time I load a page I have a different session ID. On the other domains everything is working just fine.
The website is running ruby 1.8.7 and rails 3.0.0.
I am really convinced that the problem comes from the "-" in the domain name but I just can't find anything (or almost anything) on the subject through the web.
Hopefully I am being clear enough, if not just tell me.
Here is the answer :
From Module ActionDispatch::Http::URL (Rails 3.0.x), be sure to read the comments ;-)
# Returns the \domain part of a \host, such as "rubyonrails.org" in "www.rubyonrails.org".
# You can specify a different <tt>tld_length</tt>, such as 2 to catch rubyonrails.co.uk in "www.rubyonrails.co.uk".
def domain(tld_length = 1)
return nil unless named_host?(host)
host.split('.').last(1 + tld_length).join('.')
end
Well, calling the domain method with the appropriate _tld_lenght_ argument did not play the trick, the request.domain (abrisud-enclosure.co.uk) was good, but not the session_domain (still co.uk).
So I had to add the following lines as a before filter to my application_controller :
def set_session_domain
request.session_options[:domain] = request.domain
end
If you have a better solution I am open to it as I think this is a really dirty fix.
Thanks
I have taken a peak at your site, the cookie is set with: domain=co.uk;path=/
So the problem is within your rails stack and not the browser(s) - time to do some debugging :-)
I am using Ruby on Rails 3 and a MYSQL database. I would like to retrieve a regex from the database and then use that value to validate email addresses.
I aim to not put the regex value in line in my RoR application code, but outside so that the value can be recalled for other usages and from other places.
In order to populate the database, I put in my 'RAILS_ROOT/db/seed.rb' the following:
Parameter.find_or_create(
:param_name => 'email_regex',
:param_value => "[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?",
)
Notice: in the 'seed.rb' file I edited a little bit the original regex from www.regular-expressions.info adding two \ just before $. Here it is the difference:
#original from www.regular-expressions.info
[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?
#edited by me
[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?
After run rake db:seed in the Terminal, in MYSQL database I have this value (without \ near $):
[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?
Then in my RoR application I use the regex this way:
def validate(string)
email_regex = Regexp.new(Parameter.find_by_param_name('email_regex').param_value)
if email_regex.match(string)
return true
else
return false
end
end
The problem using the above regex is that I can successfully validate also email addresses with double '#' or without the final part like these:
name#surname#gmail.com # Note the double '#'
test#gmail
Of course I would like to refuse those email addresses. So, how can I adjust that? Or, how can I get what I want?
I tried also to seed these regex:
#case 1
\A[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\Z
#case 2
\\A[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\\Z
#case 3
^[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$
that in the MYSQL database become respectively:
#case 1
A[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?Z
#case 2
\A[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\Z
#case 3
^[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$
but also them don't work as expected.
UPDATE
Debugging I have
--- !ruby/regexp /[a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:.[a-z0-9!#$%&'*+\/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?/
that means that just before of all / characters Ruby added a \ character. Can be that my problem? In the 'seed.rb' file I tryed to escape all / adding \ statements but the debug output is always the same.
There are so many things wrong on so many levels here…
Storing application configuration in your database isn't recommended; slower performance, potential catch 22s (like how do you configure your database, from your database), etc. Try something like SettingsLogic if you don't want to have to build your own singleton configuration or use an initializer.
Rails has built in validation functionality as a mixin that's automatically part of any models inheriting from ActiveRecord::Base. You should use it, rather than define your own validation routines, especially for basic cases like this.
You can actually have an email address with multiple # signs, provided the first is escaped with a backslash or the local portion of the address is quoted.
Why are you escaping $ characters in a character class where they have no special meaning?
Regular expressions are okay for a very basic validation of an email address to make sure you didn't get complete garbage data to pass off to your mail server, but they aren't the best way to verify an email address.
I suggest you have a good look at the validations guide at RubyOnRails.org.
You shouldn't reinvent this wheel. See http://lindsaar.net/2010/1/31/validates_rails_3_awesome_is_true for a standard way to validate email addresses in Rails 3.
If you do choose to reinvent the wheel, don't use a regular expression. The gory details of why this is a bad idea are explained in http://oreilly.com/catalog/9780596528126, along with a very, very complicated regular expression that almost does it.