As a beginner to rails, I'm finding the generation of sitemaps on Heroku to be extremely daunting due to its read-only limitations. However, a sitemap is fundamental to my website as its success is based on SEO.
I have tried dynamic_sitemaps gem however soon removed it as I realised it had no documentation for heroku use. I then used the sitemap_generator gem which had coverage of heroku integration using several gems and external platforms such as Amazon S3. The problem however is that as a beginner I'm running into issues and finding it hard to get past them.
Is there a solution I can use easily generate sitemaps for consistent content such as blog posts on the heroku platform? I really want to get up and running and feel this could take a while to configure if I have to use the methods I've already attempted.
Thanks!
I figured out a small trick that makes it possible to dynamically generate the sitemap file but persist it for later calls on Heroku.
It works great for small\medium size projects, if you have a big\huge project and thousand of pages that changes endlessly , please consider using S3 to store the sitemap file.
Those are the steps:
use the sitemap_generator gem as instructed here https://github.com/kjvarga/sitemap_generator
after bundle, run rake sitemap:install, it will create a config/sitemap.rb file for you
edit the config/sitemap.rb file to look like this
SitemapGenerator::Sitemap.default_host = [your host name goes here]
SitemapGenerator::Sitemap.public_path = File.join(Rails.root, 'tmp').to_s
SitemapGenerator::Sitemap.compress = false
SitemapGenerator::Sitemap.create do
[all your site pages add commands goes here]
end
regarding the host name, I suggest it to be "#{ENV['HOST_PROTOCOL']}://#{ENV['HOST_NAME']}" (and of course add the appropriate environment variables) so you could change it on different environments.
regarding compress, start with false, make sure all is working great for you and change it later if it is a big file.
create your sitemap controller file - app/controllers/sitemap_controller.rb
Edit the sitemap controller file to look like this
require 'rake'
class SitemapController < ApplicationController
def index
file_name = File.join(Rails.root, 'tmp', 'sitemap.xml').to_s
unless File.exist?(file_name)
Rails.application.load_tasks
Rake::Task['sitemap:refresh:no_ping'].invoke
end
# it's better to be safe than sorry
if File.exist?(file_name)
respond_to do |format|
format.xml { render file: file_name }
end
else
render file: 'public/404.html', status: :not_found, layout: false
end
end
end
Add the index action to your routes.rb file
resources :sitemap, only: %i[index], constraints: ->(req) { req.format == :xml }
restart/deploy you server and go to /sitemap.xml
Enjoy 😊
Have taken a look to Dynamic Site Maps Gem this is really simple to set up just read the read me in the Github for more features you can also look at Site Map Generator gem
Wish you the best of luck
try like this
Controller
class SitemapController < ApplicationController
layout nil
def index
#static_pages = [jobs_url, advertising_url, join_url]
#offers = Offer.all
respond_to do |format|
format.xml
end
end
end
View
xml.instruct!
xml.urlset(
) do
#static_pages.each do |page|
xml.url do
xml.loc "#{page}"
xml.changefreq("monthly")
end
end
#offers.each do |offer|
xml.url do
xml.loc
xml.changefreq("daily")
end
end
end
Route
get 'sitemap.xml', :to => 'sitemap#index', :defaults => { :format => 'xml' }
I want to be able to upload an Excel file that contains contact information. I then went to be able to parse it and create records for my Contact model.
My application is a Rails application.
I am using the paperclip gem on heroku, I've been able to parse vim cards into the Contact model, and am looking for something similar, but will go through all lines of the Excel file.
Gems that simplify the task and sample code to parse would be helpful!
Spreadsheet is the best Excel parser that I have found so far. It offers quite a lot of functionality.
You say you use Paperclip for attachments which is good. However, if you store the attachments in S3 (which I assume since you use Heroku) the syntax for passing the file to spreadsheet is a little different but not difficult.
Here is an example of the pure syntax that can be used and not placed in any classes or modules since I don't know how you intend to start the parsing of contacts.
# load the gem
require 'spreadsheet'
# In this example the model MyFile has_attached_file :attachment
#workbook = Spreadsheet.open(MyFile.first.attachment.to_file)
# Get the first worksheet in the Excel file
#worksheet = #workbook.worksheet(0)
# It can be a little tricky looping through the rows since the variable
# #worksheet.rows often seem to be empty, but this will work:
0.upto #worksheet.last_row_index do |index|
# .row(index) will return the row which is a subclass of Array
row = #worksheet.row(index)
#contact = Contact.new
#row[0] is the first cell in the current row, row[1] is the second cell, etc...
#contact.first_name = row[0]
#contact.last_name = row[1]
#contact.save
end
I had a similar requirement in one of my Rails 2.1.0 application. I solved it in the following manner:
In the 'lib' folder I wrote a module like this:
require 'spreadsheet'
module DataReader
def read_bata(path_to_file)
begin
sheet = book.worksheet 0
sheet.each 2 do |row|
unless row[0].blank?
# Create model and save it to DB
...
end
end
rescue Exception => e
puts e
end
end
end
Had a model Upload:
class Upload < AR::Base
has_attached_file :doc,
:url => "datafiles/:id",
:path => ":rails_root/uploads/:id/:style/:basename.:extension"
# validations, if any
end
Generated an UploadsController which would handle the file upload and save it to appropriate location. I used Paperclip for file upload.
class UploadsController < AC
include DataReader
def new
#upload = Upload.new
end
def create
#upload = Upload.new(params[:upload])
#upload.save
file_path = "uploads/#{#upload.id}/original/#{#upload.doc_file_name}"
#upload.read = DataReader.read_data(file_path)
# respond_to block
end
end
Read about 'spreadsheet' library here and here. You can make appropriate improvements and make the technique work in Rails 3. Hope this helps.
I made a gem to achieve this easily. I called it Parxer and...
It's built on to of roo gem.
It allows you to parse xls, xlsx and csv files.
Has a DSL to handle:
Column mapping.
File, row, and column/cell validation.
Column/cell formatting.
I'm using this code (taken from here) in ApplicationController to detect iPhone, iPod Touch and iPad requests:
before_filter :detect_mobile_request, :detect_tablet_request
protected
def detect_mobile_request
request.format = :mobile if mobile_request?
end
def mobile_request?
#request.subdomains.first == 'm'
request.user_agent =~ /iPhone/ || request.user_agent =~ /iPod/
end
def detect_tablet_request
request.format = :tablet if tablet_request?
end
def tablet_request?
#request.subdomains.first == 't'
request.user_agent =~ /iPad/
end
This allows me to have templates like show.html.erb, show.mobile.erb, and show.tablet.erb, which is great, but there's a problem: It seems I must define every template for each mime type. For example, requesting the "show" action from an iPhone without defining show.mobile.erb will throw an error even if show.html.erb is defined. If a mobile or tablet template is missing, I'd like to simply fall back on the html one. It doesn't seem too far fetched since "mobile" is defined as an alias to "text/html" in mime_types.rb.
So, a few questions:
Am I doing this wrong? Or, is there a better way to do this?
If not, can I get the mobile and tablet mime types to fall back on html if a mobile or tablet file is not present?
If it matters, I'm using Rails 3.0.1. Thanks in advance for any pointers.
EDIT: Something I forgot to mention: I'll eventually be moving to separate sub-domains (as you can see commented out in my example) so the template loading really needs to happen automatically regardless of which before_filter has run.
Possible Duplicate of Changing view formats in rails 3.1 (delivering mobile html formats, fallback on normal html)
However, I struggled with this exact same problem and came up with a fairly elegant solution that met my needs perfectly. Here is my answer from the other post.
I think I've found the best way to do this. I was attempting the same thing that you were, but then I remembered that in rails 3.1 introduced template inheritance, which is exactly what we need for something like this to work. I really can't take much credit for this implementation as its all laid out there in that railscasts link by Ryan Bates.
So this is basically how it goes.
Create a subdirectory in app/views. I labeled mine mobile.
Nest all view templates you want to override in the same structure format that they would be in the views directory. views/posts/index.html.erb -> views/mobile/posts/index.html.erb
Create a before_filter in your Application_Controller and do something to this effect.
before_filter :prep_mobile
def is_mobile?
request.user_agent =~ /Mobile|webOS|iPhone/
end
def prep_mobile
prepend_view_path "app/views/mobile" if is_mobile?
end
Once thats done, your files will default to the mobile views if they are on a mobile device and fallback to the regular templates if a mobile one is not present.
You need to do several things to wire this up, but the good news is that Rails 3 actually makes this a lot simpler than it used to be, and you can let the router do most of the hard work for you.
First off, you need to make a special route that sets up the correct mime type for you:
# In routes.rb:
resources :things, :user_agent => /iPhone/, :format => :iphone
resources :things
Now you have things accessed by an iphone user agent being marked with the iphone mime type. Rails will explode at you for a missing mime type though, so head over to config/initializers/mime_types.rb and uncomment the iphone one:
Mime::Type.register_alias "text/html", :iphone
Now you're mime type is ready for use, but your controller probably doesn't yet know about your new mime type, and as such you'll see 406 responses. To solve this, just add a mime-type allowance at the top of the controller, using repsond_to:
class ThingsController < ApplicationController
respond_to :html, :xml, :iphone
Now you can just use respond_to blocks or respond_with as normal.
There currently is no API to easily perform the automatic fallback other than the monkeypatch or non-mime template approaches already discussed. You might be able to wire up an override more cleanly using a specialized responder class.
Other recommended reading includes:
https://github.com/plataformatec/responders
http://www.railsdispatch.com/posts/rails-3-makes-life-better
Trying removing the .html from the .html.erb and both iPhone and browser will fallback to the common file.
I have added a new answer for version 3.2.X. This answer is valid for <~ 3.0.1.
I came to this question while looking to be able to have multiple fallbacks on the view. For example if my product can be white-labeled and in turn if my white-label partner is able to sell sponsorship, then I need a cascade of views on every page like this:
Sponsor View: .sponsor_html
Partner View: .partner_html
Default View: .html
The answer by Joe, of just removing .html works (really well) if you only have one level above the default, but in actual application I needed 5 levels in some cases.
There did not seem to be anyway to implement this short of some monkey patching in the same vein as Jeremy.
The Rails core makes some fairly wide ranging assumptions that you only want one format and that it maps to a single extension (with the default of NO extension).
I needed a single solution that would work for all view elements -- layouts, templates, and partials.
Attempting to make this more along the lines of convention I came up with the following.
# app/config/initializers/resolver.rb
module ActionView
class Base
cattr_accessor :extension_fallbacks
##extension_fallbacks = nil
end
class PathResolver < Resolver
private
def find_templates_with_fallbacks(name, prefix, partial, details)
fallbacks = Rails.application.config.action_view.extension_fallbacks
format = details[:formats].first
unless fallbacks && fallbacks[format]
return find_templates_without_fallbacks(name, prefix, partial, details)
end
deets = details.dup
deets[:formats] = fallbacks[format]
path = build_path(name, prefix, partial, deets)
query(path, EXTENSION_ORDER.map {|ext| deets[ext] }, details[:formats])
end
alias_method_chain :find_templates, :fallbacks
end
end
# config/application.rb
config.after_initialize do
config.action_view.extension_fallbacks = {
html: [:sponsor_html, :partner_html, :html],
mobile: [:sponsor_mobile, :partner_mobile, :sponsor_html, :partner_html, :html]
}
# config/initializers/mime_types.rb
register_alias 'text/html', :mobile
# app/controllers/examples_controller.rb
class ExamplesController
respond_to :html, :mobile
def index
#examples = Examples.all
respond_with(#examples)
end
end
Note: I did see the comments around alias_method_chain, and initially did make a call to super at the appropriate spot. This actually called ActionView::Resolver#find_templates (which raises a NotImplemented exception) rather than the ActionView::PathResolver#find_templates in some cases. I wasn't patient enough to track down why. I suspect its because of being a private method.
Plus, Rails, at this time, does not report alias_method_chain as deprecated. Just that post does.
I do not like this answer as it involves some very brittle implementation around that find_templates call. In particular the assumption that you only have ONE format, but this is an assumption made all over the place in the template request.
After 4 days of trying to solve this and combing through the whole of the template request stack its the best I can come up with.
The way that I'm handling this is to simply skip_before_filter on those requests that I know I want to render the HTML views for. Obviously, that will work with partials.
If your site has a lot of mobile and/or tablet views, you probably want to set your filter in ApplicationController and skip them in subclasses, but if only a few actions have mobile specific views, you should only call the before filter on those actions/controllers you want.
If your OS has symlinks you could use those.
$ ln -s show.html.erb show.mobile.erb
I am adding another answer now that we have updated to 3.2.X. Leaving the old answer as it was in case someone needs that one. But, I will edit it to direct people to this one for current versions.
The significant difference here is to make use of the "new" (since 3.1) availability of adding in custom path resolvers. Which does make the code shorter, as Jeroen suggested. But taken a little bit further. In particular the #find_templates is no longer private and it is expected that you will write a custom one.
# lib/fallback_resolver.rb
class FallbackResolver < ::ActionView::FileSystemResolver
def initialize(path, fallbacks = nil)
#fallback_list = fallbacks
super(path)
end
def find_templates(name, prefix, partial, details)
format = details[:formats].first
return super unless #fallback_list && #fallback_list[format]
formats = Array.wrap(#fallback_list[format])
details_copy = details.dup
details_copy[:formats] = formats
path = Path.build(name, prefix, partial)
query(path, details_copy, formats)
end
end
# app/controllers/application_controller.rb
class ApplicationController < ActionController::Base
append_view_path 'app/views', {
mobile: [:sponsor_mobile, :mobile, :sponsor_html, :html],
html: [:sponsor_html, :html]
}
respond_to :html, :mobile
# config/initializers/mime_types.rb
register_alias 'text/html', :mobile
Here's a simpler solution:
class ApplicationController
...
def formats=(values)
values << :html if values == [:mobile]
super(values)
end
...
end
It turns out Rails (3.2.11) adds an :html fallback for requests with the :js format. Here's how it works:
ActionController::Rendering#process_action assigns the formats array from the request (see action_controller/metal/rendering.rb)
ActionView::LookupContext#formats= gets called with the result
Here's ActionView::LookupContext#formats=,
# Override formats= to expand ["*/*"] values and automatically
# add :html as fallback to :js.
def formats=(values)
if values
values.concat(default_formats) if values.delete "*/*"
values << :html if values == [:js]
end
super(values)
end
This solution is gross but I don't know a better way to get Rails to interpret a request MIME type of "mobile" as formatters [:mobile, :html] - and Rails already does it this way.
Yes, I'm pretty sure this is the right way to do this in rails. I've defined iphone formats this way before. That's a good question about getting the format to default back to :html if a template for iphone doesn't exist. It sounds simple enough, but I think you'll have to add in a monkeypath to either rescue the missing template error, or to check if the template exists before rendering. Take a look a the type of patches shown in this question. Something like this would probably do the trick (writing this code in my browser, so more pseudo code) but throw this in an initializer
# config/initializers/default_html_view.rb
module ActionView
class PathSet
def find_template_with_exception_handling(original_template_path, format = nil, html_fallback = true)
begin
find_template_without_exception_handling(original_template_path, format, html_fallback)
rescue ActionView::MissingTemplate => e
# Template wasn't found
template_path = original_template_path.sub(/^\//, '')
# Check to see if the html version exists
if template = load_path["#{template_path}.#{I18n.locale}.html"]
# Return html version
return template
else
# The html format doesn't exist either
raise e
end
end
end
alias_method_chain :find_template, :exception_handling
end
end
Here is another example of how to do it, inspired by Simon's code, but a bit shorter and a bit less hacky:
# application_controller.rb
class ApplicationController < ActionController::Base
# ...
# When the format is iphone have it also fallback on :html
append_view_path ExtensionFallbackResolver.new("app/views", :iphone => :html)
# ...
end
and somewhere in an autoload_path or explicitly required:
# extension_fallback_resolver.rb
class ExtensionFallbackResolver < ActionView::FileSystemResolver
attr_reader :format_fallbacks
# In controller do append_view_path ExtensionFallbackResolver.new("app/views", :iphone => :html)
def initialize(path, format_fallbacks = {})
super(path)
#format_fallbacks = format_fallbacks
end
private
def find_templates(name, prefix, partial, details)
fallback_details = details.dup
fallback_details[:formats] = Array(format_fallbacks[details[:formats].first])
path = build_path(name, prefix, partial, details)
query(path, EXTENSION_ORDER.map { |ext| fallback_details[ext] }, details[:formats])
end
end
The above is still a hack because it is using a private API, but possibly less fragile as Simon's original proposal.
Note that you need to take care of the layout seperately. You will need to implement a method that chooses the layout based on the user agent or something similar. The will only take care of the fallback for the normal templates.
Rails 4.1 includes Variants, this is a great feature that allow you to set different views for the same mime. You can now simply add a before_action and let the variant to do the magic:
before_action :detect_device_variant
def detect_device_variant
case request.user_agent
when /iPad/i
request.variant = :tablet
when /iPhone/i
request.variant = :phone
end
end
Then, in your action:
respond_to do |format|
format.json
format.html # /app/views/the_controller/the_action.html.erb
format.html.phone # /app/views/the_controller/the_action.html+phone.erb
format.html.tablet do
#some_tablet_specific_variable = "foo"
end
end
More info here.
You can in this case for the format to html. By example you want always use the html in user show method
class UserController
def show
..your_code..
render :show, :format => :html
end
end
In this case, if you request show on User controller you render all the time the html version.
If you want render JSON too by example you can made some test about your type like :
class UserController
def show
..your_code..
if [:mobile, :tablet, :html].include?(request.format)
render :show, :format => :html
else
respond_with(#user)
end
end
end
I made a monkey patch for that, but now, I use a better solution :
In application_controller.rb :
layout :which_layout
def which_layout
mobile? ? 'mobile' : 'application'
end
With the mobile? method you can write.
So I have a different layout but all the same views, and in the mobile.html.erb layout, I use a different CSS file.
I need the same thing. I researched this including this stack overflow question (and the other similar one) as well as followed the rails thread (as mentioned in this question) at https://github.com/rails/rails/issues/3855 and followed its threads/gists/gems.
Heres what I ended up doing that works with Rails 3.1 and engines. This solution allows you to place the *.mobile.haml (or *.mobile.erb etc.) in the same location as your other view files with no need for 2 hierarchies (one for regular and one for mobile).
Engine and preparation Code
in my 'base' engine I added this in config/initializers/resolvers.rb:
module Resolvers
# this resolver graciously shared by jdelStrother at
# https://github.com/rails/rails/issues/3855#issuecomment-5028260
class MobileFallbackResolver < ::ActionView::FileSystemResolver
def find_templates(name, prefix, partial, details)
if details[:formats] == [:mobile]
# Add a fallback for html, for the case where, eg, 'index.html.haml' exists, but not 'index.mobile.haml'
details = details.dup
details[:formats] = [:mobile, :html]
end
super
end
end
end
ActiveSupport.on_load(:action_controller) do
tmp_view_paths = view_paths.dup # avoid endless loop as append_view_path modifies view_paths
tmp_view_paths.each do |path|
append_view_path(Resolvers::MobileFallbackResolver.new(path.to_s))
end
end
Then, in my 'base' engine's application controller I added a mobile? method:
def mobile?
request.user_agent && request.user_agent.downcase =~ /mobile|iphone|webos|android|blackberry|midp|cldc/ && request.user_agent.downcase !~ /ipad/
end
And also this before_filter:
before_filter :set_layout
def set_layout
request.format = :mobile if mobile?
end
Finally, I added this to the config/initializers/mime_types.rb:
Mime::Type.register_alias "text/html", :mobile
Usage
Now I can have (at my application level, or in an engine):
app/views/layouts/application.mobile.haml
and in any view a .mobile.haml instead of a .html.haml file.
I can even use a specific mobile layout if I set it in any controller:
layout 'mobile'
which will use app/views/layouts/mobile.html.haml (or even mobile.mobile.haml).
I solved this problem by using this before_filter in my ApplicationController:
def set_mobile_format
request.formats.unshift(Mime::MOBILE) if mobile_client?
end
This puts the mobile format to the front of the list of acceptable formats. So, the Resolver prefers .mobile.erb templates, but will fall back to .html.erb if no mobile version is found.
Of course, for this to work you need to implement some kind of #mobile_client? function and put Mime::Type.register_alias "text/html", :mobile into your config/initializers/mime_types.rb
I did a quick Google search and didn't see anything super-great to automate creation and updating of my google sitemap for the ruby on rails app. Any suggestions?
I really wouldn't recommend using controller and simply routing to it like '/sitemap.xml', because if your sitemaps will grow it will waste resourses. It would be best to make a rake task and regenerate sitemaps every night or so, depending on how frequently the content changes.
If your site does not have anywhere near 50000 pages and you're not planing to have more then you can fit in one sitemap file then I suggest you use "Google Sitemaps with Ruby on Rails, Capistrano, and Cron" I know it works decently because I'm using it sucessfully atm.
Note: I would not suggest using authors way of copying sitemaps every time when deployment task is run, in my opinion it's not clean. It's better to use shared directory and symlink to it on deployment, then you'll be sure to have only one version of files and now wasting space if extra copies get left behind. I have this in my deploy.rb:
desc "Symlink the upload directories"
task :before_symlink do
run "rm -drf #{release_path}/public/sitemaps"
run "ln -s #{shared_path}/sitemaps #{release_path}/public/sitemaps"
end
Also you can use "Big sitemap" gem, but I have found it just now and can't tell you how good it is. It looks like it's really easy to setup this gem and it also uses your rails routes for link generation, so you'll have one problem less to solve than with the first solution.
I added a dynamic sitemap to a blog application just recently. These steps should get you started.
Add this route towards the bottom of your config/routes.rb file (more specific routes should be listed above it):
map.sitemap '/sitemap.xml', :controller => 'sitemap'
Create the SitemapController (app/controllers/sitemap_controller):
class SitemapController < ApplicationController
layout nil
def index
headers['Content-Type'] = 'application/xml'
last_post = Post.last
if stale?(:etag => last_post, :last_modified => last_post.updated_at.utc)
respond_to do |format|
format.xml { #posts = Post.sitemap } # sitemap is a named scope
end
end
end
end
—As you can see, this is for a blog, so is using a Post model.
This is the view template (app/views/sitemap/index.xml.builder):
base_url = "http://#{request.host_with_port}"
xml.instruct! :xml, :version=>'1.0'
xml.tag! 'urlset', 'xmlns' => 'http://www.sitemaps.org/schemas/sitemap/0.9' do
for post in #posts do
xml.tag! 'url' do
xml.tag! 'loc', "#{base_url}#{post.permalink}"
xml.tag! 'lastmod', post.last_modified
xml.tag! 'changefreq', 'monthly'
xml.tag! 'priority', '0.5'
end
end
end
That's it! You can test it by bringing up http://localhost:3000/sitemap.xml (if using Mongrel) in a browser, or perhaps by using cURL.
Note that the controller uses the stale? method to issue a HTTP 304 Not Modified response if there are no new posts sinces the sitemap was last requested.
The Google Sitemap Generator is Google's official release. It's easy to set up and configure.
http://code.google.com/p/googlesitemapgenerator/
I would recommend that you check out the sitemap_generator gem. It handles all of these issues for you...and really, who wants to mess around authoring XML?
Here is an example sitemap to show how you use your Rails models and path helpers to generate your sitemap URLs:
# config/sitemap.rb
SitemapGenerator::Sitemap.default_host = "http://www.example.com"
SitemapGenerator::Sitemap.create do
add '/contact_us'
Content.find_each do |content|
add content_path(content), :lastmod => content.updated_at
end
end
Then you use Rake tasks to refresh as often as you would like. It really is that simple :)