I am currently working on an application where I scrape information from a number of different sites. To get the deeplink for the desired topic on a site I rely on the sitemap that is provided (e.g. "Forum"). As I am expanding I came across some sites that don't provide a sitemap themselves, so I was wondering if there was any way to generate it within Rails from the top level domain?
I am using Nokogiri and Mechanize to retrieve data, so if there is any functionality that could help to tackle that task it would be easier to integrate.
This can be done with the Spidr gem like so:
url_map = Hash.new { |hash,key| hash[key] = [] }
Spidr.site('http://intranet.com/') do |spider|
spider.every_link do |origin,dest|
url_map[dest] << origin
end
end
Related
I have a rails app using the ruby-graphql and graphql-fragment_cache gem and a query for pages:
field :all_pages, PageType.connection_type, null: true
When I resolve and cache it using the graphql-fragment_cache gem all the pages get cached - which makes sense of course.
def all_pages
cache_fragment {
chain = Page
chain = scope_status(chain, context[:stage])
chain = lookahead_for_collection(chain, lookahead)
chain = search_records(chain, search)
chain = order_records(chain, order)
chain.all
}
end
I am using the api in a Gatsby website. So whenever there is an update in my rails app a webhook gets posted to Gatsby Cloud to rebuild the site. This rebuild will fetch all pages with all available fields which is very heavy. Also because they are a lot.
Is there an option to cache all pages but only invalidate a single one (similar to rails.cache.fetch_multi) so that all pages that did not change get fetched from the cache except the one that changed? This would speed up my app significantly.
Thank you!
As i mentioned on the question title, I want to know the best approach to get this to work and i'll need and example ( very simple one as the follow: )
Let's say i have an API which has 1 controller and 1 action for example simplicity
root 'main#index'
and inside the index action i have
def index
#date = Data.today
end
Then i have another rails app which will work for front-end rendering
How can i pass this #date as JSON from the API to the other app to render it ?
Should i have same controller on the other app ?
How can i connect and send http request and receive response ?
Thanks in advance
For such a simple example, you can do something as simple as:
def index
#date = Date.today
respond_to do |format|
format.json #date
end
end
However, you're most likely going to want to deal with more complicated JSON responses, so before long you'll probably want to use something like the Jbuilder gem or ActiveModel Serializers (my preferred approach).
On the other end, your front-end will need to make an HTTP GET request. Lots of ways (and gems) to do this, but one common approach is just to use the built in Net::HTTP class.
require 'net/http'
url = URI.parse('http://backend.dev/main/index')
request = Net::HTTP::Get.new(url.to_s)
response = Net::HTTP.start(url.host, url.port) do |http|
http.request(request)
end
raise response.body.inspect
In your situation, a better approach might be to use the Active Resource gem. This gem allows you to create models that are backed by a REST API rather than a database. For example, if your API app provides basic Create-Read-Update-Destroy actions for a particular model (let's call it Widget) at the following URLs:
GET http://backend.dev/widget # listing of widgets
GET http://backend.dev/widget/1 # Read for widget id: 1
POST http://backend.dev/widget # Create new widget
UPDATE http://backend.dev/widget/1 # Update widget id: 1
DELETE http://backend.dev/widget/1 # Destroy widget id: 1
then in your front-end app you could declare an Active Resource like this:
class Widget < ActiveResource::Base
self.site = "http://backend.dev"
end
which will auto-magically access all of those methods in your API, and behave much like a regular Active Record model. That way, you basically design your front-end app like a "normal" rails app, but using ActiveResource-based models in place of ActiveRecord.
I would note, however, that a more common thing to do these days would be to build your API in Rails, and build your front-end with client-side Javascript, using something like JQuery or Angular to make requests from the API. I'm not sure what you're gaining by splitting API and front-end, where both of them are Rails apps - unless you've got a compelling reason, I'd just build one Rails app that handles both API and front-end, or build a Rails API + Angular (or similar) front-end.
I'm a Rails noob, but I'd like to use it as a backend for an Ember application with Ember Data. Unfortunately, I have some unknown unknowns.
The RESTAdapter documentation says:
Comments for a post can be loaded by post.get('comments'). The REST
adapter will send a GET request to /comments?ids[]=1&ids[]=2&ids[]=3.
It will generate similar urls if you use something like App.Post.find({title: "Some Title"}), in about the format you'd expect: /posts?title=Some+Title
Is there some option, or gem I can use to handle that sort of simple query, or do I have to go parse parameters in my controllers manually?
To clarify, I'm aware that I can tell my Rails controller to return a set like:
#comments = Comment.find(params[:ids])
respond_with(#comments)
But it seems like querying on ids or accessible attributes like that would be a common enough use case for REST APIs that something would be built in, or have a gem written to handle it.
Can anyone point me in the right direction?
Thanks.
This might be helpful in your case:
https://github.com/ernie/ransack/
Or
https://github.com/ernie/squeel
How has Github managed to get friendly URLs for representing repos of users? For a project called abc by username foo, how do they work around with a URL like: http://github.com/foo/abc. Are they fetching the abc model for the DB from the title in the URL (which sounds unreasonable as they are modifying the titles). How are they transferring the unique ID of the abc repo which they can fetch and show in the view?
The reason I ask is that I am facing a similar problem of creating friendlier URLs to view a resource. MongoDB's object IDs are quite long and make the URL look horrific. Is there a workaround? All the tutorials that demonstrate CRUD (or REST) URLs for a resource always include the object's unique ID(e.g. http://mysite.org/post/1 or http://mysite.org/post/1/edit. Is there a better way to do it?
Not having seen their code, I couldn't tell you exactly how they do it, but if you're using Rails there are at least two Ruby gems that will give you similar results:
Take a look at Slugged and friendly_id
http://github.com/foo/abc is a unique repository identifier (for that repo's master branch). I'd assume that somewhere they have a table that looks like:
repository-id | user-id | project-id
and are just looking up based on user and project rather than repository-id.
You'd need to do some domain-specific mapping between internal and user-friendly ids, but you'd need to make sure that was a 1:1 mapping.
See this rails cast on methods, gems and solutions to common problems you might get while modifying the application to use friendly urls.
http://railscasts.com/episodes/314-pretty-urls-with-friendlyid?view=asciicast
(although Ryan Bates deserves the rep+ for this)
I mocked a structure like this using FriendlyID and Nested Resources.
Essentially, use friendly ID to get the to_param-ish slugs in your routes, then set up nested resources. Using GitHub as an example:
routes.rb
resources :users do
resources :repositories
end
Then in your controller, say, for repositories, you can check the existence of params[:user_id] and use that to determine the user from the route. The reason I check for existence is because I did something like (roughly):
/myrepositories/:repository_id
/:user_id/:repository_id
So my controller does:
def show
#user = params[:user_id] ? User.find(params[:user_id]) : current_user
end
I followed this tutorial here to get started with this same project.
This is called URL rewriting if the web server does it (such as Apache), and routing when it happens in a web application framework (such as Ruby on Rails).
http://www.sinatrarb.com/intro#Routes
http://httpd.apache.org/docs/current/mod/mod_rewrite.html
Currently developing a RoR mobile web application for displaying information about particular places.
Let's call these places 'installations'. Each installation has at least one 'venue', which is a specific instance of the installation. So for example an installation could be a city park, and one of its corresponding venues could be the park during a concert or event. If this example is confusing, disregard, it doesn't matter much.
The data for these installations and their corresponding venues is hosted on Amazon Web Service S3 as json objects. I use HTTParty to read in the AWS S3 json objects. I am currently reading them in like so:
In the application controller I have:
inlcude HTTParty
base_uri 's3.amazonaws.com/our_bucket'
before_filter :set_up_httparty
def set_up_httparty
#installation = "INSTALLATION"
#venue = "VENUE"
#dir_url = "/#{#installation}/#{#venue}/"
end
In my corresponding controllers, where I get separate information I have:
response =
ApplicationController.get("#{#dir_url}/ConstantUniqueName.json")
Currently I am hardcoding INSTALLATION and VENUE values to point to a specific set of folders. This works great, but I need to be able to take in these values from the url like so:
www.themobilewebapp.com/INSTALLATION/VENUE/index
Now I am sure it is most likely possibly to pass in these values with params like so:
www.themobilewebapp.com/index?installtion=INSTALLATION&venue=VENUE
But if possible I would like to set my URL's up the previous way. I need to have the URL's be friendly for users to put in themselves and have QRC codes point directly to a specific installation and corresponding venue.
If not possible to be able to do something like www.mobwebapp.com/INSTALL/VEN/index then is it possible to set up sub domains for each venue and then pull in that venue as a string?
Any help is appreciated!! Thanks!
If you put this in your config/routes.rb file then you will be able to use the URL pattern that you want:
match ':installation/:venue/index', :controller => :venue, :action => :index