I'm trying to create a rails app that fetches album scores from PitchforkMedia.com
For example, I want to grab the score 8.0 from this album as shown below in the image.
In my brief research, I came across a similar question asked from asp.net point of view.
Generally, the answers suggested
using http request
parse the HTML source
identifying unique flags around the variable score
Some answers came with use of built in asp.net functions.
Are there equivalent functions available in Ruby or Ruby on Rails?
Or can someone suggest some other way doing what I want to do?
Any suggestion will be appreciated. Thank you.
Edit 1: This is completely for personal use and education. Assume that there is no legal ramification for data mining from websites.
I would use Mechanize to scrape the content that you are interested in. Ryan Bates has a great screencast on how to use Mechanize.
You need to make sure you're reading the site's terms of use before you start screen-scraping data, especially if it's not for personal use.
This is a portion of Pitchfork's terms of use, and most other sites have something similar.
"Any reproduction, copying or redistribution of the Materials or design elements of the Website for any purpose are strictly prohibited without the express written consent of Pitchfork or its owner if Pitchfork is not the owner."
Screen scraping definitely falls in this category.
Related
So I looked around and found nothing that helps or clarify this question, suppose you have a ruby on rails (5.2) API and you want to save some URLs as part of a model definition, how to determine that the URL you are saving is safe? and by safe I mean, avoid URLs/links that may contain phishing and deceptive sites.
Thx in advance 👍.
Here is what I found so far, hope this help someone (that have the same requirement):
As pointed by #debugger there are multiple services that provide these functionalities, the best fit in my case are the ones below:
Safe browsing google API no commercial purposes
Web Risk for commercial purposes
The above are google APIs that can be used to check if a URL is safe or not.
In the last case you will be charged after a certain amount of request so maybe is a good idea to check if the URL/link is valid: valid URL gem
I am considering implementing a Facebook-integration to my web app. I want to be able to import friends names, their ages and their interests/likes.
A. First off, is this possible? Can I access this information?
This import will, more or less, be a one-time import.
I also want to be able to use Facebook-login and to use it parallell to my "normal" login (auth) functionality. I assume this should be quite straightforward since most websites have it this way (e.g. Fiverr.com).
B. These two things being my basic needs of my Facebook-integration, which gem would you recommend me to use?
C. I am 1 1/2 years into RoR and consider myself decent at Rails-programming but hardly know any JavaScript and very little jQuery. Will this integration be very difficult for me, you think?
Receommendations of useful blog posts etc will also be appreciated!
A. Check facebook doc on permissions. I'd say you can get a user friends list (id and names), but nothing more : the friends would have to allow your app in order for you to retrieve their data. I've stumbled upon this issue a few weeks ago, but we were retrieving albums and pictures. The data you want is less sensitive, so maybe you can do it anyway. Bottom line : check.
B. I'd suggest using devise for managing everything related to authentication. It is a well known gem, used by many and more. You can add support for facebook via omniauth; there's a wiki page on devise about how to achieve this.
C. Once you get your grasp around the OAuth concepts, you'll be good. You can use facebook connect without javascript/jQuery. Some features though, as the "like button", will require to use the js SDK. Besides these ones, you can do pretty much everything server-side. For more advanced queries, the koala gem can do that.
Hope this is enough for you.
I was hoping to get some advice. I want to create an application which is like a library that holds a lot of different information including documents, pics etc where a user can log in and search for different topics and see the definition of terminology. For example users would be able to click on the word 'electical current' and a definition of electrical current would appear with also links to supporting information. The user will not be allowed to edit any information only read it.
I am wondering what is the best language to use to do this? I know that storing all the information in a html file is definitely not the way to go. But I am not sure what to use - whether html linked to a database is a good option (I've heard that this is also not great because your database code is still accessible) or whether ror is a good option. I decided that ror would be a good framework to use but I still have some doubts such as:
the application will not be hosted on the internet but only on an intranet would ror still be good for this?
the 'site' is basically static except for a few users who will have admin access although it may have more user specific functions later on.
Any advice would be great. Thanks in advance.
It seems to me that a wiki type system would be fit for your purposes. You can create one without writing any code by using a free service such as Google Sites.
I agree with #Finbarr that your problem statement sounds like it could be solved by a wiki.
I suggest mediawiki since it is open source--you can add to it or change the core software if necessary.
Or any other opensource wiki could be used. See wikipedia article
Re: which language: your problem statement is pretty broad. From what you've told us, any language would do.
Best advice: find the open source wiki that most closely meets your needs and then learn the language that it was developed in. Usually that's php or perl. But you can also find Ruby on Rails wikis such as Instiki
Added: Welcome to Stack Overflow. Note that your question is quite broad. So it may get closed for not being specific enough. Stack Overflow is aimed at answering specific questions about software programming. See the faq.
I'm developing a web application using Spring, JSF 2 and Primefaces 3. I want the user to be able to choose from different languages. I don't want to use Google Translate. Which is the best approach to translate my website?
Take a look at http://www.coreservlets.com/JSF-Tutorial/jsf2/#Properties
Basically, you create a properties file for each language, like messages.properties, messages_fr.properties, messages_es.properties etc. and the appropriate one is loaded according to the locale.
The user can also select the language herself, take a look at the next tutorials at that site to see how to implement it.
Are you asking how to implement this technically or how to get the content of your site translated?
For the former I trust you can find a guide or five.
For the latter if you don't want to use an automated service like Google Translate then you're going to need humans to do it for you. Unless you have multilingual friends who owe you a favor or, say, a giant, enthusiastic userbase like Facebook or Twitter has, you're not going to get this for free. Your options, basically, are to pay for the services of one of the many, many companies out there that do this, or find multilingual individuals on your own, e.g. by calling local universities' foreign language departments or international student organizations, or posting on Craigslist and the like. As with web development contractors, I think it's safe to assume you get what you pay for.
When you say you don't want to use Google Translate, do you men GT specifically, or any such service? Because if you're not using any translation service, you'll have to maintain versions of each page on your site in different languages. And maintain them.
Come to think of it, you probably have to take this approach as none of the online translation services provide a good enough native translation.
There's probably some commercial machine-translation packages out there, but they don't come cheap, I imagine.
Alternatively, employ an army of translators - there's websites out there where you can hire translators.
You'll probably want to do a search for translation software, then figure out how much it'd cost you.
I'm curious to learn Ruby and Ruby on Rails. I have previous experience with PHP and some basic C/Objective C and Java.
I have done a couple of 'Get started with Rails' tutorials and want to try do create a complete site now. I do have some questions.
Is the entire website an app? I mean, sure, the simple 'blog-with-comments' examples fit in a single app. But what if you want more? Say, a forum, a blog, community, shop, task manager and so on (only random examples here... :) ).
Should I split these things into different 'apps' or not? I would still need them to communicate with each other and share user data and so on...
Any ideas?
Regards
Linus
In general, your core competency as a rails app -- that which you provide the most value for your users. You can then supplement from there, so your support section might be GetSatisfaction and live on support.yourdomain.com.
Your online shop could be a link to your shopify page at shop.yourdomain.com, etc.