I'm learning rails and would like to make an image scraper for 4chan. I don't really know where to start though, so I was wondering if anyone could point me towards anything that I could look into to make this happen or anything I could study to become familiar with image scraping.
Well, first of all, you don't need to make a Rails App only for this image-scraping functionality. You could just write a script that does this. For that I would suggest using Nokogiri. You then need to find the way 4chan displays images on it's pages (inspecting the page from your browser), how are they structured in order to be able to get to them.
Related
I have a RoR application that given a set of N URLs to parse, will perform N shell calls for a given PhantomJS (actually is a CasperJS) script.
So,
Right now I have something like this:
urls_to_parse = ['first.html', 'second.html',...]
urls_to_parse.each do |url|
parse_results = \`casperjs parse_urls.js '#{url}'\`
end
I have never done this before. Launching shell scripts from a RoR/Ruby application, so I am wondering if this is a good approach and what alternative may I have. So, why I use PhantomJS in combination with RoR?
I basically have an API (RoR app) that keeps receiving urls that need to be parsed. They need to be parsed in a headless browser manner. The page actually needs to be rendered (that's why I don't use Nokogiri or any other HTML parser).
I am concerned about putting this up to production performance wise, and before going forward I would like to know if I am doing this correctly, or I can do it in a better way.
It's possible I thought about doing the same thing, but even with a headless browser I would be really concerned about the speed and bandwidth your server is going to need to have. I use capser in conjuction with Python and it works very well for me. I read stdout spit back from firing the casper scripts, but I don't parse and scrape on the fly like you're talking about doing. I would imagine it's okay, but ideally you already have a cached database of results when people search. Maybe if it is a very very basic search you'll be okay, but I don't know.
I'm a pretty big RoR novice but I've had a lot of problems with finding out even where to begin.
I'm looking for any tutorials or examples of apps that lets a user fill out a simple form in ruby, takes that information, redirects to another page, and posts the information. The user would be able to share that unique link with others, which shows that information.
I've taken a look at various gems such as Formtastic and Simpleforms but the tutorials on Railscast are outdated and it's difficult to get working. I believe it's because I'm on Rails 3.
I'm pretty lost here and if anything can just prod me in the general direction of where to start / give me general strategy of how I should tackle this, it'd be greatly appreciated. This seems incredibly easy but I'm making it harder for myself than what it is.
I'm actually looking to take an intensive RoR course in a few weeks but I need to get this simple MVP app done (it'll include video by nimbb API) for a deadline... so while I should definitely be putting in the effort to learn rather than cheat and drift, your help this time around would really be great :)
To a large extent this functionally is baked into rails. The simplest way to do this is without any extra gems (besides the default ones):
In your terminal run:
rails g scaffold form name:string message:text
Then the user can go to example.com/forms/new and enter their info. Once they submit the form rails auto redirects them to the view page for that new data. The URL of that page which will be something like /forms/n where n is some integer is the link to share.
If your struggling with rails concepts I would look at the rails guides, especially: http://guides.rubyonrails.org/getting_started.html
I am planning to build a website. I can build the front end with html,css and javascript with the help of twitter bootstrap and jquery ui. Actually i have done a bit of work and it looks good to me.
But I am short of knowledge in case of server side language, (just learning python and django in fact).
So is it advised to build by blog without the database things and all? or should i delay it by 3-4 months until i learn server side programming?
Can I do thing like searching posts, sorting them , comment on post with out the use of databases?
Is it easy to transform my blog to a database based one in later stage easily without taking it off the web?
ps: i dont want to use blogging platforms like wordpress.
you can make a website using static web pages. But that wont be really good enough for your users or readers. Besides that you really have to put a lot of time in manually writing the code for each page.
But if you choose to build a dynamic blog that will be better for the readers. You deploy functions like search , comment more easily.
Using a free and open source blogging platform like wordpress will cut down your task. It has ready made themes, plug ins available to get your tasks done and customize your site the way you want.Again it will be far more secure and attractive.
So my suggestion will be to go with wordpress.
And as far as the learning par is concerned you will learn more stuff while working with wordpress than building a static site
This seems a little silly to ask but some background:
I'm a designer who just got added last minute to a rails project— I pretty much have no knowledge of rails other than what I've learned this week, and even that is REALLY slim. The literal extent to my project is in the views, so it's not like I need too much.
100% of my experience is limited to purely static html/css/js or wordpress/expression engine projects, so I don't go this deep usually.
That said, a lot of my front end work hasn't even been designed in the backend, so I can't access it from navigating it on the live version locally. Is there a way that I can manually load this via url?
haml path/to/page.haml
Generates the .html file for you.
Im a newbie RoR kid, and I'm trying to make a web gallery app just like the onyx http://www.hulihanapplications.com/projects/onyx since it's outdated and i couldn't get it to work with my 2.3.8 rails' version.
So, are plugins like paperclip or carrierwave any good for me to start with. Or should i take another route and writh the app from scratch.
please provide me with good links regarding the subject.
thank you in advance
Paperclip is good way to start when it comes to any kind of images you'll add during runtime. You do have to install ImageMagick to change images into various sizes (thumbnails, etc), and it wouldn't be taking very long to create a webgallery with it (took me a day to figure it out).
http://www.cordinc.com/blog/2009/04/multiple-attachments-with-vali.html