I've manually created a template and a task on Mechanical Turk. What's the easiest way to now programmatically (in Ruby) create a new task, where:
I reuse the task template I've already created
I upload to MTurk a CSV file with some different data
I can download the raw results CSV programmatically as well
? Is there some Ruby library that already makes this easy, or would I have to dig into the API itself?
I've seen rturk and Turkee, but they seem a little complicated -- I don't actually want my questions to reside on an external site (in the rturk case), and I don't need a Rails app (in the Turkee case -- I couldn't actually get Turkee working with a Rails app anyways).
You might want to check out this fork of rturk. As you can see in the specs it lets you build Amazon-hosted QuestionForms using either XML or a Ruby DSL.
Hope that helps!
I spoke to some folks more familiar with the Amazon MTurk API, and apparently the API doesn't allow to use templates (like in the web UI). Rather, you have to loop through submitting a bunch of individual HIT items.
Related
We are planning to make a "large" website for I'd say 5000 up to many more users. We think of putting in lots of real time functionality, where data changes instantly propagate to all connected clients. New frameworks like Meteor and DerbyJS look really promising for this kind of stuff.
Now, I wonder if it is possible to do typical backend stuff like sending (bulk) emails, cleaning up the database, generating pdfs, etc. with those new frameworks. And in a way that is productive and doesn't suck. I also wonder how difficult it is to create complex forms with them. I got used to the convenient Rails view helpers and Ruby gems to handle those kind of things.
Meteor and DerbyJS are both quite new, so I do expect lots of functionality will be added in the near future. However, I also wonder if it might be a good idea to combine those frameworks with a "traditional" Rails app, that serves up certain complex pages which do not need realtime updates. And/or with a Rails or Sinatra app that provides an API to do the heavy backend processing. Those Rails apps could then access the same databases then the Meteor/DerbyJS app. Anyone thinks this is a good idea? Or rather not? Why?
It would be nice if anyone with sufficient experience with those new "single page app realtime" frameworks could comment on this. Where are they heading towards? Will they be able to handle "complete" web apps with authentication and backend processing? Will it be as productive/convenient to program with them as with Rails? Well, I guess no one can know that for sure yet ;-) Well, any thoughts, guesses and ideas are welcome!
For things like sending bulk emails and generating PDFs, Derby let's you simply use normal Node.js modules. npm now has over 10,000 packages, so there are packages for most things you might want to do on the server. Derby doesn't control your server, and it works on top of any normal Express server. You should probably stick with Node.js code as much as possible and not use Rails along with Derby. That is not to say that you can't send messages to a separate Rails app, but since you already have to have a Node.js app running to host Derby, you might as well use it for stuff like this.
To communicate with such server-side code, you can use Derby's model events. We are still exploring how this kind of code works and we don't have a lot of examples, but it is something that we will have a clear story around. We are building an app ourselves that communicates with an email server, so we should have some real experience with this pretty soon.
You can also just use a normal AJAX request or send a message over Socket.IO manually if you don't want to use the Derby model to do this kind of communication. You are free to make your own server-side only routes with Express along with your Derby app routes. We think it is nice to have this kind of flexibility in case there are any use cases that we didn't properly anticipate with the framework.
As far as creating forms goes, Derby has a very powerful templating system, and I am working on making it a lot better still. We are working on a new UI components feature that will make it possible to build libraries of self-contained UI widgets that can simply be dropped into a Derby app while still playing nicely with automatic view-model bindings and data syncing. Once this feature is completed, I think form component libraries will be written rather quickly.
We do expect to include all of the features needed for a normal app, much like Rails does. It won't look like Rails or work like Rails, but it will be similarly feature complete eventually.
For backend tasks (such as sending emails, cleaning up the database, generating pdfs) it's better to use resque or sidekiq
Now, I wonder if it is possible to do typical backend stuff like
sending (bulk) emails, cleaning up the database, generating pdfs, etc.
with those new frameworks. And in a way that is productive and doesn't
suck. I also wonder how difficult it is to create complex forms with
them. I got used to the convenient Rails view helpers and Ruby gems to
handle those kind of things.
Also, my question is not only about background jobs, but also about stuff one can might do during a request, like generating a pdf, or simply rendering complex views with rails helpers or code from gems. –
You're mixing metaphors here - a single page app is just a site where the content is loaded without doing a full page reload, be that a front end in pure js or you could use normal html and pjax.
The kind of things you are describing would be done in a background task regardless of the fornt-end framework you used. But +1 for sidekiq if you're using ruby.
As for notifying all the other users of things that have changed, you can look into using http://pusher.com or http://pubnub.com if you don't want to maintain a websocket server.
I have a Rails app that has 2 "clients" -- an iOS app reads/writes JSON, and Web browsers that read HTML.
Now, if I said "I want different output for different browsers/clients", we could use different ERB files and render based on User-Agent or similar.
In this way Rails was imagined in the Web world -- where I type "cap deploy", and all of my "instances" are upgraded to the latest version (save active session AJAX calls).
As an iOS dev, however, I'm very used to if blocks in code to deal with different data versions & client app versions. It's painful (but required).
I would love to be able to say "I will magically think of the perfect data structure in version 1.0", but we probably all know that's not true -- I'll want to add & deprecate attributes or even models as time goes.
I don't want to mix all of that versioning logic with my normal HTML (which can upgrade gracefully, per above), so I've thought about:
Having a separate controller or just actions for my API calls from iOS
Writing a "forwarder" that treats the HTML/Web version as a layer on top of the API version
Am I trying to solve a Solved Problem? Are there any resources or guiding philosophies I should know about in undertaking this project?
I suggest you create 2 applications, one for the webapp and one for the API.
You could have an external library with all the models, tests and business logic used in those 2 applications.
You can then lock the api to a certain version of the library if needed.
You can create multiple versions of the API without affecting older versions.
You can implement caching at library level if needed and it will automatically affect all applications.
With this model you can also create specific needs, for example, you may want shorter urls for the api than the real webapp, since you don't care about SEO in the API.
What do you think?
Edit :
Here is what I have in mind, library is in the middle.
Hypothetical question (at the moment!)
Suppose I have a great idea for an application. It acts on data which can be well-represented by tables in a relational database, using interlinked objects which represent those tables. It supports a well-defined API for interacting with (Creating, Reading, Updating, Deleting) those objects, and viewing information about them.
In short, it's a perfect fit for Rails... except it doesn't want to be a web-app. Perhaps it wants a Command Line interface; or an OS-native dialog-based interface; or perhaps it wants to present itself as a resource to other apps. Whatever - it just isn't designed to present itself over HTTP.
These questions suggest it's certainly possible, but both approach the problem from the point of view of adapting an existing web-app to have an additional, non-web, interface.
I'm interested in knowing what the best way to create such an app would be. Would you be best to rails new non_web_app, in order to get the skeleton built "for free", then write some "normal" Ruby code that requires config/environment - but then you have a lot of web-centric cruft that you don't need? Or would it be better to roll up your sleeves and build it from whole cloth, taking just the libraries you need and manually writing any required configuration?
If the latter, what exactly is needed to make a Rails app, but without the web bits?
If you want to access the Rails ORM to develop a CRUD non-web application, just include ActiveRecord in your own Ruby script; you will avoid using a lot of Rails modules you probably don't need (routing, template generator, ...) Here is an example of how to do it.
If you prefer to have the full Rails stack, do not run your Rails web app in an application server (WEBrick, Passenger, Mongrel, ...) to avoid any HTTP exposure, and interact with your application using tasks or the rails console.
I would avoid taking Rails too far off the rails. If I were doing this and felt that the gains of rails w/o the web stuff I'd do the following:
rails new non_web_app
and ignore the webbish cruft and use rails to generate models. In this way you get the tight, comfortable database behavior and can tie various gems in as you want to augment those models. I'd not bother implementing views, of course, and I'd consider implementing controllers in which the various render bits are removed and to use you instantiate an instance of the controller and call the action directly. This means the controller represents your API into your business logic still but the "views" it now "renders" are simply the return of the data output.
Then you could simply strip out the bits you do not need...the public directory, the view structure under app, config/routes.rb, etc. You'll need to test those changes incrementally and make sure that removing some now extraneous bit doesn't throw the Rails world into chaos.
Rails is for Web apps. That means HTTP. Now, you could package a Web app so that it runs on the desktop instead, or you could use ActiveRecord with a desktop application framework like Monkeybars.
I've only just started learning ruby on rails and I would like to create an application that will allow me to add files to the database. Currently, I'm developing the rails application using the Aptana plugin for Eclipse and the application is using the default sqllite db.
I have tried generating a scaffold with the following parameters: documents title:string file:varbinary. Then I do a 'rake'->'db'->'migrate'. When I migrate to localhost/documents and click on 'New Document' the application fails and displays an error.
What I would like to do is click on 'New Document', have a field that will allow me to browse for a document on my local computer, select the document and add it to the db on the rails application.
Paperclip is more recommended than attachment_fu these days. It is really simple and easy to use with your active record model.
Is it a particular kind of file you want to add?
I just ask because if it's not data of a kind that benefits from being in a database ( textual data might be searchable, binary data is not ) then you are much better storing it in the filesystem and serving it up straight - especially for stuff like images or video - rather than inserting it into a database and having to go through your application every time a user requests it.
I'm not saying that there aren't any reasons you might want to have a file in the database, but I treat that as a last resort and in ten years of web programming I've not come across a case where it was necessary.
I would highly recommend the attachment_fu plugin as this lets you create models with attachments pretty nicely, Paperclip plugin is another good one also!
If you have trouble deciding which one to use, as far as i can remember, Paperclip makes it easier for multiple attachments, such as an Album has many Photos, and Attachment_fu is easier for single attachments such as a User has one display picture.
We do something like this on a site I'm managing. Instead of storing these files in a database, I'd agree with the other posters here and recommend you try something like Paperclip.
One caveat: if you want access control, make sure that paperclip doesn't save your files somewhere under /public, where anyone could possibly access them if they knew the URL. Deliver files to the user via send_file in your controller.
The rails books and web pages I've been following have all stuck to very simple projects for the sake of providing complete examples. I'm moving away from the small project app and into a realm of non-browser clients and need to decide where to put code that is shared by all involved parties.
The non-browser client is a script that runs on any machine which can connect to the database. Browser clients write commands into the database, which the script examines and decides what to do. Upon completion, the script then writes its result back. The script is not started by the RoR server, but has access to its directory structure.
Where would be the best place for shared code to live, and how would the RoR loader handle it? The code in question doesn't really belong in a model, otherwise I'd drop it in there and be done with it.
I'd put the shared code in the Rails project's /lib directory and consider making it a custom Rake task.
It really depends on how much you use this shared code. If you use it everywhere, then throw it in the lib folder (as has already been stated here). If you are only using it in a few places, you might want to consider making a plugin out of it and loading it only in the places that use it. It's nice to only load what you need (one of the reasons I'm loving merb).