I am trying to build a free web application using ruby/rails It should be able to send sms through online forms of various mobile operators. (like this one (in russian)).
So, I need to
wait for the user, who wants to send an sms through my website.
establish connection to operator website. Probably, using Mechanize.
retrieve captcha
show captcha to the user
allow user to enter a message and captcha
submit form on operators website (with message, captcha, phone number)
The connection to the operator website should be alive during all this process (otherwise captcha will change). As far as I understand, I need to create a (sub)process each time sms is sent.
Could you please advise what is the best way of handling this in rails\ruby?
I am still rather new to web-development...
Should I use threads? forks? popen? using PTY? some external gem? How should I communicate with my process?
Assuming there's nothing special about the operator's web site, no, you don't need to keep a connection alive during the whole process. Generally speaking, forms on web pages work like this: You visit the URL, your web browser downloads the page with the form on it. In your case, it will also have an <img> tag or similar to show the CAPTCHA. Once your browser has downloaded the page, the connection is severed. After you fill out the form and click on Submit, your web browser opens a new connection to the server and sends the data, and the server sends its response (whatever page is shown after you click Submit).
All your program has to do is emulate this experience. So: 1) Download the page with the form on it. Scrape the form fields (make sure you don't miss any hidden fields--with a CAPTCHA there will probably be some) and the CAPTCHA. 2) Build a page to show your user that includes the CAPTCHA and a form with all the fields they need to fill out. If there were hidden fields in the original form, make sure you include their values (as hidden fields in your form) as well, because when the user submits your form you'll need them. 3) Then, when the user submits your form, send the data, including the hidden values and what the user entered for the CAPTCHA, to the operator. 4) Finally, check if the operator indicated success, and build a page to tell your user.
If you're doing this in Rails, you'll probably have two methods in your controller: One called e.g. 'show' (steps 1 and 2 above) that will scrape the CAPTCHA and other info from the operator's site and show the user your form view, and one called e.g. 'send' (step 3 and 4 above) that the form will submit to, and which will take their data and send it to the operator's web site, collect the response and tell your user if it was successful or not.
Note: You'll want to read the operators' terms of service before you bother with any of this. I'm fairly certain that this kind of thing will be against their TOSes and if they notice your server sending a lot of requests their way they're going to block you pretty quick.
To answer another question of yours, you can use DRb or background_job (aka BJ) to actually accomplish the sending in the background so that after your user submits the captcha they don't have to wait for the response. Or you could wrap this in ajax and have the DRb/BJ process notify you when the sms sending has happened so you can notify the user of success or any problems.
Typically opening threads in Ruby is something to avoid as there are so many great gems that do what we need. Not to say that you shouldn't use threads, just that for the most part it's probably already been done really well.
Related
I'm working on ASP.NET MVC project with C#.
Ok so I have a layout view where I put my partial view which contains just a div that displays notification messages.
Now from some view I have a button that generate a report in 5 minutes in async manner. While the report being generated I need to allow the user to use other areas of the website.
My action method, once the report is generated successfully, simply returns a string "Success", o/w "Fail".
What I want to do is assign that returned string to the div of the partial view which is on the layout page. So this way the user can see the notification from wherever he is within the website.
How can I do this? Thanks.
There's a number of different things going on here. First, you want the server to update the user with the "success" or "fail" status. This requires 1) using web sockets to create a persistent connection between the client and server, allowing the server to talk to the client without requiring the client to first send a request, or 2) long-polling, which is means the client continuously sending requests at a defined interval to see if the server has any updates.
Long-polling (with AJAX) was the only way to achieve this before the advent of web sockets, which are relatively new, and not universally supported. In particular, IIS8+ is required on the server side, and client side, you need a modern browser, which is really any except IE 9 and below. If you can't run the site on IIS8+ or you need to support legacy versions of IE, then you're stuck with long-polling.
However, with either approach, you're tied to a single page. If the user navigates away, web socket connections are closed and long-polling stops. If the user is still on your site, the next page would need to re-establish all this functionality to keep it working. That's not really difficult - just something to be aware of. It just means that you'll need some universal script running on page load across your site for this.
Now as far as replacing the content of your "partial view" goes. You shouldn't look at it that way. I encourage you to read my post: There's no such thing as a "partial view" client-side, where I get into more detail. The TL;DR version is that all of this updating of the client is happening client-side, and at that point, all you have is the browser DOM. There's no concept of a "partial view". If you want to replace a part of the DOM, you must select it and manipulate it. That's all done with JavaScript and it's all on you. There's no easy "replace this partial view" button.
How do miscellaneous account management pages fit into a RESTful design in Rails 3?
For example, a user registers (create action) and is then forwarded to a registration success page (? action) where they are asked to now verify their email address via a url with a token (emailed to them).
When they click the link in the email, technically they are "updating" their account as part of the verification process right? So I'm thinking that would somehow map to the "update" action but the update action is expecting a PUT request. Is that correct? How do you make that work via the email?
I'm also wondering how forgot password, reset password, etc also fit into a RESTful design? Just trying to wrap my head around this.
Just because you have a result design, doesn't mean you HAVE to restrict yourself to only CRUD verbs that map 1:1 to Get/Post/Put/Delete. That said, if you want to get really RESTful, you can start to think of some of these things in terms of being their own resources. For example user verification:
User signs up, and gets sent a verification email, you already have that all squared away RESTfully it looks like
Verification url looks like: http://app.com/user_verifications/new?token=foobar (GET)
They follow the url and maybe are presented with a "Hello Dan, welcome back! Click here to verify your account" at that point you submit a form to http://app.com/user_verifications to trigger the create action there. Now on the backend, you can perform whatever actions you want, updating the user, setting them to active, or actually creating a "UserVerification" model.
Not a perfect example, but the idea is that the RESTful interface you are providing has an additional resource, in this case "user_verifications" and a user is acting upon it via HTTP methods in order to achieve the user's goals. You can apply similar logic to reset/forgot password either with a "UserSession" type resource or even as specific as a specific "ForgotPassword" resource.
Success page is just create.html.erb file. Usually you are redirecting from create action, but here you can just render success template.
Verifying. If you want to stay REST you should add one more step: GET verify, where is the form with your token present, which will lead to PUT update action. User recieves a link to this page.
But I prefer to use simple GET request here, which will update information without any additional clicks.
The same way you work with restoring passwords and other functionality. You add a page to with form that gets email, then you send a letter with link to a page with form filled with tokens and so on.
what design approach would you take for security and scalability (and perhaps level of effort) when posting an ASP.NET MVC HTML form that also has a Flash component, as described in the scenario below?
I have a scenario where there's an ASP.NET MVC site, and a page that requires user authentication & authorization to access (using ASP.NET forms authentication). On that page there's an HTML form. The form also has a Flash component. The form has a few text fields, and the Flash component has binary data that needs to get submitted to the server in tandem with the HTML form fields. When the user hits the submit button on the HTML form, the form contents and binary data from the Flash need to get submitted as part of one atomic unit, so to speak.
I know that I can use HTML / Flash JavaScript bridging to post the form either through JavaScript, or through the Flash component. I could even do both, perhaps posting the binary data from the Flash component when the user clicks submit, and then posting the HTML form content following that.
From your experience, what approach would be the path of least resistance to post the form with? Considering the user authentication and authorization part, I imagine that Flash would higher effort than HTML. What about the user authentication aspect? If the page posted from Flash to the server, would Flash also have to authenticate the user, in addition to the standard HTML authentication form?
My Flash binary data should not typically be greater than 300KB, often less...any opinions / insights are greatly appreciated!
EDIT:
I also vaguely remember that with Flash 10.1, ExternalInterface seemed to not work right when transferring binary data to JavaScript, am I mistaken with that, and that you can easily transfer 300KB or more of binary data from Flash to a JavaScript variable that is then posted to the server?
Thanks!
That's one way of doing it:
Make your flash component call a javascript function through ExternalInterface.call().
Make the called javascript function change a hidden field inside the form.
When the form is posted, the hidden field will send the desired value to the server.
I guess this is also the safest way - since the Flash component will not communicate with the server. Everything you need is a standard, non-AJAX form post.
Edit
Sorry, I guess I should have paid more attention... If there is too much data being sent from the flash component, maybe you should post it directly to the server.
Still, you could use ExternalInterface to synchronize the whole process. Make Flash call a server-side method (I would use FluorineFX for that, but your opinion may vary). Then .NET will return an ID, meaning it has received and saved the binary data for future use. Then call ExternalInterface to set that ID to a hidden field. After that, when the HTML is posted to the server, the server-side action method just need to retrieve the binary data using the posted ID...
The problem here is that you will end up with some binary data that will never be associated to any form post... But that's OK I guess, just run some "garbage-collector" script from time to time.
About the authentication issue: FluorineFX does implement .NET authentication, and it is able to retrieve the current logged in user. Of course, there are some issues.
A common web problem is where a user clicks the submit button of a form multiple times so the server processes the form more than once. This can also happen when a user hits the back button having submitted a form and so it gets processed again.
What is the best way of stopping this from happening in ASP.NET MVC?
Possibilities as I see it are:
Disable the button after submit - this gets round the multiple clicks but not the navigation
Have the receiving action redirect immediately - browsers seem to leave these redirects out of the history
Place a unique token in the session and on the form - if they match process the form - if not clear the form for a fresh submit
Are there more?
Are there some specific implementations of any of these?
I can see the third option being implemented as an ActionFilter with a HtmlHelper extension in a similar manner to the anti-forgery stuff.
Looking forward to hearing from you MVC'ers out there.
Often people overlook the most conventional way to handle this which is to use nonce keys.
You can use PRG as others have mentioned but the downside with PRG is that it doesn't solve the double-click problem, it requires an extra trip to the server for the redirect, and since the last step is a GET request you don't have direct access to the data that was just posted (though it could be passed as a query param or maintained on the server side).
I like the Javascript solution because it works most of the time.
Nonce keys however, work all the time. The nonce key is a random unique GUID generated by the server (also saved in the database) and embedded in the form. When the user POSTs the form, the nonce key also gets posted. As soon as a POST comes in to the server, the server verifies the nonce key exists in its database. If it does, the server deletes the key from the database and processes the form. Consequently if the user POSTs twice, the second POST won't be processed because the nonce key was deleted after processing the first POST.
The nonce key has an added advantage in that it brings additional security by preventing replay attacks (a man in the middle sniffs your HTTP request and then replays it to the server which treats it as a legitimate).
You should always return a redirect as the HTTP response to a POST. This will prevent the POST from occuring again when the user navigates back and forth with the Forward/Back buttons in the browser.
If you are worried about users double-clicking your submit buttons, just have a small script disable them immediately on submit.
You might want to look at the Post-Redirect-Get (PRG) pattern:
This really isn't MVC specific, but the pattern we follow on our web pages is that actions are performed with AJAX calls, rather than full page POSTs. So navigating to a url never performs an action, just displays the form. The AJAX call won't be in the history
Along with the disabling of buttons, you can add a transparent div over the entire web page so that clicking does nothing. We do this at my work and add a little friendly label saying processing request..
The most elegant solution I found was to use ActionFilters:
Blog post is here
I have made a little app for signing up for an event. User input their data and click "sign me in".
Now sometimes people are double in the database, the exact same data that got inserted 2 times very quickly after each other. This can only mean someone clicked the button twice, which caused two posts to happen.
This is common web problem, as credit card apps and forum apps often say: "Clicking once is enough!".
I guess you could solve it by checking for the exact same data to see if the post is unique, but I wonder if there are other methods.
This ofcourse does not count for ASP.NET webforms, because POST doesn't matter as much.
While JavaScript solutions can disable the submit button after it has been clicked, this will have no effect on those people who have JavaScript disabled. You should always make things work correctly without JavaScript before adding it in, otherwise there's no point as users will still be able to bypass the checks by just disabling JavaScript.
If the page where the form appears is dynamically generated, you can add a hidden field which contains some sort of sequence number, a hash, or anything unique. Then you have some server-side validation that will check if a request with that unique value has already come in. When the user submits the form, the unique value is checked against a list of "used" values. If it exists in the list, it's a dupe request and can be discarded. If it doesn't exist, then add it to the list and process as normal. As long as you make sure the value is unique, this guarantees the same form cannot be submitted twice.
Of course, if the page the form is on is not dynamically generated, then you'll need to do it the hard way on the server-side to check that the same information has not already been submitted.
Most of the answers so far have been client-side. On the server-side, you can generate a hidden field with a GUID when you first produce the form, and then record that GUID as a submitted form when the post is received. Check it before doing any more processing.
Whenever a page is requested from the server , generate a unique requestToken , save it in server side,mark status as NOT Processed and pass it along with the current requested page. Now whenever a page submit happens , get the requestToken from the "POST"ed data and check the status and save the data or take alternate action.
Most of the banking applications use this technique to prevent double "POST"ing.So this is a time proven & reliable way of preventing double submissions.
A user-side solution is to disable the submission button via Javascript after the first click.
It has drawbacks, but I see it often used on e-commerce websites.
But, it won't never replace a real server-side validation.
Client side techniques are useful, but you may want to couple it with some server side techniques.
One way to do this is to include a unique token in the form (e.g. a GUID or similar), so that when you come to process the form you can check to see whether the token has already been used, preventing a double submission.
In your case, if you have a table with event visitors, you might include this token as a column.
A client-only solution won't be enough, as stated in many of the answers here. You need to go with a server-side fail-safe.
An often overlooked reason that disabling the submit button doesn't work is, the user can simply refresh the submit target (and click OK on the "are you sure you want to resubmit the POST data?" dialog). Or even, some browsers may implicitly reload the submitted page when you try to save the page to disk (for example, you're trying to save a hard-copy of an order confirmation).
Almost no one has js disabled.
Think about coding your e-commerce website for the 70 year old woman who double clicks every link and button.
All you want to do is add a javascript to prevent her clicking "Order Now" twice.
Yes - check this at the server side too "be defensive" - but don't code for that case. But for the sake of a better UI do it on the client side too.
Here are some scripts that I found:
//
// prevent double-click on submit
//
jQuery('input[type=submit]').click(function(){
if(jQuery.data(this, 'clicked')){
return false;
}
else{
jQuery.data(this, 'clicked', true);
return true;
}
});
and
// Find ALL <form> tags on your page
$('form').submit(function(){
// On submit disable its submit button
$('input[type=submit]', this).attr('disabled', 'disabled');
});
None of the solutions address a load-balance server.
If you have some load balancer, send a UUID (or any type of unique number) to the server to store and read again will not work well if the server is not aware of other servers, because each request could be processed by a different server in a stateless environment. These servers need to read/write to the same place.
If you have multiple servers you will need to have some shared cache (like a Redis) among the servers to read/write the unique value in the same place (what could be an over-engineering solution, but works).
Client side alteration is a common technique:
Disable submit button
Change the screen to a "please wait" screen
If the form was modal, changing the screen back to their usual process (this has the benefit of making things look really slick)
But it's not perfect. It all relies on JS being available and if that's not the case, without back-end duplication detection, you'll get duplicates still.
So my advice is to develop some sort of detection behind the scenes and then improve your form to stop people with JS being able to double-submit.
You can track the number of times the form's been submitted and compare it to the number of unique visits to the page with the form on it in the session.
Beside the many good techniques already mentioned, another simple server-side method, that has the drawback of requiring a session, is to have a session variable that is switched off on the first submit.