is it possible to submit forms with yahoo pipes?
i basically need to log in somewhere, and get some stuff from the members area of a website into a feed.
Although this is not exactly programming related... I guess it is close enough.
No, logging into somewhere is impossible with Yahoo Pipes. Sending the username/password isn't even the only problem here.
The real problem is that most, if not all, web sites that require a log-in depend on a session cookie or something similar. Yahoo pipes can do a GET request, and that's about it. Even if it was possible to send your user name/password in the URL, you would not be able to use the session cookie, so subsequent requests would fail.
So... If you have access to a hosted web site somewhere: Write a small proxy script (in PHP or whatever is available) that does the login and fetches the data. Let Yahoo pipes read from your proxy page. But if you are that far, you can just as well produce RSS format right away. ;-)
I did a pipe that can log in and extract info. is working ok on a simple web form using POST.
Related
I'm trying to access and store a database of my read books and/or uploaded documents. I don't really mind what the method is as long as it's not scraping the info with some javascript extension in the browser. That defeats my purpose of automation.
This is what I've tried so far:
Oauth2: Gives me VERY limited information. Email, name, and not much more. So apparently the way is to scrape for that info, so I tried the following:
Mechanize: I tried to load the amazon login form and post it with my personal information stored on my server, but it seems this info is not sufficient, not sure why. Some hidden values also needed apparently.
I tried using an iframe to load the amazon site and the store the session cookies but can't find the way to make this work. Also, is this method allowed? I´m having some concerns about the legitimacy of this.
I know I asked about Amazon I think it also applies to the more general question of how to extract some data from a website that requires authentication.
Thanks a lot.
Capybara would be a good choice. It uses Selenium to open a browser and navigate to a URL, post form info, etc.
Previously if I request app.asana.com/api/1.0/workspaces into a browser (after I had logged into Asana) it was returning all of my workspace ID's.
But currently it is not working and shows the error :- "{"errors":[{"message":"Not Authorized"}]}".
Is there any other solutions to get my asana workspace IDs?
Thank You,
(I work at Asana.) We're sorry for the inconvenience - this was a security hole and we had to close it, as we wrote on our API updates feed. If you're writing an app for others to use, Oauth is the most appropriate and secure way to access the users' data. If you're writing a simple script, you can use your API key. This can be done through a browser, but by using JavaScript and not just making requests in the browser location bar.
I have awebsite, lets just call it search, in one of my browserpages open. search has a form, which when submitted runs queries on a database to which I don't have direct access. The problem with search is that the interface is rather horrible (one cannot save the aforementioned queries etc.)
I've analyzed the request (with a proxy) which is send to the server via search and I am able to replicate it. The server even sends back the correct result, but the browser is not able to open it. (Same origin policy). Do you have any ideas on how I could tackle this problem?
The answer to your question is: you can't. At least not without using a proxy as suggested in the answer by Walter, and that would mean your web site visitors would have to knowingly login to your web site using their other web site's credentials (hmm doesn't sound good...)
The reason you can't do this is related to security, if you could run a script on the tab next to the one with the site open (which is what I'm guessing you want to do), you would be able to do a CSRF attack and get any data you wish and send it to hack.com
This is, of course, assuming that there has to be a login somewhere in the process, otherwise there's no reason for you to not be able to create a simple form which posts the required query and gets the info.
If you did have access to the mentioned website, you would be able to support cross domain xml using JSONP.
It is not possible to bypass the same origin policy in javascript (assuming that you want to do it with that considering your question). You need to set up a proxy server side that is doing the request for you and returns the html.
A simple way of doing this in PHP would be like this:
<?php
echo file_get_contents("http://searchdomainname.com" . "?" . http_build_query($_GET, '', '&'));
?>
An architectural question.
My site needs to allow the user to record video and upload it to the "site". I've been poking around a fair bit and it seems I have to use some kind of media server to achieve this aim. As I'm introducing this secondary server into the system (I seek to embed the flash app residing on this server into the HTML delivered by the site) it occurs to me that this broadens the scope of security a lot. What scares me is attackers trying to embed the flash app themselves or attempting to impersonate clients (or anything else I haven't thought of yet!).
I was therefore wondering how people secure their applications with such an architecture. Sure I can do what is suggested here, a decent band-aid for now but afaik the domain information can technically be falsified by the client.
I could separate out the auth of the site giving me a WebServer, an AuthServer and a MediaServer enabling the MediaServer to separately auth. Getting the user to log into both sites is obviously onerous and passing around the user's login creds and securing all connections sounds ugly and averse to best practice.
As far as I can see my best bet is some kind of temporary token that the auth server creates. So the website kicks the auth server after logging in to generate the token which the site can then pass to the media server (as part of the flash vars) and the MediaServer itself can use to double check against the auth server.
I'm relatively new to Red5, Flash and web security so I was wondering if the following sounds sane, secure and/or necessary. Also if anyone knows of decent tools to use for such an auth system and whether there is something already kicking about in ASP.NET auth for such a purpose.
the solution provided in your link ... you should read my second comment.
The first about virtual hosts is wrong! My comment does actually tell you (at least one) solution to secure your app.
You could for example pass a SESSION_ID in the connect method to Red5. The user would get the SESSION_ID from another webservice call before he invokes the record or playback method.
The SESSION_ID might be even some kind of temporary token, that is only valid for 15 minutes and only usable a single time for exactly that video. How far you implement that is a matter of how secure your mechanism needs to be.
Sebastian
So I'm writing a post on my wall and type a URL into the main body of the post. As soon as I finish the URL, Facebook creates a little section underneath which has the title, description, and an image from the url I typed.
Without getting too indepth, how is this done and what is the best way of make something similar myself?
jQuery (or some other framework that lets you do Ajax easily) to communicate between browser client and webserver
PHP/ASP.NET/Python (or some other scripting framework on the backend) to fetch the url
Facebook also has a meta data specification you might be interested in, to let developers further define what gets shown in a Facebook page.
I believe Facebook is written in PHP. And PHP does this easily.
FOpen can be used to access files on other sites. There are other functions but this will get you started. Then it's a matter of parsing the html you get from the url to get what you want.
http://php.net/manual/en/function.fopen.php
You have a couple choices. You can fetch it using Ajax from the client; or you can fetch it from your server.
If doing it from your server in asp.net then you need to use HttpWebRequest.
FB does an asynchronous JavaScript call to fetch that data without reloading the window you're on. Lookup ajax and libraries like jquery do this: http://api.jquery.com/category/ajax/