I have a simple server-side command line python script, based on the now deprecated FT SQL API using Client Login for authorization.
Every few hours I refresh my FT by inserting the updated data. It's a small table, 3000 rows.
Read the FT Migration Guide, have a Google User id, enabled the FT API and have an API key, etc.
(Switching my Google Maps application to the new API was straight forward)
But I'm confused about how to proceed with migrating a command line python script. I see that OAuth2 is the recommended approach for authentication but it appears to me that this is a much more complicated approach than I truly need. It's not even clear to me that it will work for command line scripts.
Truly no need to prompt the user for login info (it's only me). I realize that this must be an "Installed app" and I found the Hello,python example at
https://developers.google.com/fusiontables/docs/sample_code#ftAPIv1
Haven't tried this yet but wanted to see if anyone has experience with migrating command line scripts. (PHP would work as well)
In particular I was intrigued by a quote I found here:
https://developers.google.com/fusiontables/docs/v1/using
"The Fusion Tables API also supports older authorization options, such as OAuth 1.0, AuthSub, or ClientLogin; however, in most cases we don't recommend using those other options. If your application already uses those options, we recommend migrating to OAuth 2.0 if possible."
Because, frankly I'd rather not switch to OAuth 2 for such a simple task.
UPDATE
Forgot to mention, I run this from cron, every 3 hours. So user input is not really possible. Though as David suggested below a one time user input is doable.
You should be able to use your Client Login token as your auth token for your existing server-side command line app with the new API. You'll also need to include your developer key in the request as the "key" parameter.
There is a very simple python implementation of the OAuth2 Installed Application flow here https://developers.google.com/fusiontables/docs/samples/python as you may have found, and yes, you should switch to this. ClientLogin deprecated just like the SQL API.
The Code requires you to open a browser and prompts you every time you use it and I'm going to speak to the author about optimizing it and having it store the Refresh Token and so it would only prompt the first time you use it.
Related
I was following this tutorial, https://towardsdatascience.com/how-to-download-twitter-friends-or-followers-for-free-b9d5ac23812, which was written in 2021. It should've worked fine, however, they have to 'fix' the things that just work.
Specifically, running this line
for fid in Cursor(api.followers_ids, screen_name=screen_name, count=5000).items():
ids.append(fid)
gives the error:
"tweepy.error.TweepError: [{'message': 'You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve', 'code': 453}]"
I could have pulled the data in five minutes. Now debugging this already cost one hour+ because they just break the things that work. Is there anyway to make this old code snippet work? The application to use API 1.1 takes weeks, and I don't have time to watch their bad documents of how to migrate from API 1.1 to 2.0 and then the documents of migrating from Tweepy 3.9.0 to 4.0.0. Five minutes' task would just become half a day. Thanks in advance for any help.
First of all, have you at least tried to apply for the Elevated access?
It can take some time, it's true, but it can also be instantaneous.
The other solution would be to use the Twitter API V2.
You don't need any tutorial, just read the documentation:
Here for the authentication ;
Here for the retrieval of the followers ;
Here for the pagination.
And you should get something like that:
import tweepy
client = tweepy.Client("Bearer Token here")
paginator = tweepy.Paginator(
client.get_users_followers,
id=..., # ID only, no screename
max_results=1000
).flatten()
for follower in paginator:
print(follower.id)
Finally, even if I understand your frustration (and developing Twitter applications can be very frustrating), I think that you should try to keep it out your SO questions. Good luck!
I am trying to create a Google Assistant for my Raspberry Pi in Kotlin. I implemented a OAuth flow using the so called "device flow" proposed in this IETF draft, since my Raspberry shall later just expose a web interface and does not have any input devices or graphical interfaces.
Google does support this flow (of course) and I obtain a valid access token with user consent in the end. For testing purpose I also tried a default authorization flow that will just forward the user to localhost, as it is normally done but it did not solve the problem.
I tested the access token using this tool and it confirmed validity of scope and token. So the token itself should work.
Scope is: https://www.googleapis.com/auth/assistant-sdk-prototype as documented here
This actually does not point to any valid web resource but is referenced in every documentation.
Then I tried to stream audio data to the assistant SDK endpoint using the gRPC provided java stubs. As took a third party reference implementation as a guide how to authenticate the rpc stub. But neither the reference implementation nor my own one works. They both report
io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
The stub is authenticated this way:
embeddedAssistantStub.withCallCredentials(
MoreCallCredentials.from(OAuth2Credentials
.newBuilder()
.setAccessToken(
myAccessToken,
myAccessTokenExpirationDate))
.build()))
and the authenticated request is performed like this:
val observer = authenticatedEmbeddedAssistantStub.converse(myStreamObserverImplementation)
observer.onNext(myConfigConverseRequest)
while(more audio data frames available) {
observer.onNext(myAudioFrameConverseRequest)
}
observer.onCompleted()
(I prefixed pseudo variables with "my" for clarity, they can consist of more code in the actual implementation.)
I even contacted the author of this demo implementation. He told me, last time he checked (several months ago) it was working perfectly fine. So I finally ran out of options.
Since the client implementation I took as reference used to work and I do actually authenticate the stub (although the error message suggests the opposite) Probably, either my valid access token with correct scope is not suitable chosen for the assistant API (though I followed the suggestions of google) or the API servers had a change not properly documented in the getting started articles by google.
So: Did anyone ran in the same problem and know how to fix it? I have the project on github. So if anyone needs the broken source code, I can do a temporary commit that produces the error.
Note, to save some works for mods: This issue referres to this and this question, both unresolved and using different languages but describing a similar problem.
Well, seems I was right about my second assumption: The error is server side. Here is the github issue, let's just wait for the fix.
https://github.com/googlesamples/assistant-sdk-python/issues/138
Parse.com's REST API docs (https://www.parse.com/docs/rest) say: Authentication is done via HTTP headers. The X-Parse-Application-Id header identifies which application you are accessing, and the X-Parse-REST-API-Key header authenticates the endpoint. In the examples with curl that follow, the headers are stored in shell variables APPLICATION_ID and REST_API_KEY, so to follow along in the terminal, export these variables.
I am building a Sencha Touch app as a native app on iOS and Android using Phonegap, and I was wondering whether it is secure to expose these keys to the client while making the REST calls?
Also, can someone explain to me how does security work in this scenario? Help is much appreciated! Thanks!
Without phonegap , in a proguard , post processed android apk , the string values of the 2 headers you mention are exposed client-side . not a big issue. TLS covers the http header values during network leg and far more important for app security, you have Full ACL at the DB row level(parse/mongo) contingent on permissions of 'current user()'. So with no access to logon, some outsider doesn't have any more than obfuscated string value to an app-level access token.
. One odd thing is that with parse the lease time on the client-side token value foapi key is permanent rather than say a month.
Parse REST security is robust n well executed.
Can't speak to what PG framework offers in obfuscate/minify/uglify area but you should check that.
I have written a client in C to upload files to Google Docs, I get an authentication token, and use it for the upload. However, I have 4 Google accounts, the upload worked on 2 accounts, but not the other 2. And now, it only works on one account.
On authentication, all accounts behave the same, and ClientLogin issues an "Auth=xxx" token. I use the token 100% successfully on one Google account, it used to work on another, and has never worked on the other 2.
The only error is "HTTP/1.1 401 Token invalid".
One account used to work, and now does not, which does not make much sense. The ClientLogin calls never seem to ask for a CAPTCHA.
Currently I can't invest the time to move to the Drive API, and that may present the same issues anyway.
Any thoughts?
Thank you
Garry
Try them individually first, if this works you can start combining untill all 4 works,
doing it step by step is the way to succes, I can't help you with programming code, according to the fact that I'm just a noobie :) tho I thought myself PHP by just doing it step by step, if you got 1 done, then it's just copy/paste and change abit :)
Goodluck tho :)
You should move your code away from ClientLogin and use OAuth 2.0 instead. You didn't mention what language your script uses but Google has some samples for performing OAuth 2.0 at:
https://developers.google.com/google-apps/documents-list/#authorizing_requests_with_oauth_20
and
https://developers.google.com/api-client-library/
Jay
I'm trying to find my way around the OAuth spec, its requirements and any implementations I can find and, so far, it really seems like more trouble than its worth because I'm having trouble finding a single resource that pulls it all together. Or maybe it's just that I'm looking for something more specialized than most tutorials.
I have a set of existing APIs--some in Java, some in PHP--that I now need to secure and, for a number of reasons, OAuth seems like the right way to go. Unfortunately, my inability to track down the right resources to help me get a provider up and running is challenging that theory. Since most of this will be system-to-system API usage, I'll need to implement a 2-legged provider. With that in mind...
Does anyone know of any good tutorials for implementing a 2-legged OAuth provider with PHP?
Given that I have securable APIs in 2 languages, do I need to implement a provider in both or is there a way to create the provider as a "front controller" that I can funnel all requests through?
When securing PHP services, for example, do I have to secure each API individually by including the requisite provider resources on each?
Thanks for your help.
Rob, not sure where you landed on this but wanted to add my 2 cents in case anyone else ran across this question.
I more or less had the same question a few months ago and hearing about "OAuth" for the better part of a year. I was developing a REST API I needed to secure so I started reading about OAuth... and then my eyes started to roll backwards in my head.
I probably gave it a good solid day or 2 of skimming and reading until I decided, much like you, that OAuth was confusing garbage and just gave up on it.
So then I started researching ways to secure APIs in general and started to get a better grasp on ways to do that. The most popular way seemed to be sending requests to the API along with a checksum of the entire message (encoded with a secret that only you and the server know) that the server can use to decide if the message had been tampered with on it's way from the client, like so:
Client sends /user.json/123?showFriends=true&showStats=true&checksum=kjDSiuas98SD987ad
Server gets all that, looks up user "123" in database, loads his secret key and then (using the same method the client used) re-calculates it's OWN checksum given the request arguments.
If the server's generated checksum and the client's sent checksum match up, the request is OK and executed, if not, it is considered tampered with and rejected.
The checksum is called an HMAC and if you want a good example of this, it is what Amazon Web Services uses (they call the argument 'signature' not 'checksum' though).
So given that one of the key components of this to work is that the client and server have to generate the HMAC in the same fashion (otherwise they won't match), there have to be rules on HOW to combine all the arguments... then I suddenly understood all that "natural byte-ordering of parameters" crap from OAuth... it was just defining the rules for how to generate the signature because it needed to.
Another point is that every param you include in the HMAC generation is a value that then can't be tampered with when you send the request.
So if you just encode the URI stem as the signature, for example:
/user.json == askJdla9/kjdas+Askj2l8add
then the only thing in your message that cannot be tampered with is the URI, all of the arguments can be tampered with because they aren't part of the "checksum" value that the server will re-calculate.
Alternatively, even if you include EVERY param in the calculation, you still run the risk of "replay attacks" where a malicious middle man or evesdropped can intercept an API call and just keep resending it to the server over and over again.
You can fix that by adding a timestamp (always use UTC) in the HMAC calculation as well.
REMINDER: Since the server needs to calculate the same HMAC, you have to send along any value you use in the calculation EXCEPT YOUR SECRET KEY (OAuth calls it a consumer_secret I think). So if you add timestamp, make sure you send a timestamp param along with your request.
If you want to make the API secure from replay attacks, you can use a nonce value (it's a 1-time use value the server generates, gives to the client, the client uses it in the HMAC, sends back the request, the server confirms and then marks that nonce value as "used" in the DB and never lets another request use it again).
NOTE: 'nonce' are a really exact way to solve the "replay attack" problem -- timestamps are great, but because computers don't always have in-sync timestamp values, you have to allow an acceptable window on the server side of how "old" a request might be (say 10 mins, 30 mins, 1hr.... Amazon uses 15mins) before we accept or reject it. In this scenario your API is technically vulnerable during the entire window of time.
I think nonce values are great, but should only need to be used in APIs that are critical they keep their integrity. In my API, I didn't need it, but it would be trivial to add later if users demanded it... I would literally just need to add a "nonce" table in my DB, expose a new API to clients like:
/nonce.json
and then when they send that back to me in the HMAC calculation, I would need to check the DB to make sure it had never been used before and once used, mark it as such in the DB so if a request EVER came in again with that same nonce I would reject it.
Summary
Anyway, to make a long story short, everything I just described is basically what is known as "2-legged OAuth". There isn't that added step of flowing to the authority (Twitter, Facebook, Google, whatever) to authorize the client, that step is removed and instead the server implicitly trusts the client IF the HMAC's they are sending match up. That means the client has the right secret_key and is signing it's messages with it, so the server trusts it.
If you start looking around online, this seems to be the preferred method for securing API methods now-adays, or something like it. Amazon almost exactly uses this method except they use a slightly different combination method for their parameters before signing the whole thing to generate the HMAC.
If you are interested I wrote up this entire journey and thought-process as I was learning it. That might help provide a guided thinking tour of this process.
I would take a step back and think about what a properly authenticated client is going to be sending you.
Can you store the keys and credentials in a common database which is accessible from both sets of services, and just implement the OAuth provider in one language? When the user sends in a request to a service (PHP or Java) you then check against the common store. When the user is setting up the OAuth client then you do all of that through either a PHP or Java app (your preference), and store the credentials in the common DB.
There are some Oauth providers written in other languages that you might want to take a look at:
PHP - http://term.ie/oauth/example/ (see bottom of page)
Ruby - http://github.com/mojodna/sample-oauth-provider
.NET http://blog.bittercoder.com/PermaLink,guid,0d080a15-b412-48cf-b0d4-e842b25e3813.aspx