I am running logstash 7.16.2 on an Amazon Linux 2 AMI with the following command:
bin/logstash -f /etc/logstash/conf.d/twitter.conf
Here is the content of twitter.conf:
input {
twitter {
id => "twitter_plugin_1"
consumer_key => "MY_CONSUMER_KEY"
consumer_secret => "MY_CONSUMER_SECRET"
oauth_token => "MY_OAUTH_TOKEN"
oauth_token_secret => "MY_OAUTH_TOKEN_SECRET"
use_samples => true
}
}
output {
stdout {
codec => json
}
}
Here is the error:
[WARN ] 2022-01-18 01:11:58.701 [[main]<twitter] twitter - Twitter client error {:message=>"", :exception=>Twitter::Error::Forbidden, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/response.rb:24:in `on_headers_complete'", "org/ruby_http_parser/RubyHttpParser.java:370:in `<<'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/response.rb:19:in `<<'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/connection.rb:20:in `stream'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/client.rb:119:in `request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/client.rb:59:in `sample'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-twitter-4.0.3/lib/logstash/inputs/twitter.rb:146:in `do_run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-twitter-4.0.3/lib/logstash/inputs/twitter.rb:131:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:409:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:400:in `block in start_input'"], :options=>nil}
What am I doing wrong??
According to an issue that I shared on twitter community, a new plugin for v2 should be developed.
For more details, refer to this topic: https://twittercommunity.com/t/connect-logstash-to-twitter-app-v2/172503
So, the problem was that I have credentials for Twitter API V2 and the Logstash Twitter Plugin was trying to reach Twitter API V1 endpoints. The problem will remain until a plugin is developed for Twitter API V2.
Related
I am using CKAN 2.7.2.
I have added the following configurations in my development.ini file of ckan:
ckan.oauth2.authorization_endpoint = https://wso2IP/oauth2/authorize
ckan.oauth2.token_endpoint = https://wso2IP/oauth2/token
ckan.oauth2.profile_api_url = https://wso2IP/userinfo
ckan.oauth2.client_id = *client-id*
ckan.oauth2.client_secret = *clientsecret*
ckan.oauth2.profile_api_user_field = abc
ckan.oauth2.profile_api_mail_field = abc#gmail.com
Also, have exported the following while running ckan using paster serve :
export OAUTHLIB_INSECURE_TRANSPORT=True
Also, I have added an application (ckan provider) in WSO2 Identity Server also with callback URL = (http://ckan-url:5000/oauth2/callback) where the CKAN instance is running (i.e a private IP of 172.30.66.XX type running on port 5000).
So, what configuration need to be done in WSO2 Identity Server and ckan development.ini file?
Also, configuring on both sides, I get the error Client authentication Failed in CKAN GUI.
I am using Ckan 2.7.2 , WSO2-IS 5.1 and ckanext-oauth2 (oauthlib==0.8.0)
will it be giving a problem in authentication process??Can it be a version dependency issue??
I have followed the given steps. Kindly correct if I am missing or doing something wrong:
I have used API store and published ckan application.
Entered the ckan callback url as http://oauth2/callback
Entered the generated client ID and client secret from step 1 in ckan configuration.
Also, made the given changes mentioned by you regarding claim configurations and ckanext-oauth2 configurations.
Now, it gives above error.or sometimes Invalid Client (Client Authentication Failed)
Inbound settings:
enter image description here
Error Logs :
[2018-07-25 19:29:25,432] INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} - 'admin#carbon.super [-1234]' logged in at [2018-07-25 19:29:25,431+0530]
[2018-07-25 21:55:15,930] INFO {org.wso2.carbon.identity.application.authentication.framework.store.OperationCleanUpService} - Session Operation Data cleanup task is running successfully for removing expired Operation Data
[2018-07-25 23:40:15,929] INFO {org.wso2.carbon.identity.application.authentication.framework.store.SessionCleanUpService} - Session Data cleanup task is running successfully for removing expired Data
[2018-07-26 09:21:18,948] INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} - 'admin#carbon.super [-1234]' logged in at [2018-07-26 09:21:18,948+0530]
[2018-07-26 09:21:39,307] INFO {org.wso2.carbon.core.services.authentication.AuthenticationAdmin} - 'admin#carbon.super [-1234]' logged out at [2018-07-26 09:21:39,0306]
[2018-07-26 09:55:15,975] INFO {org.wso2.carbon.identity.application.authentication.framework.store.OperationCleanUpService} - Session Operation Data cleanup task is running successfully for removing expired Operation Data
After configuring the settings I receive the following Message:
Your application settings at WSO2-IS seems to be correct. The only thing to take into consideration, it is that ckanext-oauth2 is only able to use the Code and, the Refresh Token grant types, so only those types need to be allowed.
Using this configuration for ckanext-oauth2:
ckan.oauth2.authorization_endpoint = https://wso2IP/oauth2/authorize
ckan.oauth2.token_endpoint = https://wso2IP/oauth2/token
ckan.oauth2.profile_api_url = https://wso2IP/userinfo
ckan.oauth2.client_id = *** OAuth Client Key from the Inbound Authentication Configuration at WSO2 ***
ckan.oauth2.client_secret = *** OAuth Client Secret from the Inbound Authentication Configuration at WSO2 ***
ckan.oauth2.scope = all_info openid
ckan.oauth2.profile_api_user_field = email
ckan.oauth2.profile_api_fullname_field = name
ckan.oauth2.profile_api_mail_field = email
and using your claim mapping configuration (http://wso2.org/claims/fullname and http://wso2.org/claims/emailaddress); I was able to sign in into CKAN. I have tested using CKAN v2.7.4, v2.8.0 and v2.8.1; ckanext-oauth2 v0.6.1 and WSO2-IS v5.5.0 (although it should work using WSO2-IS v5.1).
I am working on a basic grails app protected by OAuth2 authentication -- code here. It is nothing more than what spring-security and spring-security-oauth plugins have for a getting started app. In my bootstrap I have (as per getting started):
Role roleUser = new Role(authority: 'ROLE_USER').save(flush: true)
User user = new User(
username: 'user',
password: 'password',
enabled: true,
accountExpired: false,
accountLocked: false,
passwordExpired: false
).save(flush: true)
UserRole.create(user, roleUser, true)
new Client(
clientId: 'my-client',
authorizedGrantTypes: ['authorization_code', 'refresh_token', 'implicit', 'password', 'client_credentials'],
authorities: ['ROLE_CLIENT'],
scopes: ['read', 'write'],
redirectUris: ['http://example.com']
).save(flush: true)
I am trying to understand the OAuth login flow following the example flows listed in the documentation.
Whenever I try to authenticate using the "Resource Owner Password Credentials Grant" with this request:
$ curl -X POST -d "client_id=my-client" \
-d "grant_type=password" -d "username=user" \
-d "password=password" -d "scope=read" \
http://localhost:9090/grails-oauth-service/oauth/token
I get the following message:
{"error":"invalid_client","error_description":"Bad client credentials"}
What am I doing wrong? How would I (or my client) correctly make a single call to authenticate to the service?
It's amazing what sleep can do to solve a problem. Came in ready to recreate the whole setup figuring I messed something up following the getting started guides.
Started from the plugin download page and looked at the latest plugin version:
spring-security-oauth2-provider:2.0-RC5
and checked that against my app where for some reason I had
spring-security-oauth2-provider:2.0-RC3
I upgraded the plugin, re-ran the s2-init-oauth2-provider script, fired it up and the curl post above WORKED! Glancing through the change log for RC-5 I see
Resolve minor problems affecting stateless access of OAuth 2.0 resources
So basically I was running into a bug with that version of the oauth provider plugin. Fun times.
I am creating an API for a backend service with Rails 4.
The service needs to upload an image file to an amazon s3 bucket.
I'd like to use a direct upload url, so that the clients manage the uploads to s3 and the server is not kept busy.
Currently I have the following prototypical rails action
def create
filename = params[:filename]
s3_direct_post = S3_BUCKET.presigned_post(key: "offers/#{SecureRandom.uuid}/#{filename}", acl: 'public-read')
s3p = s3_direct_post.fields
url = "#{s3_direct_post.url}/#{filename}?X-Amz-Algorithm=#{s3p['x-amz-algorithm']}&X-Amz-Credential=#{s3p['x-amz-credential']}&X-Amz-Date=#{s3p['x-amz-date']}&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=#{s3p['x-amz-signature']}"
render json: {success: true, url: url}, status: :ok
end
This generates such an url:
https://my-bucket.s3.eu-central-1.amazonaws.com/test.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=MYKEY/20150420/eu-central-1/s3/aws4_request&X-Amz-Date=20150420T162603Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=MYSIGNATURE
Now I try to post the test.png to this url with the following:
curl -v -T test.png "url"
and I get the following error response:
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>MYKEY</AWSAccessKeyId>...
I believe the problem comes from the fact, that the specified X-Amz-SignedHeaders Header is wrong. I am not sure which headers are used by default from the amazon rails sdk gem.
How should I change my url generation, so that a mobile client can just take the url and post a file to it?
Here is a solution:
In config/initializers/aws.rb:
AWS_CREDS = Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY'])
Aws.config.update({
region: 'eu-central-1',
credentials: AWS_CREDS
})
S3 = Aws::S3::Resource.new('eu-central-1')
S3_BUCKET_NAME = ENV['S3_BUCKET_NAME']
S3_BUCKET = S3.bucket(S3_BUCKET_NAME)
In your model/controller/concern/or whatever:
obj = S3_BUCKET.object("offers/#{user.id}/#{self.id}")
url = obj.presigned_url(:put) # obj.presigned_url(:put, acl: 'public-read') #if you want to make the file public
Then to upload you can use a mobile client or curl:
curl -X PUT -T file_to_upload "url from above"
Note that you will have to add the x-amz-acl: public-read header if you used the public-read acl option:
curl -H "x-amz-acl: public-read" -X PUT -T file_to_upload "url from above"
I am making a Ruby On Rails application and I am attempting to use the Google Plus API's for user sign in. To do this I am using Signet an OAuth helper library. I am looking at this code as an example of Signet with Google APIs.
Here is my code:
require "signet/oauth_2/client"
require "google/api_client"
oathClient = Signet::OAuth2::Client.new(
:authorization_uri => "https://accounts.google.com/o/oauth2/auth",
:token_credential_uri => "https://accounts.google.com/o/oauth2/token",
:client_id => Rails.application.secrets.gapi_client_id,
:client_secret => Rails.application.secrets.gapi_client_secret,
:redirect_uri => Rails.application.secrets.gapi_redirect_uri,
:scope => "https://www.googleapis.com/auth/plus.login")
gapi_client = Google::APIClient.new(
:application_name => "Branches",
:application_version => "0.0.1")
oathClient.code = request.body.read
oathClient.fetch_access_token!#Error on this line
gapi_client.authorization = oathClient
An error occurs on the second to last line:
oathClient.fetch_access_token!
This is the error:
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
I have done some research and it seams like this is caused by the lack of certificate information. However none of the solution's shown address this issue when dealing with Signet.
OS: Windows 8 x64
RoR version: 4.1.1
Signet Version: 0.5.1
Google API Client Version: 0.6.2
Looks related to https://github.com/googleapis/google-api-ruby-client/issues/235, where the solutions were:
update openssl
if you're using a custom .pem file, set it with: export SSL_CERT_FILE=/path/to/custom/certificate/authority.pem
I recently forked https://github.com/fortuity/rails3-mongoid-omniauth and tried to get login working for different providers. It works for Twitter and Facebook (You can try it out at http://jgodse-omniauth-mongoid.heroku.com/), but I couldn't get it working for Github. The code snapshot is here at github.
My environment looks like this:
$ heroku info
=== jgodse-omniauth-mongoid
Web URL: http://jgodse-omniauth-mongoid.heroku.com/
Git Repo: git#heroku.com:jgodse-omniauth-mongoid.git
Dynos: 1
Workers: 0
Repo size: 7M
Slug size: 5M
Stack: bamboo-mri-1.9.2
Data size: (empty)
Addons: Basic Logging, MongoHQ MongoHQ Free, Shared Database 5MB
Owner: xxxxx#yyy.com
Jay#JAY-PC ~/rapps/rails3-mongoid-omniauth (master)
$ heroku config --long
BUNDLE_WITHOUT => development:test
DATABASE_URL => postgres://xxxxxxxxxxxxxxxxxxxx.compute-1.amazonaws.com/rrretnhwhj
FACEBOOK_APP_ID => xxxxxxxxxxxxxxxxxxxx
FACEBOOK_APP_SECRET => xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
GITHUB_CLIENT_ID => xxxxxxxxxxxxxxxxxxxxx
GITHUB_SECRET => xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
LANG => en_US.UTF-8
MONGOHQ_URL => mongodb://heroku:9xxxxxxxxxxxxxxxx.mongohq.com:27098/app527030
RACK_ENV => production
SHARED_DATABASE_URL => postgres://xxxxxxxxxxxxxxxxxxxxx.compute-1.amazonaws.com/rrretnhwhj
TWITTER_KEY => xxxxxxxxxxxxxxxxxxx
TWITTER_SECRET => xxxxxxxxxxxxxxxxxxxxxxxxxxxx
My github information for "OAuth Application: Jay's Rails3 Mongoid OAuth" is as follows (from my app profile page):
Authorization Token: https://github.com/login/oauth/authorize
Access Token URL: https://github.com/login/oauth/access_token
URL: http://jgodse-omniauth-mongoid.heroku.com/
Callback URL: http://jgodse-omniauth-mongoid.heroku.com/
Client ID: xxxxxxx
Secret: xxxxxxxxxxxxxxxx
The client and secret are set as environment variables in omniauth.rb
The authentication happens, but it redirects to http://jgodse-omniauth-mongoid.heroku.com/?error=redirect_uri_mismatch and I haven't apparently logged in. If I change the Callback URL to http://jgodse-omniauth-mongoid.heroku.com/callback, the application crashes.
What am I missing to get github authentication to work?
I don't fu%^&*g believe this.
I went to github.com where the application secret, key, url, callback is configured and removed the trailing slash from the "Callback URL" and "URL", and it worked.
URL: http://jgodse-omniauth-mongoid.heroku.com
Callback URL: http://jgodse-omniauth-mongoid.heroku.com
This is nuts! Twitter wants the trailing slash on the callback but Github does not. Github and twitter should allow both and then trim it automatically if needed.