How to validate Dataflow Java code using Google Service Account?
Currently we are passing user credential to select/validate the Google project for executing Dataflow runner. Instead we need to validate the Dataflow code with Service Account.
It would be good if some one share the Java code to validate the Dataflow Job with Service Account.
To use the service account to communicate to Google Cloud all you have to do is to download the JSON file that contains your service account key and change the environment variable of the system to:
For Linux/MacOS:
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"
For Windows:
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\[FILE_NAME].json"
You can get this key from the GCP console, select the Service Account you require, select JSON type and then click on create. You now have the JSON file containing your key on your machine.
You then run the job with:
--serviceAccount=service-account#my-project.iam.gserviceaccount.com
This flag runs with the service account mail. You can get the service account mail from the IAM panel of the Google Console. Also, remember to give the service account the appropriated roles and permissions.
To change the name of the job, you can use --jobName=[your_job_name] in order to give an specific name. If you are running the example from the
Quickstart Using Java and Apache Maven documentation, then your job name is going to use part of your username as default.
Related
I want to create my first project in Watson Studio; however, I can not do it because when I click on the "Create" bottom I receive the following message "Unable to configure credentials for your project in the selected Cloud Object Storage instance". Does anyone know what this means?
Thanks,
You need to be IBM Cloud account Administrator or Owner if you are using shared account within your organization.
If you are not account Administrator, you should ask account administrator to enable storage delegation for cloud object storage service used for project so that non-administrator users under that account can create project using that Cloud Object storage service.
Please check this documentation link:-
https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/projects.html?audience=wdp
(See Requirements section).
https://dataplatform.cloud.ibm.com/docs/content/wsj/console/wdp_admin_cos.html?audience=wdp
There are also chances that this may be a temporary issue caused by some intermediate problem for IBM Cloud IAM service as this project creation attempts to create credentials for the storage service(IBM Cloud Object storage) you select, so you as account owner may also run into this issue. In such case, please reach out to IBM Support.
I'm trying to set up the TFS2018 search Service, but it reports a problem in the configuration wizard
I tried to change the user account to enable basic autentication in Serach Service and the Service Account, but always shows the same error.
Any suggestions?
Solved:
I needed to use Administrator as search service user, still using the Network service account for the service.
In this scenario, Elasticsearch service are using Network Service.
I am attempting to get get a Ruby on Rails project that uses the Google AdWords API.
What I did so far, following the steps in this guide:
I created an AdWords Manager account.
I created a test account that is part of the AdWords Manager account.
I set up the Ruby client library in my Rails project.
I then attempted to set up OAuth2 authentication with the example code from the guide.
However since the guide was written (and the video version of the guide was made) it seems that the interface has changed. I am able to create a Client ID client_secrets.json-file, or a Service Account .json file. I am able to export these and read the settings from .
I added the required settings, using an OAUTH2_SERVICE_ACCOUNT .json file.
Now, when attempting to connect, I get back the AdwordsAPIException AuthenticationError.NOT_ADS_USER.
I therefore know that the actual authentication works. However, the authorization does not.
How can I turn on AdWords API support for the oAuth credentials from my google accounts? The Google Credentials Console lists many APIs that you can turn on, but the AdWords API is not in there. The AdWords guide does not mention turning on an API at all, and only tells you to create a new Credential.
What is going on here?
The Adwords API does not need to be added to your project in the Google Cloud console (it's always enabled)—as indicated by the error message, the actual problem lies in the fact that your service account does not have access to any Adwords accounts.
As a matter of fact, the only way to use service accounts to authenticate against the Adwords API is when you're also using a G Suite domain (see the corresponding documentation, section "Prerequisites".
If you have a G Suite domain, you'll need to
Enable "G Suite Domain-wide Delegation" on your service account key
Add the project ID of the Google cloud project to your G Suite domain's authorized API client list
Use your service account to impersonate any user from your G suite domain that has Adwords access
As you can see, it's quite an involved process. My recommendation (that is shared by the above article) is to use an OAuth2 installed application flow for any user that has Adwords access. This requires to store the obtained refresh token on your end, but is more flexible (and arguably safer) than a delegation-enabled service account and easier to set up.
My pipeline doesn't start due to error:
(8e45efed0ad51300): Workflow failed. Causes: (8e45efed0ad51e7b): There was a problem refreshing your credentials.
Please check:
1. Dataflow API is enabled for your project.
2. There is a robot service account for your project: [project number]#cloudservices.gserviceaccount.com should have access to your project. If this account does not appear in the permissions tab for your project, contact Dataflow support.
I assume that Dataflow API is enabled as I'm able to reach Dataflow monitoring console, so first requirement is fulfilled. Second isn't. There is no a single account in the domain cloudservices.gserviceaccount.com.
Where can I ask for such help without paid support plan?
If you disable and then re-enable the Dataflow API for your project that should create the missing service account.
Hope that helps!
I am following below link to setup Service account and use GMail API to send emails by running java code from an application.
https://developers.google.com/identity/protocols/OAuth2ServiceAccount#delegatingauthority
But here i am unable to find out Security from the list of controls. And so unable to enter ClientID in Manage API client access screen.
Please help me to navigate properly.
Thanks
In order to perform what you want, you need to own a G Suite account. If you already own a G Suite account, then all you have to do is follow the steps here. If you do not own a G Suite account then you will have to impelement a solution without a service account, therefore you will need to follow the guide here