Passing secrets to web activity in Azure Data Factory - azure-keyvault

I am using web activity to get a bearer token from an authentication endpoint. I need to pass secrets such as user id and password in the header as below.
Headers in web activity
I want to fetch user id and password from Azure key vault and pass it to web activity. Is there a way I can access the key vault in the expression?
Currently, I am fetching the secrets from an azure function and passing them as parameters to the pipeline.

Currently you can´t access directly to Azure Key vault.
You can create a json file like:
{
"IC-USER-ID": "abc",
"IC-USER-PASSWORD": "abc123",
}
And read from it. In release process you can read the values from key vault and update json file. You have to ensure that only the right person can access the file.

Related

Can't understand aqueduct auto-generated db and auth

I created an Aqueduct project using aqueduct create -t db_and_auth but I did not understand how registration and authentication with OAuth 2.0 works. Can someone explain how to register from OAuth2.0 and DB template auto-created by aqueduct and what steps I have to do to register and then authenticate?
From a client application, you POST /register with a JSON payload containing a user. Depending on what version of the template you have, this may just be {"username": "bob", "password": "password"} - check the definition of your _User type.
When you are authenticating an already existing user, you invoke POST /auth/token and pass the username, password and other required fields as x-www-form-urlencoded data. The format of that request - written in Dart code - is here: http://aqueduct.io/docs/auth/controllers/.
Whether you are registering a new user or authenticating an existing user, you have to provide a client identifier (and optionally client secret) as a Basic Authorization header. The client identifier must have already been registered with your application and stored in its database.
To store client identifiers in a database, you'll need to first run your application's database migrations on a database instance (see http://aqueduct.io/docs/db/db_tools/ for running database migrations). This will create tables to store OAuth 2.0 client identifiers and tokens.
Then you'll need to add OAuth2.0 client identifiers to your database. This is best accomplished using the aqueduct auth CLI, and there is documentation on it here: http://aqueduct.io/docs/auth/cli/.

'Insufficient Privileges' error while using 'addKey' action in Azure AD Graph API

I have an application registered in Azure AD which uses certificates. I am trying to write a script which would add a new certificate to the application. This can be used to add a new certificate when the existing certificate is going to expire.
I am trying to use AddKey function of Azure AD Graph API. The request body of this api as a parameter 'proof' which is a JWT assertion signed by the existing certificate of the application. The doc says the "aud" claim in JWT should be set to "AAD Graph SPN". Here what is meant by "AAD Graph SPN"?
I tried with a JWT where "aud" was set to "00000002-0000-0000-c000-000000000000". But I am getting the following error,
{
"odata.error": {
"code":"Authorization_RequestDenied",
"message":{
"lang":"en",
"value":"Insufficient privileges to complete the operation."
}
}
}
Any thoughts on this?
I am getting the access token to call the Azure AD Graph API via "Resource Owner Credentials Grant" flow . To get the access token i am using the client_id "1950a258-227b-4e31-a9cf-717495945fc2" (The Well Known Client ID for Azure PowerShell")
My script (For deployment purpose) does something like below,
i) Get the access token as described above and registers a new application in Azure AD with a initial certificate.
ii) When the initial certificate is about to expire it should add a new certificate to the created application.
According to the documentation, you must use a self-signed JWT token to access that API:
As part of the request validation for this service action, a proof of
possession of an existing key is verified before the action can be
performed. The proof is represented by a self-signed JWT token. The
requesting application needs to generate a self-signed JWT token with
the following requirements...
The "Resource Owner Credentials Grant" won't work here in this situation. Instead, use the "Client Credentials Grant":
https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-protocols-oauth-service-to-service
The application you want to update should be the Client ID used to get this access token.
The other option is to update the application directly using an PATCH request on the Application Object:
https://msdn.microsoft.com/en-us/library/azure/ad/graph/api/entity-and-complex-type-reference#application-entity
Using this method, you should be able to update using the method you described above (user credentials and an external Client ID)
Let me know if this helps.

Proper method of getting a server auth access token for a client to use with google analytics

I have a global account that has several views that I want to use on the server side to embed dashboards for the various views on the client side. From what I understand, I get an access token using a service account on the server side and can then send the access token to the client side whenever needed. I was wondering, is this the correct flow? Should the access token be per session?
The authorization on the client side shown here has a field for a server auth access token, but couldn't find documentation on the exact flow I wanted. Basically I'm unsure what the proper way of generating that server auth access token is. Any help/pointers would be very much appreciated.
[Here][1] is an example of how to set up server side auth. The above code creates a new token when anyone visits the site. You can see the endpoint that gets that access token [here][2].
Below are the general steps to get to a working version:
Step 1: Create a service account and download the JSON key
Step 2: Add the service account as a user in Google Analytics
Step 3: Use the JSON key data to request an access token
# service-account.py
import json
from oauth2client.client import SignedJwtAssertionCredentials
# The scope for the OAuth2 request.
SCOPE = 'https://www.googleapis.com/auth/analytics.readonly'
# The location of the key file with the key data.
KEY_FILEPATH = 'path/to/json-key.json'
# Load the key file's private data.
with open(KEY_FILEPATH) as key_file:
_key_data = json.load(key_file)
# Construct a credentials objects from the key data and OAuth2 scope.
_credentials = SignedJwtAssertionCredentials(
_key_data['client_email'], _key_data['private_key'], SCOPE)
# Defines a method to get an access token from the credentials object.
# The access token is automatically refreshed if it has expired.
def get_access_token():
return _credentials.get_access_token().access_token
Back to the client side:
Step 4: Load the Embed API library.
<script>
(function(w,d,s,g,js,fs){
g=w.gapi||(w.gapi={});g.analytics={q:[],ready:function(f){this.q.push(f);}};
js=d.createElement(s);fs=d.getElementsByTagName(s)[0];
js.src='https://apis.google.com/js/platform.js';
fs.parentNode.insertBefore(js,fs);js.onload=function(){g.load('analytics');};
}(window,document,'script'));
</script>
Step 5: Add HTML containers to host the dashboard components.
<div id="chart-1-container"></div>
<div id="chart-2-container"></div>
Step 6: Write the dashboard code.
Use the access token obtained in step 3 to authorize the Embed API.
gapi.analytics.ready(function() {
/**
* Authorize the user with an access token obtained server side.
*/
gapi.analytics.auth.authorize({
'serverAuth': {
'access_token': '{{ ACCESS_TOKEN_FROM_SERVICE_ACCOUNT }}'
}
});
...
The additional work of creating an endpoint which returns the token depends on your back end implementation but the source code of how the demo does it can be found [here][2].
[1]: https://ga-dev-tools.appspot.com/embed-api/server-side-authorization/
[2]: https://github.com/googleanalytics/ga-dev-tools/blob/abb3c5a18160327a38bf5c7f07437dc402569cac/lib/controllers/server_side_auth.py
I'm not very familiar with Google Analytics, but as far as OAuth goes, the handling of access tokens and refresh tokens should all be on the server-side. The client receives an authorization code and provides that to the server, which then obtains the tokens and uses the tokens to obtain the data necessary. There shouldn't be any need to send an access token to the client.
It might be helpful to read this, which describes the standard OAuth flow:
https://developers.google.com/identity/protocols/OAuth2

How to make API requests with an access_token for a Service Account

My end goal is to be able to retrieve place details from Google's API.
I need to do this as a Service Account, since this is kicked off as a background task on my server. Service Accounts require you to exchange a JWT (JSON Web Token) for an access_token. I finally got that working and am receiving an access_token. Phew.
Now however, I don't know what to do with this access_token.
The Place Details API says that the key parameter is required, but I don't have a key. Just an access_token. Using that value for key or changing the name of the paramater to access_token is not working.
Ultimately I need to be able to hit a URL like so:
https://maps.googleapis.com/maps/api/place/details/json?reference={MY_REFERENCE}&sensor=false&key={MY_ACCESS_TOKEN}
How do I use my Access Token to make a request to the Google Place Detail APIs?
Update 1
Still no success, but I thought I'd post the details of my request in case there's something wrong with what I'm submitting to Google.
I'm using the JWT Ruby library, and here are the values of my claim set:
{
:iss => "54821520045-c8k5dhrjmiotbi9ni0salgf0f4iq5669#developer.gserviceaccount.com",
:scope => "https://www.googleapis.com/auth/places",
:aud => "https://accounts.google.com/o/oauth2/token",
:exp => (Time.now + 3600),
:iat => Time.now.to_i
}
Looks sane to me.
Create the service account and its credentials
You need to create a service account and its credentials. During this procedure you need to gather three items that will be used later for the Google Apps domain-wide delegation of authority and in your code to authorize with your service account. These three items are your service account:
• Client ID.
• Private key file.
• Email address.
In order to do this, you first need a working Google APIs Console project with the Google Calendar API enabled. Follow these steps:
Go to the Google APIs Console.
Open your existing project or create a new project.
Go to the Service section.
Enable the Calendar API (and potentially other APIs you need access to).
You can now create the service account and its credentials. Follow these steps:
Go to the API Access section.
Create a client ID by clicking Create an OAuth 2.0 client ID...
Enter a product name, specify an optional logo and click Next.
Select Service account when asked for your Application type and click Create client ID.
At this point you will be presented with a dialog allowing you to download the Private Key as a file (see image below). Make sure to download and keep that file securely, as there will be no way to download it again from the APIs Console.
After downloading the file and closing the dialog, you will be able to get the service account's email address and client ID.
You should now have gathered your service account's Private Key file, Client ID and email address. You are ready to delegate domain-wide authority to your service account.
Delegate domain-wide authority to your service account
The service account that you created now needs to be granted access to the Google Apps domain’s user data that you want to access. The following tasks have to be performed by an administrator of the Google Apps domain:
Go to your Google Apps domain’s control panel. The URL should look like: www.google.com/a/cpanel/mydomain.com
Go to Advanced tools... > Manage third party OAuth Client access.
In the Client name field enter the service account's Client ID.
In the One or More API Scopes field enter the list of scopes that your application should be granted access to (see image below). For example if you need domain-wide access to the Google Calendar API enter: www.googleapis.com/auth/calendar.readonly
Click the Authorize button.
Your service account now has domain-wide access to the Google Calendar API for all the users of your domain, and potentially the other APIs you’ve listed in the example above.
Below is a description that uses a service account to access calendar data in PHP
The general process for service account access to user calendars is a follows:
• Create the Google client
• Set the client application name
• If you already have an Access token then check to see if it is expired
• If the Access token is expired then set the JWT assertion credentials and get a new token
• Set the client id
• Create a new calendar service object based on the Google client
• Retrieve the calendar events
Note: You must save the Access token and only refresh it when it is about to expire otherwise you will receive an error that you have exceeded the limit for the number of access tokens in a time period for a user.
Explanation of Google PHP Client library functions used:
The client object has access to many parameters and methods all of the following are accessed through the client object:
Create a new client object:
$client = new Google_Client();
Set the client application name:
$client->setApplicationName(“My Calendar App”);
Set the client access token if you already have one saved:
$client->setAccessToken($myAccessToken);
Check to see if the Access token has expired, there is a 30 second buffer, so this will return true if the token is set to expire in 30 seconds or less. The lifetime of an Access token is one hour. The Access token is actually a JSON object which contains the time of creation, it’s lifetime in seconds, and the token itself. Therefore no call is made to Google as the token has all of the information locally to determine when it will expire.
$client->isAccessTokenExpired();
If the token has expired or you have never retrieved a token then you will need to set the assertion credentials in order to get an Access token:
$client->setAssertionCredentials(new Google_AssertionCredentials(SERVICE_ACCOUNT_NAME,array(CALENDAR_SCOPE), $key,'notasecret','http://oauth.net/grant_type/jwt/1.0/bearer',$email_add));
Where:
SERVICE_ACCOUNT_NAME is the the service account email address setup earlier.
For example:’abcd1234567890#developer.gserviceaccount.com’
CALENDAR_SCOPE is the scope setup in the Google admin interface.
For example: ‘https://www.googleapis.com/auth/calendar.readonly’
$key is the content of the key file downloaded when you created the project in Google apps console.
$email_add is the Google email address of the user for whom you want to retrieve calendar data.
Set the client id:
$client-setClientId(SERVICE_CLIENT_ID);
Where:
SERVICE_CLIENT_ID is the service account client ID setup earlier.
For example: ‘abcd123456780.apps.googleusercontent.com’
Create a new calendar service object:
$cal = new Google_CalendarService($client);
Several options can be set for calendar retrieval I set a few of them in the code below, they are defined in the api document.
$optEvents = array('timeMax' => $TimeMax, 'timeMin' => $TimeMin, 'orderBy' => 'startTime', 'singleEvents' => 'True');
Get the list of calendar events and pass the above options to the call:
$calEvents = $cal->events->listEvents('primary', $optEvents);
Loop through the returned event list, the list is paged so we need to fetch pages until the list is exhausted:
foreach ($calEvents->getItems() as $event) {
// get event data
$Summary = $event->getSummary();
$description = $event->getDescription();
$pageToken = $calEvents->getNextPageToken();
if ($pageToken) { // if we got a token the fetch the next page of events.
$optParams = array('pageToken' => $pageToken);
$calEvents = $cal->events->listEvents('primary', $optParams);
} else {
break;
}
}
Get the Access token:
$myAccessToken=$client->getAccessToken();
Save the access token to your permanent store for the next time.
The language isn't important php, ruby, .net, java the process is the same. The api's console shows the Places API as supporting service accounts so it should be possible to access it.
As far as using the token please have a look at https://code.google.com/p/google-api-ruby-client/ code as the usage is clearly defined in the code repository. Doesn't make any difference if the access token is for a service account or a single user the process for using the token is the same. See the section titled "Calling a Google API" in the following link: https://developers.google.com/accounts/docs/OAuth2InstalledApp
The access token is sent in the http authorization header along with the request.For a calendar request it would look something like the following:
GET /calendar/v3/calendars/primary HTTP/1.1
Host: www.googleapis.com
Content-length: 0
Authorization: OAuth ya29.AHES6ZTY56eJ0LLHz3U7wc-AgoKz0CXg6OSU7wQA

How can I get the quantity_available field to be returned from calls to the EventBrite API?

The Eventbrite documentation on the ticket object indicates that it can contain a quantity_available or quantity_sold field, but that to see either of these fields "requires authentication". It doesn't give any more detail than that, though, and when I make calls to the event_search method using my app key, the tickets objects in the returned events do not contain quantity_available or quantity_sold keys.
What authentication is required to see these fields? Are they only visible to the owners of the event, or is it possible in some way for me to have the API return the number of tickets available for somebody else's event?
If this is not possible through the API, is the number of tickets remaining for an event publicly visible anywhere else on Eventbrite where I could get to it with a web scraper?
This needs to be called as an expansion. There are some more details here:
https://groups.google.com/forum/#!msg/eventbrite-api/sjMO-gV8-Go/uzw7GHq2_SEJ
Basically, calling it like so will populate the proper fields using your Apps OAuth token in python3:
import requests
eventbrite_response = requests.get(
"https://www.eventbriteapi.com/v3/events/<YOUR EVENT ID HERE>/?expand=ticket_classes",
headers = {
"Authorization": "Bearer <YOUR APP OAUTH TOKEN>",
},
verify = True, # Verify SSL certificate
)
print(eventbrite_response.json()['ticket_classes'][0]['quantity_sold'])
You can tailor the print function at the end to include more of the json data if you wish.
In order to read or write private data using the Eventbrite API, you will need to supply additional user-authentication tokens. This extra information lets Eventbrite know who should be authorized to access private data (including quantity_available and quantity_sold values) during the request.
Whenever you provide additional user access tokens, both public and private data will be available.
Authentication parameters include:
app_key: An application key (also referred to as an API key), identifies the application that is contacting the API. All API requests must include some form of application identification. If this is the only authentication token provided, the API request will be limited to publicly available data. Application keys have a default rate-limit of 1000 requests per day. You can get and manage your API keys here: https://www.eventbrite.com/api/key/
access_token: Recommended. OAuth2 access tokens are tied to a user account and an application key. Since the user-authorized application can also be identified via this token, it is the only authentication parameter that does not require an application key to be provided as well. Be careful not to expose these tokens to other users! Additional request headers are required when using access_tokens to contact our API: “Authorization: Bearer YOUR_ACCESS_TOKEN_HERE“. You can learn more about how to configure your application for OAuth2.0 here: http://developer.eventbrite.com/doc/authentication/oauth2/
user_key: Each Eventbrite account has an associated user_key. This token provides access to the related user’s account data, in addition to our publicly available data. This authentication method is preferred for use-cases that require private data-access where OAuth2.0 workflows are not possible. This token unlocks sensitive information, so be very careful not to expose this token to other users!
Here is an example of an API call that is using both the app_key and user_key parameters to return private data (remember to substitute in your own app_key and user_key):
https://www.eventbrite.com/json/user_list_events -G -d app_key=APPKEY -d user_key=USERKEY
You can also see the authentication documentation here: http://developer.eventbrite.com/doc/authentication/

Resources