Azure KeyVault Encrypt API Endpoint Not Returning Base64 Encoded String - azure-keyvault

In PowerShell I'm calling the KeyVault Encrypt API endpoint to encrypt a connection string.
First I create a byte array of the connection string and then I create a base64 string. Then I call the /encrypt endpoint
[byte[]]$bytes = [System.Text.Encoding]::Unicode.GetBytes($targetConnectionString)
echo "Plain Bytes" $bytes
#$base64Array = [Convert]::ToBase64String($bytes)
#$EncryptedTargetConnectionString = Encrypt-ByteArray -accessToken $kvToken -vaultName $kvName -keyID $encryptionKeyId -plainArray $bytes
# converting the byte array to Base64 string
# Note from [Chris Clayton (AzureCAT)].
# "When performing the post methods there is a common theme of converting to Base64 strings which just makes sense so we do not introduce any invalid characters etc. and creating a JSON body as you can see here"
$base64Array = [Convert]::ToBase64String($bytes)
echo "Base 64 Array" $base64Array
# constructing the url for the vault encrypt operation by specifying the KeyID and Key version. If key version is not specified, then the most recent key version will be used
$queryUrl = "$encryptionKeyId" + '/encrypt?api-version=2016-10-01'
#echo "Query URL: " $queryUrl
# injecting the access token we obtained in the authorization header
$headers = #{ 'Authorization' = "Bearer $kvToken"; "Content-Type" = "application/json" }
echo "KV Token" $kvToken
# construct the body with our data to be encrypted and specifying RSA as encryption alg.
$bodyObject = #{ "alg" = "RSA-OAEP"; "value" = $base64Array }
$bodyJson = ConvertTo-Json -InputObject $bodyObject
# invoking a POST call to do the magic
$encryptionResponse = Invoke-RestMethod -Method POST -Ur $queryUrl -Headers $headers -Body $bodyJson
That seems to be working, because when I use Postman to call the Decrypt endpoint I get the response I'd expect.
Then I call an Azure Function which will need to be able to decrypt this connection string.
$EncryptedTargetConnectionString = $encryptionResponse.value
$webRequestBody = #"
{
"BlobUrl" : "$blobUrl",
"Environment" : "$env",
"State" : "$state",
"EncryptedConnectionStrings" : [
{
"EncryptedConnectionString": "$EncryptedTargetConnectionString"
}
]
}
"#
$webRequest = Invoke-WebRequest http://localhost:7071/api/DisableReplication -Headers #{"Accept" = "application/json"} -Method 'POST' -ContentType 'application/json; charset=utf-8' -Body $webRequestBody -UseBasicParsing
What's crazy here, though, is that the RESPONSE from the /encrypt endpoint doesn't seem to be in base64 format.
When I try to run Convert.FromBase64String(encryptedString) in C# against the encrypted string, it doesn't work. It tells me about it not being it base64 format.
C# Code
byte[] inputAsByteArray = Convert.FromBase64String(encryptedString);
// Encryption algorithm MUST match what was used in the PowerShell task in the Release Pipeline to encrypt the string
// Ex: $bodyObject = #{ "alg" = "RSA-OAEP"; "value" = $base64Array }
DecryptResult decryptResult = await cryptographyClient.DecryptAsync(JsonWebKeyEncryptionAlgorithm.RSAOAEP, inputAsByteArray);
string decryptedString = Encoding.UTF8.GetString(decryptResult.Plaintext);
return decryptedString;
None of the Key Vault SDK Clients that I've found for .NET Core support decrypting a string directly, and I can't seem to convert the string into a byte array the way I'd expect to be able to.
Any idea what kind of mind games Microsoft is playing here?

Key Vault uses Base64Url Encoding which is slightly different than Base64 encoding.
Base64URL encoding is nearly identical to Base64 encoding except for the fact that it substitutes characters that are not safe in a URL ('+','~', etc.) with other characters. In addition, it does not use padding ('=').

Related

How do you send MIME format emails using Microsoft Graph Java SDK?

The official documentation does not provide an example for any SDK's (including the Java SDK): https://learn.microsoft.com/en-us/graph/api/user-sendmail?view=graph-rest-1.0&tabs=java#example-4-send-a-new-message-using-mime-format. As there is no example, I have tried in vain to send the MIME content using the SDK (microsoft-graph 5.0.0):
Message sending = new Message();
ItemBody body = new ItemBody();
final String mimeMessageRFC822 = input.getMimeMessageRFC822();
body.content = Base64.getMimeEncoder().encodeToString(mimeMessageRFC822.getBytes());
sending.body = body;
GraphServiceClient service = getService(acHost, configuration);
service
.me()
.sendMail(UserSendMailParameterSet.newBuilder().withMessage(sending).withSaveToSentItems(true).build())
.buildRequest(new HeaderOption("Content-Type", "text/plain"))
.post();
The above code sets the request's content-type to text/plain, however the request body that is being sent is JSON (xxxxxx below is a placeholder for a valid Base64 encoded MIME content string).
{
"message":
{
"body":
{
"content": xxxxxx
}
},
"saveToSentItems": true
}
The response is a 404, stating:
GraphServiceException: Error code: ErrorMimeContentInvalidBase64String
Error message: Invalid base64 string for MIME content.
I can understand why it is responding with this error as the graph endpoint is parsing the text/plain content as base64 encoded MIME but finds the JSON structure instead. I have been on a video call with a Microsoft Graph support agent, and they have seen that my MIME content is valid. Sadly, they are not able to help with the Microsoft Graph Java SDK even though it is developed by Microsoft!
This suggests that we are not supposed to use the Java SDK at all for sending MIME formatted emails. Is this correct? Surely it can't be otherwise what is the point of a library that can receive MIME formatted emails but can't send them? Does anyone have a solution?
For now at least the solution is to send a CustomRequest with MIME content instead of using the fluent API provided by the Graph client.
final String encodedContent = Base64.getMimeEncoder().encodeToString(mimeMessageRFC822.getBytes());
CustomRequest<String> request = new CustomRequest<>(requestUrl, service, List.of(new HeaderOption("Content-Type", "text/plain")), String.class);
request.post(encodedContent);

Unsupported grant type in Google OAuth

I am getting an error "unsupported grant type" when I try to request an OAuth token for a service account using curl. I'm following the example for OAuth 2.0 for service accounts (https://developers.google.com/identity/protocols/OAuth2ServiceAccount) and I think I have everything setup correctly. I have a service account setup in Google Cloud and I'm using that email address in the OAuth request.
The documentation says to use the URL encoded grant type "urn:ietf:params:oauth:grant-type:jwt-bearer" but it isn't clear if this is the only option for the grant type or what other options might be.
I am sending the the base64 encoded header
{"alg":"RS256","typ":"JWT"}
and "."
and base64 encoded claims
{
"iss":"chargepubadmin#xxxxxxxx.iam.gserviceaccount.com",
"scope":"https://www.googleapis.com/auth/pubsub",
"aud":"https://www.googleapis.com/oauth2/v4/token",
"exp":1497159875,
"iat":1497156275
}
and "."
and base64 encoded signature
{base64 header}.{base64 claims}
.
curl -X POST -d 'grant_type=http%3A%2F%2Foauth.net%2Fgrant_type%2Fdevice%2F1.0%26assertion=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.ew0KICAiaXNzIjoiY2.......' "https://www.googleapis.com/oauth2/v4/token"
I'm using an online base64 encoding tool which matches the example base64 encoding.
Can anyone enlighten me as to the what the grant type is or should be?
The grant type should be set as urn:ietf:params:oauth:grant-type:jwt-bearer documented here under the REST API Making the access token request section.
Working example using google-auth library
It will be very easy and simple, if you used the google-auth library which automatically takes care of parsing the private key json file, fetching access tokens, refreshing them and actually including them as part of the requests.
You only need to provide the request URL and body, the library takes care of the rest. Here is a simplified example:
#!/usr/bin/env python
from google.auth.transport.requests import AuthorizedSession
from google.oauth2.service_account import Credentials
# BEGIN CONFIGURATION - change as needed.
# Path to the JSON file containing the service account private key and email.
PRIVATE_KEY_JSON = '/path/to/json/file'
# The API scope this token will be valid for.
API_SCOPES = ['https://www.googleapis.com/auth/pubsub']
# END CONFIGURATION
if __name__ == '__main__':
credentials = Credentials.from_service_account_file(
PRIVATE_KEY_JSON, scopes=API_SCOPES)
authed_session = AuthorizedSession(credentials)
url = 'https://pubsub.googleapis.com/v1/<SOMETHING>'
response = authed_session.get(url)
print str(response.content)
Working example without additional libraries
If you do not want to use any additional libraries but can use the standard python libraries, here is a working sample (tested personally with a service account of my own) in Python (supports both 2.x and 3.x versions) which takes care of all the steps:
#!/usr/bin/env python
import Crypto.PublicKey.RSA as RSA
import Crypto.Hash.SHA256 as SHA
import Crypto.Signature.PKCS1_v1_5 as PKCS1_v1_5
import base64
import json
import time
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
try:
from urllib.parse import urlencode
except ImportError:
from urllib import urlencode
# BEGIN CONFIGURATION - change as needed.
# Path to the JSON file containing the service account private key and email.
PRIVATE_KEY_JSON = '/path/to/json/file'
# The API scope this token will be valid for.
API_SCOPE = 'https://www.googleapis.com/auth/pubsub'
# The validity of the token in seconds. Max allowed is 3600s.
ACCESS_TOKEN_VALIDITY_SECS = 3600
# END CONFIGURATION
class OauthAccessTokenGetter:
"""Fetches a new Google OAuth 2.0 access token.
The code is based on the steps described here: https://developers.go
ogle.com/identity/protocols/OAuth2ServiceAccount#authorizingrequests
"""
ACCESS_TOKEN_AUD = 'https://www.googleapis.com/oauth2/v4/token'
REQUEST_URL = 'https://www.googleapis.com/oauth2/v4/token'
GRANT_TYPE = 'urn:ietf:params:oauth:grant-type:jwt-bearer'
def __init__(self, private_key_json_file, scope, token_valid_secs=3600):
self.private_key_json = self.LoadPrivateKeyJsonFromFile(
private_key_json_file)
self.scope = scope
self.token_valid_secs = token_valid_secs
#classmethod
def Base64UrlEncode(cls, data):
"""Returns the base64url encoded string for the specified data."""
return base64.urlsafe_b64encode(data)
#classmethod
def LoadPrivateKeyJsonFromFile(cls, private_key_json_file):
"""Returns JSON object by parsing the specified private key JSON
file."""
with open(private_key_json_file) as private_key_json_file:
return json.load(private_key_json_file)
def GetPrivateKey(self):
"""Returns the imported RSA private key from the JSON data."""
return RSA.importKey(self.private_key_json['private_key'])
def GetSigner(self):
"""Returns a PKCS1-V1_5 object for signing."""
return PKCS1_v1_5.new(self.GetPrivateKey())
#classmethod
def GetEncodedJwtHeader(cls):
"""Returns the base64url encoded JWT header."""
return cls.Base64UrlEncode(json.dumps({'alg': 'RS256', 'typ': 'JWT'}).encode('utf-8'))
def GetEncodedJwtClaimSet(self):
"""Returns the base64url encoded JWT claim set."""
current_time_secs = int(time.time())
jwt_claims = {
'iss': self.private_key_json['client_email'],
'scope': self.scope,
'aud': self.ACCESS_TOKEN_AUD,
'exp': current_time_secs + self.token_valid_secs,
'iat': current_time_secs
}
return self.Base64UrlEncode(json.dumps(jwt_claims).encode('utf-8'))
def GetJwtSignature(self, message):
"""Returns signature of JWT as per JSON Web Signature (JWS) spec."""
signed_message = self.GetSigner().sign(SHA.new(message))
return self.Base64UrlEncode(signed_message)
def GetSignedJwt(self):
"""Returns signed JWT."""
header = self.GetEncodedJwtHeader()
jwt_claim_set = self.GetEncodedJwtClaimSet()
signature = self.GetJwtSignature(header + b'.' + jwt_claim_set)
return header + b'.' + jwt_claim_set + b'.' + signature
def SendRequest(self, body):
"""Returns the response by sending the specified request."""
return urlopen(self.REQUEST_URL, urlencode(body).encode('utf-8')).read()
def GetAccessToken(self):
"""Returns the access token."""
body = {
'grant_type': self.GRANT_TYPE,
'assertion': self.GetSignedJwt()
}
response = json.loads(self.SendRequest(body))
return response['access_token']
if __name__ == '__main__':
print (OauthAccessTokenGetter(PRIVATE_KEY_JSON, API_SCOPE,
ACCESS_TOKEN_VALIDITY_SECS).GetAccessToken())
After you get the access token, you need to include it as the Bearer header in the requests you send as described here.
GET /drive/v2/files HTTP/1.1
Authorization: Bearer <access_token>
Host: www.googleapis.com/
Equivalently in curl as:
curl -H "Authorization: Bearer <access_token>" https://www.googleapis.com/drive/v2/files
Although it is described here that you can specify the token using access_token= parameter, I could not get it working at least for Google Compute Engine APIs, may be it works with PubSub, but the Bearer header approach has worked always in my experience.
UPDATE: As per the discovery doc for PubSub API, there seems to be a query parameter for access_token=, so it might very well work too.
"access_token": {
"description": "OAuth access token.",
"type": "string",
"location": "query"
},
And the discovery doc for Compute Engine APIs indicate the use of oauth_token query parameter instead and I did verify that it worked.
"oauth_token": {
"type": "string",
"description": "OAuth 2.0 token for the current user.",
"location": "query"
},

Single-quoted body in API request

I am making an iOS app that makes requests to an API. The request is in this format:
curl -X PUT -H "Content-Type: application/json" -d '{"username":"blabla#hotmail.com","password":"blabla"}' "https://server.mywebsite.com/login"
The API can only accept single-quoted strings in the body but I can't make a string with single quotes in Swift without it adding backslashes and making the string unreadable by the API.
"\'{\"email\": \"blabla#hotmail.com\", \"password\": \"blabla\"}\'"
Is there a way I can pass this string in Swift without the backslashes? Or is there a String or JSON encoding that is in that format?
The single quotes in your curl are required just in the Unix shell (to quote the double quotes on the command line), they are not actually transmitted to the server. The server just sees this JSON payload:
{"username":"blabla#hotmail.com","password":"blabla"}
So in your Swift API request you can remove the single quotes from your string:
let auth = "{\"email\": \"blabla#hotmail.com\", \"password\": \"blabla\"}"
Is there are way to avoid the escaping of the double-quotes here? No. In Swift you can't switch between ' and " like you can in say Python. Nor does it have """.
Since it is easy to make quoting errors when building the JSON on your own, you may want to use JSONSerialization instead, like so:
let jsonAuth = [ "email": "blabla#hotmail.com",
"password": "blabla" ]
let jsonData = JSONSerialization.data(withJSONObject: jsonAuth)
let jsonString = String(data: data, encoding: .utf8)

Twitter Application Only Auth

I'm trying to get an Application Only Auth token following the steps of this link:
https://dev.twitter.com/docs/auth/application-only-auth
I'm using Ruby on Rails and Rest Client to make the POST request needed and I'm setting the headers (I think) properly.
The step-by-step says:
URL encode the consumer key and the consumer secret according to RFC
1738. Note that at the time of writing, this will not actually change the consumer key and secret, but this step should still be performed
in case the format of those values changes in the future.
Concatenate the encoded consumer key, a colon character ":", and the
encoded consumer secret into a single string.
Base64 encode the string from the previous step.
And my code is:
require 'rest_client'
key = URI::encode('app_key')
secret = URI::encode('app_secret')
encoded = Base64.encode64("#{key}:#{secret}")
res = RestClient::Resource.new "https://api.twitter.com/oauth2/token/"
response = ''
options = {}
options['Authorization'] = "Basic #{encoded}"
options['Content-Type'] = 'application/x-www-form-urlencoded;charset=UTF-8'
res.post('grant_type=client_credentials', options) do |response, request, result|
response << "#{CGI::escapeHTML(response.inspect)}<br /><br />"
response << "#{CGI::escapeHTML(request.inspect)}<br /><br />"
response << "#{CGI::escapeHTML(result.inspect)}<br />"
end
render :text => txt
And I print out this:
"{\"errors\":[{\"label\":\"authenticity_token_error\",\"code\":99,\"message\":\"Unable to verify your credentials\"}]}"
#<RestClient::Request:0x9ece5d8 #method=:post, #headers={"Authorization"=>"Basic bXlfa2V5Om15X3NlY3JldA==\n", "Content-Type"=>"application/x-www-form-urlencoded;charset=UTF-8"}, #url="https://api.twitter.com/oauth2/token/", #cookies={}, #payload="", #user=nil, #password=nil, #timeout=nil, #open_timeout=nil, #block_response=nil, #raw_response=false, #verify_ssl=false, #ssl_client_cert=nil, #ssl_client_key=nil, #ssl_ca_file=nil, #tf=nil, #max_redirects=10, #processed_headers={"Accept"=>"*/*; q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Basic bXlfa2V5Om15X3NlY3JldA==\n", "Content-Type"=>"application/x-www-form-urlencoded;charset=UTF-8", "Content-Length"=>"29"}, #args={:method=>:post, :url=>"https://api.twitter.com/oauth2/token/", :payload=>"grant_type=client_credentials", :headers=>{"Authorization"=>"Basic bXlfa2V5Om15X3NlY3JldA==\n", "Content-Type"=>"application/x-www-form-urlencoded;charset=UTF-8"}}>
#<Net::HTTPForbidden 403 Forbidden readbody=true>
My key and secret are valid.
Am I missing something?
Thanks!
EDIT:
Updating with the solution I've found.
The problem was on the Base64 convertion and string encoding.
I had to add a forced encoding parameter to the key+secret combination, for UTF-8 convertion:
encoded = Base64.encode64("#{key}:#{secret}".force_encoding('UTF-8'))
The Rails Base64.encode64 inserts a line break every 60 encoded characters.
The workaround was:
For Ruby 1.9+ (strict_ was included in Ruby 1.9)
Base64.strict_encode64(string)
For Ruby 1.9-
Base64.encode64(string).gsub('/\n/') # To remove the line break
Are you trying to implement Authorization with Tweeter (as OAuth Provider). Instead of writing it from the scratch following the API documentation, I would suggest to use OmniAuth. The setup & boilerplate code is fairly easy to use.
Read more about it at http://www.omniauth.org/ & https://github.com/intridea/omniauth/wiki
Let us know, if that helped you or not.

Convert Rails Net::HTTP request to MD5 Hex Digest

In order to use a third-party API, I need to encode the Net::HTTP::Post request as an MD5 hex digest, which is then used as part of the signature. However, when I try to simply Digest::MD5.hexdigest(req), it throws a "Cannot convert to string error", and when I explicitly req.to_s, it just gives the MD5 of #<Net::HTTP::Post:0x112a0eef8>
I'm simply:
request = Net::HTTP::Post.new(url.path)
request.body = {
"key" => "val"
}.to_json
# later...
hexDigest = Digest::MD5.hexdigest(request)
which is the documented spec, I think: "[with the] JSON body containing the new information."
This is the relevant sample Java code they supply:
ByteArrayOutputStream requestOutputStream = new ByteArrayOutputStream();
httpMethod.getEntity().writeTo(requestOutputStream);
DigestUtils.md5Hex(requestOutputStream.toByteArray()).toLowerCase();
Any ideas?
Thanks!
Try to call 'to_s' method explicitly, it should help:
hexDigest = Digest::MD5.hexdigest(request.to_s)
The equivalent ruby code for those lines is:
OpenSSL::Digest::MD5.hexdigest(request.body)
httpMethod.getEntity() will return the json defined as the request body.
requestOutputStream.toByteArray() will return the array of bytes corresponding to the request body.

Resources