Can't decode saml message from shibboleth SP - invalid byte 1 of 1-byte UTF-8 sequence - java - xml-parsing

In some installations we have a Service Provider Shibboleth Sp v3 and a Shibbleth idp V4 and all works fine.
For some reasons in an other environments we have to rely on a customized idp that we are writing in java (a sort of proxy between different sp and different third party idps), we still have the shibboleth service providers v3 like in the other environment, but this time they call our custom idp instead of the shibboleth one.
Usually in our custom idp we handle saml messages like this:
String samlmessage = request.getParameter("SAMLResponse");
or
String samlmessage = request.getParameter("SAMLRequest");
Where "request" is an javax.servlet.http.HttpServletRequest object.
And decode it in this manner:
byte[] base64DecodedMessagge = Base64.decodeBase64(samlmessage);
ByteArrayInputStream is = new ByteArrayInputStream(base64DecodedMessagge);
DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();
documentBuilderFactory.setNamespaceAware(true);
DocumentBuilder docBuilder = documentBuilderFactory.newDocumentBuilder();
Document document = docBuilder.parse(is);
etc.
We have a problem decoding the logout request sent by the shibboleth sp to our idp:
At this instruction:
Document document = docBuilder.parse(is);
this exception is raised:
org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence.
(note that for example decoding with the same code the authentication response of the third party idp does not raise this exception)
With saml tracer I see the saml of all messages, also the one that in java gives me the exception.
But if I take the value of these parameters (always from saml tracer), for example:
SAMLResponse: PHNhbWxwOlJlc3BvbnNlIER...etc etc
SAMLRequest: nVNdb6JAFP0rhNdGgUGFTtSE...etc etc
and i test in a decoder like this with utf-8:
https://www.base64decode.org/
Are decoded except the one that in java gives me an exception.
If saml tracer shows me the saml message value, it somehow managed to decode it..where am I wrong in the decoding? Is there a way to write equivalent code with opensaml libraries and have more reliable code in decoding?
Thanks for any helps.

I found a solution, based on this :
https://stackoverflow.com/a/60813998/3147345
Instead of using:
byte[] base64DecodedMessagge = org.apache.commons.codec.binary.Base64.Base64.decodeBase64(samlmessage);
ByteArrayInputStream is = new ByteArrayInputStream(base64DecodedMessagge);
DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();
documentBuilderFactory.setNamespaceAware(true);
DocumentBuilder docBuilder = documentBuilderFactory.newDocumentBuilder();
Document document = docBuilder.parse(is);
etc.
I used for this particular message:
byte[] decodedMessage = org.apache.xml.security.utils.Base64.decode(samlmessage);
byte[] inflatedData = new byte[(10 * decodedMessage.length)];
Inflater decompresser = new Inflater(true);
decompresser.setInput(decodedMessage, 0, decodedMessage.length);
int inflatedBytesLength = decompresser.inflate(inflatedData);
decompresser.end();
String inflated = new String(inflatedData, 0, inflatedBytesLength);
ByteArrayInputStream is = new ByteArrayInputStream(inflated.getBytes());
DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();
documentBuilderFactory.setNamespaceAware(true);
DocumentBuilder docBuilder = documentBuilderFactory.newDocumentBuilder();
Document document = docBuilder.parse(is);
etc.

Related

Can we use same auth cookie (generated in dot net core api) in a different api applications but in the same domain

I have an application(core 2.2.0) which uses a cookie for authentication, below is the code from the Startup.cs file
services.AddAuthentication(cookieConfig.AuthScheme)
.AddCookie(cookieConfig.AuthScheme, options => {
options.LoginPath = new PathString(cookieConfig.LoginPath);
options.AccessDeniedPath = new PathString(cookieConfig.AccessDeniedPath);
options.Cookie = cookie;
options.Events = cookieEvents;
});
And below code is from the Sign-in API
await HttpContext.SignInAsync(_cookieConfig.AuthScheme, userPrincipal, authProps);
Suppose this application generates a cookie 'ABC', and it has a URL1 - https://somedomain.com/api
and I have another API hosted at URL2 - https://somedomain.com/another_api/whatever
But remember both APIs are a different project.
When I pass this cookie to URL1, in the OnAuthorization() I can see the Identities and the Claims properties with correct values of that user and it authorizes the user perfectly. But when I pass the same cookie to URL2 all claims and identity properties show null and it does not authorize.
My question: Is it possible to share authentication between different domains? If yes, then how? If not possible then please suggest an alternative approach.
Additional Details -
We have the above code which generates (at App1) a cookie ABC=some_encrpyted_value.
And in App2 I am trying to unprotect that cookie in below manner
string cookieValue = context.HttpContext.Request.Cookies["ABC"];
var provider = DataProtectionProvider.Create(new DirectoryInfo(#"C:\temp-keys\"));
var dataProtector = provider.CreateProtector(typeof(CookieAuthenticationMiddleware).FullName, "ABC", "v2");
UTF8Encoding specialUtf8Encoding = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true);
byte[] protectedBytes = Base64UrlTextEncoder.Decode(cookieValue);
byte[] plainBytes = dataProtector.Unprotect(protectedBytes);
string plainText = specialUtf8Encoding.GetString(plainBytes);
TicketDataFormat ticketDataFormat = new TicketDataFormat(dataProtector);
AuthenticationTicket ticket = ticketDataFormat.Unprotect(cookieValue);
I have placed the data protection key from App1 in the folder 'C:\temp-keys'.
I found this code somewhere in StackOverflow itself and it throws 'CryptographicException: The payload was invalid.' exception at line
byte[] plainBytes = dataProtector.Unprotect(protectedBytes);
My understanding of the App1 code is that the Identity and Claim values (with some other values) are encrypted and cookie ABC is generated, and when we send a request this cookie is decrypted and the Identity and Claims are get populated in the context.HttpContext.User
I actually wanted it to work the same way in the App2, I want to pass the cookie and the identity values should be populated in the context.
It is possible. To deal with this type of scenarios we have been using ASP.NET machineKey. That is does is it encrypts/descriptor the auth data using the same key so that different sites can share the same authenticated sessions and data.
In Dotnet Core this machineKey concept has evolved and now called - new data protection system. Microsoft.AspNetCore.DataProtection.SystemWeb package is used to implement the data protection system. To read more on this.
There are various ways how you can store and use the key:
ProtectKeysWithAzureKeyVault (if you are using Azure)
PersistKeysToFileSystem (this is the easier one)
ProtectKeysWith*
UnprotectKeysWithAnyCertificate
Details of the individual scenario is described here.

Authentication fail error on convert Html to Pdf

I try to convert Html to PDF use "HtmlToPdf" nuget , It was work fine on local test but when i upload site to host i get this error :
Conversion error: Authentication error.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Exception: Conversion error: Authentication error.
This is my Convert method Code
[AllowAnonymous]
public ActionResult Convert(int id)
{
HtmlToPdf converter = new HtmlToPdf();
var context = System.Web.HttpContext.Current;
string baseUrl = context.Request.Url.Host + ":"+context.Request.Url.Port + "/Doctor/DietTherapy/LineRegimePrint/";
PdfDocument doc = converter.ConvertUrl(baseUrl + id);
// save pdf document
byte[] pdf = doc.Save();
// close pdf document
doc.Close();
// return resulted pdf document
FileResult fileResult = new FileContentResult(pdf, "application/pdf");
fileResult.FileDownloadName = "Document.pdf";
return fileResult;
}
How can i authorize user for this convert ?
It sounds like you just need to authenticate the request being made by the PDF library. For example, if it's using Basic HTTP Authentication:
HtmlToPdf converter = new HtmlToPdf();
converter.Options.Authentication.Username = "some username";
converter.Options.Authentication.Password = "some password";
// the rest of your code...
The linked documentation also contains examples for other authentication methods.
Did you check to see if you're getting the proper value on this line of code (when running on host server)?
string baseUrl = context.Request.Url.Host + ":"+context.Request.Url.Port + "/Doctor/DietTherapy/LineRegimePrint/";
While your development machine's IIS Express uses Anonymous Authentication, your hosting server is probably using Windows Authentication or other kinds. Check the IIS Managers to see the difference.
SelectPDF's library somehow creates a separate process to perform the HtmlToPdf conversion, which is outside the IUser, and therefore, the server asks for authentication.
For Windows Authentication setting, you may just use a generic user login account for this purpose and populate the required properties as mentioned in above thread and all would be fine.
converter.Options.Authentication.Username = "WindowsUser";
converter.Options.Authentication.Password = "WindowsPassword";

OAuth 1.0, javascript and server-to-server authentication

We are trying to integrate a "sign in with twitter" function in our application and for this purpose we are using javascript (hello.js).
Unfortunately, twitter is using oauth 1.0 so we can't have only a javascript solution but we need to implement a server-to-server communication to sign the request. The hello.js author provided an auth-server implementation for demo purposes based on node.js.
In our application for the backend part we are using java and I was wondering if a java solution exists for this purpose. Could I use for instance signpost or similar to do the job of auth-server ?
[UPDATE]
I tried to set the proxy used by hello.js to a mine servlet (so, instead of herokuap now I'm using a localhost servlet with oauth in the buildpath).
This servlet is doing the following:
OAuthConsumer consumer = new DefaultOAuthConsumer(
"xxxx",
"yyyyyyyyyyy");
OAuthProvider provider = new DefaultOAuthProvider(
"https://api.twitter.com/oauth/request_token",
"https://api.twitter.com/oauth/access_token",
"https://api.twitter.com/oauth/authorize");
System.out.println("Fetching request token from Twitter...");
// we do not support callbacks, thus pass OOB
String authUrl = provider.retrieveRequestToken(consumer, "http://localhost:8080/oauth1/twitter/response_server");
URL url = new URL(authUrl);
HttpURLConnection req = (HttpURLConnection) url.openConnection();
req.setRequestMethod("GET");
req.connect();
BufferedReader rd = new BufferedReader(new InputStreamReader(req.getInputStream()));
StringBuilder d = new StringBuilder();
String line = null;
while ((line = rd.readLine()) != null){
d.append(line + '\n');
}
System.out.println(d);
PrintWriter out = response.getWriter();
out.println(d);
and it prints the twitter login page in the hello.js popup. In this way I got some encoding error but it is quite working.
Anyway the callback url is mapped to another servlet where I should simply "sign" the request but I think I am missing something because sometimes I got the error
"The server understand the request but it is still refusing it"
but if I close all browser window it works.
However the response servlet is similar to this one
OAuthConsumer consumer = new AbstractOAuthConsumer(
"xxxxx",
"yyyyyyyyyyy"){
#Override
protected HttpRequest wrap(Object arg0) {
return (HttpRequest)arg0;
}
};
consumer.sign(request);
But this code does not work because I don't know how to sign a tomcat request. In oauth homepage is explained how to sign jetty and apache common http request but not the tomcat one. However, is it correct my approach ?
Please checkout the spec of node-oauth-shim which is used my //auth-server and which HelloJS delivers.

ASP.Net MVC & WebAPI encryption

I want to utilise some form of "simple" encryption that is reasonably secure but very low friction in terms of impact on development process.
Supposing I own both sides of the conversation in a client <> web service situation. My application is a windows phone/win8/silverlight/desktop app and the server is ASP.Net MVC or WebAPI.
In my mind, I want something as simple as:-
<security encryption="off|sometype|someothertype">
<privatekey>12345MyKey54321</privatekey>
</security>
as some form of configuration parameter on both the client and server. Additionally an authentication routine will return and store some form of public key.
Doing so will enable the 'encryption mode' and result in any http requests being encrypted & hashed in the selected manner using the provided keys. The end result being anything sniffed on the local, proxy or remote machines would not be able to view the data without the key and decryption method. On the server, data is decrypted using the same key before hitting controller actions.
Other than swapping out HttpRequest/WebClient calls for something like EncryptedHttpRequest and adding the appropriate hook on the MVC/WebAPI side of things, all other client code and controller actions would be ignorant to the fact the data was encrypted.
Am I missing something or could setup not be this simple? As far as I have searched there is nothing that offers this level of simplicity so I figure I'm missing some gaping flaw in my logic?
All you are looking for can be achieved by simply using HTTPS. Just buy a certificate (or use a self-signed certificate) and there is your encryption.
Do not re-invent the wheel.
I've done this successfully. It isn't too difficult and works well. I use it for activating a license for a product. The most important thing is that your truly control the client and server - no one can extract your private key from your code on the client.
Step 1: Create an MVC controller action method that takes no arguments:
[HttpPost] public ActionResult Activate() { ... }
Step 2: In the controller just use the HttpRequest.InputStream to get ahold of the bytes sent from the client.
var stream = this.HttpContext.Request.InputStream;
Step 3: Create a CryptoStream to deserialize.
I've included creating both encryption and decryption examples here. The sharedSecret is a byte[] of sufficient length (512 bytes) of random bytes - this is what you protect!
public CryptoStream CreateEncryptionStream(Stream writeStream)
{
TripleDESCryptoServiceProvider cryptoProvider = new TripleDESCryptoServiceProvider();
PasswordDeriveBytes derivedBytes = new PasswordDeriveBytes(this._sharedSecret, null);
CryptoStream cryptoStream = new CryptoStream(writeStream, cryptoProvider.CreateEncryptor(derivedBytes.GetBytes(16), derivedBytes.GetBytes(16)), CryptoStreamMode.Write);
return cryptoStream;
}
public CryptoStream CreateDecryptionStream(Stream readStream)
{
TripleDESCryptoServiceProvider cryptoProvider = new TripleDESCryptoServiceProvider();
PasswordDeriveBytes derivedBytes = new PasswordDeriveBytes(this._sharedSecret, null);
CryptoStream cryptoStream = new CryptoStream(readStream, cryptoProvider.CreateDecryptor(derivedBytes.GetBytes(16), derivedBytes.GetBytes(16)), CryptoStreamMode.Read);
return cryptoStream;
}
Step 4: Use your CryptoStream another stream reader to decrypt.
I use an XmlReader so that all my existing serialization code can work either in the clear (when reading/writing to disk or database on the server) or encrypted (when transmitting).
using (var reader = XmlReader.Create(decryptionStream, settings)) { ... }
Step 5: Formulate a secure response in your controller.
This is doing the reverse of Steps 1-4 to encrypt your response object. Then you just write your encrypted response to a memory stream and return it as a File result. Below, I've shown how I do this for my license response object.
var responseBytes = GetLicenseResponseBytes(licenseResponse);
return File(responseBytes, "application/octet-stream");
private byte[] GetLicenseResponseBytes(LicenseResponse licenseResponse)
{
if (licenseResponse != null)
{
using (MemoryStream memoryStream = new MemoryStream())
{
this._licenseResponseSerializer.Write(memoryStream, licenseResponse);
return memoryStream.ToArray();
}
}
return null;
}
Step 6: Implement your client request response.
You can use HttpWebRequest or the WebClient classes to formulate the request. Here's a couple of examples from the code I use.
byte[] postBytes = GetLicenseRequestBytes(licenseRequest);
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(licenseServerUrl);
request.Method = "POST";
request.ContentType = "application/octet-stream";
request.Proxy = WebRequest.DefaultWebProxy;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(postBytes, 0, postBytes.Length);
}
return request;
private LicenseResponse ProcessHttpResponse(HttpWebResponse response)
{
if ((response.StatusCode == HttpStatusCode.OK) && response.ContentType.Contains("application/octet-stream"))
{
var stream = response.GetResponseStream();
if (stream != null)
{
var licenseResponse = this._licenseResponseSerializer.Read(stream);
return licenseResponse;
}
}
return new LicenseResponse(LicensingResult.Error);
}
Summary and Tips
Use the streams in the request/responses on the client and server to communicate binary octet-stream data
Use CryptoStream along with an encryption algorithm (consider using the strongest encryption possilbe) and a good private key to encrypt data when you serialize/deserialize it.
Make sure to check the size and format all incoming data to the client and server (avoid buffer overruns and throw exceptions early)
Protect your private key on your client using obfuscation if possible (take a look at the DeepSea obfustactor)

Getting 401 Unauthorized from Google API Resumable Update

I am trying to integrate upload of arbitrary files to Google Docs into an existing application. This used to work before using resumable upload became mandatory. I am using Java client libraries.
The application is doing the upload in 2 steps:
- get the resourceId of the file
- upload the data
To get the resourceId I am uploading a 0-size file (i.e. Content-Length=0). I am passing ?convert=false in the resumable URL (i.e. https://docs.google.com/feeds/upload/create-session/default/private/full?convert=false).
I am passing "application/octet-stream" as content-type. This seems to work, though I do get different resourcesIds - "file:..." resourceIds for things like images, but "pdf:...." resourceIds for PDFs.
The second step constructs a URL based on the resourceId obtained previously and performs a search (getEntry). The URL is in the form of https://docs.google.com/feeds/default/private/full/file%3A.....
Once the entry is found the ResumableGDataFileUploader is used to update the content (0-byte file) with the actual data from the file being uploaded. This operation fails with 401 Unauthorized response when building ResumableGDataFileUploader instance.
I've tried with ?convert=false as well as ?new-revision=true and both of these at the same time. The result is the same.
The relevant piece of code:
MediaFileSource mediaFile = new MediaFileSource(
tempFile, "application/octet-stream");
final ResumableGDataFileUploader.Builder builder =
new ResumableGDataFileUploader.Builder(client, mediaFile, documentListEntry);
builder.executor(MoreExecutors.sameThreadExecutor());
builder.requestType(ResumableGDataFileUploader.RequestType.UPDATE);
// This is where it fails
final ResumableGDataFileUploader resumableGDataFileUploader = builder.build();
resumableGDataFileUploader.start();
return tempFile.length();
The "client" is an instance of DocsService, configured to use OAuth. It is used to find "documentListEntry" immediately before the given piece of code.
I had to explicitly specify request type, since it seems the client library code contains a bug causing NullPointerException for "update existing entry" case.
I have a suspicion that the issue is specifically in the sequence of actions (upload 0-byte file to get the resourceId, then update with actual file) but I can't figure out why it doesn't work.
Please help?
This code snippet works for me using OAuth 1.0 and OAuth 2.0:
static void uploadDocument(DocsService client) throws IOException, ServiceException,
InterruptedException {
ExecutorService executor = Executors.newFixedThreadPool(10);
File file = new File("<PATH/TO/FILE>");
String mimeType = DocumentListEntry.MediaType.fromFileName(file.getName()).getMimeType();
DocumentListEntry documentEntry = new DocumentListEntry();
documentEntry.setTitle(new PlainTextConstruct("<DOCUMENT TITLE>"));
int DEFAULT_CHUNK_SIZE = 2 * 512 * 1024;
ResumableGDataFileUploader.Builder builder =
new ResumableGDataFileUploader.Builder(
client,
new URL(
"https://docs.google.com/feeds/upload/create-session/default/private/full?convert=false"),
new MediaFileSource(file, mimeType), documentEntry).title(file.getName())
.requestType(RequestType.INSERT).chunkSize(DEFAULT_CHUNK_SIZE).executor(executor);
ResumableGDataFileUploader uploader = builder.build();
Future<ResponseMessage> msg = uploader.start();
while (!uploader.isDone()) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) {
throw ie; // rethrow
}
}
DocumentListEntry uploadedEntry = uploader.getResponse(DocumentListEntry.class);
// Print the document's ID.
System.out.println(uploadedEntry.getId());
System.out.println("Upload is done!");
}

Resources