Getting 401 Unauthorized from Google API Resumable Update - google-docs-api

I am trying to integrate upload of arbitrary files to Google Docs into an existing application. This used to work before using resumable upload became mandatory. I am using Java client libraries.
The application is doing the upload in 2 steps:
- get the resourceId of the file
- upload the data
To get the resourceId I am uploading a 0-size file (i.e. Content-Length=0). I am passing ?convert=false in the resumable URL (i.e. https://docs.google.com/feeds/upload/create-session/default/private/full?convert=false).
I am passing "application/octet-stream" as content-type. This seems to work, though I do get different resourcesIds - "file:..." resourceIds for things like images, but "pdf:...." resourceIds for PDFs.
The second step constructs a URL based on the resourceId obtained previously and performs a search (getEntry). The URL is in the form of https://docs.google.com/feeds/default/private/full/file%3A.....
Once the entry is found the ResumableGDataFileUploader is used to update the content (0-byte file) with the actual data from the file being uploaded. This operation fails with 401 Unauthorized response when building ResumableGDataFileUploader instance.
I've tried with ?convert=false as well as ?new-revision=true and both of these at the same time. The result is the same.
The relevant piece of code:
MediaFileSource mediaFile = new MediaFileSource(
tempFile, "application/octet-stream");
final ResumableGDataFileUploader.Builder builder =
new ResumableGDataFileUploader.Builder(client, mediaFile, documentListEntry);
builder.executor(MoreExecutors.sameThreadExecutor());
builder.requestType(ResumableGDataFileUploader.RequestType.UPDATE);
// This is where it fails
final ResumableGDataFileUploader resumableGDataFileUploader = builder.build();
resumableGDataFileUploader.start();
return tempFile.length();
The "client" is an instance of DocsService, configured to use OAuth. It is used to find "documentListEntry" immediately before the given piece of code.
I had to explicitly specify request type, since it seems the client library code contains a bug causing NullPointerException for "update existing entry" case.
I have a suspicion that the issue is specifically in the sequence of actions (upload 0-byte file to get the resourceId, then update with actual file) but I can't figure out why it doesn't work.
Please help?

This code snippet works for me using OAuth 1.0 and OAuth 2.0:
static void uploadDocument(DocsService client) throws IOException, ServiceException,
InterruptedException {
ExecutorService executor = Executors.newFixedThreadPool(10);
File file = new File("<PATH/TO/FILE>");
String mimeType = DocumentListEntry.MediaType.fromFileName(file.getName()).getMimeType();
DocumentListEntry documentEntry = new DocumentListEntry();
documentEntry.setTitle(new PlainTextConstruct("<DOCUMENT TITLE>"));
int DEFAULT_CHUNK_SIZE = 2 * 512 * 1024;
ResumableGDataFileUploader.Builder builder =
new ResumableGDataFileUploader.Builder(
client,
new URL(
"https://docs.google.com/feeds/upload/create-session/default/private/full?convert=false"),
new MediaFileSource(file, mimeType), documentEntry).title(file.getName())
.requestType(RequestType.INSERT).chunkSize(DEFAULT_CHUNK_SIZE).executor(executor);
ResumableGDataFileUploader uploader = builder.build();
Future<ResponseMessage> msg = uploader.start();
while (!uploader.isDone()) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) {
throw ie; // rethrow
}
}
DocumentListEntry uploadedEntry = uploader.getResponse(DocumentListEntry.class);
// Print the document's ID.
System.out.println(uploadedEntry.getId());
System.out.println("Upload is done!");
}

Related

Authentication fail error on convert Html to Pdf

I try to convert Html to PDF use "HtmlToPdf" nuget , It was work fine on local test but when i upload site to host i get this error :
Conversion error: Authentication error.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Exception: Conversion error: Authentication error.
This is my Convert method Code
[AllowAnonymous]
public ActionResult Convert(int id)
{
HtmlToPdf converter = new HtmlToPdf();
var context = System.Web.HttpContext.Current;
string baseUrl = context.Request.Url.Host + ":"+context.Request.Url.Port + "/Doctor/DietTherapy/LineRegimePrint/";
PdfDocument doc = converter.ConvertUrl(baseUrl + id);
// save pdf document
byte[] pdf = doc.Save();
// close pdf document
doc.Close();
// return resulted pdf document
FileResult fileResult = new FileContentResult(pdf, "application/pdf");
fileResult.FileDownloadName = "Document.pdf";
return fileResult;
}
How can i authorize user for this convert ?
It sounds like you just need to authenticate the request being made by the PDF library. For example, if it's using Basic HTTP Authentication:
HtmlToPdf converter = new HtmlToPdf();
converter.Options.Authentication.Username = "some username";
converter.Options.Authentication.Password = "some password";
// the rest of your code...
The linked documentation also contains examples for other authentication methods.
Did you check to see if you're getting the proper value on this line of code (when running on host server)?
string baseUrl = context.Request.Url.Host + ":"+context.Request.Url.Port + "/Doctor/DietTherapy/LineRegimePrint/";
While your development machine's IIS Express uses Anonymous Authentication, your hosting server is probably using Windows Authentication or other kinds. Check the IIS Managers to see the difference.
SelectPDF's library somehow creates a separate process to perform the HtmlToPdf conversion, which is outside the IUser, and therefore, the server asks for authentication.
For Windows Authentication setting, you may just use a generic user login account for this purpose and populate the required properties as mentioned in above thread and all would be fine.
converter.Options.Authentication.Username = "WindowsUser";
converter.Options.Authentication.Password = "WindowsPassword";

How to create GoogleCredential object referencing the service account json file in Dataflow?

I have written a pipeline to extract G suite activity logs by referring the G suite java-quickstart where the code reads client_secret.json file as below,
InputStream in = new FileInputStream("D://mypath/client_secret.json");
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, new InputStreamReader(in));
Pipeline runs as expected in local(runner=DirectRunner) but the same code fails with java.io.FileNotFoundException expection when executed on cloud(runner=DataflowRunner)
I understand local path is invalid when executed on cloud. Any suggestion here?
Updated:
I have modified the code as below and I am able to read the client_secrets.json file
InputStream in =
Activities.class.getResourceAsStream("client_secret.json");
Actual problem is in creating the credential object
private static java.io.File DATA_STORE_DIR = new java.io.File(System.getProperty("user.home"),
".credentials/admin-reports_v1-java-quickstart");
private static final List<String> SCOPES = Arrays.asList(ReportsScopes.ADMIN_REPORTS_AUDIT_READONLY);
static {
try {
HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
DATA_STORE_FACTORY = new FileDataStoreFactory(DATA_STORE_DIR);
} catch (Throwable t) {
t.printStackTrace();
System.exit(1);
}
}
public static Credential authorize() throws IOException {
// Load client secrets.
InputStream in =
Activities.class.getResourceAsStream("client_secret.json");
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, new InputStreamReader(in));
// Build flow and trigger user authorization request.
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(HTTP_TRANSPORT, JSON_FACTORY,
clientSecrets, SCOPES).setDataStoreFactory(DATA_STORE_FACTORY).setAccessType("offline").build();
Credential credential = new AuthorizationCodeInstalledApp(flow, new LocalServerReceiver()).authorize("user");
System.out.println("Credentials saved to " + DATA_STORE_DIR.getAbsolutePath());
return credential;
}
Observations:
Local execution:
On initial execution, program attempts to open browser to authorize the request and stores the authenticated object in a file - "StoredCredential".
On further executions, the stored file is used to make API calls.
Running on cloud(DataFlowRunner):
When I check logs, dataflow tries to open a browser to authenticate the request and stops there.
What I need?
How to modify GoogleAuthorizationCodeFlow.Builder such that the credential object can be created while running as dataflow pipeline?
I have found a solution to create GoogleCredential object using the service account. Below is the code for it.
public static Credential authorize() throws IOException, GeneralSecurityException {
String emailAddress = "service_account.iam.gserviceaccount.com";
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(emailAddress)
.setServiceAccountPrivateKeyFromP12File(Activities.class.getResourceAsStream("MYFILE.p12"))
.setServiceAccountScopes(Collections.singleton(ReportsScopes.ADMIN_REPORTS_AUDIT_READONLY))
.setServiceAccountUser("USER_NAME")
.build();
return credential;
}
Can you try running the program multiple times locally. What I am wondering is, if the "StoredCredential" file is available, will it just work? Or will it try to load up the browser again?
If so, can you determine the proper place to store that file, and download a copy of it from GCS onto the Dataflow worker? There should be APIs to download GCS files bundled with the dataflow SDK jar. So you should be able to use those to download the credential file.

Response Header issue on Azure Web Application

I am not sure what is happening here.
When I run my web application locally and click a button to download a file, the file is downloaded fine and Response header as you can see in the attached screenshot where it says local.
But when I publish the application to azure web app. Somehow the download button stops working. I checked the Response Header and you can see the difference.
What would cause this problem? The code is the same? Is there any settings that I should be setting in azure web app in azure portal?
Updated to add code
I have debugged remotely to figure out what is going on as #Amor suggested.
It is so strange that When I debug on my local machine first ExportTo action gets hit which prepares the TempData then Download action gets called once the first action completed with ajax call.
However, this is not the case when I debug remotely. Somehow the ExportTo action never gets called. It directly calls the Download action. As a result the TempData null checking is always null.
But why? Why on earth and how that is possible? Is there something cached somewhere?
I have wiped the content of web application on the remote and re-publish evertyhing to ensure everything is updated. But still no success.
here is the code:
[HttpPost]
public virtual ActionResult ExportTo(SearchVm searchVm)
{
var data = _companyService.GetCompanieBySearchTerm(searchVm).Take(150).ToList();
string handle = Guid.NewGuid().ToString();
TempData[handle] = data;
var fileName = $"C-{handle}.xlsx";
var locationUrl = Url.Action("Download", new { fileGuid = handle, fileName });
var downloadUrl = Url.Action("Download");
return Json(new { success = true, locationUrl, guid = handle, downloadUrl }, JsonRequestBehavior.AllowGet);
}
[HttpGet]
public ActionResult Download(string fileGuid, string fileName)
{
if (TempData[fileGuid] != null)
{
var fileNameSafe = $"C-{fileGuid}.xlsx";
var data = TempData[fileGuid] as List<Company>;
using (MemoryStream ms = new MemoryStream())
{
GridViewExtension.WriteXlsx(GetGridSettings(fileNameSafe), data, ms);
MVCxSpreadsheet mySpreadsheet = new MVCxSpreadsheet();
ms.Position = 0;
mySpreadsheet.Open("myDoc", DocumentFormat.Xlsx, () =>
{
return ms;
});
mySpreadsheet.Document.Worksheets.Insert(0);
var image = Server.MapPath("~/images/logo.png");
var worksheet = mySpreadsheet.Document.Worksheets[0];
worksheet.Name = "Logo";
worksheet.Pictures.AddPicture(image, worksheet.Cells[0, 0]);
byte[] result = mySpreadsheet.SaveCopy(DocumentFormat.Xlsx);
DocumentManager.CloseDocument("myDoc");
Response.Clear();
//Response.AppendHeader("Set-Cookie", "fileDownload=true; path=/");
Response.ContentType = "application/force-download";
Response.AddHeader("content-disposition", $"attachment; filename={fileNameSafe}");
Response.BinaryWrite(result);
Response.End();
}
}
return new EmptyResult();
}
here is the javascript:
var exportData = function (urlExport) {
console.log('Export to link in searchController: ' + urlExport);
ExportButton.SetEnabled(false);
var objData = new Object();
var filterData = companyFilterData(objData);
console.log(filterData);
$.post(urlExport, filterData)
.done(function (data) {
console.log(data.locationUrl);
window.location.href = data.locationUrl;
});
};
When Export button is clicked exportData function is called:
var exportToLink = '#Url.Action("ExportTo")';
console.log('Export to link in index: '+exportToLink);
SearchController.exportData(exportToLink);
As I mentioned that this code works perfectly on the local machine. something weird is happening on azure webapp that ExportTo action breakpoint is never gets hit.
I am not sure what else I could change to get the ExportTo action hit?
Based on the Response Header of Azure Web App, we find that the value of Content-Length is 0. It means that no data has been sent from web app server side.
In ASP.NET MVC, we can response file using following ways.
The first way, send the file which hosted on server. For this way, please check whether the excel file has been uploaded to Azure Web App. You could use Kudu or FTP to the folder to check whether the file is exist.
string fileLocation = Server.MapPath("~/Content/myfile.xlsx");
string contentType = System.Net.Mime.MediaTypeNames.Application.Octet;
string fileName = "file.xlsx";
return File(fileLocation, contentType, fileName);
The second way, we can read the file from any location(database, server or azure storage) and send the file content to client side. For this way, please check whether the file has been read successfully. You can remote debug your azure web app to check whether the file content hasn't been read in the right way.
byte[] fileContent = GetFileContent();
string contentType = System.Net.Mime.MediaTypeNames.Application.Octet;
string fileName = "file.xlsx";
return File(fileContent, contentType, fileName);
5/27/2017 Update
Somehow the ExportTo action never gets called. It directly calls the Download action. As a result the TempData null checking is always null.
How many instances does your Web App assigned? If your Web App have multi instances, the ExportTo request is handled by one instance and the Download request is handled by another instance. Since the TempData is store in memory of dedicated instance, it can't be got from another instance. According to the remote debug document. I find out the reason why the ExportTo action never gets called.
If you do have multiple web server instances, when you attach to the debugger you'll get a random instance, and you have no way to ensure that subsequent browser requests will go to that instance.
To solve this issue, I suggest you response the data directly from the ExportTo action or save the temp data in Azure blob storage which can't be accessed from multi instances.

Read HTTP Status Code from MVC controller

I am uploading a file to the server. To do that I am using the SOAP Web Service that the developers of the site created.
This service CustomerWebService contains only one void method transferFile. In about 20-30 seconds after the upload I am getting the confirmation email from the site.
It works fine, however I would like to get at least HTTP 200 status right after the execution (like I see in Fiddler right away.)
How can I return one in code?
public ActionResult UploadFile(string fileName)
{
StreamReader sr = new StreamReader(fileName);
String line = sr.ReadToEnd();
CustomerWebService ws = new CustomerWebService ();
ws.transferFile("user", "password", line);
return Json("{Success}", JsonRequestBehavior.AllowGet);
}

XSSFWorkbook#write() results in empty response

I'm trying to get my JSF to export a spreadsheet for download. I'm using Apache's POI library for the Excel document writing. I get the following error in an alert box when the code runs:
emptyResponse: An empty response was received from the server.
The method generating the spreadsheet and exporting to the OutputStream is below (I have renamed classes, methods etc for simplicity sake).
private void generateSpreadsheet(Object object) throws Exception {
FacesContext context = FacesContext.getCurrentInstance();
HttpServletResponse response = (HttpServletResponse)context.getExternalContext().getResponse();
String fileName = object.getProperty() + ".xlsx";
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment;filename=\"" + fileName +"\"");
OutputStream os = response.getOutputStream();
Workbook wb = new XSSFWorkbook();
Sheet sheet = wb.createSheet("Sheet 1");
Row row = sheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("test");
wb.write(os);
os.flush();
os.close();
FacesContext.getCurrentInstance().responseComplete();
}
Any advice much appreciated, thanks.
If it makes a difference, I'm using AJAX (<f:ajax> tag) on the form submit that calls this method.
It makes definitely difference. You can not download files by Ajax. Ajax is executed by JavaScript code. But JavaScript has no facilities to force a Save As dialogue or to execute the platform default application associated with the file's mime type (thankfully; that would have been a major security/intrusion problem). Further, the JSF Ajax API expects a XML response in a specified structure conform the JSF specs. When you send a complete Excel file instead, the whole Ajax response would be ignored as ununderstandable garbage by the JSF Ajax API.
You need to send a normal synchronous request. Remove the <f:ajax> tag from the command link/button. The current page will remain the same anyway if the download is sent as an attachment.

Resources