There exists a DocsClient.get_resource_by_id function to get the document entry for a single ID. Is there a similar way to obtain (in a single call) multiple document entries given multiple document IDs?
My application needs to efficiently download the content from multiple files for which I have the IDs. I need to get the document entries to access the appropriate download URL (I could manually construct the URLs, but this is discouraged in the API docs). It is also advantageous to have the document type and, in the case of spreadsheets, the document entry is required in order to access individual worksheets.
Overall I'm trying to reduce I/O waits, so if there's a way I can bundle the doc ID lookup, it will save me some I/O expense.
[Edit] Backporting AddQuery to gdata v2.0 (from Alain's solution):
client = DocsClient()
# ...
request_feed = gdata.data.BatchFeed()
request_entry = gdata.data.BatchEntry()
request_entry.batch_id = gdata.data.BatchId(text=resource_id)
request_entry.batch_operation = gdata.data.BATCH_QUERY
request_feed.add_batch_entry(entry=request_entry, batch_id_string=resource_id, operation_string=gdata.data.BATCH_QUERY)
batch_url = gdata.docs.client.RESOURCE_FEED_URI + '/batch'
rsp = client.batch(request_feed, batch_url)
rsp.entry is a collection of BatchEntry objects, which appear to refer to the correct resources, but which differ from the entries I'd normally get via client.get_resource_by_id().
My workaround is to convert gdata.data.BatchEntry objects into gdata.docs.data.Resource objects ilke thus:
entry = atom.core.parse(entry.to_string(), gdata.docs.data.Resource)
You can use a batch request to send multiple "GET" requests to the API using a single HTTP request.
Using the Python client library, you can use this code snippet to accomplish that:
def retrieve_resources(gd_client, ids):
"""Retrieve Documents List API Resources using a batch request.
Args:
gd_client: authorized gdata.docs.client.DocsClient instance.
ids: Collection of resource id to retrieve.
Returns:
ResourceFeed containing the retrieved resources.
"""
# Feed that holds the batch request entries.
request_feed = gdata.docs.data.ResourceFeed()
for resource_id in ids:
# Entry that holds the batch request.
request_entry = gdata.docs.data.Resource()
self_link = gdata.docs.client.RESOURCE_SELF_LINK_TEMPLATE % resource_id
request_entry.id = atom.data.Id(text=self_link)
# Add the request entry to the batch feed.
request_feed.AddQuery(entry=request_entry, batch_id_string=resource_id)
# Submit the batch request to the server.
batch_url = gdata.docs.client.RESOURCE_FEED_URI + '/batch'
response_feed = gd_client.Post(request_feed, batch_url)
# Check the batch request's status.
for entry in response_feed.entry:
print '%s: %s (%s)' % (entry.batch_id.text,
entry.batch_status.code,
entry.batch_status.reason)
return response_feed
Make sure to sync to the latest version of the project repository.
Related
I am attempting to programmatically retrieve a list of users (principalType = "User") and their associated appRoleId values for an enterprise app using itsresourceId value from Azure AD. There is a total of ten Users with a combined total of twenty appRoleId values associated with the app. However, when I run my query I receive data for just two users and a combined total of four appRoleId values.
Here's my C# code:
GraphServiceClient myGraphClient = GetGraphServiceClient([scopes]);
// Retrieve the [Id] value for the app. Note [Id] is a pseudonym for the [resourceId] required to retrieve users and app roles assigned.
var servPrinPage = await myGraphClient.ServicePrincipals.Request()
.Select("id,appRoles")
.Filter($"startswith(displayName, 'Display Name')")
.GetAsync()
.ConfigureAwait(false);
// Using the first [Id] value from the [ServicePrincipals] page, retrieve the list of users and their assigned roles for the app.
var appRoleAssignedTo = await myGraphClient.ServicePrincipals[servPrinPage[0].Id].AppRoleAssignedTo.Request().GetAsync().ConfigureAwait(false);
The query returns a ServicePrincipalAppRoleAssignedToCollectionPage (as expected) but the collection only contains four pages (one per User/appRoleId combination).
As an aside, the following query in Microsoft Graph Explorer produces an equivalent result:
https://graph.microsoft.com/v1.0/servicePrincipals/[resourceId]/appRoleAssignedTo
What am I missing here? I need to be able to retrieve the complete list of users and assigned app roles. Any assistance is greatly appreciated.
The issue I was confronting has to do with the pagination feature employed by Azure AD and MS Graph. In a nutshell, I was forced to submit two queries in order to retrieve all twenty records I was expecting.
If you have a larger set of records to be retrieved you may be faced with submitting a much larger number of successive queries. The successive queries are managed using a "skiptoken" passed as a request header each time your query is resubmitted.
Here is my revised code with notation....
// Step #1: Create a class in order to strongly type the <List> which will hold your results.
// Not absolutely necessary but always a good idea when working with <Lists> in C#.
private class AppRoleByUser
{
public string AzureDisplayName;
public string PrincipalDisplayName;
}
// Step #2: Submit a query to acquire the [id] for the Service Principal (i.e. your app).
// Note the [ServicePrincipals].[id] property is synonymous with the [resourceId] needed to
// retrieve [AppRoleAssignedTo] values from Microsoft Graph in the next step.
// Initialize the Microsoft Graph Client.
GraphServiceClient myGraphClient = GetGraphServiceClient("Directory.Read.All");
// Retrieve the Service Principals page containing the app [Id].
var servPrinPage = await myGraphClient.ServicePrincipals.Request().Select("id,appRoles").Filter($"startswith(displayName, 'Your App Name')").GetAsync().ConfigureAwait(false);
// Store the app [Id] in a local variable (for readability).
string resourceId = servPrinPage[0].Id;
// Step #3: Using the [Id]/[ResourceId] value from the previous step, retrieve a list of AppRoleId/DisplayName pairs for your app.
// Results of the successive queries are typed against the class created earlier and are appended to the <List>.
List<AppRoleByUser> appRoleByUser = new List<AppRoleByUser>();
// Note, unlike "Filter" or "Search" parameters, it is not possible to
// add a "Skiptoken" parameter directly to your query in C#.
// Instead, it is necessary to insert the "skiptoken" as request header using the Graph QueryOption class.
// Note the QueryOption List is passed as an empty object on the first pass of the while loop.
var queryOptions = new List<QueryOption>();
// Initialize the variable to hold the anticipated query result.
ServicePrincipalAppRoleAssignedToCollectionPage appRoleAssignedTo = new ServicePrincipalAppRoleAssignedToCollectionPage();
// Note the number of user/role combinations associated with an app is not always known.
// Consequently, you may be faced with the need to acquire multiple pages
// (and submit multiple consecutive queries) in order to obtain a complete
// listing of user/role combinations.
// The "while" loop construct will be utilized to manage query iteration.
// Execution of the "while" loop will be stopped when the "bRepeat" variable is set to false.
bool bRepeat = true;
while (bRepeat == true)
{
appRoleAssignedTo = (ServicePrincipalAppRoleAssignedToCollectionPage) await myGraphClient.ServicePrincipals[resourceId].AppRoleAssignedTo.Request(queryOptions).GetAsync().ConfigureAwait(false);
foreach (AppRoleAssignment myPage in appRoleAssignedTo)
{
// I was not able to find a definitive answer in any of the documents I
// found but it appears the final record in the recordset carries a
// [PrincipalType] = "Group" (all others carry a [PrincipalType] = "User").
if (myPage.PrincipalType != "Group")
{
// Insert "User" data into the List<AppRoleByUser> collection.
appRoleByUser.Add(new AppRoleByUser{ AzureDisplayName = myPage.PrincipalDisplayName, AzureUserRole = myPage.AppRoleId.ToString() });
}
else
{
// The "bRepeat" variable is initially set to true and is set to
// false when the "Group" record is detected thus signaling
// task completion and closing execution of the "while" loop.
bRepeat = false;
}
}
// Acquire the "nextLink" string from the response header.
// The "nextLink" string contains the "skiptoken" string required for the next
// iteration of the query.
string nextLinkValue = appRoleAssignedTo.AdditionalData["#odata.nextLink"].ToString();
// Parse the "skiptoken" value from the response header.
string skipToken = nextLinkValue.Substring(nextLinkValue.IndexOf("=") + 1);
// Include the "skiptoken" as a request header in the next iteration of the query.
queryOptions = new List<QueryOption>()
{
new QueryOption("$skiptoken", skipToken)
};
}
That's a long answer to what should have been a simple question. I am relatively new to Microsoft Graph but it appears to me Microsoft has a long way to go in making this API developer-friendly. All I needed was to know the combination of AppRoles and Users associated with a single, given Azure AD app. One query and one response should have been more than sufficient.
At any rate, I hope my toil might help save time for someone else going forward.
Could you please remove "Filter" from the code and retry the operation. Let us know if that worked.
In converting from an older bit of code that uses the EWS (ews-java-api v 2.0) SDK/API/Scope to Graph (microsoft-graph v5.4.0), I found that I could search (say by InternetMessageId) across multiple folder hierarchies at once in EWS with (simplifying how to get FolderId values a bit):
SearchFilter.SearchFilterCollection filter =
new SearchFilter.SearchFilterCollection(LogicalOperator.And);
filter.add(new SearchFilter.IsEqualTo(EmailMessageSchema.InternetMessageId, msgId));
List<FolderId> folders = Arrays.asList(new FolderId("AllItems"), new FolderId("Deletions"));
ItemView view = new ItemView(10);
ServiceResponseCollection<FindItemResponse<Item>> findResultsCollection =
service.findItems(searchFolders, filter, null, view, null, ReturnErrors);
With that EWS search whether my message of interest is in the Inbox, some user-created sub-folder, JunkEmail, DeletedItems, RecoverableItemsDeletions I find it by InternetMessageId in one go.
With Graph I issue two calls to be able to ensure the message does not exist
UserRequestBuilder u = GraphServiceClient
.builder()
.authenticationProvider(authenticationProvider)
.buildClient()
.users(user);
for (String folderTree : Arrays.asList("AllItems", "RecoverableItemsDeletions")) {
MessageCollectionPage mcp = u.mailFolders(folderTree)
.messages()
.buildRequest()
.filter("internetMessageId eq '" + msgId + "'")
.get();
Is there a way to search multiple trees in one go with Graph to be more like the EWS path that took a List?
Use List Messages endpoint to get the messages in the signed-in user's mailbox (including the Deleted Items and Clutter folders).
Depending on the page size and mailbox data, getting messages from a mailbox can incur multiple requests. The default page size is 10 messages. Use $top to customize the page size, within the range of 1 and 1000.
To improve the operation response time, use $select to specify the exact properties you need. Refer documentation here.
Java Code Snippet -
GraphServiceClient graphClient = GraphServiceClient.builder().authenticationProvider( authProvider ).buildClient();
MessageCollectionPage messages = graphClient.me().messages()
.buildRequest()
.select("sender,subject")
.get();
The text here: https://developers.google.com/fusiontables/docs/v2/migration_guide implies that the 10MB limit not in effect for API v2, or that an alternative service "Media download" could be used for large responses.
The API Reference here: https://developers.google.com/fusiontables/docs/v2/reference/ does not have any information regarding the 10MB limit, or how you use "media download" to recieve your request.
How do I work around the 10MB limit for Fusion Tables API v2? I can't seem to find documentation that explains it.
To use media-download simply add the parameter alt=media to the URL
For those who use Google's API Client Libraries, the 'media download' is specified by using a specific method. For the Python library, there are two versions of the SQL query methods: sql*() and sql*_media() (and this is very likely true for the other client libraries as well).
Example usage:
# Build the googleapiclient service
FusionTables = build('fusiontables', 'v2', credentials=credentials);
query = 'select * from <table id>';
# "standard" query, returning fusiontables#sqlresponse JSON:
jsonRequest = FusionTables.query().sqlGet(sql = query);
jsonResponse = jsonRequest.execute();
# alt=media query, returning a CSV-formatted bytestring (in Python, at least):
bytestrRequest = FusionTables.query().sqlGet_media(sql = query);
byteResponse = bytestrRequest.execute();
As Kerry mentions here, media format queries that are too large to be sent as a GET request will fail (while regular format queries of the same length succeed provided the query result is less than 10 MB). In the python client, this failure appears as a HTTP 502: Bad Gateway error.
Also note that ROWIDs are currently not returned in the media response format.
I have a MVC 5 internet application, that uploads files to Azure blob storage. I have currently implemented the code to create a SAS for blobs in a container.
I have a MVC view, that lists many images via the <img src> tag for many different objects. Each of these images are the same image. By that I mean they have the same fileName and are in the same container.
Is it possible to check if a blob already has a current SAS url, and if so, retrieve that SAS url?
Thanks in advance.
EDIT
Here is my code to get a sas url:
public string GetBlobSasUri(string containerName, string fileName)
{
CloudBlobContainer container = GetCloudBlobContainer(containerName);
//Get a reference to a blob within the container.
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
//Set the expiry time and permissions for the blob.
//In this case the start time is specified as a few minutes in the past, to mitigate clock skew.
//The shared access signature will be valid immediately.
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessStartTime = DateTime.UtcNow.AddMinutes(-5);
sasConstraints.SharedAccessExpiryTime = DateTime.UtcNow.AddHours(4);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read;
//Generate the shared access signature on the blob, setting the constraints directly on the signature.
string sasBlobToken = blob.GetSharedAccessSignature(sasConstraints);
//Return the URI string for the container, including the SAS token.
return blob.Uri + sasBlobToken;
}
If I pass in a containerName with "Test" and a FileName with "Example.png", an SAS is created and returned. If I then pass the same parameters into the function, can I check to see if the FileName already has a SAS created for it, and if so, return the same SAS url?
SAS is Shared Access Signature. That means - it is just digital signature identifying given resource. It is not attached in any way to the resource. SAS is not just some randomly generated token, that is assigned to the resource. SAS is verified during the request - evaluated, signature check, resource check, action check. Thus the service itself (blob service) and the resources in that service (containers, blobs) have no idea of whether SAS exists or not.
Having said that you have couple of possible approaches:
Have cache that stores SAS, where cache key would be your blob URI. Appropriate cache expiry time shall be configured with respect to SAS lifetime
Create SAS for every request (in case you cannot co-relate requests)
Here is MSDN doc on Constructing Shared Access Signature URI.
I am trying to develop an IMAP email client using the javax.mail API. I have been able to contact the servers, fetch emails, attachments and other operations without any problem.
However, we would not want to fetch the attachment until the user wants to view it explit in order to improve the performance. This would mean that we would need information which would indicate if the email has an attachment, if yes, then the filename(s) and the size(s)but would not send me the actual content of the email. However, I was not able to find a method in the API description which would return just the multi-part content.
Is there a way I could fetch just the body and the details of the attachment but not the actual attachment?
Thanks,
Aravind
The method getContent() of javax.Part returns
the content as a Java object. The type
of the returned object is of course
dependent on the content itself. For
example, the object returned for
"text/plain" content is usually a
String object. The object returned for
a "multipart" content is always a
Multipart subclass.
See http://download.oracle.com/javaee/6/api/javax/mail/Part.html#getContent%28%29
If a Message (which is a Part) contains attachments, the getContent method will return a Multipart object. In addition you can know the MIME type of the part with getContentType.
The information that is missing from the documentation is that this MultiPart object returned by getContent is just an empty representation of the Message's structure. The content of each Part of the Multipart will only be fetched from the server when you specifically ask for it. For example, to parse a multipart Message you would do:
if( p.isMimeType("multipart/*") ) {
Multipart mp = (Multipart)p.getContent();
// the content was not fetched from the server
// parse each Part
for (int i = 0; i < mp.getCount(); i++) {
Part inner_part = mp.getBodyPart(i)
if( inner_part.isMimeType("text/plain") ) {
String text = inner_part.getText();
// the content of this Part was fetched from the server
}
}
}
Something else to consider is that for optimal performance you should get the body structure information from the server in batch for the collection of messages you want to parse (see http://download.oracle.com/javaee/6/api/javax/mail/FetchProfile.Item.html#CONTENT_INFO). If not, each
Multipart mp = (Multipart)p.getContent();
will result in a request to the server to fetch the body structure. If the FetchProfile approach is used, the body structure for the collection of messages will be fetched with only one request. You can see the requests to the server in the log if you activate debug mode on the session:
session.setDebug(true);
This said, to get size and filenames of attachments just use Part.getSize() and Part.getFileName(), respectively.