Until now we used a SharePoint on-premise custom web service which delivered all users (approx 15,000) inclusive properties like aboutMe, skills, etc. and the direct manager. That job took approx. 15 minutes.
All the data was stored in a Lucene search index.
Now we have to switch to O365.
I am able to get all the desired information from Microsoft Graph but it would take way too long (3 - 5 hours):
Fetch all users via /v1.0/users (with paging)
Iterate through the collection and
get manager for given user via /v1.0/[user-id]/manager
get properties like aboutMe, skills for given User via /v1.0/[user-Id]?$select=aboutMe,skills
Is there any efficient way to do that task?
Ideally, you should just call Microsoft Graph for the data you want on-demand rather than attempting to sync it to your own database.
Assuming you can't do that, you can deduce the time this takes using /delta endpoint (Get incremental changes for users). When you use a delta token, you will only get back resources that have changed (adds, deletes, edits) since your previous request. So your first pass might take a few hours, but subsequent passes should take seconds.
You can control which properties you're "tracking changes" against using the $select query parameter. For example, if you only care about changes to the displayName then using /v1.0/users/delta?$select=displayName will ensure you only receive changes to that property. From the documentation:
If a $select query parameter is used, the parameter indicates that the client prefers to only track changes on the properties or relationships specified in the $select statement. If a change occurs to a property that is not selected, the resource for which that property changed does not appear in the delta response after a subsequent request.
Also, consider batching requests to improve your processes' overall performance. Batching allows you to send multiple queries to Microsoft Graph in a single request and get the complete results back in a single response.
Related
I am ideally looking for an API that returns all the messages posted(including replies) since a given timestamp.
conversations.history is supposed to be the one I should be using, but it does not return replies, it only returns parent message (only if the timestamp of the parent message satisfies the "oldest" param I supply - i.e. if the oldest supplied in the query is later than parent's ts but earlier than replies, neither parent nor replies will be returned).
Hence, I am trying to find if there is any other API that returns the threads based on "oldest" timestamp. i.e. all the threads which have replies since a given timestamp.
Note: I looked at conversations.replies, it is only useful if you know which thread's replies you are fetching.
Currently there is no API to do what you aspire to do.
The best work around is manually fetching all threads data in-memory and then applying filter.
Did you find an alternative solution to this question? I have the same use case and when contacting Slack support I received the same response that we need to use the combination of conversations.history & conversations.replies. This will be quite an intensive and continuously growing number of calls if we need to call conversations.replies for all threaded messages just to filter out the timestamps that fit the date range. This would be catastrophic in the long run.
Ideally slack need to update conversations.replies API to support getting all replies between oldest & latest parameter just like in history.
Another alternative I am considering is to change the implementation and use the Events API instead of the Web Client API and use queueing to store all incoming messages then this will make sure that all messages are captured and stored then apply the required filters.
I was reading the Microsoft Graph API Documentation to batch queries right here and did not find what I need.
Basically I need to combine two or more requests but one depends to another value. I know there is a "dependsOn" feature to wait the other request, it is not what I am looking for.
Request one: GET '/me/joinedTeams';
Request two: GET 'teams/{groupId}/channels';
The "Request one" returns an array of groups and inside these array values there's an id property. Can I batch these two requests using the value of ther first one to get the second?
I am searching a way to do a GET and return all values of one teant like the Microsoft Teams Application does, returning all teams, all chats, etc. Batching requests is the more closer we can get it I think.
Or there is another way to generate the token to https://chatsvcagg.teams.microsoft.com/api/v1/teams/users/me url like Microsoft does?
#Gaspar, multiple api calls can be batched using json batching but any interdependent calls batch can not handle.
If you have any dependency, you have to make separate calls.
When using the user delta endpoint for the initial read with expand on Manager property the query returns duplicate results. Some of these differ by either having the Manager Id or not but in some cases there are exact copies in the result pages. Some processed data suggests that sometimes even key properties like DisplayName end up missing in one of the duplicates.
Anyone knows what is a reliable strategy to resolve these merge issues to arrive at a consistent state? Or ideally how to avoid having to do this at all?
$expand is not supported for users, groups and other identity entities with delta queries. I authored a pull request to fix the documentation. If you want to track changes for the manager you should use $select=manager instead.
I am trying to make an ABAP OData, that receive a request, does some calculation, then, should return a message to the End User and make an decision based on the user input. So basically, the OData service should be put "on hold", before its receives a response.
Does anyone have a good idea?
Appreciate your response.
Regards!
OData is a special kind of REST. REST is stateless. What you want is stateful.
The good way to do turn this stateful flow into a stateless one is:
Send a first request (REST: POST, OData: CREATE) that creates and saves(!) a document that represents the calculation and its result. That first request may return the calculation's result to be presented to the user.
The user's choice then sends a second request that addresses the previously created document (e.g. via a GUID) and includes the user's choice. This means the second request neither has to send the computation input again, nor does it actually perform any calculations; it only changes the existing object's state.
If the calculation is not needed anymore afterwards, that second request may delete it. To prevent data leakage, removing older calculations after a time limit (e.g. 24h) may be a wise move.
Is there a way to get ALL the responses for a single day in one transaction for a specific survey? on the API doc, I know there is the /surveys/{id}/responses/bulk option, and even I can send the start_created_at variable.
But I think that the API response has a max number of records/data it can send, it that case, what could the solution be? Paging through the results?
I'm using the .net API, found at this site, but I can build my own wrapper if necessary.
Reference link to API doc: /Surveys/SURVEY_ID/responses/bulk
Yes you're right the /surveys/{id}/responses/bulk endpoint is what you're looking for, and you can use the start_created_at and end_created_at to filter data to a date range.
The SurveyMonkey API doesn't allow a full dump of all your data, it will always be paginated. By default it'll paginate 50 at a time, but you can change that by using the per_page GET parameter.
That max per_page varies by endpoint, for responses BULK it is 100. So you'll have to fetch 100 at a time, looping through the pages to get all your data.
One alternative is to use webhooks and set up a subscriber, that way you can get new responses in real time and fetch them one by one. That way you can keep your data updated on your side as new responses come in, rather than running a script or endpoint to bulk dump all your data. But this depends on your use case, if you're building something like an export feature, then you'll have to go through the paginated route.