Microsoft Graph API duplicate users - microsoft-graph-api

Question

I’m calling the Microsoft graph restful api in powershell to receive users from my organisation and all works splendidly when i want to use a full import. But when I try /users/delta I receive a large number of duplicates.
In my test environment it’s not an issue (the script runs for a couple of seconds long for a thousand users) but in the production environment of 70k users it returns 160k records. I need to use delta so that I can store the delta token somewhere and for future delta not require reading all 70k users again needlessly.
I have tried adding /users/delta?$Filter=&$distinct(UserPrincipalName) also with id and it still returns the duplicate users. Does anyone have a clue about how I can filter out the duplicates in the api call?
$Version = ‘Beta’
$BatchSize = 500
$Uri = "https://graph.microsoft.com/$Version/Users/delta?top=$BatchSize"
$users = $null
do{#read all pages into a single array
try
{ #Get api data
$response2 = Invoke-RestMethod -Method Get -Uri $Uri -Headers $Global:AuthHeader
}
catch
{
Log -ExtraLogging $Global:Debug -LoggingLevel 1 -ErrorCode 2 -Message "$($_.Exception). terminating run profile"
exit
}
if($response2 -ne $null) #if multiple pages read all pages at once
{
$Uri = $response2.'#odata.nextlink'
$users = $users + $response2.value
}
}until ($Uri -eq $null -or $Uri -contains 'Error')

Answer 1

How are you returning all the users? since there is a 999 return limit per page for the users graph endpoint. I have a feeling that your own code is causing duplicates and not the graph endpoint. I tried running the command on a huge tenant and I don't get duplicates at all, but then again it only shows max 999 at a time. then you have to use the nextlink to get the next batch of users. most likely somewhere in your powershell logic you ended up returning the same thing twice somewhere.
But you would have to give more details about your script if you want more help.