I am making a call to the GRAPH API using the delta method to get Users
/beta/users/delta?$skipToken=[SKIPTOKEN]
the length of my skip token returned on the last delta request using a delta token is 17005 characters, this in turn makes the URL too long.
What methods can we use to reduce the size of the skip token?
There are over 37000 users in our AAD what we are querying and this may be the problem but not sure how the skip and delta tokens are generated.
Any help or advise would be greatly appreciated.
Is there anyway to put the skip / delta tokens in a header value to mitigate this issue?
The skipToken generated from server-side due to server-side paging, try to use $top and odata:nextLink to do paging when querying large data.
Check here for more details
This is something the team is aware of and we're investigating ways to fix it. In the meantime, you can workaround the issue by wrapping your requests in a batch request, which moves the too-long URL into the body of the POST. Not ideal, but it gets around the issue while we work on making it better :)
Related
I have an application that was doing call to Azure Application Gateway and it was failing when the following rule was enforced:
RuleId: 200003
Description: Multipart Request Body Strict Validation
The call was to a PUT endpoint, passing 2 files in the body. Disabling this rule makes the call work.
Can't find a proper description about this rule. Just wondering if someone could provide more info about it? Is it safe to disable it?
I had this issue and MS support response was this is a bug and to disable the rule. See their response below.
I went through the WAF logs, and it seems we are hitting rule 200003 matching your request.
After investigation, there is basically a bug with this rule that is a new multipart formatting body check in 3.2. (it was only added a couple weeks ago).
What our expert says about is the following:
“This rule it's looking at the stream of the file and trying to determine if the formatting of the multipart request is valid. If it thinks it's not, the body check fails and the rest of the request can't be scanned. This particular rule causes a lot of false positives and should be disabled if that is the case here.
The 200002/200003/200004 family of rules are very difficult to workaround because they are looking at the formatting of the request, however, how the request is formatted is almost impossible to predict because the streaming of a file this way get encoded into a bunch of random chars, and if those chars break the typical formatting boundaries of a multipart request these rules will trigger. But.. since you can't predict how a file will get encoded during a stream, it's almost impossible to "fix" from a code /request level. Disable the rule the profit.“
I have a Jmeter script where during replay, Post request is displaying as Get request and the parameters in the request are not sent to the server. Due to this, correlations are failing at this request.
One of the parameters in the request is ViewState with so many characters. Is this large parameter value causing the above issue? How to proceed now?
Most probably you're sending a malformed request therefore instead of properly responding to a POST request you're being redirected somewhere (most probably to Login page)
Use View Results Tree listener in HTML or Browser mode to see what page you're hitting in the reality
With regards to the ViewState, "so many characters" is not a problem, the problem is that these are not random characters. ViewState is being used for client-side state management and if you fail to provide the proper value you won't be able to move further so you need to design your test as follows:
Open first page
Extract ViewState using a suitable Post-Processor
Open second page
here you need to pass viewstate from the step 1 along with other parameters
More information: ASP.NET Login Testing with JMeter
Also don't forget to add HTTP Cookie Manager to your Test Plan
What I'm able to understand is the request may be getting redirected. This happens usually when the server expects a unique request. If you recorded the request, you may be possibly using older headers that carry old cookie information. Check your headers and then reconstruct the request.
Make sure you are not using old cookies anywhere. remove that cookie part from HTTP Header Manager everywhere.
I encountered an odd problem working with the Microsoft GraphAPI beta Delta Query endpoint for users.
The first 2 requests worked as expected, supplying a "nextLink" in the response.
Then, the last request (which was supposed to be the last, and according to documentation should have returned a "deltaLink" in the response) fails returning a 404 (non-JSON response).
Is this a known issue with the beta Delta Query?
The request:
https://graph.microsoft.com/beta/users/delta?$skiptoken={TOKEN_HERE}
The response:
404 - File or directory not found.
The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.
There is a known issue in the API where the skip token size, hence the query string size, is sometimes large (> 256 characters) and service does not handle it. We are actively working on a fix. Please check your skip token size and let us know in case it does not satisfy above criteria.
This issue has been fixed. Let us know if you have any more issues or feedback on the capability.
I am trying to code an API which has a long running process to which an end user may make a POST request:
POST /things { "some":"json" }
The actual creation process can take some time, will often be queued. It might take many minutes. As a result I am not sure what I should be returning or when. Is it the usual 201 plus object, returned after whatever time it takes my API to create the object? Isn't this going to cause problems at the client end? Is there some other standard way to do this - such as an intermediate step?
I'm using Rails & Grape for my API if that helps.
Consider whether the Post-Redirect-Get pattern suits your needs. For example, you can return a 303 redirect to some sort of status page where the client can check the progress of the request. In general, 201+object is a poor choice if the client has to wait for any appreciable period, because too many things can go wrong (what if out of annoyance or impatience he kills the browser window, or refreshes, or resubmits?)
How do I play a track from a SoundCloud URL, which, for example, I got from the xml response from a query
<stream-url>https://api.soundcloud.com/tracks/31164607/stream</stream-url>
I should have thought that it would have been as easy as:
https://api.soundcloud.com/tracks/31164607/stream&client_id=my_client_id
yet I get
<error>401 - Unauthorized</error>
All I want to do is consume it in a Silverlight MediaElement, so all I need is set some url to the MediaElement's Source property.
I've checked an application that I wrote about 2 years ago, and THEN, accessing the stream url was as easy as this for a public track:
http://api.soundcloud.com/tracks/18163056/stream&consumer_key=MY_CONSUMER_KEY
however this no longer seems to work.
For example, all I had to do then in C# was:
MediaElement me = new MediaElement();
me.Source= new Url("http://api.soundcloud.com/tracks/18163056/stream&consumer_key=MY_CONSUMER_KEY");
me.Play();
Any hints would be appreciated.
I had a reply on a Microsoft forum that seems to imply that SoundCloud might not be possible to stream to Windows 8 Metro devices without consuming the whole stream before playback starts - which is quite worrying and would seem to imply that to make authentication possible, it would have to be done entirely in the url querystring insterad of using the header:
(The following reply is the answer to the following question: 'I am able to access an audio stream by http using the MediaElement, however I need to access it via https in which I need to add the oAuth info to the header of the initial request.
How is this done when using a MediaElement, and if it cannot be done, what is the workaround for consuming an audio feed in Metro 8 that requires header authentication to stream?')
"Direct access to the underlying network stream is not currently permitted by the MediaElement. Because of this there is currently no way to modify the header of the HTTP request to include any additional authentication information. That said, you do have control over the URL. You could theoretically setup an HTTP proxy service that translated the HTTP GET request parameters into the necessary oAuth credentials. Keep in mind that this is just a theoretical workaround. You may find different behavior in practice. Another theoretical workaround would be to handle the oAuth yourself via a raw stream socket and pass the retuned media data to the MediaElement via "Set Source" and a "Random Access Stream". Please keep in mind that this method has major limitations. in order to use a "Random Access Stream" with the ME you need to make sure all of the data is available before passing it to the ME."
The proxy service is not scalable for an application that is merely distributed for free as every stream would need to come via the proxy. And the raw stream socket, although getting around this, would mean that playback could not start until the whole file had downloaded - and this goes against all current UX (User Experience) guidelines.
So once again, if anyone has any tips, or info about how the whole authentication thing can be achieved in a querystring instead of using headers, I'd appreciate it!
I'm a little confused about whether you're referring to a public or a private track? If it's a public track, then you shouldn't need to send any authentication information, just your client id.
When I request https://api.soundcloud.com/tracks/31164607/stream?client_id=YOUR_CLIENT_ID then I get a 302 redirect to the proper mp3 stream.
Remember, adding parameters to a URL must start with a ? not &. This could (more than likely) be the reason why you are getting a 401 (SC is not picking up the client_id).
After authentication the link like this
http://api.soundcloud.com/tracks/103229681/stream?consumer_key=d61f17a08f86bfb1dea28539908bc9bf
is working fine. I am using Action Script.
I'm following up on Tom's reply because he calls attention to url character specificity. My HTTP requests randomly started failing today, and I was prefacing my client_Id with a ?. As soon as I changed that single ? to &, it started working. So in my case, SC wasn't picking up my client_Id because I used the wrong character. I think depending on where in the request we're talking about specifically, it's worth noting that differences between ? and & do make a difference.