F# using FSharp.Data not able to get basicAuth working - f#

I am new to F# and trying to automate APIs using FSharp.Data but I am not able to get basic auth working. I know there is some syntax issue that I am not able to crack. Please suggest how to get rid of this error on the "(BasicAuth "username_api" "pwd_api")" part of the code: This expression was expected to have type 'string' but here has type 'string * string'.
Also, I have attached an image for the quick reference.
Here is my test:
[<Test>]
let getBasicAuthUsingData () =
let Authorization (credentials:string) = "Authorization", credentials
let BasicAuth (username:string) (password:string) =
let base64Encode (s:string) =
let bytes = Encoding.UTF8.GetBytes(s)
Convert.ToBase64String(bytes)
sprintf "%s:%s" username password |> base64Encode |> sprintf "Basic %s" |> Authorization
let apiURL = "http://sampleapi"
let response = Http.RequestString( apiURL, httpMethod = "GET", headers = [ "Authorization", (BasicAuth "username_api" "pwd_api")])
printfn "Response of get is %s: " response

You're adding the name of the header twice: first in the Authorization function, and then in the list of headers for some reason.
As a result, the element of the header list ends up being ("Authorization", ("Authorization", "user:pass")) instead of just ("Authorization", "user:pass")
Just get rid is one of the "Authorization" strings:
let response = Http.RequestString( apiURL, httpMethod = "GET", headers = [BasicAuth "username_api" "pwd_api"])

Related

Azure Function: Event Grid Output Binding Throwing InvalidOperationException: 'Can't convert from type 'Azure.Messaging.EventGrid.EventGridEvent

I'm trying to build an Azure Function in F# with an Event Grid output binding. The code I have builds fine, but whenever I try and run it, I get the runtime exception: InvalidOperationException: 'Can't convert from type 'Azure.Messaging.EventGrid.EventGridEvent.
This is my code (note, I'm not currently using the outputEvents argument in the function, as I first wanted to make sure I could run the function before adding events to it):
[<FunctionName("EventListener")>]
let run ([<HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)>]req: HttpRequest)
([<EventGrid(TopicEndpointUri = "TopicEndpoint", TopicKeySetting = "TopicKey")>]outputEvents: IAsyncCollector<EventGridEvent>)
(log: ILogger) =
async {
log.LogInformation("F# HTTP trigger function processed a request.")
let nameOpt =
if req.Query.ContainsKey(Name) then
Some(req.Query.[Name].[0])
else
None
use stream = new StreamReader(req.Body)
let! reqBody = stream.ReadToEndAsync() |> Async.AwaitTask
let data = JsonConvert.DeserializeObject<NameContainer>(reqBody)
let name =
match nameOpt with
| Some n -> n
| None ->
match data with
| null -> ""
| nc -> nc.Name
let responseMessage =
if (String.IsNullOrWhiteSpace(name)) then
"This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response."
else
"Hello, " + name + ". This HTTP triggered function executed successfully."
return OkObjectResult(responseMessage) :> IActionResult
} |> Async.StartAsTask
What could be causing this issue?

F#- Using HttpFs.Client and Hopac: How do I get a response code, response headers and response cookies?

I am using F# with HttpFs.Client and Hopac.
I am able to get Response body and value of each node of JSON/XML response by using code like:
[<Test>]
let ``Test a Create user API``() =
let response = Request.createUrl Post "https://reqres.in/api/users"
|> Request.setHeader (Accept "application/json")
|> Request.bodyString ReadFile
|> Request.responseAsString
|> run
printfn "Response of get is %s: " response
let info = JsonValue.Parse(response)
let ID = info?id
printfn "ID in Response is %i: " (ID.AsInteger())
But how do I get a response code, response headers, and response cookies? I need to get this inside the same method as shown above so that I can do the assertion on these items too.
I did try response.StatusCode, response.Cookies.["cookie1"] but there are no such methods comes up when I add period after response.
let response =
Request.createUrl Post "https://reqres.in/api/users"
|> Request.setHeader (ContentType (ContentType.create("application", "json")))
|> Request.bodyString token //Reading content of json body
|> HttpFs.Client.getResponse
|> run
Please read the doc https://github.com/haf/Http.fs
Point 3 shows how to access cookies and headers in the response.
response.StatusCode
response.Body // but prefer the above helper functions
response.ContentLength
response.Cookies.["cookie1"]
response.Headers.[ContentEncoding]
response.Headers.[NonStandard("X-New-Fangled-Header")]

How to specify headers for a web request

Trying to set headers for a http call but running into issues. Need guidance for both Authorization and a custom header x-api-key.
let url = "http://example.com"
let token = requestToken()
let request = WebRequest.Create(url) :?> HttpWebRequest
request.Method <- "GET"
request.Accept <- "application/json;charset=UTF-8"
request.Headers.Authorization <- sprintf "%s %s" token.token_type token.access_token
request.Headers["x-api-key"] <- "api-key" // custom headers
// or this
request.Headers["Authorization"] <- sprintf "%s %s" token.token_type token.access_token
The error I'm getting is
error FS3217: This expression is not a function and cannot be applied. Did you intend to access the indexervia expr.[index] instead?
The error message that you are getting actually tells you what the problem is. In F#, the syntax for using an indexer is obj.[idx] - you need to have a . between the object and the square bracket. The correct syntax in your specific case would be:
request.Headers.["x-api-key"] <- "api-key"
request.Headers.["Authorization"] <- sprintf "%s %s" token.token_type token.access_token

Parsing http responses in Elm 0.18

I am parsing http responses from my server (Phoenix 1.3) on my Elm 0.18 frontend.
The response looks like this:
error: BadStatus { status = { code = 422, message = "Unprocessable Entity" }, headers = Dict.fromList [("cache-control","max-age=0, private, must-revalidate"),("content-type","application/json; charset=utf-8")], url = "http://localhost:4000/api/v1/sessions", body = "{\"error\":\"No user could be found\"}" }
I would like to extract the three-digit HTTP code as a String....in this case, "422".
What is the best way to parse this in Elm? I am using a very hacky method and I'd like to know what tools are best applied here.
errorCode : String -> String
errorCode =
error
|> Debug.log "error"
|> toString
|> String.split "code = "
|> List.drop 1
|> String.join ""
|> String.split ","
|> List.take 1
|> String.join ""
|> Debug.log "Error"
It looks like you have an Error from the elm-lang/http package. The string you quoted is just how Errors are rendered as strings in the console / debugger, I believe -- I don't think there's any parsing to be done, as such.
In other words, I think the function you want doesn't operate on Strings at all:
errorCode : Error -> Maybe String
errorCode err =
case err of
BadStatus response ->
Just response.status.code
_ ->
Nothing
(The elm compiler should of course tell you whether your String -> String or my Error -> Maybe String signature is correct.)

Download Windows Phone app reviews using F#

I am learning F# on my own (this is for fun, it is not for work/school) and I am trying to write a simple parser which count the number of reviews across multiple markets for a Windows Phone app. There's no doubt that the code I have so far is ugly, but I am trying to improve it and follow functional programming paradigm. Since I come from the C, C++, C# world, it is pretty hard.
Coming from C world, I like null values. I know that functional programming / F# doesn't encourage the use of null, but I can't figure out a way to not use it. For example, in the function parse there's a null check. How do I not do that?
Right now my code only count the number of reviews on the first page, but it is possible that an app has more than 10 reviews and as a result multiple pages. How do I recursively go through all page (functuion downloadReviews or parse).
How could we extend this code to be entirely async?
Below is the code I have so far. In addition to the questions above, I would really like if someone could help me and give me directions on how to improve the overall structure of my code.
open System
open System.IO
open System.Xml
open System.Xml.Linq
open Printf
type DownloadPageResult = {
Uri: System.Uri;
ErrorOccured: bool;
Source: string;
}
type ReviewData = {
CurrentPageUri: System.Uri;
NextPageUri: System.Uri;
NumberOfReviews: int;
}
module ReviewUrl =
let getBaseUri path =
new Uri(sprintf "http://cdn.marketplaceedgeservice.windowsphone.com/%s" path)
let getUri country locale appId =
getBaseUri(sprintf "/v8/ratings/product/%s/reviews?os=8.0.0.0&cc=%s&oc=&lang=%s&hw=520170499&dm=Test&chunksize=10" appId country locale)
let downloadPage (uri: System.Uri) =
try
use webClient = new System.Net.WebClient()
printfn "%s" (uri.ToString())
webClient.Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
webClient.Headers.Add("Accept-Encoding", "zip,deflate,sdch")
webClient.Headers.Add("Accept-Language", "en-US,en;q=0.8,fr;q=0.6")
webClient.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1482.0 Safari/537.36")
{ Uri = uri; Source = webClient.DownloadString(uri); ErrorOccured = false }
with error -> { Uri = uri; Source = String.Empty; ErrorOccured = true }
let downloadReview country locale appId =
let uri = ReviewUrl.getUri country locale appId
downloadPage uri
let parse(pageResult: DownloadPageResult) =
if pageResult.ErrorOccured then { CurrentPageUri = pageResult.Uri; NextPageUri = null; NumberOfReviews = 0 }
else
let reader = new StringReader(pageResult.Source)
let doc = XDocument.Load(reader)
let ns = XNamespace.Get("http://www.w3.org/2005/Atom")
let nextUrl = query { for link in doc.Descendants(ns + "link") do
where (link.Attribute(XName.Get("rel")).Value = "next")
select link.Value
headOrDefault }
if nextUrl = null then
{ CurrentPageUri = pageResult.Uri; NextPageUri = null; NumberOfReviews = doc.Descendants(ns + "entry") |> Seq.length }
else
{ CurrentPageUri = pageResult.Uri; NextPageUri = ReviewUrl.getBaseUri(nextUrl); NumberOfReviews = doc.Descendants(ns + "entry") |> Seq.length }
let downloadReviews(locale: string) =
let appId = "4e08377c-1240-4f80-9c35-0bacde2c66b6"
let country = locale.Substring(3)
let pageResult = downloadReview country locale appId
let parseResult = parse pageResult
parseResult
[<EntryPoint>]
let main argv =
let locales = [| "en-US"; "en-GB"; |]
let results = locales |> Array.map downloadReviews
printfn "%A" results
0
I was playing with this problem a bit more and tried using the XML type provider and other features from F# Data. It is not complete code, but it should be enough to give you the idea (and to show that type providers are really nice :-)):
First, I need some references:
#r "System.Xml.Linq.dll"
#r "FSharp.Data.dll"
open FSharp.Data
open FSharp.Net
Next, I wrote the following code to download one sample page.
let data =
Http.Request
( "http://cdn.marketplaceedgeservice.windowsphone.com//v8/ratings/product/4e08377c-1240-4f80-9c35-0bacde2c66b6/reviews",
query=["os", "8.0.0.0"; "cc", "US"; "lang", "en-US"; "hw", "520170499"; "dm", "Test"; "chunksize", "10" ],
headers=["User-Agent", "F#"])
I saved the sample as D:\temp\appstore.xml and then used the XML type provider to get a nice type for parsing the page:
type PageDocument = XmlProvider< #"D:\temp\appstore.xml" >
Then you can download & parse the page like this (this shows how to get the number of reviews and information about the next link):
let parseAsync (locale:string) appId = async {
let country = locale.Substring(3)
// Make the request (asynchronously) using the parameters specified
let! data =
Http.AsyncRequest
( "http://cdn.marketplaceedgeservice.windowsphone.com//v8/ratings/product/"
+ appId + "/reviews",
query=[ "os", "8.0.0.0"; "cc", country; "lang", locale;
"hw", "520170499"; "dm", "Test"; "chunksize", "10" ],
headers=["User-Agent", "F#"])
// Parse the result using the type-provider generated type
let page = PageDocument.Parse(data)
// Now you can type 'page' followed by '.' and explore the results!
// page.GetLinks() returns all links and page.GetEntries() returns
// review entries. Each link also has 'Rel' and 'Href' properties:
let nextLink =
page.GetLinks()
|> Seq.tryFind (fun link -> link.Rel = "next")
|> Option.map (fun link -> link.Href)
let reviewsCount = page.GetEntries().Length
return (reviewsCount, nextLink) }
The general pattern for making code asynchronous is to find the I/O expensive operation (somewhere down in the call tree) and then go "up" from there and make all code that uses it asynchronous too until you reach a point where you need to block.
In your example, the primitive operation is downloading, so you would start by making downloadPage asynchronous:
let downloadPage (uri: System.Uri) = async {
try
use webClient = new System.Net.WebClient()
printfn "%s" (uri.ToString())
// (Headers omitted)
let! source = webClient.AsyncDownloadString(uri)
return { Uri = uri; Source = source; ErrorOccured = false }
with error ->
return { Uri = uri; Source = String.Empty; ErrorOccured = true } }
You need to wrap code in async { ... }, make call to asynchronous version of DownloadString using let! and return the results using return (in both branches).
Then you need to make functions like downloadReview and downloadReviews (again, wrap them in async block, call other asynchronous operations like downloadPage using let! or using return!).
In the end, if you're writing console application you'll need to block, but you can run downloads for different locales in parallel. Assuming downloadReviews is asynchronous:
let locales = [| "en-US"; "en-GB"; |]
let results =
locales
|> Array.map downloadReviews // Build an array of asynchronous computations
|> Async.Parallel // Compose them into a single, parallel computation
|> Async.RunSynchronously // Run the computation and wait
To answer other questions, I think using null in the example above is probably okay (you are calling LINQ which returns it, so there is no easy way to avoid that). It is actually possible to use option type instead, but it is a bit tricky - see this snippet if you're interested.
Also, you could use the Http.AsyncRequest method from F# Data Library which gives you a bit simpler way to construct complex HTTP requests (but I'm one of the contributors to that library, so I'm biased!)
As Tomas said, it would be more "functional" to create an async-based version of DownloadString (or just use his FSharp.Data library to handle it).
You could also combine FSharp.Data with ExtCore to take advantage of the asyncMaybe or asyncChoice workflows in ExtCore. Those workflows provide very easy-to-use error handling on top of the normal async workflow.
Anyway, I spent a few minutes cleaning up your code. It's not much, but it does simplify your code in a few spots:
open System
open System.IO
open System.Xml
open System.Xml.Linq
open Printf
type DownloadPageResult = {
Uri : System.Uri;
ErrorOccured : bool;
Source : string;
}
type ReviewData = {
CurrentPageUri : System.Uri;
NextPageUri : System.Uri option;
NumberOfReviews : uint32;
}
module ReviewUrl =
let baseUri = Uri ("http://cdn.marketplaceedgeservice.windowsphone.com/", UriKind.Absolute)
let getUri country locale (appId : System.Guid) =
let localUri =
let appIdStr = appId.ToString "D"
sprintf "/v8/ratings/product/%s/reviews?os=8.0.0.0&cc=%s&oc=&lang=%s&hw=520170499&dm=Test&chunksize=10" appIdStr country locale
Uri (baseUri, localUri)
let downloadPage (uri : System.Uri) =
try
use webClient = new System.Net.WebClient()
printfn "%s" (uri.ToString())
webClient.Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
webClient.Headers.Add("Accept-Encoding", "zip,deflate,sdch")
webClient.Headers.Add("Accept-Language", "en-US,en;q=0.8,fr;q=0.6")
webClient.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1482.0 Safari/537.36")
{ Uri = uri; Source = webClient.DownloadString uri; ErrorOccured = false }
with error ->
{ Uri = uri; Source = String.Empty; ErrorOccured = true }
let parse (pageResult : DownloadPageResult) =
if pageResult.ErrorOccured then
{ CurrentPageUri = pageResult.Uri; NextPageUri = None; NumberOfReviews = 0u }
else
use reader = new StringReader (pageResult.Source)
let doc = XDocument.Load reader
let ns = XNamespace.Get "http://www.w3.org/2005/Atom"
let nextUrl =
query {
for link in doc.Descendants(ns + "link") do
where (link.Attribute(XName.Get("rel")).Value = "next")
select link.Value
headOrDefault }
{ CurrentPageUri = pageResult.Uri;
NextPageUri =
if System.String.IsNullOrEmpty nextUrl then None
else Some <| Uri (ReviewUrl.baseUri, nextUrl);
NumberOfReviews =
doc.Descendants (ns + "entry") |> Seq.length |> uint32; }
let downloadReviews (locale : string) =
System.Guid "4e08377c-1240-4f80-9c35-0bacde2c66b6"
|> ReviewUrl.getUri (locale.Substring 3) locale
|> downloadPage
|> parse
[<EntryPoint>]
let main argv =
let locales = [| "en-US"; "en-GB"; |]
let results = locales |> Array.map downloadReviews
printfn "%A" results
0

Resources