Web scraping stock dividend data with F# - f#

Question

I am attempting to scrape stock dividend data from web pages using F# and the FSharp.Data library. An example page can be seen at http://www.nasdaq.com/symbol/ibm/dividend-history.
To request the web page, my code is setup as a simple console app as an example and is as follows:
open FSharp.Data
[<EntryPoint>]
let main argv =
let url = "http://www.nasdaq.com/symbol/ibm/dividend-history"
let result = Http.RequestString(url)
System.Console.ReadLine() |> ignore
0 // return an integer exit code
When run, the RequestString method errors with:
"An unhandled exception of type 'System.ArgumentOutOfRangeException' occurred in FSharp.Core.dll
Additional information: Length cannot be less than zero."
It looks like the page is formatted in a way to that "traditional" scraping approaches won't work. Any ideas or thoughts would be appreciated.

Answer 1

This is the full stacktrace I get when I run the code:
System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
at System.String.Substring(Int32 startIndex, Int32 length)
at FSharp.Data.HttpHelpers.getAllCookiesFromHeader#671.Invoke(Int32 i, String cookiePart) in C:\Git\FSharp.Data\src\Net\Http.fs:line 675
at Microsoft.FSharp.Collections.ArrayModule.IterateIndexed[T](FSharpFunc`2 action, T[] array)
at FSharp.Data.HttpHelpers.getAllCookiesFromHeader(String header, Uri responseUri, CookieContainer cookieContainer) in C:\Git\FSharp.Data\src\Net\Http.fs:line 671
at <StartupCode$FSharp-Data>.$Http.InnerRequest#803-5.Invoke(WebResponse _arg2) in C:\Git\FSharp.Data\src\Net\Http.fs:line 803
at Microsoft.FSharp.Control.AsyncBuilderImpl.args#835-1.Invoke(a a)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.FSharp.Control.AsyncBuilderImpl.commit[a](Result`1 res)
at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronously[a](CancellationToken token, FSharpAsync`1 computation, FSharpOption`1 timeout)
> at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
at <StartupCode$FSI_0004>.$FSI_0004.main#() in C:\Users\helgeu.COMPODEAL\AppData\Local\Temp\~vs2B9.fsx:line 8
Stopped due to error
I think you unfortunately have stumbled upon an bug related to this cookie handling code:
https://github.com/fsharp/FSharp.Data/issues/904
<rant>
I have tried to look into that code, but it gives me a headache from the evil cut and paste of some google answer on how to handle cookies in C# and then badly translated to F#.
</rant>
Think maybe adding info to that github case might be a better option than here.