I'm trying to use the FSharp.Data third party library but am getting an error The type 'XmlProvider' is not defined on the XmlProvider class.
namespace KMyMoney
open FSharp.Data
module Read =
let xml = File.ReadAllText("KMyMoneySampleFile.xml")
type KMyMoneySource = XmlProvider<xml>
I'm using NuGet to get the library. Library is 'FSharp.Data 1.1.8'
When I type FSharp.Data. There are four options given: Csv, FreebaseOperators, Json, and RuntimeImplementation.
Am I missing something? I'm relatively new to F#. So, sorry for the simple question. I've looked on GitHub but haven't seen any mention of this problem. I am creating a library in F#.
The parameter between <> is the Sample parameter of the type provider, which has to be a compile time constant. That sample is used to infer the structure of the xml.
Try this instead:
namespace KMyMoney
open FSharp.Data
module Read =
type KMyMoneySource = XmlProvider<"KMyMoneySampleFile.xml">
and then do
let xml = KMyMoneySource.Load("KMyMoneySampleFile.xml")
or if you're reading the same file you used as the XmlProvider sample parameter, just do this:
let xml = KMyMoneySource.GetSample()
Note that Type Providers are a feature of F# 3.0, so this only works in VS2012 or upper. If you're using VS2010, you'll just get a bunch of syntax errors.
The data has to be available at compile-time which is achieved by putting a file reference in the angle brackets like this (notice that it is a string literal containing a file path, not a string binding containing the data). You can also achieve this by putting a string literal containing the format in the brackets:
type Stocks = CsvProvider<"../docs/MSFT.csv">
let csv = new CsvProvider<"1,2,3", HasHeaders = false, Schema = "Duration (float<second>),foo,float option">()
See here for more information.
Check out this link. Basically you need to add System.Xml.Linq.dll also as reference to your project.
Related
I am working on this Kaggle competition. The Jupyter notebooks on Kaggle only support R and Python and I wanted to use F# locally. The problem is that the datasets are .json files and both the F# Json Type Provider and Newtonsoft libraries fail when trying to parse the files.
Here are examples of the code failing in F#:
open FSharp.Data
type Context = JsonProvider<"train.json">
let context = Context.
and
open System
open System.IO
open Newtonsoft.Json
open Newtonsoft.Json.Linq
let object = JObject.Parse(File.ReadAllText("train.json"));
object
This Python example uses these line of code to parse them correctly:
train = pd.read_json('../input/stanford-covid-vaccine/train.json', lines=True)
test = pd.read_json('../input/stanford-covid-vaccine/test.json', lines=True)
In the notebook, the author says that without the "lines=True" parameter, the read_json method fails with this trailing error.
My question: assuming tis is the same error, is there a way to apply that same kind of "lines=true" to the .NET libraries to parse the json?
I've seen a few datasets where the format was one valid JSON record per line:
{"event":"nothing 1"}
{"event":"nothing 2"}
{"event":"nothing 3"}
This is not valid JSON overall. I think you can either parse it line-by-line or you can turn it into valid JSON. For line-by-line parsing (which may be more efficient as you can do this in a streaming fashion), I would use:
open FSharp.Data
type Log = JsonProvider<"""{"event":"nothing 1"}""">
for line in File.ReadAllLines("some.json") do
let l = Log.Parse(line)
printfn "%s" l.Event
In order to create a Json provider I need to pass a literal with the path. There are several people working on the project from different locations, and the paths are different in each case. (Actually only the beginning of each path). I tried to create a literal with pattern matching but the compiler does not accept it. Is there another way to do this?
My failed attempt is below:
open FSharp.Data
[<Literal>]
let bitbucketRoot = // Error message: This is not a valid constant expression
let computerName = Environment.MachineName
match computerName with
| "DESKTOP-G3OF32U" -> "C:\\Users\\Fernando"
| "HPW8" -> #"H:\Dropbox\"
| _ -> failwith "Unknown computer"
[<Literal>] // Error message: This is not a valid constant expression
let projDataPath = bitbucketRoot + #"Bitbucket\VSProjects\Fractal10\Fractal10\data\"
[<Literal>] // Error message: This is not a valid constant expression
let jsonPath = projDataPath + "fractal.json"
type PathInfo = JsonProvider<Sample=jsonPath>
I would advise that you store it in source control and make it a path relative to your project root, assuming you are working out of a common source control repository.
Either that, or host the sample on a public URL. (I wouldn't actually recommend this because including it in your source repository allows versioning and doesn't publicly expose your data)
You cannot create a conditional literal as the other comments point it out. However this is a fairly frequent use case and the way to deal with it is as follows:
#r #"..\packages\FSharp.Data\lib\net40\FSharp.Data.dll"
open FSharp.Data
open System
open System.IO
[<Literal>]
let JsonSource = __SOURCE_DIRECTORY__ + #"\test.json"
type JSonType = JsonProvider<JsonSource>
let json1 = JSonType.GetSamples()
let anotherPath = #"C:\tmp"
let anotherJson = anotherPath + #"\test.json"
let json2 = JSonType.Load(anotherJson)
The __SOURCE_DIRECTORY__ directive will point to the project root (just display it in the REPL) and then you can add the filename to it and make that a literal. If you check in this file into a git repo, then everyone who checks it out can have it in a relative path, and you can refer it when generating the type. When actually using the type or referring to the full file you can just use the .Load() method to load any file, and this doesn't have to be a literal.
There is actually a second way, which could work for you depending on the circumstances, compile a sample, and distribute it as a .dll. You can refer to this and use it directly without having access to the actual file. Please see the Using the JSON Provider in a Library section at the end of the documentation.
I have not tried referring to the json in a config file, it might also be possible.
I'm using the FSharp.Data library, how can I specify the following option:
LoadOptions.PreserveWhitespace
with the Load method? I've tried passing it in but the compiler complains there's no such overload.
type Detailed = XmlProvider<"./samples/sample.xml">
let test = Detailed.Load(fi, LoadOptions.PreserveWhitespace)
fi is just a file stream object.
How do I pass a variable path to a type provider?
I have a file that I want to load using the SAS type provider that is in different locations on different PCs.
For example, I would like to do something like:
[<Literal>]
let saspath =
match System.Environment.MachineName with
| "a" -> "c:/sas.sas7bdat"
| "b" -> "d:/sas.sas7bdat"
let sasfile = new SasFileTypeProvider<saspath>()
But this is not valid. It's related to this Type provider and static argument in F# and For an F# Type Provider, how do I make a relative path work as a static parameter?, but I do not have the option of using relative paths.
Unfortunately, this is not easily possible. Parameters passed to type providers have to be (syntactically) constants. There are various workarounds.
Use relative path. Can you change the code to use a relative path that will always work? This would be ideal - assuming the type provider supports this.
Improve the type provider. You could send a PR to the SAS type provider so that it can take the file name from an local config file or and environment variable. Many of the SQL type providers do this, because it is a good solution if you need some configuration.
Generate file with a constant. This is a bit of a hack, but you can generate a separate fsx file with the constant and #load it. You'll get red squigglies in the rest of your code (until you run the first part of the script), but if you're happy with that:
open System.IO
let text =
match System.Environment.MachineName with
| "a" -> "module Constants\n[<Literal>] let path = \"c:/sas.sas7bdat\""
| "b" -> "module Constants\n[<Literal>] let path = \"c:/sas.sas7bdat\""
File.WriteAllText("C:/temp/constants.fsx", text)
;;
#load "C:/temp/constants.fsx"
let sasfile = new SasFileTypeProvider<Constants.path>()
I am using the type expression:
type dbSchema = SqlDataConnection<ConnectionStringName="X1", ConfigFile="App.config">
This works great at compile time (I have full access to all the db types), but it fails at run time. I presume it's because the config file generated in the console application's bin directory is named something else, such as MyAppName.exe.config, and therefore the App.config file is not found.
Certainly, for an ASP.NET MVC type app that uses web.config, there's no issue because the compile and runtime config filenames are the same.
Fortunately, placing a duplicate App.config in the bin directory does remediate the problem, but is that what we are expected to do? Any thoughts?
The description of how the type provider definition works is misleading - the value in the typedef really only matters at code/compile time, and as a default at runtime. However, as you've noted, it isn't very smart about finding the correct config file at runtime.
You can accomplish what you want by passing the connection string as a parameter to GetDataContext:
type dbSchema = SqlDataConnection<ConnectionStringName="X2">
let db = dbSchema.GetDataContext(ConfigurationManager.ConnectionStrings.["X2"].ConnectionString)
...or if you also want to make it work in F# interactive, wrap it like so:
type dbSchema = SqlDataConnection<ConnectionStringName="X2">
#if COMPILED
let db = dbSchema.GetDataContext(ConfigurationManager.ConnectionStrings.["X2"].ConnectionString)
#else
let db = dbSchema.GetDataContext()
#endif
(Note that you will need a reference to System.Configuration.)
I don't have a VS2012 on this PC but this should be what you're looking for :
let exeConfigFile = Path.GetFileName(System.Reflection.Assembly.GetEntryAssembly().Location) + ".config"
let defaultConfigFile = "App.config"
let configFile = if File.Exists(exeConfigFile) then exeConfigFile else defaultConfigFile
type dbSchema = SqlDataConnection<ConnectionStringName="X1", ConfigFile=configFile>