Compile with standalone flag gives compilation errors in client code - f#

I'm attempting to compile Zero29 with the --standalone compiler flag. The project itself compiles fine, but I have a unit test project that exercises some code in the Zero29 project, even though it's an executable program (.exe).
Everything works fine without the --standalone compilation flag.
However, when I add the --standalone compilation flag to the Zero29 project, the Zero29 project compiles fine, but in the unit test project, the compiler complains about this Discriminated Union defined in the Zero29 project:
namespace Ploeh.ZeroToNine
open System
open Ploeh.ZeroToNine.Versioning
type Arg =
| Assign of Version
| AssignRank of Rank * int
| Increment of Rank
| ListVersions
| ShowHelp
| Unknown of string list
The unit test project directly references the Zero29 project:
Zero29.UnitTests --references--> Zero29 (where --standalone is added)
When I attempt to compile the entire solution, the Zero29 project compiles with the --standalone flag, but then compilation of Zero29.UnitTests fails. There are several errors, but they are all the same, so here's a single example:
error FS0039: The value or constructor 'Assign' is not defined
Which points to the third line of this code:
let ParseAssignVersionReturnsCorrectResult(version : string) =
let actual = [| "-a"; version |] |> Args.Parse
verify <# [Assign(Version version)] = (actual |> Seq.toList) #>
The strange thing is that while the compiler complains about Assign in the third line of this code snippet, it doesn't complain about the use of Args.Parse, even though it's defined in the same code file as the Arg Discriminated Union.
Why does it do that, and how can I resolve this issue?
(I've attempted to distil the problem here, but the links I've provided point to the actual code files on GitHub, if more information is required.)

Libraries compiled with the --standalone switch cannot expose any F# datatypes. This is, for one, expressly stated in Pickering (2007), p. 210. In your case, a discriminated union is one of these prohibited types. The fact that the file is an executable changes nothing here: it becomes a library the moment you attempt to use it as one.
There have been also multiple reports (for example, here and here) that even libraries compiled with --standalone behave, quoting one of these sources, “funky.” It would be safe to say that the use of this switch should perhaps be limited to stand-alone executables only (and they cannot pretend to be a library even when under unit tests).
Pickering R. (2007). Foundations of F#. Apress.

Related

Some exported symbols disappear after creating a dynamic framework with the ios_framework rule

I am facing quite a strange situation trying to build a mixed-language dynamic iOS framework with Bazel.
The source code of the framework consists of *.c, *.cpp, *.m, *.mm, *.h, *.hpp files.
My first naive attempt was to declare a single objc_library rule referencing all the sources. That failed with an error message reading something about conflicting rules. Then I declared four distinct objc_library rules for *.m, *.mm, *.c, and *.cpp files respectively, and then referenced all these four rules as dependencies for the final ios_framework rule.
At this point, everything compiled and linked just fine (barring several compiler warnings which were expected). However, now the black magic begins:
The static library resulting from compiling ObjectiveC++ sources does contain all the necessary symbols (verified with the "nm" tool)
The .apple_binary_lipobin file resulting from the linking phase does not contain symbols from ObjectiveC++ sources
I believe I carefully went through the command lines used for compiling and linking, both for Bazel and xcodebuild.
The only anomalies I was able to spot:
xcodebuild passes a -single_module argument when linking, while Bazel cross tool does not.
Bazel cross tool adds '-stdlib=libc++' '-std=gnu++11' command line arguments when invoking wrapped_clang_pp during linking, while xcodebuild does not pass these arguments. I guess Bazel does this because there're these lines in the default Apple CROSSTOOL:
action_config {
config_name: "objc++-executable"
action_name: "objc++-executable"
tool {
tool_path: "wrapped_clang_pp"
execution_requirement: "requires-darwin"
}
flag_set {
flag_group {
flag: "-stdlib=libc++"
flag: "-std=gnu++11"
}
but I am not sure if these flags are really needed when invoking the linker.
I must admit I've run out of ideas except for trying to patch the CROSSTOOl file to make it behave as close as xcodebuild as possible.
Please help.
Can you try adding alwayslink = 1 to the objc_library target containing the C++ symbols? The linker is deadstripping the C++ symbols as they are not being referenced anywhere in the binary.

Release build fails when types leak from transitive dependency, Debug is fine

I'm trying to explain a weird F# compiler behavior between Release and Debug configurations regarding transitive dependencies. I will use Newtonsoft.Json package as the base dependency here, because that's the furthest I managed to get in pinpointing the issue, and it makes the example a bit less abstract.
Let's create a library project called SerializerProject, referencing Newtonsoft.Json via paket. In this project there is only one module:
module Serializer =
open System.IO
open System.Text
open Newtonsoft.Json
type OptionConverter() =
inherit JsonConverter()
(* only the signature is important, the implementation
and other methods are not interesting *)
override x.WriteJson(writer: JsonWriter, value: obj, serializer: JsonSerializer) =
(* ... *)
let fromJson<'a> (bytes: byte []): 'a =
let s = Encoding.UTF8.GetString(bytes)
use sr = new StringReader(s)
use jr = new JsonTextReader(sr)
let serializer = JsonSerializer()
serializer.Converters.Add(OptionConverter())
serializer.Deserialize<'a>(jr)
Now let's create a second project in the same solution and reference SerializerProject via project reference. I'd like to use fromJson in my new project, that's why I referenced SerializerProject in the first place.
module MyModule =
open Serializer
(* This is here just so we reference the fromJson function *)
let deserializer bytes = fromJson bytes
This is the minimal example to reproduce the behavior.
Now when I build the solution in Debug configuration, everything compiles and works fine. But when I switch to Release, the compilation fails in the second project, in MyModule in the deserializer definition. The exact error message is this:
The type referenced through 'Newtonsoft.Json.JsonWriter' is defined in an assembly that is not referenced. You must add a reference to assembly 'Newtonsoft.Json'
I'm using Visual Studio 2015 Update 3, F# tools (fsc, fsi) show version 14.0.23413.0.
It kind of makes sense, because it's reading metadata of the SerializerProject and finds that public OptionConverter type leaks the type JsonWriter on its public WriteJson method (as well as other types and other methods, but this one is encountered first), but what makes me wonder is why this works in Debug mode and is only a problem in the Release mode.
What kind of extra operations does the compiler do that affect this?
Why is this not a problem in Debug build when the type defined in Newtonsoft.Json really leaks transitively into the second project?
As suggested in the comments I tried referencing Newtonsoft.Json and decompiling the second assembly with ILSpy to see whether inlining turned on by compiler optimizations is to blame here, but even in Release configuration the second assembly looks like this:
call !!0 [SerializerProject]Serializer::fromJson<!a>(uint8[])
The fromJson function has not been inlined to expose the JsonWriter type directly, so there seem to be something more subtle going on.
This isn't a blocking issue, I just made the converter types private as I don't want to use them from the outside anyway, but I'd like to dig deeper in F# compiler inner workings.

F# function changes type when compiled with standalone switch and referenced from another project

In a Visual Studio project for an F# library I have defined a function as
let inline Estimate (s : ^a seq) (f : float) (w : int) : float * float = ..
The type of Estimate is
val Estimate : s:seq<'a> -> f:float -> w:int -> float*float
Calling Estimate from a script within that project works as expected.
Now if I compile the project with the --standalone switch and reference the output DLL from another project, Estimate is shown to be
Estimate<'a,'a>(s: Collections.Generic.IEnumerabls<'a>, f: float, w:int) : float*float
i.e. it some reason now takes tuple arguments.
Thus the following does not work
let q, p = EstimationQuality.Estimate x f 1 // This value is not a function and cannot be applied
but calling it with tuple parameters works fine
let q, p = EstimationQuality.Estimate (x, f, 1) // No problem.
What's wrong here? Is it a bug in the compiler?
EDIT:
Digging a little deeper, it appears that the problem is linked with the use of LanguagePrimitives.GenericZero.
While the problem actually compiles with the tuple parameter call, i get a runtime error when Estimate is called.
An unhandled exception of type 'System.TypeInitializationException'
occurred in LibraryTest.dll
Additional information: The type initializer for
'GenericZeroDynamicImplTable`1' threw an exception.
Compiling an F# DLL which is intended to be used from F#, with the standalone switch is not a good idea.
Why? Because all the F# metadata is lost since the whole set of F# types are included in your DLL so those types get a different identity from the types of the F# application that call your DLL or fsi.
The caller assembly uses the types in Fsharp.Core.dll which now are not the same as the ones used in your standalone compiled DLL.
That's why you see the tupled arguments, as seen from C# which doesn't understand F# metadata at all.
Generic inline functions using static constraints break as well since they need the metadata to inline at the call site.
Compiling also the caller assembly as standalone would make things worse, then you will have 3 sets of Fsharp types with different identities.
I think the standalone switch is fine when used only in the 'end-user' application.

F# integer file directive

I've been using fslex and fsyacc, and the F# source files (.fs they generate from the lexer (.fsl) and parser (.fsp) rules refer to the original .fsl (and sometimes to the same .fs source file) all over the place with statement such as this (numbers are line numbers):
lex.fs
1 # 1 "/[PROJECT-PATH-HERE]/lex.fsp
...
16 # 16 "/PROJECT-PATH-HERE]/lex.fs
17 // This is the type of tokens accepted by the parser
18 type token =
19 | EOF
...
Also, the .fs files generated by pars.fsp do the same kind of thing, but additionaly reference to the F# signature file (.fsi) generated alongside it. What does any of this do/mean?
The annotations you see in the generated code are F# Compiler Directives (specifically, the 'line' directive).
The 'line' directive makes it so that when the F# compiler needs to emit a warning/error message for some part of the generated code, it has a way to determine which part of the original file corresponds to that part of the generated code. In other words, the F# compiler can generate a warning/error message referencing the original code which is the basis of the generated code causing the error.

Defining Modules VS.NET vs F# Interactive

I have written this code which compiles and works perfectly in VS.NET 2010
module ConfigHandler
open System
open System.Xml
open System.Configuration
let GetConnectionString (key : string) =
ConfigurationManager.ConnectionStrings.Item(key).ConnectionString
however when I do a control + A and Alt + Enter to send this to FSI I get an error
ConfigHandler.fs(2,1): error FS0010: Unexpected start of structured construct in definition. Expected '=' or other token.
OK.
So I change my code to
module ConfigHandler =
open System
open System.Xml
open System.Configuration
let GetConnectionString (key : string) =
ConfigurationManager.ConnectionStrings.Item(key).ConnectionString
Now Control + A, Alt + Enter is successful and I FSI nicely tells me
module ConfigHandler = begin
val GetConnectionString : string -> string
end
However now If I try to compile my code in VS.NET 2010, I get an error message
Files in libraries or multiple-file applications must begin with a namespace or module declaration, e.g. 'namespace SomeNamespace.SubNamespace' or 'module SomeNamespace.SomeModule'
How can I have both? Ability to compile in VS.NET and the ability to send modules to FSI?
There is a tiny -- but crucial -- difference between your two snippets of code which is to blame here.
F# has two ways to declare a module. The first, a "top-level module", is declared like this:
module MyModule
// ... code goes here
The other way to declare a module is as a "local module", like so:
module MyModule =
// ... code goes here
The main differences between the "top-level" and "local" declarations are that the local declaration is followed by an = sign and the code within a "local" module must be indented.
The reason you get the ConfigHandler.fs(2,1): error FS0010: Unexpected start of structured construct in definition. Expected '=' or other token. message for the first snippet is that you can't declare top-level modules in fsi.
When you added the = sign to your module definition, it changed from a top-level module to a local module. From there, you got the error Files in libraries or multiple-file applications must begin with a namespace or module declaration, e.g. 'namespace SomeNamespace.SubNamespace' or 'module SomeNamespace.SomeModule' because local modules must be nested within a top-level module or a namespace. fsi doesn't allow you to define namespaces (or top-level modules), so if you want to copy-paste the entire file into fsi the only way it'll work is if you use the compilation directives as #pad mentioned. Otherwise, you can simply copy-paste the local module definitions (without the containing namespace) into fsi and they should work as expected.
Reference:
Modules (F#) on MSDN
The common solution is to keep the first example and create a fsx file which references the module:
#load "ConfigHandler.fs"
You have advantage of loading multiple modules and writing plumbing code for experiment.
If you really want to load ConfigHandler.fs directly to F# Interactive, you can use INTERACTIVE symbol and compiler directives:
#if INTERACTIVE
#else
module ConfigHandler
#endif
which works for both fsi and fsc.

Resources