FAKE Fsc task is writing build products to wrong directory - f#

I'm just learning F#, and setting up a FAKE build harness for a hello-world-like application. (Though the phrase "Hell world" does occasionally come to mind... :-) I'm using a Mac and emacs (generally trying to avoid GUI IDEs by preference).
After a bit of fiddling about with documentation, here's how I'm invoking the F# compiler via FAKE:
let buildDir = #"./build-app/" // Where application build products go
Target "CompileApp" (fun _ -> // Compile application source code
!! #"src/app/**/*.fs" // Look for F# source files
|> Seq.toList // Convert FileIncludes to string list
|> Fsc (fun p -> // which is what the Fsc task wants
{p with //
FscTarget = Exe //
Platform = AnyCpu //
Output = (buildDir + "hello-fsharp.exe") }) // *** Writing to . instead of buildDir?
) //
That uses !! to make a FileIncludes of all the sources in the usual way, then uses Seq.toList to change that to a string list of filenames, which is then handed off to the Fsc task. Simple enough, and it even seems to work:
...
Starting Target: CompileApp (==> SetVersions)
FSC with args:[|"-o"; "./build-app/hello-fsharp.exe"; "--target:exe"; "--platform:anycpu";
"/Users/sgr/Documents/laboratory/hello-fsharp/src/app/hello-fsharp.fs"|]
Finished Target: CompileApp
...
However, despite what the console output above says, the actual build products go to the top-level directory, not the build directory. The message above looks like the -o argument is being passed to the compiler with an appropriate filename, but the executable gets put in . instead of ./build-app/.
So, 2 questions:
Is this a reasonable way to be invoking the F# compiler in a FAKE build harness?
What am I misunderstanding that is causing the build products to go to the wrong place?

This, or a very similar problem, was reported in FAKE issue #521 and seems to have been fixed in FAKE pull request #601, which see.
Explanation of the Problem
As is apparently well-known to everyone but me, the F# compiler as implemented in FSharp.Compiler.Service has a practice of skipping its first argument. See FSharp.Compiler.Service/tests/service/FscTests.fs around line 127, where we see the following nicely informative comment:
// fsc parser skips the first argument by default;
// perhaps this shouldn't happen in library code.
Whether it should or should not happen, it's what does happen. Since the -o came first in the arguments generated by FscHelper, it was dutifully ignored (along with its argument, apparently). Thus the assembly went to the default place, not the place specified.
Solutions
The temporary workaround was to specify --out:destinationFile in the OtherParams field of the FscParams setter in addition to the Output field; the latter is the sacrificial lamb to be ignored while the former gets the job done.
The longer term solution is to fix the arguments generated by FscHelper to have an extra throwaway argument at the front; then these 2 problems will annihilate in a puff of greasy black smoke. (It's kind of balletic in its beauty, when you think about it.) This is exactly what was just merged into the master by #forki23:
// Always prepend "fsc.exe" since fsc compiler skips the first argument
let optsArr = Array.append [|"fsc.exe"|] optsArr
So that solution should be in the newest version of FAKE (3.11.0).
The answers to my 2 questions are thus:
Yes, this appears to be a reasonable way to invoke the F# compiler.
I didn't misunderstand anything; it was just a bug and a fix is in the pipeline.
More to the point: the actual misunderstanding was that I should have checked the FAKE issues and pull requests to see if anybody else had reported this sort of thing, and that's what I'll do next time.

Related

How to issue Message Before Build--or seq problems

I'm trying to add helpful messages for arbitrary builds. If the build fails the user can, for example, install the package with different arguments.
My interface idea is to provide a function, build-with-message, that would be called with something like this:
build-with-message
''Building ${pkg.name}. Alternative invocations are: ..''
pkg
My implementation is based on builtins.seq
build-with-message = msg : pkg :
seq
(self.runCommand "issue-message" {} ''mkdir $out; echo ${msg}'')
pkg;
When I build a package with build-with-message I never see the message. My hunch is that seq evaluates the runCommand far enough to see that a set is returned and moves on to building the package. I tried with deepSeq as well, but a deepSeq build fails on runCommand. I also tried calling out some attributes from the runCommand, e.g.
(self.runCommand "issue-message" {} ''mkdir $out; echo ${msg}'').drvPath
(self.runCommand "issue-message" {} ''mkdir $out; echo ${msg}'').out
My thought being that calling for one of these would prompt the rest of the build. Perhaps I'm not calling the right attribute, but in any case the ones I've tried don't work.
So:
Is there a way to force the runCommand to build in the above scenario?
Is there already some builtin that just lets me issue messages on top of arbitrary builds?
Here's me answering my own question again, consider this a warning.
Solution:
I've in-lined some numbered comments to help with the explanation.
build-with-message = msg : pkg :
let runMsg /*1*/ = self.runCommand "issue-message"
{ version = toString currentTime; /*2*/ } ''
cat <<EOF
${msg}
EOF
echo 0 > $out /*3*/
'';
in seq (import runMsg /*4*/) pkg; /*5*/
Explanation:
runMsg is the derivation that issues the message.
Adding a version based on the current time ensures that the build of runMsg will not be in /nix/store. Otherwise, each unique message will only be issued for the first build.
After the message is printed, a 0 is saved to file as the output of the derivation.
The import loads runMsg--a derivation, and therefore serialized as the path $out. Import expects a nix expression, which in this case is just the number 0 (a valid nix expression).
Now, since the runMsg output will not be available until after it has been built, the seq command will build it (issuing the message) and then build pkg.
Discussion:
I take note of Robert Hensing's comment to my question--this may not be something Nix was not intended for. I'm not arguing against that. Moving on.
Notice that issuing a message like so will add a file to your nix store for every message issued. I don't know if the message build will be garbage collected while pkg is still installed, so there's the possibility of polluting the nix store if such a pattern is overused.
I also think it's really interesting that the result of the runMsg build was to install a nix expression. I suppose this opens the door to doing useful things.

F# How to implement parameterized CI – execution loop using FAKE

The question is mostly about step #3.
I need to implement the following loop in F#:
Load some input parameters from database. If input parameters specify that the loop should be terminated, then quit. This is straightforward, thanks to type providers.
Run a model generator, which generates a single F# source file, call it ModelData.fs. The generation is based on input parameters and some random values. This may take up to several hours. I already have that.
Compile the project. I can hard code a call to MSBuild, but that does not feel right.
Copy executables into some temporary folder, let’s say <SomeTempData>\<SomeModelName>.
Asynchronously run the executable from the output location of the previous step. The execution time is usually between several hours to several days. Each process runs in a single thread because this is the most efficient way to do it in this case. I tried the parallel version, but it did not beat the single threaded one.
Catch when the execution finishes. This seems straightforward: https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.process.exited?view=netframework-4.7.2 . The running process is responsible for storing useful results, if any. This event can be handled by F# MailBoxProcessor.
Repeat from the beginning. Because execution is async, monitor the number of running tasks to ensure that it does not exceed some allowed number. Again, MailBoxProcessor will do that with ease.
The whole thing will run on Windows, so there is no need to maintain multiple platforms. Just NET Framework (let’s say 4.7.2 as of this date) will do fine.
This seems like a very straightforward CI-like exercise and F# FAKE seemed as a proper solution. Unfortunately, none of the provided scarce examples worked (even with reasonable tweaks) and the bugs were cryptic. However, the worst part was that the compiling feature did not work at all. The provided example: http://fake.build/fake-gettingstarted.html#Compiling-the-application cannot be run at all and even after accounting for something like that: https://github.com/fsharp/FAKE/issues/1579 : it still silently choses not to compile the project. I’d appreciate any advice.
Here is the code that I was trying to run. It is based on the references above:
#r #"C:\GitHub\ClmFSharp\Clm\packages\FAKE.5.8.4\tools\FakeLib.dll"
#r #"C:\GitHub\ClmFSharp\Clm\packages\FAKE.5.8.4\tools\System.Reactive.dll"
open System.IO
open Fake.DotNet
open Fake.Core
open Fake.IO
open Fake.IO.Globbing.Operators
let execContext = Fake.Core.Context.FakeExecutionContext.Create false "build.fsx" []
Fake.Core.Context.setExecutionContext (Fake.Core.Context.RuntimeContext.Fake execContext)
// Properties
let buildDir = #"C:\Temp\WTF\"
// Targets
Target.create "Clean" (fun _ ->
Shell.cleanDir buildDir
)
Target.create "BuildApp" (fun _ ->
!! #"..\SolverRunner\SolverRunner.fsproj"
|> MSBuild.runRelease id buildDir "Build"
|> Trace.logItems "AppBuild-Output: "
)
Target.create "Default" (fun _ ->
Trace.trace "Hello World from FAKE"
)
open Fake.Core.TargetOperators
"Clean"
==> "BuildApp"
==> "Default"
Target.runOrDefault "Default"
The problem is that it does not build the project at all but no error messages are produced! This is the output when running it in FSI:
run Default
Building project with version: LocalBuild
Shortened DependencyGraph for Target Default:
<== Default
<== BuildApp
<== Clean
The running order is:
Group - 1
- Clean
Group - 2
- BuildApp
Group - 3
- Default
Starting target 'Clean'
Finished (Success) 'Clean' in 00:00:00.0098793
Starting target 'BuildApp'
Finished (Success) 'BuildApp' in 00:00:00.0259223
Starting target 'Default'
Hello World from FAKE
Finished (Success) 'Default' in 00:00:00.0004329
---------------------------------------------------------------------
Build Time Report
---------------------------------------------------------------------
Target Duration
------ --------
Clean 00:00:00.0025260
BuildApp 00:00:00.0258713
Default 00:00:00.0003934
Total: 00:00:00.2985910
Status: Ok
---------------------------------------------------------------------

Fake Glob operator (!!) not expanding directory paths

I have a situation where in a Fake script I am trying to grab all the unit test dll's from a path using the Glob (!!) operator.
The issue is that on my machine the glob expansion doesn't work, on other similar Windows 10 machines at work, its fine and finds the dlls.
Below is an example:
let path = [function to generate path]
trace path [would look something like "c:\git\project\src\**\*UnitTest*"]
!! path.ToLower()
|> Seq.iter (fun file -> trace file ) [this would not output anything]
I've tried numerous things:
uninstalling older versions of F#
reinstalling the lastest version
ensuring F# is in my Path
The versions of software I am using are:
Fake v4.63.2
Windows 10
F#4.1
No errors or exceptions are thrown.
Whats the best way to trouble shoot if its an F# or a Fake issue?
How could I work out what version of F# Fake is using?
Update
I've reinstalled F# 4.1 and performed a test using fsi.exe with the following command:
Microsoft (R) F# Interactive version 4.1
<snip>
#r #"packages/FAKE/tools/FakeLib.dll";;
open Fake;;
!! "**\*UnitTests.dll" |> Seq.iter (fun x -> trace x);;
C:\git\project1\bin\Debug\project1.UnitTests.dll
C:\git\project2\bin\Debug\project2.UnitTests.dll
!! "**\*UnitTests.dll".ToLower() |> Seq.iter (fun x -> trace x);;
C:\git\project1\bin\Debug\project1.UnitTests.dll
C:\git\project2\bin\Debug\project2.UnitTests.dll
All the test dlls were found, both with and without the call to ToLower().
When I remove the ToLower() from the script, it now works on my machine.
However, on other peoples machines removing ToLower() on the path causes them not to find any files.
So, is Fake using a different version of the fsi.exe?
I've opened a github issue to see if that sheds any light on it: https://github.com/fsharp/FAKE/issues/1772
In F# as in all .NET languages, the backslash is used for escape sequences in strings.
You need to escape the backslash or use a verbatim string, eg :
let path = "c:\\git\\project\\src\\**\\*UnitTest*"
or
let path = #"c:\git\project\src\**\*UnitTest*"
Fake can work with forward slashes as well :
let path = "c:/git/project/src/**/*UnitTest*"
You'll have to use forward slashes anyway if you want your build script to run on Linux.
An even better option is to use relative paths. Your build script most likely is stored in your project folder. You can write
let path = "src/**/*UnitTest*"
Using the following script, I was able to reproduce the issue and work out that the issue was due to how Windows 10 handles the original casing of the company name, in the path.
I confirmed this by changing company name to ** in the file path expression, the operator worked and found all the dlls.
I remember changing the capitalisation of the company name, from all caps to lower case. If I remove the ToLower() on the path, then the script works fine and finds all the dlls.
This hidden issue, combined with how FAKE does a case sensitive search, doesn't help either.
Powershell
packages\FAKE\tools\FAKE.exe glob.test.fsx
glob.test.fsx
#r #"packages/FAKE/tools/FakeLib.dll"
open Fake
let thePath = """C:\git\company/projectname/**/bin/Debug/*UnitTests.dll"""
sprintf "the path is %s" thePath |> trace
!! thePath.ToLower() |> Seq.iter (fun f -> trace f)
I had a look at the process executing in ProcMon and did not see the original casing of the directory. The NTFS file system is still see this directory as its original casing (see comments below).
I re-image my machine every few months, so this will disappear soon but it was good to understand what was going on.
Thanks to all those that helped narrow the issue down.

Running F# xUnit Fact from TestDriven.NET reporting "It looks like you're trying to execute an xUnit.net unit test."

I am trying to run xUnit tests (from an F# module, if it makes any difference) using TestDriven.NET, but whatever I do I get this error:
It looks like you're trying to execute an xUnit.net unit test.
For xUnit 1.5 or above (recommended):
Please ensure that the directory containing your 'xunit.dll' reference also contains xUnit's
test runner files ('xunit.dll.tdnet', 'xunit.runner.tdnet.dll' etc.)
For earlier versions:
You need to install support for TestDriven.Net using xUnit's 'xunit.installer.exe' application.
You can find xUnit.net downloads and support here:
http://www.codeplex.com/xunit
I tried following the suggestions, i.e. I copied the files
xunit.dll.tdnet
xunit.extensions.dll
xunit.gui.clr4.exe
xunit.runner.tdnet.dll
xunit.runner.utility.dll
xunit.runner.utility.xml
xunit.xml
to the folder with xunit.dll and I ran xunit.installer.exe. How can I get it to work?
I just figured out that I forgot to make the test a function in F# (so it was just a value). The error message can't be more misleading though!
You have two problems:
your Fact is broken:-
If you hover over the
please work
bit, you'll see something like: unit -> int
For a Fact to be picked up by an xUnit runner, it needs to yield `unit (void).
Hence, one key thing to get right first is to not return anything. In other words, replace your 123 with () (or an Assertion).
You can guard against this by putting a :unit stipulation on the test:-
[<Fact>]
let ``please work`` () : unit = 123
This will force a compilation error.
TestDriven.NET is reporting it cannot find the xunit.tdnet modules
It's critical to get step 1 right first. Then retry and the problem should be gone
If it remains...
Either try the VS-based runner which should work as long as it's installed and xunit.dll is getting to your output dir or look at the docs for your version of TD.NET for detailed troubleshooting notes (exec summary is if the .tdnet file was in your out dir or you undo and redo the xunit.installer from the folder containing the packages it should just work, esp if you are on latest)

F#'s "Hello, world" with 2 fs files

I come from C# background to F#. So far I wrote simple programs and spent a lot of time in F# interactive.
I'm stuck creating a VS F# project with two .fs files.
Sample code:
// part 1: functions
let rec gcd (a : uint64) (b : uint64) =
if b = 0UL then a
else gcd b (a % b)
// part 2: main()
let a, b = (13UL, 15UL)
do printfn "gcd of %d %d = %d" a b (gcd a b)
I'd like to have two .fs files, namely, Alg.fs and Program.fs, so that Program.fs would contain the code I'm working and Alg.fs having algorithms.
Taken steps:
I've created the two files. Compiler gave an error: Files in libraries or multiple-file applications must begin with a namespace or module declaration, e.g. 'namespace SomeNamespace.SubNamespace' or 'module SomeNamespace.SomeModule'
I've inserted module Program and module Alg. The complied program executes only the code from Alg.fs completely ignoring Program.fs...
I'm using F# 2.0 in Visual Studio 2010.
P.S. I've googled and checked some posts, and read documentation on modules and saw relative questions before asking.
Sounds like this is an order-of-files-in-the-project issue. The last file is the entry point ("main method"), sounds like you have Alg.fs last, and you need Program.fs last. You can re-order them via the right-click context menu in VS Solution Explorer.
There are at least three separate things that need to be looked at here:
As mentioned by #Brian, the order of source control files is also the compile order. This matters in F# where type inference is heavily used. Make sure Alg.fs comes before Program.fs in your Visual Studio file list (try this: select Program.fs and hit Alt+Down Arrow until it's at the bottom).
Since Alg.fs and Program.fs are now in modules, you need to actually open the Alg module in Program to get access to its bindings (open Alg), or add the [<AutoOpen>] attribute on Alg.
As #Daniel says, the last problem could be the definition of the entry point to the program. You need either an [<EntryPoint>] attribute on a top level binding that is also the last function in the last file. Alternatively, this defaults to the last binding in the last file anyway, just make sure it has the right signature (see Daniel's link).

Resources