Fake Glob operator (!!) not expanding directory paths - f#

I have a situation where in a Fake script I am trying to grab all the unit test dll's from a path using the Glob (!!) operator.
The issue is that on my machine the glob expansion doesn't work, on other similar Windows 10 machines at work, its fine and finds the dlls.
Below is an example:
let path = [function to generate path]
trace path [would look something like "c:\git\project\src\**\*UnitTest*"]
!! path.ToLower()
|> Seq.iter (fun file -> trace file ) [this would not output anything]
I've tried numerous things:
uninstalling older versions of F#
reinstalling the lastest version
ensuring F# is in my Path
The versions of software I am using are:
Fake v4.63.2
Windows 10
F#4.1
No errors or exceptions are thrown.
Whats the best way to trouble shoot if its an F# or a Fake issue?
How could I work out what version of F# Fake is using?
Update
I've reinstalled F# 4.1 and performed a test using fsi.exe with the following command:
Microsoft (R) F# Interactive version 4.1
<snip>
#r #"packages/FAKE/tools/FakeLib.dll";;
open Fake;;
!! "**\*UnitTests.dll" |> Seq.iter (fun x -> trace x);;
C:\git\project1\bin\Debug\project1.UnitTests.dll
C:\git\project2\bin\Debug\project2.UnitTests.dll
!! "**\*UnitTests.dll".ToLower() |> Seq.iter (fun x -> trace x);;
C:\git\project1\bin\Debug\project1.UnitTests.dll
C:\git\project2\bin\Debug\project2.UnitTests.dll
All the test dlls were found, both with and without the call to ToLower().
When I remove the ToLower() from the script, it now works on my machine.
However, on other peoples machines removing ToLower() on the path causes them not to find any files.
So, is Fake using a different version of the fsi.exe?
I've opened a github issue to see if that sheds any light on it: https://github.com/fsharp/FAKE/issues/1772

In F# as in all .NET languages, the backslash is used for escape sequences in strings.
You need to escape the backslash or use a verbatim string, eg :
let path = "c:\\git\\project\\src\\**\\*UnitTest*"
or
let path = #"c:\git\project\src\**\*UnitTest*"
Fake can work with forward slashes as well :
let path = "c:/git/project/src/**/*UnitTest*"
You'll have to use forward slashes anyway if you want your build script to run on Linux.
An even better option is to use relative paths. Your build script most likely is stored in your project folder. You can write
let path = "src/**/*UnitTest*"

Using the following script, I was able to reproduce the issue and work out that the issue was due to how Windows 10 handles the original casing of the company name, in the path.
I confirmed this by changing company name to ** in the file path expression, the operator worked and found all the dlls.
I remember changing the capitalisation of the company name, from all caps to lower case. If I remove the ToLower() on the path, then the script works fine and finds all the dlls.
This hidden issue, combined with how FAKE does a case sensitive search, doesn't help either.
Powershell
packages\FAKE\tools\FAKE.exe glob.test.fsx
glob.test.fsx
#r #"packages/FAKE/tools/FakeLib.dll"
open Fake
let thePath = """C:\git\company/projectname/**/bin/Debug/*UnitTests.dll"""
sprintf "the path is %s" thePath |> trace
!! thePath.ToLower() |> Seq.iter (fun f -> trace f)
I had a look at the process executing in ProcMon and did not see the original casing of the directory. The NTFS file system is still see this directory as its original casing (see comments below).
I re-image my machine every few months, so this will disappear soon but it was good to understand what was going on.
Thanks to all those that helped narrow the issue down.

Related

F# Fake globbing operator and MSBuild

I'm learning Fake 5 (F# Make) and I'm going through the Getting Started tutorial. When I run the following code I receive an error message : tryscript.fsx (6,7)-(6,54): Error FS0001: The type 'Fake.IO.IGlobbingPattern' is not compatible with the type 'seq<'a>'
#r "paket: nuget Fake.IO.FileSystem //"
open Fake.IO.Globbing.Operators
let csProjectFiles = !! "src/app/**/*.csproj"
csProjectFiles
|> Seq.iter (fun x -> printfn "ProjectFile: %s" x)
// for projectFile in csProjectFiles do
// printfn "F# ProjectFile: %s" projectFile
But if I comment out two lines starting at csProjectFiles |> ... and uncomment the last two lines I will get the expected output of file names.
According to documentation and Ionide tooltips the !! should return a sequence of file names. Can someone advise me what I might be doing wrong?
P.S. I'm using Fake 5.3.1 installed using dotnet tool install fake-cli -g
UPD. I don't have any solution for this issue. It resolved itself after Windows 10 got an update and I removed Nuget package caches in %HOMEPATH\.nuget, %HOMEPATH%\AppData\Local\Nuget, and deleted .fake folder and lock file in the same folder as FAKE script and then reran script again.
If you are still facing similar issue developers ask for extended log fake -vv run <yourScriptName>.fsx after you clear all the caches, and archived contents of %HOMEPATH%\.nuget\packages\netstandard.library after this run.
Just for completeness sake, the reported issue can be found here: https://github.com/fsharp/FAKE/issues/2062
If anyone encounters this issue please open a new issue (and link the old one) and provide the following information:
Can you clean everything and send the output of fake -vv run tryscript.fsx and attach the logfile? Something is indeed fishy with the NetStandard.Library package
Can you also compress and attach the folder C:\Users\.nuget\packages\netstandard.library and then try to delete it (and again create a logfile for that)?
I'd assume this was either a caching issue or an F# compiler bug or both.

Integrating FxCop Gives an error : FAKE F#MAKE

I am using FxCop in FAKE but it is giving an error
i.e.
Analysis was not performed; at least one valid rules assembly and one valid target file must be specified.
* 1 total analysis engine exceptions.
While all targets are successfully build.
here is my code :
Target "FxCop" (fun _ ->
!! (buildDir + "/**/*.dll")
++ (buildDir + "/**/*.exe")
|> FxCop (fun p ->
{p with
//Override default parameters
ReportFileName = testDir + "FXCopResults.xml";
ToolPath = "D:/Fake/FAKE-Calculator/tools/FxCop/FxCopCmd.exe"})
)
It also shows : Project error : No targets were selected .
The FAKE documentation doesn't make it clear enough, but apparently you need to explicitly specify one of two things:
Which FxCop rules you want to run, or
The path to an "FxCop project file" (a file with the .FxCop extension).
I can't tell you how to write an FxCop project file since I've never done so myself, but maybe the programmer who set up the MsBuild system you've working with already did so. If he did, then you just need to add the following parameter to your FxCop call in your FAKE build script:
ProjectFile = buildDir </> "filename.FxCop"
where filename, of course, should be replaced by a real file name.
If you don't have an FxCop project file, then apparently you have to explicitly specify a list of FxCop rules in the RuleLibraries parameter. First, you need to find out which FxCop rules are available. To do that, look in your FxCop installation directory (on my system, where I have an older version of FxCop installed, it was C:\Program Files (x86)\Microsoft FxCop 1.36, but it may be different for you) for a Rules folder. That folder should contain several DLLs; for example, on my system, the Rules folder contained:
DesignRules.dll
GlobalizationRules.dll
InteroperabilityRules.dll
... and several other DLLs that I'm not going to bother typing out. Now you just make that list of filenames into an F# list:
RulesLibraries = ["DesignRules.dll"; "GlobalizationRules.dll"] // and so on
There should be sensible defaults for that, but currently it looks like you have to specify that list of rules by hand. So try writing your target to look something like this:
Target "FxCop" (fun _ ->
!! (buildDir + "/**/*.dll")
++ (buildDir + "/**/*.exe")
|> FxCop (fun p ->
{p with
//Override default parameters
ReportFileName = testDir + "FXCopResults.xml";
RulesLibraries = ["DesignRules.dll"; "GlobalizationRules.dll"] // etc.
ToolPath = "D:/Fake/FAKE-Calculator/tools/FxCop/FxCopCmd.exe"})
)
Remember to separate your list items with ; (semicolon): in F#, the , (comma) character is ONLY for tuples. And don't just copy my example verbatim, but actually look in your FxCop installation directory to see what rule DLLs are available, and include those. (As many, or as few, as your project actually needs).
Also, I don't know if you actually have to specify the .dll extension; you might be able to use ["DesignRules"; "GlobalizationRules"] (and so on). But it's probably just as simple to use the .dll extension and just copy and paste from the filenames.
I haven't tested this myself, so I hope this works for you. If it doesn't, let me know.

FAKE Fsc task is writing build products to wrong directory

I'm just learning F#, and setting up a FAKE build harness for a hello-world-like application. (Though the phrase "Hell world" does occasionally come to mind... :-) I'm using a Mac and emacs (generally trying to avoid GUI IDEs by preference).
After a bit of fiddling about with documentation, here's how I'm invoking the F# compiler via FAKE:
let buildDir = #"./build-app/" // Where application build products go
Target "CompileApp" (fun _ -> // Compile application source code
!! #"src/app/**/*.fs" // Look for F# source files
|> Seq.toList // Convert FileIncludes to string list
|> Fsc (fun p -> // which is what the Fsc task wants
{p with //
FscTarget = Exe //
Platform = AnyCpu //
Output = (buildDir + "hello-fsharp.exe") }) // *** Writing to . instead of buildDir?
) //
That uses !! to make a FileIncludes of all the sources in the usual way, then uses Seq.toList to change that to a string list of filenames, which is then handed off to the Fsc task. Simple enough, and it even seems to work:
...
Starting Target: CompileApp (==> SetVersions)
FSC with args:[|"-o"; "./build-app/hello-fsharp.exe"; "--target:exe"; "--platform:anycpu";
"/Users/sgr/Documents/laboratory/hello-fsharp/src/app/hello-fsharp.fs"|]
Finished Target: CompileApp
...
However, despite what the console output above says, the actual build products go to the top-level directory, not the build directory. The message above looks like the -o argument is being passed to the compiler with an appropriate filename, but the executable gets put in . instead of ./build-app/.
So, 2 questions:
Is this a reasonable way to be invoking the F# compiler in a FAKE build harness?
What am I misunderstanding that is causing the build products to go to the wrong place?
This, or a very similar problem, was reported in FAKE issue #521 and seems to have been fixed in FAKE pull request #601, which see.
Explanation of the Problem
As is apparently well-known to everyone but me, the F# compiler as implemented in FSharp.Compiler.Service has a practice of skipping its first argument. See FSharp.Compiler.Service/tests/service/FscTests.fs around line 127, where we see the following nicely informative comment:
// fsc parser skips the first argument by default;
// perhaps this shouldn't happen in library code.
Whether it should or should not happen, it's what does happen. Since the -o came first in the arguments generated by FscHelper, it was dutifully ignored (along with its argument, apparently). Thus the assembly went to the default place, not the place specified.
Solutions
The temporary workaround was to specify --out:destinationFile in the OtherParams field of the FscParams setter in addition to the Output field; the latter is the sacrificial lamb to be ignored while the former gets the job done.
The longer term solution is to fix the arguments generated by FscHelper to have an extra throwaway argument at the front; then these 2 problems will annihilate in a puff of greasy black smoke. (It's kind of balletic in its beauty, when you think about it.) This is exactly what was just merged into the master by #forki23:
// Always prepend "fsc.exe" since fsc compiler skips the first argument
let optsArr = Array.append [|"fsc.exe"|] optsArr
So that solution should be in the newest version of FAKE (3.11.0).
The answers to my 2 questions are thus:
Yes, this appears to be a reasonable way to invoke the F# compiler.
I didn't misunderstand anything; it was just a bug and a fix is in the pipeline.
More to the point: the actual misunderstanding was that I should have checked the FAKE issues and pull requests to see if anybody else had reported this sort of thing, and that's what I'll do next time.

F#'s "Hello, world" with 2 fs files

I come from C# background to F#. So far I wrote simple programs and spent a lot of time in F# interactive.
I'm stuck creating a VS F# project with two .fs files.
Sample code:
// part 1: functions
let rec gcd (a : uint64) (b : uint64) =
if b = 0UL then a
else gcd b (a % b)
// part 2: main()
let a, b = (13UL, 15UL)
do printfn "gcd of %d %d = %d" a b (gcd a b)
I'd like to have two .fs files, namely, Alg.fs and Program.fs, so that Program.fs would contain the code I'm working and Alg.fs having algorithms.
Taken steps:
I've created the two files. Compiler gave an error: Files in libraries or multiple-file applications must begin with a namespace or module declaration, e.g. 'namespace SomeNamespace.SubNamespace' or 'module SomeNamespace.SomeModule'
I've inserted module Program and module Alg. The complied program executes only the code from Alg.fs completely ignoring Program.fs...
I'm using F# 2.0 in Visual Studio 2010.
P.S. I've googled and checked some posts, and read documentation on modules and saw relative questions before asking.
Sounds like this is an order-of-files-in-the-project issue. The last file is the entry point ("main method"), sounds like you have Alg.fs last, and you need Program.fs last. You can re-order them via the right-click context menu in VS Solution Explorer.
There are at least three separate things that need to be looked at here:
As mentioned by #Brian, the order of source control files is also the compile order. This matters in F# where type inference is heavily used. Make sure Alg.fs comes before Program.fs in your Visual Studio file list (try this: select Program.fs and hit Alt+Down Arrow until it's at the bottom).
Since Alg.fs and Program.fs are now in modules, you need to actually open the Alg module in Program to get access to its bindings (open Alg), or add the [<AutoOpen>] attribute on Alg.
As #Daniel says, the last problem could be the definition of the entry point to the program. You need either an [<EntryPoint>] attribute on a top level binding that is also the last function in the last file. Alternatively, this defaults to the last binding in the last file anyway, just make sure it has the right signature (see Daniel's link).

Why does the Visual Studio 2010 debugger not pick up debug symbols for handling exception in F# sequence?

In Visual Studio 2010, the following F# sequence works as expected in Release mode (ignores UnauthorizedAccessException), but does not work correctly in Debug mode (breaks on UnauthorizedAccessException, even if I have set "Common Language Runtime Exceptions: Thrown = false, User-Unhandled = true").
open System
open System.IO
module private MyTestModule =
let rec private getAllFiles dir = seq {
if String.IsNullOrWhiteSpace dir |> not then
let getAuthorizedItems getItems dir =
try getItems dir
with :? UnauthorizedAccessException -> [||]
// Debugger stops here on UnauthorizedAccessException, but shouldn't...
yield! getAuthorizedItems Directory.GetFiles dir
for subDir in getAuthorizedItems Directory.GetDirectories dir do
yield! getAllFiles subDir }
// etc.
However, if I do not nest the getAuthorizedItems function inside the sequence, but place it at module level instead, the debugger works correctly.
Note:
I have looked at the generated IL code, and the exception handler is where it should be in both cases (is not modified/optimized in any way);
I know that System.Core must be preloaded in order for sequences to be displayed correctly in the debugger, but that is not related to my issue.
Are there any special rules for handling exceptions in sequences at debug time in F#?
Edit
After my reporting the issue, the F# team very quickly started to track it. It seems to be a minor bug related to the fact that, in Debug mode, some of the generated code is marked as ‘External Code’, even though it is user code. For the time being, one can use the workarounds suggested in pad's answer. Another workaround ist to turn off "Enable Just My Code" in the VS debugging options.
I'm able to reproduce the bug on my machine using F# 2.0/.NET 4.0 without VS2010 SP1 installed. It could be the case that this bug has been fixed in VS2010 SP1 as #svick couldn't reproduce it. It turns out this bug is still present after VS2010 SP1 update.
I think it's a bug of handling exception in nested functions inside sequence expression, changing the nested function to catch any exception doesn't affect the behaviour:
let getAuthorizedItems getItems dir =
try getItems dir
with ex -> [||]
This is a minor bug; you can bypass it in many ways:
It works in Release mode, inside fsi and even runs in Debug mode outside Visual Studio.
Declaring the nested function as inline makes it work in Debug mode.
Turning on Optimize code option makes it work again.
Not using nested functions inside seq also helps.
If it hasn't been fixed, I suggest you file a bug report at fsbugs at microsoft dot com.

Resources