I am writing a R package for my Msc's project, together with some .Rmd analysis that uses my package and generates some html reports.
I consider both the package code AND the resulting analysis as deliverables for my project, so I would like to showcase the two in a same location, maybe the same github which is already hosting my package. Question is, can I just add a /reports directory to the package structure and put my html files inside ? As package will be mostly internal use, I don't plan on submitting it to CRAN anytime soon if that matters. However, I would also like to avoid distributing my reports to other potential users.
I've just found out about the .Rbuildignore file which seems like it would do the trick, but I'd be curious to hear what's the standard practice in this case or any other suggestions you may have.
Thanks,
Related
BACKGROUND
One of the very useful tools for reproducible work in R is the "here" library.
https://malco.io/2018/11/05/why-should-i-use-the-here-package-when-i-m-already-using-projects/
http://jenrichmond.rbind.io/post/how-to-use-the-here-package/
https://here.r-lib.org/
https://here.r-lib.org/articles/rmarkdown.html
I was hooked by the part in the first link where they said this:
The "here" library is encoded in Anaconda as "r-here"
I'm not sure which came first, but Python has a "here" library as well.
https://pypi.org/project/pyhere/
https://github.com/wildland-creative/pyhere
"Here" makes relative paths a trivial matter, which is really useful for reproducible data-science and analysis work.
QUESTION
What is the Julia equivalent for clean handling of relative paths for files?
Is there a clean way to integrate that with project packaging, like RStudio does?
Based on the description, it sounds like DrWatson.jl does what you're looking for. From the website:
[DrWatson] is a Julia package created to help people increase the consistency of their scientific projects, navigate them and share them faster and easier, manage scripts, existing simulations as well as project source code. DrWatson helps establishing reproducibility, and in general it makes managing a scientific project a simple job.
Like the description implies, it's more ambitious than here seems to be, having functionality to also manage data, simulation runs, etc. But they're optional, and you can use only the directory handling part if you need.
The Navigating a Project describes the projectdir function which works similar to here. projectdir("foo", "bar") resolves to foo/bar under the current project's root directory, just like with here.
There's also datadir(), srcdir(), and others to directly handle common subdirectories under a project for eg. datadir("foo", "test.jld2") resolves to data/foo/test.jld2 under the project's root directory.
It doesn't exist, as far as I'm aware (Here.jl doesn't return any Google hits), but it seems like it would a simple enough for someone to implement. Maybe you!
I need to #include a file called ClearCore.h but it refuses to show up as a dependency.
https://i.stack.imgur.com/1kTvS.jpg
https://i.stack.imgur.com/dn6XT.jpg
As you can see, the file is in the library and most other files are listed as dependencies, but this one refuses to despite being #included.
Here is the error message from compilation.
https://i.stack.imgur.com/tM2wk.jpg
I am an applications engineer at Teknic and I saw your post and wanted to offer a few points to help you get your code running.
It sounds like you are having trouble adding the ClearCore library as a dependency. As this is more a function of ATMEL, there could be several reasons as to why we are running into issues here.
The easiest way to troubleshoot this would be to use one of Teknic’s already linked and ready to run example projects (we include these examples in addition to the ClearCore libraries). You can use the provided example projects as a template to sort out whatever may be causing the dependency issues.
You can find these examples here: https://teknic-inc.github.io/ClearCore-library/SdkExamples.html
Keep in mind that if you move the example projects into other directories, some of the relative file path definitions may be broken.
If you have any questions about the examples projects please feel free to give us a call at 585-784-7454, or use our "Contact Us" form online at https://www.teknic.com/contact/.
Best regards,
Mark D. – Teknic Servo Systems Engineer
I have a large application to manage consisting of of three or four executables and as many as fifty .dlls. Many of the source code files are shared across many of the projects.
The problem is a familiar one to many of us - if I change some source code I want to be able to identify which of the binaries will change and, therefore, what it is appropriate to retest.
A simple approach would be simply to compare file sizes. That is an 80% acceptable solution, but there is at least a theoretical possibility of missing something. Secondly, it gives me very little indication as to WHAT has changed; It would be ideal to get some form of report on this so I can then filter out irrelevant (e.g. dates/versions copyrights etc..)
On the plus side :
all my .dcus are in a row - I mean they are all built into a single folder
the build is controlled by a script (.bat)(easy, for example, to emit .obj files if that helps)
svn makes it easy to collect together any (two) revisions for comparison
On the minus side
There is no policy to include all used units in all projects; some units get included because they are on a search path.
Just knowing that a changed unit is used/compiled by a project is not sufficient proof that the binary is affected.
Before I begin writing some code to solve the problem I would like to ask the panel what suggestions they might have as to how to approach this.
The rules of StackOverflow forbid me to ask for recommended software, but if anyone has any positive experiences of continuous integration tools that would help - great
I am open to any suggestion or observation that is relevant in this context.
It seems to me that your question boils down to knowing which units are contained in your various executables. Since you are using search paths, it will be hard for you to work this out ahead of time. The most robust way to find out is to consult the .map file that the compiler emits. This contains a list of all units contained in your executable.
Once you know which units are contained in each executable, you need to know whether or not anything has changed in those units. That information is contained in your revision control system. Put this all together and you have the information that you need.
Of course, just because the source code for a unit has changed, you might argue that re-testing is not needed. Perhaps the only change made was the version, or the date in a copyright label or some such. But it is asking too much to be able to ask a computer to make such a judgement. At some point you need a human to step up and take responsibility.
What is odd about this though is that you are asking the question at all. It seems to me to be enormously risky to attempt partial testing. I cannot understand why you don't simply retest the entire product.
After using it for > 10 years for commercial in-house and freelancer work in large projects, I can recommend to try Apache Ant. It is a build tool which supports dependencies, and has many very helpful features.
Apache Ant also integrates nicely with CI tools such as Hudson/Jenkins, Bamboo etc.
Another suggestion - based on experience with Maven - is to design the general software architecture as modular as possible. If modules (single or multiple source or DCU files in one directory) use a version number in the directory name as a version number, it is possible to control exactly how application are composed from these modules.
If you want to program such a tool yourself the approach would be something like this:
First you need to detect wheter there were any changes made to seperate source files. As you already figured out comparing the file size is bad idea as the file size can stay the same despite lots of changes made to it (as long as there is same amount of text in pas file its size won't change). So instead you could check the last modification time for specific file or create some hash value like MD5 hash for comparison (can be quite slow).
Then you need to generate yourself a dependancy tree which will tell you which files are used for which project/subproject.
Finally based on changes detected in seperate files you check the dependancy tree to see which projects needs to be recompiled.
The problem of such approach is that you would probably have to update the dependancy tree manually each time when new unit is added to the project or an existing one is removed from the project.
But the best way would be to go and use some version controll software istead of reinventing the wheel. I myself like the way how GIT works and I belive that with proper implementation of GIT into the project mannager itself could be quite powerfull do to GIT support of branching/subbranching (each project is its own branch, each version of your software can be its own subbranch).
Now latest version of Delphi does have GIT integration done though SVN but this unfortunately limits some of best GIT functionality. So if you maybe decide to go and integrate GIT support directly into Delphi I'm first in line to use it.
We outsourced the development of Blackberry 5, 6, and 7 apps. Please bare in mind that I have absolutely no knowledge of Blackberry development at all.
Development is complete, and they have sent us the source code - a collection of .cod, .csl, .cso, .debug, .jad and .rapc files.
I would at least like to review the code in terms of it's consistency and standards - somewhat a measure of the quality. Clean code is not something specific to any one platform.
I have tried to open each of these files in notepad, but with no source code found.
Please advise me on what I need before I go pay them a visit.
The files you have been given are the files that are created as part of the build of your project and the resultant executable files. There is no source included here.
In a BB OS Build, regardless of the development environment used, the Java source files will all have the suffix .java, and the assets (images etc.) will have a suffix appropriate to the asset (like .png). If you don't see these file suffixes, then you have not been given the actual source. You should be able to view the java files using Notepad, the other files will open in an appropriate application (like paint).
To get the complete source, you should just ask the full 'project' files for your development. This will be a directory with a number of sub directories. The actual names used and the structure will depend on the development tool. If your developer is using Eclipse, then the two important directories are called src for source and res (resources) for the assets. If they are using another development environment, then the directories might have different names. So you should ask them what development environment they are using too.
Two other points:
1) If you are paying for this development and wish to review the code, but are not familiar with Java, then I would recommend that you pay someone to review the code who has knowledge of BB Java. There are two reasons for this:
(a) you will not be able to form a judgement on the appropriateness of the code without some understanding of Java, and
(b) you will not understand if the correct BB Java approaches have been used.
You need to be cautious about this, because programmers will always find fault in other developer's code. The question is how significant the faults are.
2) Some developers might be wary of giving source to their client while some payment is outstanding.
We create via "Tools | Options | Environment Variables" Variables like that:
$(Sources) = D:\Sources\Delphi
$(OurLib) = $(Sources)\OurLib\Src
$(OurApp1) = $(Sources)\Applications\App1\3.x
$(ThirdParty) = $(Sources)\ThirdPartyComponents
We use these Variables in the project search path like that:
($OurApp1)\Src\Core;($OurApp1)\Src\GUI;($OurApp1)\Src\Plugins;$(ThirdParty)\JVCL
But this is broken (meanwhile fixed) since Delphi 2009 as these variables are not evaluated completely anymore (see QC #73276). So the files in the directories are not found by the compiler. A workaround: Use only complete directories in the environment variables.
We use this approach because on all developer machines and the build servers the files can be found and we only have to point $(Sources) to the right place.
We don't have anything in our global library path (except the Delphi defaults), because that wouldn't be in the version control and isn't reflected on other developers or build machines.
One problem is: If one unit in $(OurLib) decides to include another new unit maybe in a new path, all projects break because they don't find this new unit. Then we have to go through all projects and add the search path. (BTW: I really hate the search path editor...wouldn't be a simple memo field much better to edit than this replace/add/delete logic?)
Another thing we do is not adding many units to our project. Especially everything from $(OurLib), but we often have units like plugins which add functionality only by including them. For different editions of our products, we want to include different units. As Delphi always messes up $IFDEFs in the uses clause in the .dpr we help us by including units named like "IncludePlugins" which then include the units depending on IFDEFs.
But not including units in the project makes navigating to a pain. The units don't appear in the project, they are not found by Ctrl+12 (Show Units), they are not shown in code completion etc.
Has anybody a better way to cope with these problems?
We use only relative paths, any libraries are always below the libs subdirectory while the project source code resides in the src subdir. So our search paths always look like:
..\libs\library1;..\libs\library2\common;
etc.
All libraries are added as svn:external to each project, so checking out the project will automatically check out the libraries as well and the search path will always point to the correct version of the library for that project.
Not perfect, but it works most of the time.
I have to agree about the search path editor, it is even worse for relative paths because you must not use the "..." buttons otherwise Delphi will insert an absolute path.
We use standard drive mappings.
Our current project is always on W: regardless if it is a network drive or a substitute.
This works great.
When you need to work on a different project, swap the W: and you can continue.
You can copy the search path out to an editor, modify it and then copy it back.
Your search path is much too big. It should contain only the things you want Delphi to recompile with your project. You don't really want to recompile the Jedi VCL every day, do you?
I create a single directory where all compiled units go. Say, C:\dcu. Specify that as the "unit output directory" in all packages. My "search path," then, is always just this:
$(Delphi)\Lib;C:\dcu
The compiler finds everything it needs, and it never finds any source code. The only source code it ever sees is in the files that directly belong to whatever project I'm compiling. The project's own source directories don't need to be on the search path because all of those files are already direct members of the project. The compiler knows exactly where they are.
For me, all a project's source files go in a single directory. If you want separate directories for different parts, like Core and GUI, then I would put those in separate packages so I could work on them and compile them separately. Even if the final program doesn't use the resultant BPLs, packages are still a good way of segmenting your project and defining dependencies.
When compiling units for one project doesn't automatically compile units for all the other projects, you're forced to change active projects. It takes a moment of your time, but it also serves as a mental reminder that you're "changing hats," too.
Although you're producing just one product, that doesn't mean you should have just one project in Delphi. You should have at least one project for each executable module (EXE, DLL, BPL) in your product. Use project groups to manage multiple projects in a single IDE session. No unit should be a member of more than one project.
I don't understand your part about plug-ins and different editions of your project. When you say "plug-in," I assume you're talking about separate executable modules, like DLLs or packages, that the customer can choose to include or not. Couldn't you turn your different editions' features into plug-in modules that simply don't include in the lesser editions? Then you don't have to worry about conditional compilation of your project; just have several different installer packagers that grab different sets of plug-ins.
I have always found it odd that this has never been addressed adequately. I suggested recently to David I that Delphi should allow the user to set up some sort of preferred development structure and that third party library publishers could be made aware of this so that they could automatically adjust their installers to install correctly in the preferred development framework. If the preferred development structure was stored in an XML file or similar, then, it could be copied from one computer to another on a development team.
As an alternative, it could make an interesting project to create a Delphi application that would allow a user to "refactor" their library installation in a high level way. You specify which folders on your system contain source or compiled components or whatever and where you want to keep source files or compiled units, hit Go and your system gets rearranged for you, while updating your Delphi environment so that when you start Delphi, it finds everything it should.
I've just recently discovered a way to have project specific environment variables in delphi builds using XE6, it's not quite as good as a full blown #define like in C but at least I can now have consistent search paths across multiple projects and create some shared option sets.
What I've done is setup environment variables in the same manner as the original poster and then override them in the dproj or optionset.
The BuildPaths.optset added to the project looks like
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<SVN_Root>..\..\..</SVN_Root>
<SVN_Riemann>$(SVN_Root)\Riemann</SVN_Riemann>
<SVN_Library>$(SVN_Root)\Library</SVN_Library>
<SVN_ThirdParty>$(SVN_Library)\Third Party</SVN_ThirdParty>
</PropertyGroup>
<ProjectExtensions>
<Borland.Personality>Delphi.Personality.12</Borland.Personality>
<Borland.ProjectType>OptionSet</Borland.ProjectType>
<BorlandProject>
<Delphi.Personality/>
</BorlandProject>
<ProjectFileVersion>12</ProjectFileVersion>
</ProjectExtensions>
</Project>