Currently, I am scanning java code in a repository using fortify batch file and scanning C/C++ code in the same repository using the command line with the help of visual studio integration.
Is it possible to scan both java and C/C++ code in a repository using a single batch file?
Also, is there command line options to scan both coding languages at once?
Yes, but you probably shouldn't.
One scan (one FPR file) should represent one codebase. Unless you have one application that's part Java and part C/C++, you want to produce 2 separate FPRs, one for Java and one for C/C++.
Instead, take your 2 scan scripts, write another very short script that calls them, and voila, a script that scans both your applications.
If you do have one application with both languages, here's what you do:
Fortify first translates your source code into its intermediate language (NST files), then it scans those NST files. The translation is the sourceanalyzer command where you point it to your code, and the scan step is the sourceanalyzer command with -scan in it. It uses the build ID to keep track of those intermediate files (that's the argument after -b).
To scan the whole codebase together, first translate one set of files, then translate the other set of files (using the same exact build ID), and then do the scan step (same build ID), and it'll scan all of the code together. But only do this if it really is one application.)
Related
I'm taking a glance over at the buildtools repo (https://github.com/bazelbuild/buildtools) and trying to understand the scope of its responsibilities as it relates to the three phases of a bazel build (loading, analysis, execution)
The repo's description states that it is A bazel BUILD file formatter and editor. I find much logic in the repo written in go-lang that lends complete support for an AST parser, starlark syntax interpreting capabilities, reformatting and rewriting of BUILD files and what not. Basically there's logic designed to operate upon a single starlark file at a time. Rereading that repo description in this light leads me to conclude that buildtools is really a single file scoped effort and presents tools that only intersect functionality wise (perhaps only partially) to those loading operations bazel conducts while building.
Question: Is it accurate that the focus of buildtools is upon the single starlark file?
If that's true then all the multiple starlark file analysis logic and so forth seems to actually be maintained over at https://github.com/bazelbuild/bazel/tree/master/src/main/java/com/google/devtools/build/lib and I should not expect to find any tools for the analysis phase and beyond in the buildtools repo. Is that right?
I don't work on Buildtools, but we agree: these tools seem to focus on BUILD / .bzl files in isolation. They let you process these files in parallel, to do similar operations on them.
If you wonder whether these tools understand relations between these files, the answer seems to be no.
If you further wonder what tools do then, the answer is Bazel's query, cquery, and aquery. I'm not aware of a programmable API for these queries though; you have to run Bazel to perform them.
buildtools has tools working on a syntactic level (it looks at the syntax tree). These tools are outside of Bazel and have no knowledge of Bazel build phases. In the future, we may expand the code to work on multiple files (for the static analysis), but it will still be independent from Bazel phases.
https://github.com/bazelbuild/bazel/tree/master/src/main/java/com/google/devtools/build/lib/ is the source code of Bazel. The syntax/ directory includes the code for reading and evaluating the Starlark files. The code there is called by Skyframe. The interpreter is called by Skyframe many times in parallel, both during the loading and the analysis phases.
If you have a more specific question (what are you trying to do?), I can help more. :)
So, at work, I frequently have to create virtually identical ant scripts. Basically the application we provide to our clients is designed to be easily extensible, and we offer a service of designing and creating custom modules for it. Because of the complexity of our application, with lots of cross dependencies, I tend to develop the module within our core dev environment, compile it using IntelliJ, and then run a basic ant script that does the following tasks:
1) Clean build directory
2) Create build directory and directory hierarchy based on package paths.
3) Copy class files (and source files to a separate sources directory).
4) Jar it up.
The thing is, to do this I need to go through the script line by line and change a bunch of property names, so it works for the new use case. I also save all the scripts in case I need to go back to them.
This isn't the worst thing in the world, but I'm always looking for a better way to do things. Hence my idea:
For each specific implementation I would provide an ant script (or other file) of just properties. Key-value pairs, which would have specific prefixes for each key based on what it's used for. I would then want my ant script to run the various tasks, executing each one for the key-value pairs that are appropriate.
For example, copying the class files. I would have a property with a name like "classFile.filePath". I would want the script to call the task for every property it detects that starts with "classFile...".
Honestly, from my current research so far, I'm not confident that this is possible. But... I'm super stubborn, and always looking for new creative options. So, what options do I have? Or are there none?
It's possible to dynamically generate ANT scripts, for example the following does this using an XML input file:
Use pure Ant to search if list of files exists and take action based on condition
Personally I would always try and avoid this level of complexity. Ant is not a programming language.
Looking at what you're trying to achieve it does appear you could benefit from packaging your dependencies as jars and using a Maven repository manager like Nexus or Artifactory for storage. This would simplify each sub-project build. When building projects that depend on these published libraries you can use a dependency management tool like Apache ivy to download them.
Hope that helps your question is fairly broad.
Has anybody discovered any means to fire an ant build process automatically based on file system changes?
I basically want my ant build system to begin building similar to an IDE (compile java classes) but from some sort of command line service.
If not, there's always coding one up with Java and integrating the Ant API into it.
I am familiar with continuous integration systems like Jenkins and the like, however I need the build to be fired not check-in. Also I would like it to be independent of the IDE, as that could work on post-save.
I'm looking for an independent build service without source control requirements.
Since you are using ant I assume a java based directory polling program will help here. You can write a program using IO notification api
Notes from the page
When to Use and Not Use This API
The Watch Service API is designed for applications that need to be
notified about file change events. It is well suited for any
application, like an editor or IDE, that potentially has many open
files and needs to ensure that the files are synchronized with the
file system. It is also well suited for an application server that
watches a directory, perhaps waiting for .jsp or .jar files to drop,
in order to deploy them.
This API is not designed for indexing a hard drive. Most file system
implementations have native support for file change notification. The
Watch Service API takes advantage of this support where available.
However, when a file system does not support this mechanism, the Watch
Service will poll the file system, waiting for events.
Edit
After I wrote that this question and its answer seems to be more useful here: Is there a sophisticated file system monitor for Java which is freeware or open source?
The widely practiced way is a way of "continuous build" / "continuous integration". A sample work-flow:
You check in your code into a source control repository
Continues Integration server picks up changes from the repository and starts a build process
The build process results in either success or failure giving you a fast feedback
Lot's of continuous integration servers (Bamboo, Jenkins, Go) support Ant natively.
You can also set up post-save hooks in your IDE. Most modern ones support it: IntelliJ, NetBeans, Eclipse.
Look up "continuous integration" in google.
A friend pointed me to this:
https://serverfault.com/questions/179706/how-can-i-trigger-a-script-to-run-after-the-rsyncdaemon-received-file-changes-to
A little script to monitor for changes and execute and independent task.
If anyone has a better method that works with ANT scripts directly, let me know.
I like to backup up the source code set for a project when I release a version. I use GExperts project backups, which seems to gather up all the files in the project manager into the ZIP file. You can also add arbitrary files to this file set, but I'm always conscious of the fact that I haven't necessarily got all the files. Unless I specifically go though the uses clauses and add all the units I have sources for to the project, I'll never be sure of storing all the files necessary to recreate the installable/executable.
I've thought about rolling an app to traverse a project, following all the units used and looking down all the search paths and seeing if there is a source file available for that unit, and building a list of files to back up that way, but hey - maybe someone has already done the work?
You should (highly recommend) look into Version Control.
e.g. SVN (subversion), CVS
This will allow you to control revisions of all of your source. It will allow you to add or remove source files, roll back merge and all other nice things related to managing project sources.
This WILL save your a$%# one day.
You can interpret your question in two ways:
How can I make sure that I backup at least enough files so I can build the project
How can I make sure that I backup not too many files so I can still build the project
The first is to make sure you can build the system at all, the second to allow you to clean up unused files.
For both, a version control system including a separate build system is the way to go.
You then - for each new set of changes - can use these steps to assure that both conditions hold:
On your daily development system, check in the new revision of your source code into your version control system.
On your separate build system, get the latest version of your source control system.
Build the project on the build system; if this fails, go to Step 1, and add the missing files to your version control system from your development system
Start removing (one-by-one) files from the project that you suspect are not needed, then rebuild until it fails.
When the build fails, restore that particular file from the version control system, then continue step 3 with the next candidate
When the build succeed you have the minimum set of files.
Now make a difference overview of the files in your version control system, and the build machine.
Mark the files that are in your version control system but not on your build machine as deprecated or deleted.
Most version control systems have good ways of generating a difference between the files on your development or build system against the files in the version control system (usually fine grained for each historic point in time you added/removed/updated files in your version control system).
The reason you want a separate build system (or two separate development systems) is that you want them to be independent: you use one for developing, and the other for checking if the build is still OK.
This is the first step that in the future you might want to extend this into a continuous integration system (that runs unit tests, automatically creates product setups and much more).
--jeroen
I'm not sure if you're asking about version control or how to be sure you've got all the files.
One useful utility I run occasionally is a program that makes a DirList of all of the files in my dcu output folder. Changing the extensions from .dcu to .pas gives me a list of all of the source code files.
Of course it misses .inc files and other non-.pas files, but perhaps this line of thinking would be helpful to you in some way?
The value of this utility to me is that a second housekeeping utility program then makes a list of all .pas files in my source tree that do not have corresponding .dcu files. This (after a full compile of all programs) generally reveals some "junk" .pas files that are no longer in use.
For getting a list of all units compiled into an executable, you could let the compiler generate a MAP file. This file will contain entries for all the units used.
I'm trying to figure out how the DITA Open Toolkit performs DITA to XHTML conversions, and it's difficult since the process is managed by dozens of ant targets spread over multiple ant files.
I need a tool that can provide a visualization of the execution flow plus property dependencies of an ant invocation. VizAnt and Grand only graph target invocations, so I'm looking for something heavier-duty. Ideally, such a tool would identify the order of target invocations, as well as property values live at invocation, and properties, files, directories, classpath entries, etc. that are referenced in the body of a target.
My first thought was to manually graph it all in OmniGraffle, but the complexity quickly became unmanageable. Surely there's something more recent out there?
yWorks Ant Explorer is kind of cool.
Graphical representation of the ANT build targets and dependencies http://www.yworks.com/demos/images/ae1.jpg
Execute code through the GUI explorer of the ANT build file http://www.yworks.com/demos/images/ae3.jpg
Run it by executing the jar file: java -jar antexplorer.jar
It looks like yWorks no longer supports it and have removed it from their website. Links that used to go to Ant Explorer now just take you to their product listing page.
But there appear to be several places that have it available for download:
http://yworks-ant-explorer.software.informer.com/
http://webscripts.softpedia.com/script/Development-Scripts-js/yWorks-Ant-Explorer-29247.html
http://plugins.jetbrains.com/plugin/?idea&id=135