I want to compare 2 FPR files in Fortify to check for new "critical" violations.
What I have got so far is get the count of the critical issues of the 2 FPR files and see if that count has changed.
FPRUtility.bat -project [Your .fpr file] -information -search -query "[fortify priority order]:critical"
Or another idea is to generate a PDF/XML file out of the FPR files and compare or take the diff of 2 files using any Linux command line utility to check for new "critical" issues.
Is there any better way to do this?
Related
I have a batch data parsing job where the inputs is a list of zip files and each zip file has numerous small text files to parse. In the order of 100Gb compressed across 50 zip files, each zip has 1 million text files.
I am using Apache Beam's package in Python and running the job through Dataflow.
I wrote it as
Create collection from the list of zip file paths
FlatMap with a function that yields for every text file inside the zip (one output is a bytes string for all the bytes read from the text file)
ParDo with a method that yields for every row in the data from the text file / bytes read
...do other stuff like insert each row in the relevant table of some database
I notice this is too slow - CPU resources are only a few % utilised. I suspect that each node is getting a zip file, but work is not distributed among local CPUs - so it's just one CPU working per node. I don't understand why that is the case considering I used FlatMap.
The Dataflow runner makes use of Fusion optimisation:
'...Such optimizations can include fusing multiple steps or transforms in your pipeline's execution graph into single steps.'
If you have a transform which in its DoFn has a large fan-out, which I suspect the Create transform in your description does, then you may want to manually break fusion by introducing a shuffle stage to your pipeline as described in the linked documentation.
I am using Jenkins with PMD Plugin v3.45 and Static Analysis Collector plugin 1.48. Everthing is working correctly except that the reports show all warnings as high priority. If I check in the PMD output folder on the server, the reports do have the expected different warning priorities in the html output files correctly as configured in the ruleset.xml.
Furthermore, Jenkins Static Analysis Collector shows warnings as High, Medium, or Low. I am expecting a translation to be something like this:
PMD priority ==> Jenkins Priority
1 ==> High
2 ==> High
3 ==> Normal
4 ==> Normal
5 ==> Low
I have been searching all over where to configure it and cannot find it. Has anyone else faced this problem?
Answering my own question - PMD.xml has a property:
If it is set to 2, only the priority or higher (1 or 2) will be reported.
I'm trying to undelete all .wav files in a work space. I've got code to filter all the wave files that were deleted by using:
cm hist <sound folders> --format={1}
to get all the changesets affecting my sound folders and then filtering through log outputs
cm log cs:<changeset>
for the D deleted file names. Now that I have the names, I intended to use
cm undelete
but it's asking for revid or itemid and cs information I don't have and can't seem to query since the item is deleted.
I'm hoping there is an awesome command line that might allow me to undelete all files of a certain type. If this is possible from plastic PCM or Plastic Gluon, that is an option also.
Thanks!
When running the "cm log" command, you can get the revision ids of the items to undelete. This revid is necessry to undelete the items with the "cm undelete" command.
cm log cs:1 --itemformat="{newline} {revid} {shortstatus} {path}"
I have hundred of files downloaded from Hive, their schema are consistent. I have tried to import one file, which contains 3M nodes, use the v2.2.0's new feature,Neo4j-import tool, to import it. It' pretty fast.
This is my command
Neo4jImport.bat --into ../data/weibo.db --nodes:User "c:\000023_1-hearder.csv,c:\000023_1.csv" --delimiter "TAB"
Adhering to the command usage, if I need to import multiple homogeneous schema files, I should use this:
Neo4jImport.bat --into ../data/weibo.db --nodes:User "c:\1-hearder.csv,c:\1.csv,c:\2.csv,c:\3.csv,c:\4.csv..." --delimiter "TAB"
What if I have hundred or thousand of small-partitioned files to import, should I cascade their file names? Or Is there any error occur?
You can just concat the files if they all have the same structure. And then zip them so they are smaller. That's what I would do.
Otherwise, yes have a file name with incremental number.
You can use a powershell/cmd script with a for loop to concatenate the file names into a one string. Although that might give a problem with the command-line length.
I have several Jenkins matrix projects in where I output benchmark results (i.e. execution times) in a CSV file. I'd like to plot these execution times as a function of the build number, so I can see if my projects are regressing over time.
I can confirm Plot Plugin is a correct and quite useful approach. BTW, it supports CSV as well: plot configuration example
I've been using it for several years without any problem. Benchmarks results were generated as a property file. Benchmark id (series id) was used as a key and result as a value. One build produces one result for each benchmark. Having that data it is quite easy to create plot configuration ant track performance.
This may help you:
https://wiki.jenkins-ci.org/display/JENKINS/Plot+Plugin
It adds plotting capabilities to Jenkins.