I started using lcov about a month back. The coverage count seems inconsistent. The first run reported about 75% line coverage where as second run reported only 19%. The test suite used was some for both the runs. I see following warning during lcov --remove. Any suggestions?
lcov: WARNING: negative counts found in tracefile all.info
Is this something to worry about?
Same known issue is reported here on GitHub.
Replacing all counts of -1 in the output with 0 (e.g. with sed -i -e 's/,-1$/,0/g' <outputfile>) causes the warning to disappear from the lcov and genhtml output while still producing the correct coverage report.
More importantly (at least for me), submitting the file with the counts set to 0 instead of -1 to codecov.io results in the results being parsed correctly and the coverage information being available through codecov.io.
Codecov also handle this kind of value error:
# Fix negative counts
$count = $2 < 0 ? 0 : $2;
if ($2 < 0)
{
$negative = 1;
}
Follow some other fixes:
Fix Undocumented Value
Remove fix for negative coverage counts
See this bug report : https://github.com/psycofdj/coverxygen/issues/6
Replacing all counts of -1 in the output with 0 (e.g. with sed -i -e 's/,-1$/,0/g' ) causes the warning to disappear from the lcov and genhtml output while still producing the correct coverage report.
More importantly (at least for me), submitting the file with the counts set to 0 instead of -1 to codecov.io results in the results being parsed correctly and the coverage information being available through codecov.io.
Related
I am using coverage.py tool to get coverage of python code. If I use the command without --branch flag like below,
coverage run test_cmd
I get the coverage report like this,
Name Stmts Miss Cover
--------------------------------------------------------------------------------------------------------------------------
/path/file.py 9 2 78%
From this I understand that the cover percentage value is derived as this
cover = (Stmts Covered/total stmts)*100 = (9-2/9)*100 = 77.77%
But when I run coverage with --branch flag enabled like this
coverage run --branch test_cmd
I get the coverage report like this,
Name Stmts Miss Branch BrPart Cover
----------------------------------------------------------------------------------------------------------------------------------------
/path/file.py 9 2 2 1 73%
From this report I am not able to understand the formula used to get Cover=73%
How is this number coming and is this correct value for code coverage?
While you probably shouldn't worry too much about the exact numbers, here is how they are calculated:
Reading from the output you gave, we have:
n_statements = 9
n_missing = 2
n_branches = 2
n_missing_branches = 1
Then it is calculated (I simplified the syntax a bit, in the real code everything is hidden behind properties.)
https://github.com/nedbat/coveragepy/blob/a9d582a47b41068d2a08ecf6c46bb10218cb5eec/coverage/results.py#L205-L207
n_executed = n_statements - n_missing
https://github.com/nedbat/coveragepy/blob/8a5049e0fe6b717396f2af14b38d67555f9c51ba/coverage/results.py#L210-L212
n_executed_branches = n_branches - n_missing_branches
https://github.com/nedbat/coveragepy/blob/8a5049e0fe6b717396f2af14b38d67555f9c51ba/coverage/results.py#L259-L263
numerator = n_executed + n_executed_branches
denominator = n_statements + n_branches
https://github.com/nedbat/coveragepy/blob/master/coverage/results.py#L215-L222
pc_covered = (100.0 * numerator) / denominator
Now put all of this in a script, add print(pc_covered) and run it:
72.72727272727273
It is then rounded of course, so there you have your 73%.
There is something very strange happening with some rosbags I have.
These rosbags contain messages of type sensor_msgs/Image among other topics.
So I do:
First scenario
On one terminal I run rostopic echo /the_image/header because I am not interested in the actual data, just the header info.
In another terminal I run rosbag play --clock the_bag.bag
With this I get
seq: 7814
stamp:
secs: 1625151029
nsecs: 882629359
frame_id: ''
---
seq: 7815
stamp:
secs: 1625151029
nsecs: 934761166
frame_id: ''
---
seq: 7816
stamp:
secs: 1625151029
nsecs: 986241550
frame_id: ''
---
seq: 7817
stamp:
secs: 1625151030
nsecs: 82884301
frame_id: ''
---
Second Scenario
I do the same as the previous scenario but instead of rosbag play I run rqt_bag the_bag.bag and once there I right click the message to publish them.
With that I get similar values but (I have reported the problem before) the first messages are skipped. (this is not the problem of this question)
Third Scenario
Here comes the weird part. Instead of doing as above I have a python script that does
timestamps=[]
for topic, msg, t in rosbag.Bag("the_bag.bag").read_messages():
if topic == '/the_image':
timestamps.append((image_idx, t))
with open("timestamps.txt",'w') as f:
for idx, t in timestamps:
f.write('{0},{1},{2}\n'.format(str(idx).zfill(8), t.secs, t.nsecs))
So as you can see I open the bag and get a list of timestamps and record it in a text file.
Which gives:
00000000,1625151029,987577614
00000001,1625151030,33818541
00000002,1625151030,88932237
00000003,1625151030,170311084
00000004,1625151030,232427083
00000005,1625151030,279726253
00000006,1625151030,363255375
00000007,1625151030,463079346
00000008,1625151030,501315763
00000009,1625151030,566104245
00000010,1625151030,586694806
As you can see the values are totally different!!!
What could be happening here?
This is a known "issue" with rostopic echo and bag files. I put issue in quotes because it's not necessarily an issue, but just a product of how rostopic works. To spare the somewhat obscure details of the implementation, this problem essentially happens because rospy.rostime does not get initialized correctly when just playing a bag file and echoing that; even if you set /use_sim_time to true.
To give some clarity on what you're seeing, the timestamps coming off your Python script are correct and the rostopic ones are not. If you need the timestamps to be 100% correct with rostopic you can use the -b flag like: rostoic echo -b the_bag.bag /my_image_topic
I am using a shell script to extract the data from 'extr' table. The extr table is a very big table having 410 columns. The table has 61047 rows of data. The size of one record is around 5KB.
I the script is as follows:
#!/usr/bin/ksh
sqlplus -s \/ << rbb
set pages 0
set head on
set feed off
set num 20
set linesize 32767
set colsep |
set trimspool on
spool extr.csv
select * from extr;
/
spool off
rbb
#-------- END ---------
One fine day the extr.csv file was having 2 records with incorrect number of columns (i.e. one record with more number of columns and other with less). Upon investigation I came to know that the two duplicate records were repeated in the file. The primary key of the records should ideally be unique in file but in this case 2 records were repeated. Also, the shift in the columns was abrupt.
Small example of the output file:
5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|200|F
5003|A3A|AAB|153.33|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|258|G
5006|A6A|ABB|147.89|154|H
5003|A7A|AAB|249.67|AAB|153.33|205|R
5004|A8A|269|F
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|215|F
Here the primary key records for 5003 and 5004 have reappeared in place of 5007 and 5008. Also the duplicate reciords have shifted the records of 5007 and 5008 by appending/cutting down their columns.
Need your help in analysing why this happened? Why the 2 rows were extracted multiple times? Why the other 2 rows were missing from the file? and Why the records were shifted?
Note: This script is working fine since last two years and has never failed except for one time (mentioned above). It ran successfully during next run. Recently we have added one more program which accesses the extr table with cursor (select only).
I reproduced a similar behaviour.
;-> cat input
5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|200|F
5003|A3A|AAB|153.33|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|258|G
5006|A6A|ABB|147.89|154|H
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|215|F
See the input file as your database.
Now I write a script that accesses "the database" and show some random freezes.
;-> cat writeout.sh
# Start this script twice
while IFS=\| read a b c d e f; do
# I think you need \c for skipping \n, but I do it different one time
echo "$a|$b|$c|$d|" | tr -d "\n"
(( sleeptime = RANDOM % 5 ))
sleep ${sleeptime}
echo "$e|$f"
done < input >> output
EDIT: Removed cat input | in script above, replaced by < input
Start this script twice in the background
;-> ./writeout.sh &
;-> ./writeout.sh &
Wait until both jobs are finished and see the result
;-> cat output
5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|200|F
5003|A3A|AAB|153.33|5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|200|F
5003|A3A|AAB|153.33|258|G
5006|A6A|ABB|147.89|154|H
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|258|G
5006|A6A|ABB|147.89|215|F
154|H
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|215|F
When I edit the last line of writeout.sh into done > output I do not see the problem, but that might be due to buffering and the small amount of data.
I still don't know exactly what happened in your case, but it really seems like 2 progs writing simultaneously to the same script.
A job in TWS could have been restarted manually, 2 scripts in your masterscript might write to the same file or something else.
Preventing this in the future can be done using some locking / checks (when the output file exists, quit and return errorcode to TWS).
I have tried to work with giza++ on window (using Cygwin compiler).
I used this code:
//Suppose source language is French and target language is English
plain2snt.out FrenchCorpus.f EnglishCorpus.e
mkcls -c30 -n20 -pFrenchCorpus.f -VFrenchCorpus.f.vcb.classes opt
mkcls -c30 -n20 -pEnglishCorpus.e -VEnglishCorpus.e.vcb.classes opt
snt2cooc.out FrenchCorpus.f.vcb EnglishCorpus.e.vcb FrenchCorpus.f_EnglishCorpus.e.snt >courpuscooc.cooc
GIZA++ -S FrenchCorpus.f.vcb -T EnglishCorpus.e.vcb -C FrenchCorpus.f_EnglishCorpus.e.snt -m1 100 -m2 30 -mh 30 -m3 30 -m4 30 -m5 30 -p1 o.95 -CoocurrenceFile courpuscooc.cooc -o dictionary
But the after getting the output files from giza++ and evaluate output, I observed that the results were too bad.
My evaluation result was:
RECALL = 0.0889
PRECISION = 0.0990
F_MEASURE = 0.0937
AER = 0.9035
Dose any body know the reason? Could the reason be that I have forgotten some parameters or I should change some of them?
in other word:
first I wanted train giza++ by huge amount of data and then test it by small corpus and compare its result by desired alignment(GOLD STANDARD) , but I don't find any document or useful page in web.
can you introduce useful document?
Therefore I ran it by small courpus (447 sentence) and compared result by desired alignment.do you think this is right way?
Also I changed my code as follows and got better result but It's still not good:
GIZA++ -S testlowsf.f.vcb -T testlowde.e.vcb -C testlowsf.f_testlowde.e.snt -m1 5 -m2 0 -mh 5 -m3 5 -m4 0 -CoocurrenceFile inputcooc.cooc -o dictionary -model1dumpfrequency 1 -model4smoothfactor 0.4 -nodumps 0 -nsmooth 4 -onlyaldumps 1 -p0 0.999 -diagonal yes -final yes
result of evaluation :
// suppose A is result of GIZA++ and G is Gold standard. As and Gs is S link in A And G files. Ap and Gp is p link in A and G files.
RECALL = As intersect Gs/Gs = 0.6295
PRECISION = Ap intersect Gp/A = 0.1090
FMEASURE = (2*PRECISION*RECALL)/(RECALL + PRECISION) = 0.1859
AER = 1 - ((As intersect Gs + Ap intersect Gp)/(A + S)) = 0.7425
Do you know the reason?
Where did you get those parameters? 100 iterations of model1?! Well, if you actually manage to run this, I strongly suspect that you have a very small parallel corpus. If so, you should consider adding more parallel data in training. And how exactly do you calculate the recall and precision measures?
EDIT:
With less than 500 sentences you're unlikely to get any reasonable performance. The usual way to do it is not find a larger (unaligned) parallel corpus, run GIZA++ on both together and then evaluate the small part for which you have the manual alignments. Check Europarl or MultiUN, these are freely available corpora, both contain a relatively large amount of English-French parallel data. The instructions on preparing the data can be found on the websites.
The "W" column on the Jenkins dashboard shows stormy for all of my PHP projects due to the contributing line
Clover Coverage: Conditionals 0% (0/0)
because PHP_CodeCoverage doesn't measure conditionals. How can I make Jenkins ignore this measurement for these projects? I have tried setting <conditionalCoverage> to 0 and -1 with no effect (yes, I remembered to reload the configuration).
<hudson.plugins.clover.CloverPublisher>
<cloverReportDir>build/logs</cloverReportDir>
<cloverReportFileName>clover.xml</cloverReportFileName>
<healthyTarget>
<methodCoverage>70</methodCoverage>
<conditionalCoverage>-1</conditionalCoverage> <!-- tried 0 too -->
<statementCoverage>80</statementCoverage>
</healthyTarget>
<unhealthyTarget/>
<failingTarget/>
</hudson.plugins.clover.CloverPublisher>
Using -1 does work, but you have to rebuild each project. The entries in the summary hover are generated as part of the build and do not change.