code coverage report for ofbiz run-tests call

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

code coverage report for ofbiz run-tests call

Adam Heath-2
I've thrown some things together this week, to implement code coverage
anaylsis for ofbiz.  The code itself is not tied to any particular
code coverage tool(there's a plugin system for it).  Currently I have
one written for cobertura, but that's gpl, so not compatible with apache.

The first run I just finished had some test failures.  I haven't
looked to see if this is because of the coverage plugin, or some other
reason.

Anyways, without further ado, here is the output:

http://www.brainfood.com/ofbiz-coverage/

Things to note are the entity, service, and minilang components.
Those have so-so coverage.

However, just about every other component has no testing whatsoever.
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Tim Ruppert
Thanks Adam - this is really interesting information and good to see that everyone is starting to push the automated testing button!

Cheers,
Ruppert
--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com

o:801.649.6594
f:801.649.6595

On Dec 9, 2009, at 6:43 PM, Adam Heath wrote:

> I've thrown some things together this week, to implement code coverage
> anaylsis for ofbiz.  The code itself is not tied to any particular
> code coverage tool(there's a plugin system for it).  Currently I have
> one written for cobertura, but that's gpl, so not compatible with apache.
>
> The first run I just finished had some test failures.  I haven't
> looked to see if this is because of the coverage plugin, or some other
> reason.
>
> Anyways, without further ado, here is the output:
>
> http://www.brainfood.com/ofbiz-coverage/
>
> Things to note are the entity, service, and minilang components.
> Those have so-so coverage.
>
> However, just about every other component has no testing whatsoever.


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Scott Gray-2
In reply to this post by Adam Heath-2
Hi Adam,

Looking at the results my first impression is that the coverage is  
under-reported.  For example, the accounting component has quite a few  
tests but no coverage is shown at all (except for the test package  
itself).  Possibly because there is lot of logic in simple methods but  
I'm 100% sure java code is also run during the tests.

But still a great start and something that will be immensely useful if  
we can up the accuracy a bit.

Thanks
Scott

HotWax Media
http://www.hotwaxmedia.com

On 10/12/2009, at 2:43 PM, Adam Heath wrote:

> I've thrown some things together this week, to implement code coverage
> anaylsis for ofbiz.  The code itself is not tied to any particular
> code coverage tool(there's a plugin system for it).  Currently I have
> one written for cobertura, but that's gpl, so not compatible with  
> apache.
>
> The first run I just finished had some test failures.  I haven't
> looked to see if this is because of the coverage plugin, or some other
> reason.
>
> Anyways, without further ado, here is the output:
>
> http://www.brainfood.com/ofbiz-coverage/
>
> Things to note are the entity, service, and minilang components.
> Those have so-so coverage.
>
> However, just about every other component has no testing whatsoever.


smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Erwan de FERRIERES-3
Hi Adam and Scott,

 From what we see, it seems that only the junit tests are executed.
Cobertura or any other code-coverage tool does not know how to test /use
simple methods, and are only testing java code.

 From what I know, it is the same for all coverage tools, so it would be
necessary to develop our tool or plugin for OFBiz simple method ...

I was planning on doing the same on our CI server,
http://selenium.ofbiz.fr/, along with the junit and selenium results.

I'm not such a technical guy, but I'm ready to test/comment/help on any
dev you'd like to plan to execute.

cheers,

Le 10/12/2009 03:16, Scott Gray a écrit :

> Hi Adam,
>
> Looking at the results my first impression is that the coverage is
> under-reported. For example, the accounting component has quite a few
> tests but no coverage is shown at all (except for the test package
> itself). Possibly because there is lot of logic in simple methods but
> I'm 100% sure java code is also run during the tests.
>
> But still a great start and something that will be immensely useful if
> we can up the accuracy a bit.
>
> Thanks
> Scott
>
> HotWax Media
> http://www.hotwaxmedia.com
>
> On 10/12/2009, at 2:43 PM, Adam Heath wrote:
>
>> I've thrown some things together this week, to implement code coverage
>> anaylsis for ofbiz. The code itself is not tied to any particular
>> code coverage tool(there's a plugin system for it). Currently I have
>> one written for cobertura, but that's gpl, so not compatible with apache.
>>
>> The first run I just finished had some test failures. I haven't
>> looked to see if this is because of the coverage plugin, or some other
>> reason.
>>
>> Anyways, without further ado, here is the output:
>>
>> http://www.brainfood.com/ofbiz-coverage/
>>
>> Things to note are the entity, service, and minilang components.
>> Those have so-so coverage.
>>
>> However, just about every other component has no testing whatsoever.
>

--
Erwan de FERRIERES
www.nereide.biz
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adrian Crum
In reply to this post by Adam Heath-2
I am working on unit tests for the framework/base component. I hope to
have them committed soon.

Thanks for the info!

-Adrian

Adam Heath wrote:

> I've thrown some things together this week, to implement code coverage
> anaylsis for ofbiz.  The code itself is not tied to any particular
> code coverage tool(there's a plugin system for it).  Currently I have
> one written for cobertura, but that's gpl, so not compatible with apache.
>
> The first run I just finished had some test failures.  I haven't
> looked to see if this is because of the coverage plugin, or some other
> reason.
>
> Anyways, without further ado, here is the output:
>
> http://www.brainfood.com/ofbiz-coverage/
>
> Things to note are the entity, service, and minilang components.
> Those have so-so coverage.
>
> However, just about every other component has no testing whatsoever.
>
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adam Heath-2
In reply to this post by Scott Gray-2
Scott Gray wrote:

> Hi Adam,
>
> Looking at the results my first impression is that the coverage is
> under-reported.  For example, the accounting component has quite a few
> tests but no coverage is shown at all (except for the test package
> itself).  Possibly because there is lot of logic in simple methods but
> I'm 100% sure java code is also run during the tests.
>
> But still a great start and something that will be immensely useful if
> we can up the accuracy a bit.

Well, it doesn't, really.  If you click thru to accounting.test,
you'll see that there aren't really that many tests.  And, upon
further investigation, the lines after the runSync calls aren't run,
due to some exception most likely.  I'm not certian if this is do to
my changes, or if the tests themselves are broken.  I'm running a
plain test run now to check that.  Plus, there actually *is* line hits
 in accounting.invoice.

Additionally, the way the classloaders are constructed is a bit odd.
The bootstrap loader only has ofbiz.jar.  The code then creates a
classloader will *all* jars in base/lib, recursively.  After the
containers and components are started, yet another classloader is
created, this time with proper libs and all other ofbiz-$foo.jar.
Both of those classloaders update the java.class.path system property.
 This is a bug, as the instrumentation changes all ofbiz-$foo.jar into
a new jar in /tmp, so the value of the property is different each
time.  Additionally, the second created classloader should not load
the base component again, as that is sub-optimal.

I'm thinking that component code should really move to start.

Another bug, is that due to one of the above bugs, the first
classloader is not instrumented at all.  Since this is the classloader
that contains framework/base, you don't see any line hits in the report.

Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Scott Gray-2
On 11/12/2009, at 6:41 AM, Adam Heath wrote:

> Scott Gray wrote:
>> Hi Adam,
>>
>> Looking at the results my first impression is that the coverage is
>> under-reported.  For example, the accounting component has quite a  
>> few
>> tests but no coverage is shown at all (except for the test package
>> itself).  Possibly because there is lot of logic in simple methods  
>> but
>> I'm 100% sure java code is also run during the tests.
>>
>> But still a great start and something that will be immensely useful  
>> if
>> we can up the accuracy a bit.
>
> Well, it doesn't, really.  If you click thru to accounting.test,
> you'll see that there aren't really that many tests.  And, upon
> further investigation, the lines after the runSync calls aren't run,
> due to some exception most likely.  I'm not certian if this is do to
> my changes, or if the tests themselves are broken.  I'm running a
> plain test run now to check that.  Plus, there actually *is* line hits
> in accounting.invoice.
The tests seem to be running fine on buildbot (http://ci.apache.org/waterfall?show=ofbiz-trunk 
), I'm guessing it's the test run problem that's causing the under  
reporting.  There may not be that many explicit accounting tests (even  
though it is a lot compared to other components) but a lot of tests  
also touch accounting indirectly.  There is just no way that only 53  
lines of java code are being executed in accounting during the full  
test run.  I know for a fact that code is executed from  
PaymentGatewayServices, FinAccountPaymentServices, PaymentWorker,  
UtilAccounting and a few others during the tests.

Regards
Scott

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Jacques Le Roux
Administrator
From: "Scott Gray" <[hidden email]>

> On 11/12/2009, at 6:41 AM, Adam Heath wrote:
>
>> Scott Gray wrote:
>>> Hi Adam,
>>>
>>> Looking at the results my first impression is that the coverage is
>>> under-reported.  For example, the accounting component has quite a  
>>> few
>>> tests but no coverage is shown at all (except for the test package
>>> itself).  Possibly because there is lot of logic in simple methods  
>>> but
>>> I'm 100% sure java code is also run during the tests.
>>>
>>> But still a great start and something that will be immensely useful  
>>> if
>>> we can up the accuracy a bit.
>>
>> Well, it doesn't, really.  If you click thru to accounting.test,
>> you'll see that there aren't really that many tests.  And, upon
>> further investigation, the lines after the runSync calls aren't run,
>> due to some exception most likely.  I'm not certian if this is do to
>> my changes, or if the tests themselves are broken.  I'm running a
>> plain test run now to check that.  Plus, there actually *is* line hits
>> in accounting.invoice.
>
> The tests seem to be running fine on buildbot (http://ci.apache.org/waterfall?show=ofbiz-trunk 
> ), I'm guessing it's the test run problem that's causing the under  
> reporting.  There may not be that many explicit accounting tests (even  
> though it is a lot compared to other components) but a lot of tests  
> also touch accounting indirectly.  There is just no way that only 53  
> lines of java code are being executed in accounting during the full  
> test run.  I know for a fact that code is executed from  
> PaymentGatewayServices, FinAccountPaymentServices, PaymentWorker,  
> UtilAccounting and a few others during the tests.
>
> Regards
> Scott

At least, I love to see all this green!

Jacques


Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adam Heath-2
In reply to this post by Scott Gray-2
Scott Gray wrote:

> On 11/12/2009, at 6:41 AM, Adam Heath wrote:
>
>> Scott Gray wrote:
>>> Hi Adam,
>>>
>>> Looking at the results my first impression is that the coverage is
>>> under-reported.  For example, the accounting component has quite a few
>>> tests but no coverage is shown at all (except for the test package
>>> itself).  Possibly because there is lot of logic in simple methods but
>>> I'm 100% sure java code is also run during the tests.
>>>
>>> But still a great start and something that will be immensely useful if
>>> we can up the accuracy a bit.
>>
>> Well, it doesn't, really.  If you click thru to accounting.test,
>> you'll see that there aren't really that many tests.  And, upon
>> further investigation, the lines after the runSync calls aren't run,
>> due to some exception most likely.  I'm not certian if this is do to
>> my changes, or if the tests themselves are broken.  I'm running a
>> plain test run now to check that.  Plus, there actually *is* line hits
>> in accounting.invoice.
>
> The tests seem to be running fine on buildbot
> (http://ci.apache.org/waterfall?show=ofbiz-trunk), I'm guessing it's the
> test run problem that's causing the under reporting.  There may not be
> that many explicit accounting tests (even though it is a lot compared to
> other components) but a lot of tests also touch accounting indirectly.
> There is just no way that only 53 lines of java code are being executed
> in accounting during the full test run.  I know for a fact that code is
> executed from PaymentGatewayServices, FinAccountPaymentServices,
> PaymentWorker, UtilAccounting and a few others during the tests.

I had some other changes in that tree that were causing tests to fail.
 I've rerun it now, all current tests pass, and I've uploaded a new
report to http://www.brainfood.com/ofbiz-coverage

Note that framework/base has almost 100% coverage.  But that's a bad
thing, because it's not explicitly testing it; all that code just
happens to be utilized during the rest of the test run.

Total coverage increased from 7% to 14%.

>
> Regards
> Scott

Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adrian Crum
Adam Heath wrote:

> Scott Gray wrote:
>> On 11/12/2009, at 6:41 AM, Adam Heath wrote:
>>
>>> Scott Gray wrote:
>>>> Hi Adam,
>>>>
>>>> Looking at the results my first impression is that the coverage is
>>>> under-reported.  For example, the accounting component has quite a few
>>>> tests but no coverage is shown at all (except for the test package
>>>> itself).  Possibly because there is lot of logic in simple methods but
>>>> I'm 100% sure java code is also run during the tests.
>>>>
>>>> But still a great start and something that will be immensely useful if
>>>> we can up the accuracy a bit.
>>> Well, it doesn't, really.  If you click thru to accounting.test,
>>> you'll see that there aren't really that many tests.  And, upon
>>> further investigation, the lines after the runSync calls aren't run,
>>> due to some exception most likely.  I'm not certian if this is do to
>>> my changes, or if the tests themselves are broken.  I'm running a
>>> plain test run now to check that.  Plus, there actually *is* line hits
>>> in accounting.invoice.
>> The tests seem to be running fine on buildbot
>> (http://ci.apache.org/waterfall?show=ofbiz-trunk), I'm guessing it's the
>> test run problem that's causing the under reporting.  There may not be
>> that many explicit accounting tests (even though it is a lot compared to
>> other components) but a lot of tests also touch accounting indirectly.
>> There is just no way that only 53 lines of java code are being executed
>> in accounting during the full test run.  I know for a fact that code is
>> executed from PaymentGatewayServices, FinAccountPaymentServices,
>> PaymentWorker, UtilAccounting and a few others during the tests.
>
> I had some other changes in that tree that were causing tests to fail.
>  I've rerun it now, all current tests pass, and I've uploaded a new
> report to http://www.brainfood.com/ofbiz-coverage

Oops, link doesn't work.

Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adam Heath-2
Adrian Crum wrote:

> Adam Heath wrote:
>> Scott Gray wrote:
>>> On 11/12/2009, at 6:41 AM, Adam Heath wrote:
>>>
>>>> Scott Gray wrote:
>>>>> Hi Adam,
>>>>>
>>>>> Looking at the results my first impression is that the coverage is
>>>>> under-reported.  For example, the accounting component has quite a few
>>>>> tests but no coverage is shown at all (except for the test package
>>>>> itself).  Possibly because there is lot of logic in simple methods but
>>>>> I'm 100% sure java code is also run during the tests.
>>>>>
>>>>> But still a great start and something that will be immensely useful if
>>>>> we can up the accuracy a bit.
>>>> Well, it doesn't, really.  If you click thru to accounting.test,
>>>> you'll see that there aren't really that many tests.  And, upon
>>>> further investigation, the lines after the runSync calls aren't run,
>>>> due to some exception most likely.  I'm not certian if this is do to
>>>> my changes, or if the tests themselves are broken.  I'm running a
>>>> plain test run now to check that.  Plus, there actually *is* line hits
>>>> in accounting.invoice.
>>> The tests seem to be running fine on buildbot
>>> (http://ci.apache.org/waterfall?show=ofbiz-trunk), I'm guessing it's the
>>> test run problem that's causing the under reporting.  There may not be
>>> that many explicit accounting tests (even though it is a lot compared to
>>> other components) but a lot of tests also touch accounting
>>> indirectly. There is just no way that only 53 lines of java code are
>>> being executed
>>> in accounting during the full test run.  I know for a fact that code is
>>> executed from PaymentGatewayServices, FinAccountPaymentServices,
>>> PaymentWorker, UtilAccounting and a few others during the tests.
>>
>> I had some other changes in that tree that were causing tests to fail.
>>  I've rerun it now, all current tests pass, and I've uploaded a new
>> report to http://www.brainfood.com/ofbiz-coverage
>
> Oops, link doesn't work.

http://www.brainfood.com/ofbiz-coverage/
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adrian Crum
Adam Heath wrote:

> Adrian Crum wrote:
>> Adam Heath wrote:
>>> Scott Gray wrote:
>>>> On 11/12/2009, at 6:41 AM, Adam Heath wrote:
>>>>
>>>>> Scott Gray wrote:
>>>>>> Hi Adam,
>>>>>>
>>>>>> Looking at the results my first impression is that the coverage is
>>>>>> under-reported.  For example, the accounting component has quite a few
>>>>>> tests but no coverage is shown at all (except for the test package
>>>>>> itself).  Possibly because there is lot of logic in simple methods but
>>>>>> I'm 100% sure java code is also run during the tests.
>>>>>>
>>>>>> But still a great start and something that will be immensely useful if
>>>>>> we can up the accuracy a bit.
>>>>> Well, it doesn't, really.  If you click thru to accounting.test,
>>>>> you'll see that there aren't really that many tests.  And, upon
>>>>> further investigation, the lines after the runSync calls aren't run,
>>>>> due to some exception most likely.  I'm not certian if this is do to
>>>>> my changes, or if the tests themselves are broken.  I'm running a
>>>>> plain test run now to check that.  Plus, there actually *is* line hits
>>>>> in accounting.invoice.
>>>> The tests seem to be running fine on buildbot
>>>> (http://ci.apache.org/waterfall?show=ofbiz-trunk), I'm guessing it's the
>>>> test run problem that's causing the under reporting.  There may not be
>>>> that many explicit accounting tests (even though it is a lot compared to
>>>> other components) but a lot of tests also touch accounting
>>>> indirectly. There is just no way that only 53 lines of java code are
>>>> being executed
>>>> in accounting during the full test run.  I know for a fact that code is
>>>> executed from PaymentGatewayServices, FinAccountPaymentServices,
>>>> PaymentWorker, UtilAccounting and a few others during the tests.
>>> I had some other changes in that tree that were causing tests to fail.
>>>  I've rerun it now, all current tests pass, and I've uploaded a new
>>> report to http://www.brainfood.com/ofbiz-coverage
>> Oops, link doesn't work.
>
> http://www.brainfood.com/ofbiz-coverage/

I like the holiday-themed colors. It would be cool if incomplete
coverage could be displayed as partially eaten sugar cookies.
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Scott Gray-2
In reply to this post by Adam Heath-2
On 11/12/2009, at 11:55 AM, Adam Heath wrote:

> Scott Gray wrote:
>> On 11/12/2009, at 6:41 AM, Adam Heath wrote:
>>
>>> Scott Gray wrote:
>>>> Hi Adam,
>>>>
>>>> Looking at the results my first impression is that the coverage is
>>>> under-reported.  For example, the accounting component has quite  
>>>> a few
>>>> tests but no coverage is shown at all (except for the test package
>>>> itself).  Possibly because there is lot of logic in simple  
>>>> methods but
>>>> I'm 100% sure java code is also run during the tests.
>>>>
>>>> But still a great start and something that will be immensely  
>>>> useful if
>>>> we can up the accuracy a bit.
>>>
>>> Well, it doesn't, really.  If you click thru to accounting.test,
>>> you'll see that there aren't really that many tests.  And, upon
>>> further investigation, the lines after the runSync calls aren't run,
>>> due to some exception most likely.  I'm not certian if this is do to
>>> my changes, or if the tests themselves are broken.  I'm running a
>>> plain test run now to check that.  Plus, there actually *is* line  
>>> hits
>>> in accounting.invoice.
>>
>> The tests seem to be running fine on buildbot
>> (http://ci.apache.org/waterfall?show=ofbiz-trunk), I'm guessing  
>> it's the
>> test run problem that's causing the under reporting.  There may not  
>> be
>> that many explicit accounting tests (even though it is a lot  
>> compared to
>> other components) but a lot of tests also touch accounting  
>> indirectly.
>> There is just no way that only 53 lines of java code are being  
>> executed
>> in accounting during the full test run.  I know for a fact that  
>> code is
>> executed from PaymentGatewayServices, FinAccountPaymentServices,
>> PaymentWorker, UtilAccounting and a few others during the tests.
>
> I had some other changes in that tree that were causing tests to fail.
> I've rerun it now, all current tests pass, and I've uploaded a new
> report to http://www.brainfood.com/ofbiz-coverage
Cool thanks, looks more in line with what I was expecting.

>
> Note that framework/base has almost 100% coverage.  But that's a bad
> thing, because it's not explicitly testing it; all that code just
> happens to be utilized during the rest of the test run.

Of course explicitly testing framework/base would be much better but  
why is the current 100% coverage a bad thing?  I mean implied testing  
is better than no testing right?  If I was to go in and incorrectly  
modify some of those base methods then there's a good chance some of  
the higher level tests would fail.

>
> Total coverage increased from 7% to 14%.



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adam Heath-2
In reply to this post by Adrian Crum
Adrian Crum wrote:
>> http://www.brainfood.com/ofbiz-coverage/
>
> I like the holiday-themed colors. It would be cool if incomplete
> coverage could be displayed as partially eaten sugar cookies.

That's standard cobertura scheme.
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adam Heath-2
In reply to this post by Scott Gray-2
Scott Gray wrote:
>> Note that framework/base has almost 100% coverage.  But that's a bad
>> thing, because it's not explicitly testing it; all that code just
>> happens to be utilized during the rest of the test run.
>
> Of course explicitly testing framework/base would be much better but why
> is the current 100% coverage a bad thing?  I mean implied testing is
> better than no testing right?  If I was to go in and incorrectly modify
> some of those base methods then there's a good chance some of the higher
> level tests would fail.

In theory such explicit base testing could be run from the command
line, very quickly, using the newly added 'tests' target defined in
common.xml.

Additionally, just because a line has been noted in cobertura, doesn't
mean all variations have been tested.  Consider the case that some
condition is doing some kind of pattern match, or looking at
Collection.contains or Map.containsKey.  It's much simpler to verify
that everything is tested when it is done explicitly.

Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Scott Gray-2
On 11/12/2009, at 12:34 PM, Adam Heath wrote:

> Scott Gray wrote:
>>> Note that framework/base has almost 100% coverage.  But that's a bad
>>> thing, because it's not explicitly testing it; all that code just
>>> happens to be utilized during the rest of the test run.
>>
>> Of course explicitly testing framework/base would be much better  
>> but why
>> is the current 100% coverage a bad thing?  I mean implied testing is
>> better than no testing right?  If I was to go in and incorrectly  
>> modify
>> some of those base methods then there's a good chance some of the  
>> higher
>> level tests would fail.
>
> In theory such explicit base testing could be run from the command
> line, very quickly, using the newly added 'tests' target defined in
> common.xml.
>
> Additionally, just because a line has been noted in cobertura, doesn't
> mean all variations have been tested.  Consider the case that some
> condition is doing some kind of pattern match, or looking at
> Collection.contains or Map.containsKey.  It's much simpler to verify
> that everything is tested when it is done explicitly.
Okay I see what you mean now, it's a bad thing that coverage is  
reported without explicit thorough testing.  Even though the indirect  
coverage is still better than no coverage whatsoever.


smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Adam Heath-2
Scott Gray wrote:
>> Additionally, just because a line has been noted in cobertura, doesn't
>> mean all variations have been tested.  Consider the case that some
>> condition is doing some kind of pattern match, or looking at
>> Collection.contains or Map.containsKey.  It's much simpler to verify
>> that everything is tested when it is done explicitly.
>
> Okay I see what you mean now, it's a bad thing that coverage is reported
> without explicit thorough testing.  Even though the indirect coverage is
> still better than no coverage whatsoever.

As a better example, let's say that there is only 10% coverage on the
entire ofbiz code base.  But base has 100% coverage.  That other 90%
of untested code may test parts of base that may not work, and would
break the higher-level code.

It's easier to write tests that are close to the code being tested.
Trying to tweak a high-level test to make certain all low-level code
is wrong is very very difficult.

Plus, if a typo gets introduced in one of those map keys, it might not
be detected until much much later in time, when explicit tests are not
 used.

In my opinion, as each new component is activated in the ofbiz system,
it should have it's own set of tests that move it close to 100%
coverage.  So, I can test just base, and get 100%, then base+sql, and
get 100% on base+sql, then base+sql+entity, and get 100% on
base+sql+entity, and so on.  You want to make certain that earlier
components are correct before utilizing later ones, or the entire test
run may fail spectactulary.
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Erwan de FERRIERES-3
In reply to this post by Scott Gray-2
Now the point would be to show the coverage of the simple methods and
also, maybe in sometime, the selenium coverage.

What would be great is finding a way to indicate which services are
tested, and then display it in the webtools. This won't give an
information as precise as cobertura but would add some quick display on
what is or can be tested.

If we agree on a syntax to use, I would be ready to add the screen in OFBiz.

Cheers,

Le 11/12/2009 00:21, Scott Gray a écrit :
../..
>
>

--
Erwan de FERRIERES
www.nereide.biz
Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Jacques Le Roux
Administrator
What about groovy scripts, are they handled by Cobertura?
And actions in Screens, is it worth do something? I guess checking "static structural" files like web.xml, ofbiz-component.xml files
(and all xml like them, controllers, menus, commonsScreens and such) is not necessary?

Jacques

From: "Erwan de FERRIERES" <[hidden email]>

> Now the point would be to show the coverage of the simple methods and also, maybe in sometime, the selenium coverage.
>
> What would be great is finding a way to indicate which services are tested, and then display it in the webtools. This won't give
> an information as precise as cobertura but would add some quick display on what is or can be tested.
>
> If we agree on a syntax to use, I would be ready to add the screen in OFBiz.
>
> Cheers,
>
> Le 11/12/2009 00:21, Scott Gray a écrit :
> ../..
>>
>>
>
> --
> Erwan de FERRIERES
> www.nereide.biz
>


Reply | Threaded
Open this post in threaded view
|

Re: code coverage report for ofbiz run-tests call

Jacques Le Roux
Administrator
In reply to this post by Adam Heath-2
From: "Adam Heath" <[hidden email]>

> Additionally, the way the classloaders are constructed is a bit odd.
> The bootstrap loader only has ofbiz.jar.  The code then creates a
> classloader will *all* jars in base/lib, recursively.  After the
> containers and components are started, yet another classloader is
> created, this time with proper libs and all other ofbiz-$foo.jar.
> Both of those classloaders update the java.class.path system property.
> This is a bug, as the instrumentation changes all ofbiz-$foo.jar into
> a new jar in /tmp, so the value of the property is different each
> time.  Additionally, the second created classloader should not load
> the base component again, as that is sub-optimal.
>
> I'm thinking that component code should really move to start.
>
> Another bug, is that due to one of the above bugs, the first
> classloader is not instrumented at all.  Since this is the classloader
> that contains framework/base, you don't see any line hits in the report.

Since the the start (my start with OFBiz) I have always wondered about complexity there

Jacques

12