Logging in selenium XML
Locked
Logging in selenium XML
 
|
Re: Logging in selenium XML
|
|
Hi Erwan,
It'll be another couple of days before I can make an informed comment, I'm still very much in the learning phase. One question I do have, does anybody have selenium setup to run in a continuous integration environment? If you have lots of tests, how long to they take to run? Is SeleniumGrid a good solution to shortening the time a test run takes? So maybe a few questions :-) I guess I'm still sitting here wondering if WebTest isn't a better solution simply because it doesn't require a browser (just another ant task) and the tests run faster. I've never used either before so I'm in the dark on these solutions. Regards Scott HotWax Media http://www.hotwaxmedia.com On 15/12/2009, at 10:07 PM, Erwan de FERRIERES wrote: > Hi all, > > As many of us are now looking into seleniumXML, I would like to > discuss a bit more with you of the logging of errors and success in > seleniumXml. > > Has anyone started something ? The changes that have to integrate > are major and is would be great to coordinate our efforts. > > What I'm thinking is adding JUnit asserts at the end of a selenium > command, to be able to create JUnit XML files and after creating a > report. This will then help us to identify errors on the interface > or in functional testcases. > > Regards, > > -- > Erwan de FERRIERES > www.nereide.biz |
smime.p7s (4K)
Re: Logging in selenium XML
|
|
Re: Logging in selenium XML
|
|
One thing that we could easily do is put together a continuous integration that has several branches - one that is run immediately upon each of the commits and one that is scheduled for a few times a day for these longer running tests. I agree that we definitely need these in place, but breaking them up might:
1. Get people to write more unit tests that are quicker to run. 2. Get people to write more of the longer ones because those will still be catching any potential issues. Just a thought. Cheers, Ruppert -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com o:801.649.6594 f:801.649.6595 On Dec 15, 2009, at 4:01 AM, Matthieu Bollot wrote: > Le mardi 15 décembre 2009 à 23:28 +1300, Scott Gray a écrit : >> Hi Erwan, >> >> It'll be another couple of days before I can make an informed comment, >> I'm still very much in the learning phase. >> >> One question I do have, does anybody have selenium setup to run in a >> continuous integration environment? > We do have some at nereide >> If you have lots of tests, how long to they take to run? Is >> SeleniumGrid a good solution to shortening the time a test run takes? >> So maybe a few questions :-) > > The results here : > http://selenium.neogia.org/ofbiz/results/result_2009.12.14_08:12:44.html > took 12 minutes for 13 tests > and the results here : > http://selenium.neogia.org/ofbizNeogia.stable/results/result_2009.12.14_08:12:45.html > took about 20 minutes for 41 tests > That's a quite fast server. 4 proc, 8GB ram (those are html testSuite > using selenium-server.jar directly, perhaps using seleniumXML can change > the duration a little bit) > > And loading a page is may be the only "long" stuff. > >> I guess I'm still sitting here wondering if WebTest isn't a better >> solution simply because it doesn't require a browser (just another ant >> task) and the tests run faster. I've never used either before so I'm >> in the dark on these solutions. > > I think that selenium is quite good because you can _really_ know how > long took a test, e.g we could also test how long it takes to purchase > 1000 products (don't know if it will be still possible with webtest) > > cheers, > >> Regards >> Scott >> >> HotWax Media >> http://www.hotwaxmedia.com >> >> On 15/12/2009, at 10:07 PM, Erwan de FERRIERES wrote: >> >>> Hi all, >>> >>> As many of us are now looking into seleniumXML, I would like to >>> discuss a bit more with you of the logging of errors and success in >>> seleniumXml. >>> >>> Has anyone started something ? The changes that have to integrate >>> are major and is would be great to coordinate our efforts. >>> >>> What I'm thinking is adding JUnit asserts at the end of a selenium >>> command, to be able to create JUnit XML files and after creating a >>> report. This will then help us to identify errors on the interface >>> or in functional testcases. >>> >>> Regards, >>> >>> -- >>> Erwan de FERRIERES >>> www.nereide.biz >> > > -- > Matthieu BOLLOT > www.nereide.biz > |
Re: Logging in selenium XML
|
|
Re: Logging in selenium XML
|
|
In reply to this post by Matthieu Bollot-4
On 16/12/2009, at 12:01 AM, Matthieu Bollot wrote:
> Le mardi 15 décembre 2009 à 23:28 +1300, Scott Gray a écrit : >> Hi Erwan, >> >> It'll be another couple of days before I can make an informed >> comment, >> I'm still very much in the learning phase. >> >> One question I do have, does anybody have selenium setup to run in a >> continuous integration environment? > We do have some at nereide >> If you have lots of tests, how long to they take to run? Is >> SeleniumGrid a good solution to shortening the time a test run takes? >> So maybe a few questions :-) > > The results here : > http://selenium.neogia.org/ofbiz/results/result_2009.12.14_08:12:44.html > took 12 minutes for 13 tests > and the results here : > http://selenium.neogia.org/ofbizNeogia.stable/results/result_2009.12.14_08:12:45.html > took about 20 minutes for 41 tests > That's a quite fast server. 4 proc, 8GB ram (those are html testSuite > using selenium-server.jar directly, perhaps using seleniumXML can > change > the duration a little bit) of tests then it will be impractical for a developer to ever run them on their local machine. I think we'll need to come up with a consistent structure for the tests so that they can easily find and run the tests that will cover their task at hand. > > And loading a page is may be the only "long" stuff. > >> I guess I'm still sitting here wondering if WebTest isn't a better >> solution simply because it doesn't require a browser (just another >> ant >> task) and the tests run faster. I've never used either before so I'm >> in the dark on these solutions. > > I think that selenium is quite good because you can _really_ know how > long took a test, e.g we could also test how long it takes to purchase > 1000 products (don't know if it will be still possible with webtest) think it just cuts out the actual rendering time. Besides this sounds more like performance testing which I don't really think Selenium is the most appropriate tool for. > > cheers, > >> Regards >> Scott >> >> HotWax Media >> http://www.hotwaxmedia.com >> >> On 15/12/2009, at 10:07 PM, Erwan de FERRIERES wrote: >> >>> Hi all, >>> >>> As many of us are now looking into seleniumXML, I would like to >>> discuss a bit more with you of the logging of errors and success in >>> seleniumXml. >>> >>> Has anyone started something ? The changes that have to integrate >>> are major and is would be great to coordinate our efforts. >>> >>> What I'm thinking is adding JUnit asserts at the end of a selenium >>> command, to be able to create JUnit XML files and after creating a >>> report. This will then help us to identify errors on the interface >>> or in functional testcases. >>> >>> Regards, >>> >>> -- >>> Erwan de FERRIERES >>> www.nereide.biz >> > > -- > Matthieu BOLLOT > www.nereide.biz > |
Re: Logging in selenium XML
|
|
In reply to this post by Tim Ruppert
That sounds like a good idea to me. What would be awesome is if the
CI server could inspect the commit, determine the components/ applications affected and then only run the applicable tests. Full tests runs could be reserved for framework commits or something like that. Regards Scott On 16/12/2009, at 3:13 AM, Tim Ruppert wrote: > One thing that we could easily do is put together a continuous > integration that has several branches - one that is run immediately > upon each of the commits and one that is scheduled for a few times a > day for these longer running tests. I agree that we definitely need > these in place, but breaking them up might: > > 1. Get people to write more unit tests that are quicker to run. > 2. Get people to write more of the longer ones because those will > still be catching any potential issues. > > Just a thought. > > Cheers, > Ruppert > -- > Tim Ruppert > HotWax Media > http://www.hotwaxmedia.com > > o:801.649.6594 > f:801.649.6595 > > On Dec 15, 2009, at 4:01 AM, Matthieu Bollot wrote: > >> Le mardi 15 décembre 2009 à 23:28 +1300, Scott Gray a écrit : >>> Hi Erwan, >>> >>> It'll be another couple of days before I can make an informed >>> comment, >>> I'm still very much in the learning phase. >>> >>> One question I do have, does anybody have selenium setup to run in a >>> continuous integration environment? >> We do have some at nereide >>> If you have lots of tests, how long to they take to run? Is >>> SeleniumGrid a good solution to shortening the time a test run >>> takes? >>> So maybe a few questions :-) >> >> The results here : >> http://selenium.neogia.org/ofbiz/results/result_2009.12.14_08:12:44.html >> took 12 minutes for 13 tests >> and the results here : >> http://selenium.neogia.org/ofbizNeogia.stable/results/result_2009.12.14_08:12:45.html >> took about 20 minutes for 41 tests >> That's a quite fast server. 4 proc, 8GB ram (those are html testSuite >> using selenium-server.jar directly, perhaps using seleniumXML can >> change >> the duration a little bit) >> >> And loading a page is may be the only "long" stuff. >> >>> I guess I'm still sitting here wondering if WebTest isn't a better >>> solution simply because it doesn't require a browser (just another >>> ant >>> task) and the tests run faster. I've never used either before so >>> I'm >>> in the dark on these solutions. >> >> I think that selenium is quite good because you can _really_ know how >> long took a test, e.g we could also test how long it takes to >> purchase >> 1000 products (don't know if it will be still possible with webtest) >> >> cheers, >> >>> Regards >>> Scott >>> >>> HotWax Media >>> http://www.hotwaxmedia.com >>> >>> On 15/12/2009, at 10:07 PM, Erwan de FERRIERES wrote: >>> >>>> Hi all, >>>> >>>> As many of us are now looking into seleniumXML, I would like to >>>> discuss a bit more with you of the logging of errors and success in >>>> seleniumXml. >>>> >>>> Has anyone started something ? The changes that have to integrate >>>> are major and is would be great to coordinate our efforts. >>>> >>>> What I'm thinking is adding JUnit asserts at the end of a selenium >>>> command, to be able to create JUnit XML files and after creating a >>>> report. This will then help us to identify errors on the interface >>>> or in functional testcases. >>>> >>>> Regards, >>>> >>>> -- >>>> Erwan de FERRIERES >>>> www.nereide.biz >>> >> >> -- >> Matthieu BOLLOT >> www.nereide.biz >> > |
Re: Logging in selenium XML
|
|
In reply to this post by Erwan de FERRIERES-3
On 16/12/2009, at 6:56 AM, Erwan de FERRIERES wrote:
> > > Le 15/12/2009 11:28, Scott Gray a écrit : > >> I guess I'm still sitting here wondering if WebTest isn't a better >> solution simply because it doesn't require a browser (just another >> ant >> task) and the tests run faster. I've never used either before so >> I'm in >> the dark on these solutions. >> > Webtest is emulating a browser and does not interface tests. It's > why it's quicker. > > I think that those two technologies are not doing the same tests : > * webtest can simulate a process, and make it run fast and doesn't > about rendering or anything. > * Selenium is more an interface test, and it's why it is more slow. > You need a browser, and rendering is slow (JS, CSS, ... everything > needs to be displayed to validate a command). and css and is actually stricter about html validity. They both ultimately use various forms of dom navigation to make assertions and perform actions. What does selenium actually do with the interface during the tests that makes it different? > Pros and cons for those 2 tools may be found on this page, which is > a bit old (2007)... > http://mguillem.wordpress.com/2007/10/29/webtest-vs-selenium-webtest-wins-13-5/ > > Webtest is under an Apache2 license, but I didn't looked deeper to > the files to see if everything really complies. > > Both can be run by ant tasks, which is fine. Well the difference here being that webtest is only using ant and java, you don't have to have a browser installed (and customized) to run the tests, anywhere you can run OFBiz you can run webtests. How difficult is it to setup a regular web server to run selenium tests? At the end of the day my only concern is that if the tests are difficult run then hardly anyone ever will and as we saw with the JUnit tests, once they stop being maintained they quickly become worthless. > We should give a try to webtest, but we have to remember that both > are different ! > > There is also webdriver which is a bit like webtest, and is > currently integrated in selenium 2.0. But still with the issues with > cssQuery and code licensed to Google... > >> Regards > -- > Erwan de FERRIERES > www.nereide.biz |
Re: Logging in selenium XML
|
Re: Logging in selenium XML
|
|
On 16/12/2009, at 10:00 AM, Nicolas Malin wrote:
> Le mercredi 16 décembre 2009 à 09:36 +1300, Scott Gray a écrit : >> That sounds like a good idea to me. What would be awesome is if the >> CI server could inspect the commit, determine the components/ >> applications affected and then only run the applicable tests. Full >> tests runs could be reserved for framework commits or something like >> that. > > It's good idea for 90% change in applications components but with > integrate system as OFBiz some change on one component can generate > error on other components. > > With Neogia we had 40 seleniums test and many alert from test came > from > business component that selenium test didn't depends. > > With OFBiz structure we can run full test when operate change in > application and framework (for me application is also a business > framework ;) ) and only a component for not important framework > component (exemple, ...) and specialpurpose. every commit but eventually (assuming people actually contribute tests) the number of tests may make this impractical and that is the point where we'll need to find ways of trimming down the test runs. > > cheers > Nicolas > >> >> Regards >> Scott >> >> On 16/12/2009, at 3:13 AM, Tim Ruppert wrote: >> >>> One thing that we could easily do is put together a continuous >>> integration that has several branches - one that is run immediately >>> upon each of the commits and one that is scheduled for a few times a >>> day for these longer running tests. I agree that we definitely need >>> these in place, but breaking them up might: >>> >>> 1. Get people to write more unit tests that are quicker to run. >>> 2. Get people to write more of the longer ones because those will >>> still be catching any potential issues. >>> >>> Just a thought. >>> >>> Cheers, >>> Ruppert >>> -- >>> Tim Ruppert >>> HotWax Media >>> http://www.hotwaxmedia.com >>> >>> o:801.649.6594 >>> f:801.649.6595 >>> >>> On Dec 15, 2009, at 4:01 AM, Matthieu Bollot wrote: >>> >>>> Le mardi 15 décembre 2009 à 23:28 +1300, Scott Gray a écrit : >>>>> Hi Erwan, >>>>> >>>>> It'll be another couple of days before I can make an informed >>>>> comment, >>>>> I'm still very much in the learning phase. >>>>> >>>>> One question I do have, does anybody have selenium setup to run >>>>> in a >>>>> continuous integration environment? >>>> We do have some at nereide >>>>> If you have lots of tests, how long to they take to run? Is >>>>> SeleniumGrid a good solution to shortening the time a test run >>>>> takes? >>>>> So maybe a few questions :-) >>>> >>>> The results here : >>>> http://selenium.neogia.org/ofbiz/results/result_2009.12.14_08:12:44.html >>>> took 12 minutes for 13 tests >>>> and the results here : >>>> http://selenium.neogia.org/ofbizNeogia.stable/results/result_2009.12.14_08:12:45.html >>>> took about 20 minutes for 41 tests >>>> That's a quite fast server. 4 proc, 8GB ram (those are html >>>> testSuite >>>> using selenium-server.jar directly, perhaps using seleniumXML can >>>> change >>>> the duration a little bit) >>>> >>>> And loading a page is may be the only "long" stuff. >>>> >>>>> I guess I'm still sitting here wondering if WebTest isn't a better >>>>> solution simply because it doesn't require a browser (just another >>>>> ant >>>>> task) and the tests run faster. I've never used either before so >>>>> I'm >>>>> in the dark on these solutions. >>>> >>>> I think that selenium is quite good because you can _really_ know >>>> how >>>> long took a test, e.g we could also test how long it takes to >>>> purchase >>>> 1000 products (don't know if it will be still possible with >>>> webtest) >>>> >>>> cheers, >>>> >>>>> Regards >>>>> Scott >>>>> >>>>> HotWax Media >>>>> http://www.hotwaxmedia.com >>>>> >>>>> On 15/12/2009, at 10:07 PM, Erwan de FERRIERES wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> As many of us are now looking into seleniumXML, I would like to >>>>>> discuss a bit more with you of the logging of errors and >>>>>> success in >>>>>> seleniumXml. >>>>>> >>>>>> Has anyone started something ? The changes that have to integrate >>>>>> are major and is would be great to coordinate our efforts. >>>>>> >>>>>> What I'm thinking is adding JUnit asserts at the end of a >>>>>> selenium >>>>>> command, to be able to create JUnit XML files and after >>>>>> creating a >>>>>> report. This will then help us to identify errors on the >>>>>> interface >>>>>> or in functional testcases. >>>>>> >>>>>> Regards, >>>>>> >>>>>> -- >>>>>> Erwan de FERRIERES >>>>>> www.nereide.biz >>>>> >>>> >>>> -- >>>> Matthieu BOLLOT >>>> www.nereide.biz >>>> >>> >> > -- > Nicolas MALIN > Consultant > Tél : 06.17.66.40.06 > Site projet : http://www.neogia.org/ > ------- > Société LibrenBerry > Tél : 02.48.02.56.12 > Site : http://www.librenberry.net/ > |
Re: Logging in selenium XML
Administrator
|
Re: Logging in selenium XML
|
|
Re: Logging in selenium XML
|
|
| Free forum by Nabble | Edit this page |