The idea of dev-box testing is simple but very effective. Let’s take an example of a development cycle:
- Developer implements the functionality along with unit and integration tests and when she’s satisfied she commits the source code in the source code repository like svn or git.
- Continuous Integration (CI) gets triggered by source code commit and CI tool like Jenkins performs the automated build which results in code compilation and execution of the test cases (unit, integration and automated functional tests).
- After multiple builds when developer thinks that functionality is ready for functional testing, she asks the tester to take a specific build and test.
- Tester starts testing. If tester finds bugs in the functionality, she informs the developer about the problem and developer again begins from step 1.
- If everything is good, tester moves the user-story into DONE state.
Working through step 1-3 takes a lot of time and if testers has to move the story again in development, it results in a lot of time-waste and frustration.
How about this?
As developer sees everything working on his machine, he asks tester to test the functionality on his machine. While testing on dev-machine if everything works, only then developer commits the code which eventually triggers the build and auto-deployment. On the committed code available in testing environment tester performs further exploratory tests. Otherwise developer continue to fix the issues on his machine. This way, a lot of valuable time is saved.
A nice idea, but I’d like to see it honed and applied in practice. I sometimes commit over 5 times per hour. Some _automated_ acceptance tests take over an hour. That would slow me down by a factor of 5 at least.
Of course this can be optimized, but that’s not trivial. It an idea with potential. I’d love to hear what interesting results you find in applying it.
@Iwein, Not sure if I got you. In dev-box testing, tester manually performs tests on developer machine. So he performs corner cases which developer may have forgotten. So instead of getting those bugs on test environment, they are caught then and there. Automated acceptance tests don’t have to do anything with this approach.
Hi Shrikanth, good thought…I can see just one problem, as one in blind to his/her own mistakes/errors, if tester is sitting beside developer he will have tendency to think from same frame of mind specially when other person is dominating or even in case both have good rapport…you might have heard of chameleon effect .check out this link…www.yale.edu/acmelab/articles/chartrand_bargh_1999.pdf..and you cannot solve a problem with the same mind that created it..working together helps when you both have to think in same direction and achieve same goal like pair programming but when one has to find mistake in other’s job it might not work.I think tester should do the testing independently so that developer’s perspective does not influence him.. that’s my thought, what do you say… DK
@DK tester will not be sitting with developer all the time. He will come to dev-machine when developer invites him to test. So the frame of mind a tester will be of a tester. Just to make sure, it’s not pair-testing or pair-programming where both people sit as driver and navigator all the time.
Got it… Thanks Shrikant..
Not a good idea. Not a practical solution.
Reason: As a tester we should not test on dev environment, developer more often customise there sandbox or there unit test environment for things to work.
you often see a comment from developer everything is working at my end.
@Rahul Nothing is 100% foolproof. Dev-box testing is not the alternative of testing in test-environment anyway. I have seen personally many times when developer says that he has committed the code and functionality is ready for test, either it doesn’t work completely or it breaks some other functionality from regression point of view. That’s very frustrating because at the end of it user-story is not DONE. As tester has view of entire functionality (not only the current user-story) and main acceptance criteria, he is able to find issues then and there and then developer realizes – “oh yes, I missed this criteria”. It shortens the feedback loop. It’s a practice you may want to use and for sure is not a silver bullet.
Would be good to test with the developer, but in an agile proyect the time to commit is short and the testing team is small comparing with dev team. In other words you won’t be able to do it because the developer cannot be waiting for an available tester. Also you have a lot of preassure to finish the tests over the software version that you have in your testing environment, to deploy it in production; so it consumes all QA resources. Don’t missundertand me, this is a good way to increase quality, but it’s hard to get an agreement with devs and product teams to work in this way.
Achieving sustainable pace is indeed necessary to keep right balance and to attain high quality. You can not blame agile, if you go on taking extra work without saying “NO”. You also definitely need to staff right developer to tester ratio. Also, when you are working in this mode since defect induction rate is very low, testers will get good time to automate tests too and hence increase coverage.
Agree with your point as it helps to make a stable and quality build before releasing it to tester for their exploratory testing or deep dive testing. We have seen a positive effect of this approach in our project. In making a good balance we share unit test cases with dev team beforehand so that before releasing to testers we were sure that major functions are working.
But during this exercise we should have to ensure that your team should be enough mature to understand purpose correctly and they understand and believe in “being-Agile” phrase. There will be more challenges if team is not taking it positively or doing it with other intention.
I think this is a good approach, and it would seem a key point is missing to address all a few of the comments regarding time and being “anchored” by the developers view point and that is writing the test cases first (TDD) so that the developer is “building it right”. This should also cut down on the defect escape rate and would give the developer higher confidence in knowing the commits meet the acceptance criteria rather than their own subjective view of what they think should be tested and passing.