I have spent some time trying to simplify the release process. For a variety of reasons, we can only release on Thursdays. The code is "frozen" on Tuesday before it can be released on Thursday. But we sometimes squeeze in a quick fix on Wednesday as well.
The question, is when should QA test the code?
Here is what I have seen happen:
- Dev writes code and sends it to QA.
- QA finds problems, sends it back to the Dev.
- Dev fixes and sends it back to QA.
I have seen a Dev fix their code on Tuesday, and then QA comes back on Wednesday with problems, when the code should have been frozen anyway.
I am looking, what should be the best solution here.
We have several problems going on at once:
- Developers test on the same server as QA tests. I am working to switch developers to a separate Dev server, but it is a long work in progress.
- We don't have an easy way to revert code back from the QA server. It is easier to build revisions than revert changes. We can try to revert code more, but it will require a culture change.
- QA don't really have a schedule when they are supposed to do functional testing vs regression testing.
I don't know what is the best way to proceed forward. Thus far, I haven't thought too much about the QA because I was focused more on getting releases out. Now that releasing is more simplified, that we can potentially do weekly releases, I am trying to see how we should proceed with the testing process.
It seems to me that your problem is that your definition of "freeze" seems to allow fixes for QA issues. So, not a freeze at all if the idea is to give QA a chance to have a clean testing framework on Wednesday.
I see two alternatives:
Make Tuesday a true freeze. Any defects found by QA drop that feature out of the Thursday release.
Stop "throwing features over the wall" to QA. Make the QA testers/process part of the development team/process. Features are considered "done" and ready for submission to production only once tested. Freeze on Thursday morning with only integration testing to be done before release.
In truth, both approaches yield the same results. If programmers have to get it right by Tuesday, then they'll need to work more closely with QA during development. Eventually, the Wednesday testing becomes little more than a rubber stamp and they'll push to move the freeze back to Wednesday.
Most importantly it seems that in this situation the "definition of done", has to be more than just "coding completed".
I like this response. Indeed, if we don't have strong expectations from the devs to finish by a certain time, then we shouldn't have such expectations on QA.
I wanted to suggest something like this. Code-freeze wise, you can have a "minor" and "major" problems, major problems block the feature, minor ones let it go (but you now have a tech debt, and make sure that THIS process to fixing up found issues is higher-prio then new features). Of course, you decide what is minor and what major. E.g. maybe a typo in the UI is acceptable, maybe not.
As for throwing features over the wall - I would actually suggest just changing the perspective - make QA involved earlier. The feature is not ready and not frozen unless it's been looked at by QA. Then when a thing is frozen, it's really ready. (Of course you'll still have regressions etc but that's another topic.)
I think that defining "done" as QA tested is way better than "the code compiles", which is essentially what most teams seem to use.
Developers need to get into the habit of not writing bugs. That's technically the answer to all of these problems. "We have issues dealing with bugs found in the QA phase". So stop writing bugs for QA to find, then the problem goes away.
If the attitude is "Bugs found in QA kick the feature out of the release", then the programmers are going to find that they work all week and end up contributing nothing to the release. Maybe the release is completely empty. Hold them responsible for it. Attitudes will change.