Grouping tests according to the iterations and purpose

I believe that organizing your test structure for modular testing is stupid, until you cover the majority of your project with isolated unit tests (not with the integration tests).

Most people group tests using the Java packages structure. Class foo.Bar is tested under the foo/BarTest file in the tests directory. This is good approach if you really test your classes with isolated unit test.

Something about 95% of my test coverage comes from the integration tests. I load minimal Spring context, minimal Hibernate session factory and so forth. I also try to treat my tests as my private (pre-QA) acceptance tests. In such case foo/BarTest hierarchy presented above sucks. It doesn't reflect what I'm testing since packages do not reflect the cross-concern nature of the integration tests.

My approach to is to create:

• directories for application version (version == development process iteration)
• many test classes with single (or few at most) testing methods describing particular testing scenario

For example:

• version_1.2/ShouldSendAndValidateDoc.java
• version_1.2/ShouldLinkUserDataToProcessMetadata.java
• version_1.3/ShouldCreateAndReadEntityOracle.java
• version_1.3/ShouldCreateAndReadEntityMySQL.java

BTW I use Should*.class pattern for localizing tests instead of *Test.class.

I believe that this is the only test structure maintainable for the long time. I associates tests with releases. This is quite logical since we write tests for current iterations, not for the system in general. Test path and name tell me everything about its purpose.