We are having an aggregated test report task specified along this lines:

task aggregatedTestReport(type: TestReport) {
testResultDirs = subprojects*.tasks*.withType(Test).flatten().binResultsDir
destinationDir = file("$buildDir/reports/tests")
allprojects.tasks*.withType(Test).flatten()*.finalizedBy it
}

Typically it works fine, except one very specific case:
1. gradle clean
2. gradle aggregatedTestReport // passes
3. gradle aggregatedTestReport // fails!

What happens is that the first time the binary test results is empty and the constructor in TestOutputStore.Reader#Reader() creates an empty 'output.bin' file. The second time arround, it sees that the outputs file exists and takes a different branch, throwing exception because the index file is missing.

If we snip the irrelevant code, it looks like this:

public Reader() {
File indexFile = getIndexFile();
File outputsFile = getOutputsFile();

if (outputsFile.exists()) {
if (!indexFile.exists()) {
throw new IllegalStateException(String.format("Test outputs data file '{}' exists but the index file '{}' does not", outputsFile, indexFile));
}
// snipped
} else { // no outputs file
if (indexFile.exists()) {
throw new IllegalStateException(String.format("Test outputs data file '{}' does not exist but the index file '{}' does", outputsFile, indexFile));
}

GFileUtils.touch(getOutputsFile());
index = new Index();
}

// snipped
}

The root cause is that we are running the report without having run the tests, but that seems to be intentionally supported behavior, so I would deem this as a bug.