[GRADLE-2546] directory scanning is inefficient when all include patterns share a common prefix Created: 04/Nov/12 Updated: 10/Feb/13 Resolved: 04/Feb/13
|Comment by Adam Murdoch [ 04/Nov/12 ]|
For example, given includes = [a/b/c/*/d] and base dir = /base, currently DirectoryFileTree does the following:
1. List the children of /base
This can be quite slow if one of these directories has a large number of children (eg when looking for an artifact in the artifact file store). Instead, in this instance, DirectoryFileTree should skip directly to step 6. above.
Also, PatternSet is currently implemented in Groovy. It should be ported to Java.
|Comment by Luke Daley [ 06/Nov/12 ]|
This is a pretty big change, that will require some heavy redesigning.
I'd be a bit uncomfortable about putting this in 1.3 at this late stage.
|Comment by Peter Niederwieser [ 23/Nov/12 ]|
There's now SingleIncludePatternFileTree, which provides a more efficient way to walk a directory hierarchy than DirectoryFileTree. Question is where to go from here. Some options:
1. Leave DirectoryFileTree as-is and switch selected clients (e.g. PatternBasedLocallyAvailableResourceFinder) to SingleIncludePatternFileTree.
I've already spiked 1. and 4., but don't have conclusive benchmarks.
|Comment by Adam Murdoch [ 05/Dec/12 ]|
For now, the cases that we're interested in are improving PathKeyFileStore and PatternBasedLocallyAvailableResourceFinder. So option 1 is certainly a good next step.
Option 4 would be worth doing. PathKeyFileStore and PatternBasedLocallyAvailableResourceFinder don't care about the guarantee, just as long as all the matching files are visited at some point. They also don't care about visiting the directories, so that offers some other potential optimisations. The same is true for FileCollection.getFiles() and a few other methods.
So, I think we should do something like this:
p.s. can you update the Javadocs to use "prefix order" instead of "breadth-wise order"?