you didn't mention version, but before 3.6 and the dirhash code,
anything over about 10k files in a directory starts getting slow.
after 100k it really becomes unreasonable.