Eh2406: I am working on cargos resolver, one of the things I don't have is realistic benchmarks. I would love to know which crates happen to have taken the longest amount of time to resolve. Or required the most backtracking.
Icefox: What do you mean by backtracing?
Eh2406: When it discovers that it can't use the newest version of a library. There is not way to get metrics on it (yet), but I could add one if you could make it available.
Adding timing info to this at least is not going to be that difficult, I think. The cargo metadata calls are in general relatively slow, so simple timing should be reasonably accurate, though the cargo_metadata crate has to parse the resulting JSON which will add some overhead.
But hey, that's just another number to measure.
Trivially done, results here: http://alopex.li/temp/cargo_metadata_times_20190318.csv
Imagine every possible disclaimer against crude benchmarks, they all apply. It's literally just the amount of time it took to run the
cargo_metadatalibrary call, and so likely to be I/O bound, have overhead from parsing the JSON output, etc, etc. Still, it's a place to start.
...I want to put this info in the results but I'm not awake enough to rearrange things nicely. Hmmm.
Eh2406 reported back that the slowest resolver times were not reproducible, I expect that they were outliers spending a lot of time on disk I/O or something. Would still like a note if we discover something consistently taking >10 seconds. The current setup is really optimized for throughput though, not timing analysis.