Rerun Benchmarks with Latest Mooncake Version

[willtebbutt]

Per the description. I believe that there was one central performance issue in Mooncake which was causing the catastrophic performance on the benchmarks. If you could re-run with the latest version (0.4.65), that would be great!

[nsiccha]

That's great! Preliminary runs seem to confirm this, where Mooncake was often several orders of magnitude slower, it now seems to be much more comparable 👍 Will rerun and upload the updated benchmarks.

[nsiccha]

Should be live now here: https://nsiccha.github.io/StanBlocks.jl/performance.html#visualization
Mooncake sometimes beats Enzyme, but I think only whenever Enzyme encounters a performance bug.
I've also changed the way I benchmark. It's a bit untested, but as the current results agree with the previous results, I think it should be fine.
I think you also mentioned on Slack that Mooncake should eventually not need any "new" allocations if called repeatedly? Is that so? I don't think I have observed such a behavior yet, but I also haven't checked very thoroughly.

[willtebbutt]

Thanks for re-running this.

I think you also mentioned on Slack that Mooncake should eventually not need any "new" allocations if called repeatedly?

I think the allocations that we're continuing to see are a mixture of small performance bugs that I've yet to pin down completely, and limitations discussed here.

My inclination is to close this issue for now. I can open another when I think it's worth running the benchmarks again.

[nsiccha]

Sounds good to me 👍