[Dataflow Streaming] Reduce contention on work submission

[arunpandianp]

The monitor guarding BoundedQueueExecutor's counters was locked by both the work submitting threads and the harness threads (300 of them). The contention can slow down the work submission loop. This change reduces the contention by collapsing counter updates from the harness threads and letting one thread to lock the monitor and update the counters.

Example wallz profile showing the stacks locking the monitor

#33578

[arunpandianp]

R: @scwhittle

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

[arunpandianp]

Run Java PreCommit

[scwhittle]

wondering if this will end up creating more threads in the pool if we have a bunch of threads blocking on this but the count is decremented by another thread and new work is admitted.

maybe we could improve to reduce the # of threads waiting on synchronized block using an atomic? Seems like something like the following would make it so there is at most one thread waiting for the synchronized block

addToQueue
bool coalesce = !atomic_coalescer.getandset(true);
if (!coalesce) return;
{
sync single
atomic_coalescer.set(false);
poll queue, return if empty
monitor {

}
}