m_cleanupImageBarriers in WrappedVulkan does not update its size

[JorenJoestar]

Add missing array resize for ImageBarriereSequence when replaying Vulkan captures, as ImageBarrierSequence::SetMaxQueueFamilyIndex does not update the arrays but just the static index.

Already wrote more informations in #3491, but will post them here as well!

Description

Vulkan crash when opening a capture, using a custom Vulkan renderer that has any queue family index > MaxQueueFamilyIndex.

The crash is happening in WrappedVulkan::ApplyInitialContents(), line 3708:

it->second.LockWrite()->ResetToOldState(m_cleanupImageBarriers, GetImageTransitionInfo());

Basically m_cleanupImageBarriers has the array that is not updated if MaxQueueFamilyIndex is updated.

I've downloaded latest source code this morning, and by debugging I found out that m_cleanupImageBarriers is not resized if the QueueFamilyIndex is > MaxFamilyIndex in the method bool WrappedVulkan::Serialise_vkCreateDevice in vk_device_funcs.cpp.

In line 4059 on:

for(uint32_t i = 0; i < createInfo.queueCreateInfoCount; i++)
{
uint32_t qidx = createInfo.pQueueCreateInfos[i].queueFamilyIndex;
m_ExternalQueues.resize(RDCMAX((uint32_t)m_ExternalQueues.size(), qidx + 1));

// This does NOT resize m_cleanupImageBarriers nor m_setupImageBarriers!!!
ImageBarrierSequence::SetMaxQueueFamilyIndex(qidx);

just adding this (with better interface, probably an utility method in ImageBarrierSequence:

for(uint32_t mbc = 0; mbc < ImageBarrierSequence::MAX_BATCH_COUNT; mbc++)
{
m_setupImageBarriers.batches[mbc].resize_for_index(qidx);
m_cleanupImageBarriers.batches[mbc].resize_for_index(qidx);
}

BEFORE the ImageBarrierSequence::SetMaxQueueFamilyIndex(qidx); call it fixes the problem.

Using a Vulkan renderer that has a QueueFamily > 4 and do an image transition with that.
I am using that with a transfer queue that has FamilyIndex = 5, thus the crash.
Capture a frame, and when opening RenderDoc will crash.

[baldurk]

This change looks good, I can merge it once CI is green.

For future note, there is no point in creating an issue if you are also immediately going to create a PR fixing the issue. It only creates more noise and extra work without adding any value.

[JorenJoestar]

You are totally right Baldur, next time I will just open a pull request! :)

Change is ready and most of all...an heartfelt thank you for creating RenderDoc.
Years I am using it, and it is quintessential to my rendering work, at home and at work!

[baldurk]

Thanks for that, and I'm glad that RenderDoc is useful to you.

As in the CONTRIBUTING documentation, please squash your changes to ensure you do not have any merge commits and that every commit individually is correctly formatted and compiles, to avoid any commits that only do reformatting or compile fixes.