Natvis for C++/WinRT doesn't work

[vmilea]

Describe the bug

Natvis for C++/WinRT doesn't work because the *.winmd files aren't deployed alonside the process *.exe. This affects both SDK and user authored types.

Steps to reproduce the bug

1. Create a solution from Blank App, Packaged (WinUI 3 in Desktop) template.
2. Set a breakpoint in App::OnLaunched()
3. Debug and pause at breakpoint.
4. Try to inspect the LaunchActivatedEventArgs argument.

Expected behavior

The debug visualizer shows object properties based on metadata.

Screenshots

Notice the output message. To enable natvis diagnostics see documentation.

NuGet package version

1.1.3

Packaging type

Packaged (MSIX)

Windows version

Windows 11 version 21H2 (22000)

IDE

Visual Studio 2022

Additional context

A workaround is to manually copy the *.winmd files from Microsoft.WindowsAppSDK to the executable folder.

[HO-COOH]

This is 4 months old, and still no reply and nobody else encountered it? Btw, I think it could be solved by modifing the project template to add a custom post build step, that copies the whatever winmd files required to the executable folder

[mgradwohl]

I'm hitting it often.

[torleifat]

Yeah, I encounter this fairly often as well. The workaround mentioned in the report works for some of the stuff, but where I struggle is collections with custom types, or just collections in general.

[ackh]

Enabling diagnostics shows that nativs only searches within the AppX of a packaged app for custom winmd files but not within the folder where the custom winmd files are actually created, i.e. one folder higher.

This does not make sense because, as far as I understand it, the winmd files should not be packaged and shipped as part of the app. They are required at compile time, not at runtime and thus should not be shipped for the same reasons that pdb files aren't shipped. From that perspective, adding a post build step to copy winmd files into the AppX folder isn't a solution.

Therefore, it is save to say that the debugging experience for custom types is broken. Why this does not receive more attention is beyond me. Especially because it seems to be fixable by just changing the search paths for winmd files.

[Scottj1s]

@ackh You're correct, we don't want to load up the AppX folder with debugging support files like pdbs and winmds, which aren't required at runtime. In my experiments, I don't see any Windows App SDK winmd files being copied to the parent of the AppX folder, so I don't think we can rely on that. Fortunately, while the diagnostics don't indicate it, the %TEMP% folder is also searched as a cache location. So, a short-term workaround is to add a custom target that copies all the Windows App SDK metadata to the %TEMP% folder. I've confirmed this works with the LaunchActivatedEventArgs repro, and it doesn't impact the AppX folder.

directory.build.targets:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <Target Name="CopyWinAppSdkMetadataToTempFolder" BeforeTargets="PrepareForRun">
        <ItemGroup>
            <MetadataFiles Include="$(_MuxWinmdDir)*.winmd" />
        </ItemGroup>
        <Message Importance="High" Text="Copying Windows App SDK metadata files to TEMP to support C++/WinRT Natvis debugging visualizations" />
        <Copy SkipUnchangedFiles="True" OverwriteReadOnlyFiles="True" SourceFiles="@(MetadataFiles)" DestinationFolder="$(TEMP)" />
    </Target>
</Project>

[torleifat]

@ackh You're correct, we don't want to load up the AppX folder with debugging support files like pdbs and winmds, which aren't required at runtime. In my experiments, I don't see any Windows App SDK winmd files being copied to the parent of the AppX folder, so I don't think we can rely on that. Fortunately, while the diagnostics don't indicate it, the %TEMP% folder is also searched as a cache location. So, a short-term workaround is to add a custom target that copies all the Windows App SDK metadata to the %TEMP% folder. I've confirmed this works with the LaunchActivatedEventArgs repro, and it doesn't impact the AppX folder.

directory.build.targets:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <Target Name="CopyWinAppSdkMetadataToTempFolder" BeforeTargets="PrepareForRun">
        <ItemGroup>
            <MetadataFiles Include="$(_MuxWinmdDir)*.winmd" />
        </ItemGroup>
        <Message Importance="High" Text="Copying Windows App SDK metdata files to TEMP to support C++/WinRT Natvis debugging visualizations" />
        <Copy SkipUnchangedFiles="True" OverwriteReadOnlyFiles="True" SourceFiles="@(MetadataFiles)" DestinationFolder="$(TEMP)" />
    </Target>
</Project>

This works for me as well. However I am wondering, are the winmd files for collections located elsewhere?

For instance if I do:
auto numbers = single_threaded_observable_vector<int>({ 1, 2, 3, 4, 5 });
I will still get
Natvis C++/WinRT: Could not find metadata for Windows.Foundation.Collections.IObservableVector`1<Int32>

[Scottj1s]

@torleifat Unfortunately, debug visualization does not support generics (collections). The work to implement this is quite involved, so I can't give any estimate if/when it will be done.

[Scottj1s]

@DefaultRyan any thoughts on supporting generics in the debug visualizer?

[DefaultRyan]

@DefaultRyan any thoughts on supporting generics in the debug visualizer?

Same thoughts as Kenny stated in the linked issue. It's not really a technical barrier, but a time+resources barrier. Even more so lately. We don't have any funding allocated for a feature like this right now. At least, not without some sort of signal that this is an impactful issue for lots of customers.

Could make a fun spare time hack project, and cppwinrt regularly accepts outside contributions.

[torleifat]

@Scottj1s @DefaultRyan Thanks for the insight guys.

Can't speak for anyone but myself with regards to impact on customers, but it does make things a bit harder to debug at least when I am used to being able to inspect collections while debugging. But I am certainly sympathetic to time/resources being a constraint.

[ackh]

@Scottj1s @DefaultRyan

At least, not without some sort of signal that this is an impactful issue for lots of customers.

What strikes me as really odd about this statement is that WinUI and I assume large parts of the Windows App SDK is written in C++/WinRT, isn't it? If Microsoft internally writes that stuff in C++/WinRT, how would the lack of ability to inspect collections in the debugger not be a daily PITA for you guys?

It's just that I'm really having a hard time understanding why we customers need to point out the obvious need for this if you guys have more C++/WinRT code to debug than everybody else. I might be wrong here and Microsoft's code base was authored in C++/CX. At least that would explain the hesitance of releasing it as open source.

[Scottj1s]

@ackh That's a good insight. I've seen a few internal complaints, but nothing compelling. Given that the visualizer is restricted to properties, adding support for generics still wouldn't provide much. You'd see the Size of IMap(View) and IVector(View), but not much else. We'd need to find a new technique to iterate over collections, in order to observe elements as you can with C# client code.

[DefaultRyan]

@Scottj1s good callout there. C# has indexed properties, while the WinRT type system does not.

Thinking about how to solve this in the general case, my first instinct would be to add a custom attribute to some types to tell diagnostic and debug tooling to treat certain methods as quasi-"indexed properties". Lacking that, one could special case the commonly-used types we know about: Windows.Foundation.Collections.* and Windows.UI.Xaml.Interop.IBindableVector (and friends).

The vector-like collections shouldn't be too hard - Size is already a property, and we could iterate over the collection from 0 to Size-1. Map-like collections don't provide a getter for the set of keys, so I think we'd have to use its IIterable to obtain an IIterator and simply iterate over the collection. That should be fine, given that the semantics of IMap are fairly well understood. I'm not sure we'd want to provide generalized support for IIterable because their iterators could potentially behave like a C++ input_iterator and have side effects from reading/advancing (like only being able to read a value once).

[DefaultRyan]

Update! I got a few free days and was able to get basic generics visualization up and running in a branch: https://github.com/microsoft/cppwinrt/tree/feature/generic_natvis

This required synthesizing parsing generic type names, synthesizing type signatures for those generic types, plumbing in these more fully-fledged type signatures (over the existing coded_index<TypeDefOrRef> design), generating a C++-friendly type name for these properties. But I've successfully tested it with an instance of IKeyValuePair<int, IKeyValuePair<string, int>>.

There's still some work to be done:

• Check in a good torture test project with lots of wild property types to make sure the visualizer displays correctly or fails gracefully.
• Use the above test project to ensure good error handling. It's no automated test suite, but it's better than what we have today.
• Improve some of the type/signature logic so we can cache property info for generic types like we already do for non-generics.
• Add special logic for IIterator<T> to merge Current/HasCurrent into a single pseudo-property so we can prevent errors from calling Current when HasCurrent is false.
  • I need to figure out how to dispatch the two calls out to the program. I think I have a vauge idea of how to do this, and if that works, then it shouldn't be too hard to do some simple logic here to simply display the value of Current or display a placeholder "empty" string.
• Add even more special logic for IIterable<T> to call First as a pseudo-property (it's a method, not a property, but it takes no params and returns a single value, just like a property), iterate the collection, and somehow display the results as children of the container.
  • This one requires the most understanding of how to communicate with VS and tell it about children element structure, which is also my weakest area of expertise in the visualizer. If I can figure out the Current/HasCurrent trick above, the traversal is probably not much harder. But I want some advice on how to collect all of that up and somehow get the results back to the debugger, tell it how many children it has, etc. @Scottj1s who worked on the original visualizer would be hugely valuable for that effort, which is presumably where we'd hit the biggest customer value. Also sending a heads-up to @kennykerr in case he's got any contacts that could help me light up this last bit.

[ackh]

@DefaultRyan I'm actually excited about what you did here! Having support for generics would be really helpful when debugging. Now I'm actively hoping that this makes it into a C++/WinRT release eventually.

[Optimierungswerfer]

Any update on this? It has been a year now and we would still like to see this feature before we can further commit to using WinRT in our product.

[DefaultRyan]

Hey all, I've finally been able to carve out enough time to get the fundamentals into a PR - microsoft/cppwinrt#1472

This allows visualizing properties, and has the aforementioned heavy lifting of type lookup, signature synthesis, and property retrieval.

Does not yet have the fakery required for iterating collections, but I've got that provisionally working in a prototype state that I'm currently working to clean up for production. But first, things first, I'm going to get the foundation committed.