createLocalTracks with video device with exact id, exact constraint is not maintained on re-enabling camera

[mdanilakis]

Describe the bug

const video = {
  deviceId: { exact: "deviceId" },
};

const audio = { ... };

const tracks = await createLocalTracks({ video, audio });

Constraining the deviceId with exact on Track creation.

Expectation: internally getUserMedia is called with exact the first time, and should be called again on restart (e.g. unmute of camera).

What happens: internally getUserMedia is called with exact the first time, but on unmute the exact constraint is removed resulting in selecting a different video stream (from a different device) on unmute (tested in Chrome).

Where I think the issue is: https://github.com/livekit/client-sdk-js/pull/773/files

Why is the browser returning a different stream given it actually returned it correctly the first time: I don't know (this is where the whole getUserMedia madness starts).

Reproduction

1. Create LocalTrack(s) using:

const video = {
  deviceId: { exact: "deviceId" },
};

const audio = { ... };

const tracks = await createLocalTracks({ video, audio });

2. Verify correct camera feed
3. Switch off camera e.g. localParticipat.setCameraEnabled(false)
4. Switch on camera
5. Camera feed is from a different device from the one specified in exact

Note: it happens under specific circumstances (hard to reproduce on some clients, reproducible with others - e.g. with a different set of devices).

Logs

No response

System Info

Mac OS, Chrome 130.0.6723.119, LiveKit Client 2.1.5

Severity

annoyance

Additional Information

No response

[iiiyx]

I have (I think) a better reproduction scenario, 100% reproducible:

1. Publish video using NOT a default camera (the default one is usually the first one in the list).

const videoTrack = await createLocalVideoTrack({
  deviceId: { exact: YOUR_SELECTED_CAMERA_ID },
});
const videoTrackPublication = await room!.localParticipant.publishTrack(
  videoTrack,
  {
    name: YOUR_SELECTED_CAMERA_ID // set the publication name for convenience
  },
);

2. Mute the camera using

const videoTrackPublication = room.localParticipant.getTrackPublicationByName(YOUR_SELECTED_CAMERA_ID);
videoTrackPublication.mute();

3. Unmute it back

const videoTrackPublication = room.localParticipant.getTrackPublicationByName(YOUR_SELECTED_CAMERA_ID);
videoTrackPublication.unmute();

Expected result: the selected camera is unmuted

Actual result: the default camera is unmuted

The issue comes from here:

client-sdk-js/src/room/track/LocalVideoTrack.ts

Line 166 in 780f98a

await this.restartTrack();

client-sdk-js/src/room/track/LocalVideoTrack.ts

Line 244 in 780f98a

async restartTrack(options?: VideoCaptureOptions) {

The constraints are vanished and the browser starts capturing the default video input device.

This is a major issue for users who use the second camera and for the apps that use all available video input devices (e.g., medical apps).

[lukasIO]

thanks for the report, could you provide some debug logs?
The exact parameter is not held on to after restarts currently, but the deviceId constraint as ideal should persist. This is due to the fact that we have to deal with scenarios in which the selected device got unplugged while muted.

[lukasIO]

I gave this a shot in #1388, could you please test with that branch and see if it resolves the issue for you?