Skip to content

Commit

Permalink
TrackerMIL optimizations (#7032)
Browse files Browse the repository at this point in the history
<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
This PR fixes some problems with TrackerMIL memory managment
- Free up some memory after we are done with tracking
- Stable memory consuption while tracking (no endlessly raising heap
size)
Open a task with quite large images 2k -> create a couple of tracks ->
track a couple of frames:

![tracking](https://github.com/opencv/cvat/assets/50956430/087910dd-a0c8-4007-a25c-6952743c2f36)



### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [ ] I submit my changes into the `develop` branch
- [ ] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
- [ ] I have updated the documentation accordingly
- [ ] I have added tests to cover my changes
- [ ] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [ ] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))

### License

- [ ] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.
  • Loading branch information
klakhov authored Oct 24, 2023
1 parent e894e97 commit c6682ff
Show file tree
Hide file tree
Showing 10 changed files with 67 additions and 54 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
### Fixed

- OpenCV.js memory leak with TrackerMIL
(<https://github.com/opencv/cvat/pull/7032>)
2 changes: 1 addition & 1 deletion cvat-ui/.eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ module.exports = {
ignorePatterns: [
'.eslintrc.js',
'webpack.config.js',
'src/assets/opencv_4.5.4_dev.js',
'src/assets/opencv*.js',
'node_modules/**',
'dist/**',
],
Expand Down
2 changes: 1 addition & 1 deletion cvat-ui/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "cvat-ui",
"version": "1.58.0",
"version": "1.58.1",
"description": "CVAT single-page application",
"main": "src/index.tsx",
"scripts": {
Expand Down
48 changes: 0 additions & 48 deletions cvat-ui/src/assets/opencv_4.5.4_dev.js

This file was deleted.

48 changes: 48 additions & 0 deletions cvat-ui/src/assets/opencv_4.8.0.js

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -207,8 +207,10 @@ class OpenCVControlComponent extends React.PureComponent<Props & DispatchToProps

public componentWillUnmount(): void {
const { canvasInstance } = this.props;
const { trackedShapes } = this.state;
canvasInstance.html().removeEventListener('canvas.interacted', this.interactionListener);
openCVWrapper.removeProgressCallback();
trackedShapes.forEach((shape: TrackedShape) => shape.trackerModel.delete());
}

private interactionListener = async (e: Event): Promise<void> => {
Expand Down
1 change: 1 addition & 0 deletions cvat-ui/src/utils/opencv-wrapper/opencv-interfaces.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ export interface TrackerModel {
init: (src: ImageData, points: number[]) => void;
reinit: (points: number[]) => void;
update: (src: ImageData) => TrackingResult;
delete: () => void;
}

export interface OpenCVTracker {
Expand Down
4 changes: 2 additions & 2 deletions cvat-ui/src/utils/opencv-wrapper/opencv-wrapper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ export class OpenCVWrapper {
}

private async inject(): Promise<void> {
const response = await fetch('/assets/opencv.js');
const response = await fetch('/assets/opencv_4.8.0.js');
if (response.status !== 200) {
throw new Error(`Response status ${response.status}. ${response.statusText}`);
}
Expand Down Expand Up @@ -101,7 +101,7 @@ export class OpenCVWrapper {

const global = window as any;

this.cv = await global.cv;
this.cv = global.cv;
}

public async initialize(onProgress: (percent: number) => void): Promise<void> {
Expand Down
6 changes: 6 additions & 0 deletions cvat-ui/src/utils/opencv-wrapper/tracker-mil.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ export default class TrackerMILImplementation implements TrackerMIL {
if (!this.imageData) {
throw Error('TrackerMIL needs to be initialized before re-initialization');
}
this.trackerMIL.delete();
this.trackerMIL = new this.cv.TrackerMIL();
this.init(this.imageData, points);
}

Expand All @@ -65,4 +67,8 @@ export default class TrackerMILImplementation implements TrackerMIL {
if (matImage) matImage.delete();
}
}

public delete(): void {
this.trackerMIL.delete();
}
}
4 changes: 2 additions & 2 deletions cvat-ui/webpack.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -210,8 +210,8 @@ module.exports = (env) => {
to : 'assets/[name][ext]',
},
{
from: 'src/assets/opencv*.js',
to : 'assets/opencv.js',
from: 'src/assets/opencv_4.8.0.js',
to : 'assets/opencv_4.8.0.js',
},
{
from: 'plugins/**/assets/*.(onnx|js)',
Expand Down

0 comments on commit c6682ff

Please sign in to comment.