Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Plugin] Live stream canvas via webrtc #976

Merged
merged 72 commits into from
Sep 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
42009f3
inline stylesheets when loaded
Juice10 May 31, 2022
b9ddfae
set empty link elements to loaded by default
Juice10 Jun 1, 2022
4c10804
Clean up stylesheet manager
Juice10 Jun 1, 2022
e9f8e06
Remove attribute mutation code
Juice10 Jun 2, 2022
c11e052
Update packages/rrweb/test/record.test.ts
Juice10 Jun 7, 2022
244a26a
Update packages/rrweb/test/record.test.ts
Juice10 Jun 7, 2022
1edb1ea
Update packages/rrweb/test/record.test.ts
Juice10 Jun 7, 2022
45bafbd
Merge branch 'master' of https://github.com/rrweb-io/rrweb into seria…
Juice10 Jun 7, 2022
401c503
Update packages/rrweb/scripts/repl.js
Juice10 Jun 7, 2022
2ec406b
Update packages/rrweb/test/record.test.ts
Juice10 Jun 7, 2022
2e8492e
Update packages/rrweb/src/record/index.ts
Juice10 Jun 7, 2022
217bd7c
Add todo
Juice10 Jun 7, 2022
c0371f0
Move require out of time sensitive assert
Juice10 Jun 7, 2022
fd38d43
Merge branch 'serialize-stylesheet-contents' of https://github.com/rr…
Juice10 Jun 7, 2022
dee14a6
Add waitForRAF, its more reliable than waitForTimeout
Juice10 Jun 7, 2022
1965152
Remove flaky tests
Juice10 Jun 7, 2022
8ff7bc6
Add recording stylesheets in iframes
Juice10 Jun 7, 2022
d5a83be
Remove variability from flaky test
Juice10 Jun 7, 2022
7e3a1a8
Make test more robust
Juice10 Jun 7, 2022
8c7a38f
Fix naming
Juice10 Jun 27, 2022
6878711
Merge branch 'master' of https://github.com/rrweb-io/rrweb into seria…
Juice10 Jun 27, 2022
d2cb411
Add test cases for inlineImages
Juice10 Jun 29, 2022
18c4475
Add test cases for inlineImages
Juice10 Jun 29, 2022
9efecfe
Merge branch 'inline-image-test-cases' of https://github.com/rrweb-io…
Juice10 Jun 30, 2022
da245bd
Record iframe mutations cross page
Juice10 Jun 30, 2022
be618d3
Test: should record images inside iframe with blob url after iframe w…
Juice10 Jun 30, 2022
9fc54b4
Merge branch 'master' of https://github.com/rrweb-io/rrweb into inlin…
Juice10 Jul 1, 2022
d63a529
Handle negative ids in rrdom correctly
Juice10 Jul 1, 2022
83102f0
Update packages/rrdom/src/diff.ts
Juice10 Jul 25, 2022
f872f76
Merge branch 'master' into rrdom-negative-ids
Juice10 Jul 25, 2022
3cabbf7
Start unserialized nodes at -2
Juice10 Jul 25, 2022
3a8ce08
Set unserialized id starting number at -2
Juice10 Jul 25, 2022
14179d6
Remove duplication
Juice10 Jul 25, 2022
7317ac3
Use turbo instead of lerna
Juice10 Jul 26, 2022
7cbb307
Skip benchmark as it is unreliable when executed in parallel
Juice10 Jul 26, 2022
eed8174
Strip port number from serialization, it can vary
Juice10 Jul 26, 2022
b43fafb
Add settimeout to virtual dom test
Juice10 Jul 26, 2022
c093928
Remove console.log and refactor blob:url serialization
Juice10 Jul 26, 2022
23c8711
Include references in tsconfig to indicate which monorepo packages ar…
Juice10 Jul 26, 2022
5006ed4
Add stream setup
Juice10 Jul 26, 2022
9291a05
Migrate project to es module
Juice10 Jul 27, 2022
f66b1c2
Add reference to rrweb from rrdom
Juice10 Jul 28, 2022
9599257
Move jest config to ESM
Juice10 Jul 28, 2022
bb1c5c5
Setup basic WebRTC canvas streaming
Juice10 Jul 28, 2022
03b6b28
Cleanup and refactor WebRTC streaming
Juice10 Jul 28, 2022
48a4ad3
Remove ? which isn't propper javascript
Juice10 Aug 22, 2022
05300fd
Yarn lock
Juice10 Aug 22, 2022
9be9176
Remove webrtc code from rrweb
Juice10 Aug 25, 2022
9925f07
Add plugin hooks
Juice10 Aug 25, 2022
7b768c7
Expose plugins with server
Juice10 Aug 25, 2022
678bd60
Use unminified version for tests
Juice10 Aug 25, 2022
9df08ea
Don't include simple-peer in rrweb main project
Juice10 Aug 25, 2022
e913c03
Add canvas webrtc plugin
Juice10 Aug 25, 2022
17164b2
Merge branch 'master' into live-stream
Juice10 Aug 25, 2022
22a5c1d
ignore tsconfig.tsbuildinfo
Juice10 Aug 25, 2022
b2065d6
Cleanup unused code
Juice10 Aug 25, 2022
41e3775
type definition files are no longer committed
Juice10 Aug 25, 2022
5ea10d7
Devtools off by default
Juice10 Aug 25, 2022
0d807e0
Extract .css into its own file
Juice10 Aug 25, 2022
43a73ed
Refactor plugin apis and fix multi canvas streaming support
Juice10 Aug 26, 2022
da14368
Add readme to rrweb canvas webrtc plugin
Juice10 Aug 26, 2022
9d950fa
Reference canvas-webrtc plugin in documentation
Juice10 Aug 26, 2022
39b6d22
Forbidden non-null assertion
Juice10 Aug 26, 2022
bc0bf13
Remove linting of each project, yarn lint:report will do this
Juice10 Aug 26, 2022
29964f5
Remove test code
Juice10 Aug 26, 2022
28c4744
Cut down line length
Juice10 Aug 26, 2022
fb0be78
Merge branch 'master' into live-stream
YunFeng0817 Sep 13, 2022
2919a20
fix CI failure and improve the zh_CN doc
YunFeng0817 Sep 13, 2022
c65bef0
Update packages/rrweb/src/plugins/canvas-webrtc/replay/index.ts
Juice10 Sep 14, 2022
0fbecb9
Cleaner styling of replay
Juice10 Sep 14, 2022
1e83835
Clean up stream.js based on @Mark-Fenng's feedback
Juice10 Sep 14, 2022
20f5f62
Remove duplicate send
Juice10 Sep 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
.idea
node_modules
package-lock.json
tsconfig.tsbuildinfo

temp

Expand Down
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ install:
script:
- yarn build:all
- yarn turbo run check-types
- xvfb-run --server-args="-screen 0 1920x1080x24" yarn lerna run test
- xvfb-run --server-args="-screen 0 1920x1080x24" yarn test
6 changes: 5 additions & 1 deletion docs/recipes/canvas.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Canvas

Canvas is a special HTML element, which will not be recorded by rrweb by default. There are some options for recording and replaying Canvas.
Canvas is a special HTML element, and will not be recorded by rrweb by default.
There are some options for recording and replaying Canvas.

Enable recording Canvas:

Expand Down Expand Up @@ -33,3 +34,6 @@ replayer.play();
```

**Enable replaying Canvas will remove the sandbox, which may cause a potential security issue.**

Alternatively you can stream canvas elements via webrtc with the canvas-webrtc plugin.
For more information see [canvas-webrtc documentation](../../packages/rrweb/src/plugins/canvas-webrtc/Readme.md)
3 changes: 3 additions & 0 deletions docs/recipes/canvas.zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,6 @@ replayer.play();
```

**回放 Canvas 将会关闭沙盒策略,导致一定风险**。

另外,您可以使用 canvas-webrtc 插件通过 WEBRTC 流式传输 Canvas 元素。
有关更多信息,请参考[canvas-webrtc 文档](../../packages/rrweb/src/plugins/canvas-webrtc/readme.md)
6 changes: 4 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"packages/*"
],
"devDependencies": {
"@monorepo-utils/workspaces-to-typescript-project-references": "^2.8.2",
"@typescript-eslint/eslint-plugin": "^5.25.0",
"@typescript-eslint/parser": "^5.25.0",
"concurrently": "^7.1.0",
Expand All @@ -32,11 +33,12 @@
},
"scripts": {
"lerna": "lerna",
"build:all": "yarn turbo run prepublish",
"test": "yarn turbo run test",
"build:all": "yarn run concurrently --success=all -r -m=1 'yarn workspaces-to-typescript-project-references' 'yarn turbo run prepublish'",
"test": "yarn run concurrently --success=all -r -m=1 'yarn workspaces-to-typescript-project-references --check' 'yarn turbo run test'",
"test:watch": "yarn turbo run test:watch",
"dev": "yarn turbo run dev",
"repl": "cd packages/rrweb && npm run repl",
"live-stream": "cd packages/rrweb && yarn live-stream",
"lint": "yarn run concurrently --success=all -r -m=1 'yarn run markdownlint docs' 'yarn eslint packages/*/src --ext .ts,.tsx,.js,.jsx,.svelte'",
"lint:report": "yarn eslint --output-file eslint_report.json --format json packages/*/src --ext .ts,.tsx,.js,.jsx"
},
Expand Down
11 changes: 10 additions & 1 deletion packages/rrdom-nodejs/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"compilerOptions": {
"composite": true,
"target": "ES6",
"module": "commonjs",
"noImplicitAny": true,
Expand All @@ -16,5 +17,13 @@
},
"compileOnSave": true,
"exclude": ["test"],
"include": ["src", "test.d.ts", "../rrweb/src/record/workers/workers.d.ts"]
"include": ["src", "test.d.ts", "../rrweb/src/record/workers/workers.d.ts"],
"references": [
{
"path": "../rrdom"
},
{
"path": "../rrweb-snapshot"
}
]
}
1 change: 1 addition & 0 deletions packages/rrdom/test/virtual-dom.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ function walk(node, mirror, blankSpace) {
`;

describe('RRDocument for browser environment', () => {
jest.setTimeout(60_000);
let mirror: Mirror;
beforeEach(() => {
mirror = new Mirror();
Expand Down
6 changes: 6 additions & 0 deletions packages/rrdom/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"compilerOptions": {
"composite": true,
"target": "ES6",
"module": "commonjs",
"noImplicitAny": true,
Expand All @@ -14,6 +15,11 @@
"declaration": true,
"importsNotUsedAsValues": "error"
},
"references": [
{
"path": "../rrweb-snapshot"
}
],
"compileOnSave": true,
"exclude": ["test"],
"include": ["src", "../rrweb/src/record/workers/workers.d.ts"]
Expand Down
8 changes: 8 additions & 0 deletions packages/rrweb-player/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,13 @@
"__sapper__/*",
"public/*",
"../rrweb/src/record/workers/workers.d.ts"
],
"compilerOptions": {
"composite": true
},
"references": [
{
"path": "../rrweb"
}
]
}
6 changes: 3 additions & 3 deletions packages/rrweb-snapshot/src/rebuild.ts
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ export function buildNodeWithSN(
mirror: Mirror;
skipChild?: boolean;
hackCss: boolean;
afterAppend?: (n: Node) => unknown;
afterAppend?: (n: Node, id: number) => unknown;
cache: BuildCache;
},
): Node | null {
Expand Down Expand Up @@ -398,7 +398,7 @@ export function buildNodeWithSN(
node.appendChild(childNode);
}
if (afterAppend) {
afterAppend(childNode);
afterAppend(childNode, childN.id);
}
}
}
Expand Down Expand Up @@ -449,7 +449,7 @@ function rebuild(
doc: Document;
onVisit?: (node: Node) => unknown;
hackCss?: boolean;
afterAppend?: (n: Node) => unknown;
afterAppend?: (n: Node, id: number) => unknown;
cache: BuildCache;
mirror: Mirror;
},
Expand Down
3 changes: 2 additions & 1 deletion packages/rrweb-snapshot/test/rebuild.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ describe('add hover class to hover selector related rules', function () {
expect(addHoverClass(cssText, cache)).toEqual(cssText);
});

it('benchmark', () => {
// this benchmark is unreliable when run in parallel with other tests
it.skip('benchmark', () => {
const cssText = fs.readFileSync(
path.resolve(__dirname, './css/benchmark.css'),
'utf8',
Expand Down
4 changes: 3 additions & 1 deletion packages/rrweb-snapshot/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"compilerOptions": {
"composite": true,
"module": "ESNext",
"moduleResolution": "Node",
"noImplicitAny": true,
Expand All @@ -11,5 +12,6 @@
"lib": ["es6", "dom"]
},
"exclude": ["test"],
"include": ["src"]
"include": ["src"],
"references": []
}
1 change: 1 addition & 0 deletions packages/rrweb/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
node_modules
package-lock.json
# yarn.lock
tsconfig.tsbuildinfo
build
dist
es
Expand Down
2 changes: 1 addition & 1 deletion packages/rrweb/jest.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/** @type {import('ts-jest/dist/types').InitialOptionsTsJest} */
module.exports = {
export default {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/**.test.ts'],
Expand Down
10 changes: 7 additions & 3 deletions packages/rrweb/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"test:headless": "PUPPETEER_HEADLESS=true npm run test",
"test:watch": "PUPPETEER_HEADLESS=true npm run test -- --watch",
"repl": "npm run bundle:browser && node scripts/repl.js",
"live-stream": "yarn bundle:browser && node scripts/stream.js",
"dev": "yarn bundle:browser --watch",
"bundle:browser": "cross-env BROWSER_ONLY=true rollup --config",
"bundle": "rollup --config",
Expand All @@ -18,6 +19,7 @@
"lint": "yarn eslint src",
"benchmark": "jest test/benchmark"
},
"type": "module",
"repository": {
"type": "git",
"url": "git+ssh://[email protected]/rrweb-io/rrweb.git"
Expand Down Expand Up @@ -45,7 +47,8 @@
"devDependencies": {
"@rollup/plugin-node-resolve": "^13.1.3",
"@types/chai": "^4.1.6",
"@types/inquirer": "0.0.43",
"@types/dom-mediacapture-transform": "^0.1.3",
"@types/inquirer": "^8.2.1",
"@types/jest": "^27.4.1",
"@types/jest-image-snapshot": "^4.3.1",
"@types/node": "^17.0.21",
Expand All @@ -57,7 +60,7 @@
"fast-mhtml": "^1.1.9",
"identity-obj-proxy": "^3.0.0",
"ignore-styles": "^5.0.1",
"inquirer": "^6.2.1",
"inquirer": "^9.0.0",
"jest": "^27.5.1",
"jest-image-snapshot": "^4.5.1",
"jest-snapshot": "^23.6.0",
Expand All @@ -69,8 +72,9 @@
"rollup-plugin-rename-node-modules": "^1.3.1",
"rollup-plugin-typescript2": "^0.31.2",
"rollup-plugin-web-worker-loader": "^1.6.1",
"simple-peer-light": "^9.10.0",
"ts-jest": "^27.1.3",
"ts-node": "^10.7.0",
"ts-node": "^10.9.1",
"tslib": "^2.3.1",
"typescript": "^4.7.3"
},
Expand Down
40 changes: 23 additions & 17 deletions packages/rrweb/rollup.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,16 @@ const baseConfigs = [
name: 'rrwebConsoleRecord',
pathFn: toPluginPath('console', 'record'),
},
{
input: './src/plugins/canvas-webrtc/record/index.ts',
name: 'rrwebCanvasWebRTCRecord',
pathFn: toPluginPath('canvas-webrtc', 'record'),
},
{
input: './src/plugins/canvas-webrtc/replay/index.ts',
name: 'rrwebCanvasWebRTCReplay',
pathFn: toPluginPath('canvas-webrtc', 'replay'),
},
{
input: './src/plugins/console/replay/index.ts',
name: 'rrwebConsoleReplay',
Expand Down Expand Up @@ -121,7 +131,7 @@ function getPlugins(options = {}) {
minify,
}),
postcss({
extract: false,
extract: true,
inject: false,
minimize: minify,
sourceMap,
Expand Down Expand Up @@ -209,33 +219,29 @@ if (process.env.BROWSER_ONLY) {
name: 'rrwebConsoleRecord',
pathFn: toPluginPath('console', 'record'),
},
{
input: './src/plugins/canvas-webrtc/record/index.ts',
name: 'rrwebCanvasWebRTCRecord',
pathFn: toPluginPath('canvas-webrtc', 'record'),
},
{
input: './src/plugins/canvas-webrtc/replay/index.ts',
name: 'rrwebCanvasWebRTCReplay',
pathFn: toPluginPath('canvas-webrtc', 'replay'),
},
];

configs = [];

// browser record + replay, unminified (for profiling and performance testing)
configs.push({
input: './src/index.ts',
plugins: getPlugins(),
output: [
{
name: 'rrweb',
format: 'iife',
file: pkg.unpkg,
},
],
});

for (const c of browserOnlyBaseConfigs) {
configs.push({
input: c.input,
plugins: getPlugins({ sourceMap: true, minify: true }),
plugins: getPlugins(),
output: [
{
name: c.name,
format: 'iife',
file: toMinPath(c.pathFn(pkg.unpkg)),
sourcemap: true,
file: c.pathFn(pkg.unpkg),
},
],
});
Expand Down
22 changes: 13 additions & 9 deletions packages/rrweb/scripts/repl.js
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
/* eslint:disable: no-console */

const fs = require('fs');
const path = require('path');
const EventEmitter = require('events');
const inquirer = require('inquirer');
const puppeteer = require('puppeteer');
import * as path from 'path';
import * as fs from 'fs';
import { EventEmitter } from 'node:events';
import inquirer from 'inquirer';
import puppeteer from 'puppeteer';
import { fileURLToPath } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const emitter = new EventEmitter();

function getCode() {
const bundlePath = path.resolve(__dirname, '../dist/rrweb.min.js');
const bundlePath = path.resolve(__dirname, '../dist/rrweb.js');
return fs.readFileSync(bundlePath, 'utf8');
}

Expand Down Expand Up @@ -165,7 +169,7 @@ void (async () => {
}

await page.addStyleTag({
path: path.resolve(__dirname, '../dist/rrweb.min.css'),
path: path.resolve(__dirname, '../dist/rrweb.css'),
});
await page.evaluate(`${code}
const events = ${JSON.stringify(events)};
Expand Down Expand Up @@ -196,10 +200,10 @@ void (async () => {
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Record @${time}</title>
<link rel="stylesheet" href="../dist/rrweb.min.css" />
<link rel="stylesheet" href="../dist/rrweb.css" />
</head>
<body>
<script src="../dist/rrweb.min.js"></script>
<script src="../dist/rrweb.js"></script>
<script>
/*<!--*/
const events = ${JSON.stringify(events).replace(
Expand Down
Loading