feat(gnodev): lazy loading & staging support #3237

gfanton · 2024-11-28T15:12:01Z

featuring:

Lazy reload: Super fast reload, only loading the needed packages for the current directory package.
Chain resolver: You can change the gnodev resolving process so it can resolve packages on-chain and configure a local fallback, allowing you to try your package against the chain before submitting it.
- This will probably be updated when feat(gnovm): packages loader #2932 & feat(gnomod)!: forbid require and find dependencies without it #3123 will be merged
Staging Mode: gnodev now starts from the current directory by default. The staging subcommand will reproduce previous behaviors by loading and monitoring the entire example folder.

Contributors' checklist...

Added new tests, or not needed, or not feasible
Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
Updated the official documentation or not needed
No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
Added references to related issues and PRs
Provided any useful hints for running manual tests

Signed-off-by: gfanton <[email protected]>

codecov · 2024-11-28T15:15:11Z

Codecov Report

Attention: Patch coverage is 74.11273% with 248 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
contribs/gnodev/pkg/proxy/path_interceptor.go	69.80%	44 Missing and 17 partials ⚠️
contribs/gnodev/pkg/packages/resolver.go	69.17%	38 Missing and 3 partials ⚠️
contribs/gnodev/pkg/packages/package.go	61.90%	22 Missing and 10 partials ⚠️
contribs/gnodev/pkg/packages/loader_glob.go	44.23%	20 Missing and 9 partials ⚠️
contribs/gnodev/pkg/dev/node.go	74.15%	20 Missing and 3 partials ⚠️
contribs/gnodev/pkg/packages/loader.go	70.96%	12 Missing and 6 partials ⚠️
contribs/gnodev/pkg/packages/resolver_remote.go	71.42%	13 Missing and 5 partials ⚠️
contribs/gnodev/pkg/packages/glob.go	89.14%	13 Missing and 1 partial ⚠️
contribs/gnodev/pkg/packages/resolver_local.go	70.58%	5 Missing ⚠️
contribs/gnodev/pkg/packages/resolver_mock.go	85.00%	3 Missing ⚠️
... and 2 more

📢 Thoughts on this report? Let us know!

Signed-off-by: gfanton <[email protected]>

Gno2D2 · 2024-11-30T08:12:44Z

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):

IGNORE the bot requirements for this PR (force green CI check)

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)

☑️ Contributor Actions:

Fix any issues flagged by automated checks.
Follow the Contributor Checklist to ensure your PR is ready for review.
- Add new tests, or document why they are unnecessary.
- Provide clear examples/screenshots, if necessary.
- Update documentation, if required.
- Ensure no breaking changes, or include BREAKING CHANGE notes.
- Link related issues/PRs, where applicable.

☑️ Reviewer Actions:

Complete manual checks for the PR, including the guidelines and additional checks if applicable.

📚 Resources:

Debug

Automated Checks
Maintainers must be able to edit this pull request (more info)
If
🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: gfanton/gno)
Then
🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request
Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)
If
🟢 Condition met
└── 🟢 On every pull request
Can be checked by

Any user with comment edit permission

Signed-off-by: gfanton <[email protected]>

leohhhn · 2025-01-27T19:30:15Z

contribs/gnodev/cmd/gnodev/setup_loader.go

+			logger.Info("guessing directory path", "path", path, "dir", dir)
+			paths = append(paths, path) // append local path
+		} else {
+			logger.Warn("invalid local path", "dir", dir)


I think this should be no package found at {pwd}, maybe even no Gno pacakge found at {pwd}.

On top of that, maybe another affirmation sentence that will tell the user that it's ok if there is no package, because lazy load kicks in for links that are requested from gnoweb/queries. Something like:

Loader ┃ W no Gno pacakge found at {pwd} Accounts ┃ I default address imported name=test1 addr=g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5 Node ┃ I packages paths=[] // rm this line in this case Node ┃ W no path to load, lazy resolving from {GNOROOT}

Or, another approach would to disable the loader output in this case; since querying the node and the given updates should be enough, maybe?
In cases where a package was found or a path was specified, keep it

leohhhn · 2025-01-27T19:47:06Z

contribs/gnodev/cmd/gnodev/setup_loader.go

+		resolver := packages.NewLocalResolver(path, dir)
+
+		if resolver.IsValid() {
+			logger.Info("guessing directory path", "path", path, "dir", dir)


Suggested change

logger.Info("guessing directory path", "path", path, "dir", dir)

logger.Info("getting package path", "pkgpath", path, "dir", dir)

Because in this case it gets it from a gno.mod file, no?

not necessarily; if no gno.mod is detected, it will still load the package using the directory as the package name.

leohhhn

I tested out most of the features manually, lgtm. Really looking forward to this being merged.

I left a few suggestions on the user messaging.

After this we can also set up a small CI job to create a preview of new example realms, or changes to gnoweb code that would use gnodev as a base.

Signed-off-by: gfanton <[email protected]>

gfanton · 2025-02-04T11:12:42Z

@leohhhn i've updated gnodev starting logic based on your feedback

Signed-off-by: gfanton <[email protected]>

wip: lazy loading & custom resolver support

16ee7d9

Signed-off-by: gfanton <[email protected]>

gfanton self-assigned this Nov 28, 2024

github-actions bot added 🧾 package/realm Tag used for new Realms or Packages. 📦 🤖 gnovm Issues or PRs gnovm related labels Nov 28, 2024

chore: fixup

fd13a5a

Signed-off-by: gfanton <[email protected]>

feat: add remote resolver & fixup

135f854

Signed-off-by: gfanton <[email protected]>

gfanton force-pushed the feat/dev/lazy-loader branch from 5b95a3e to 135f854 Compare November 30, 2024 08:13

gfanton added 8 commits December 2, 2024 15:20

wip: lazy loader

d258fd5

Signed-off-by: gfanton <[email protected]>

wip: test

3f92248

Signed-off-by: gfanton <[email protected]>

wip: staging

3e9c300

Signed-off-by: gfanton <[email protected]>

Merge remote-tracking branch 'origin/master' into feat/dev/lazy-loader

07c714a

Signed-off-by: gfanton <[email protected]>

wip: glob

0e5e5f0

Signed-off-by: gfanton <[email protected]>

feat: staging

54c8193

Signed-off-by: gfanton <[email protected]>

feat: gnodev app

ec51f78

Signed-off-by: gfanton <[email protected]>

chore: lint

d48ffd9

Signed-off-by: gfanton <[email protected]>

github-actions bot added the 📦 🌐 tendermint v2 Issues or PRs tm2 related label Dec 15, 2024

gfanton added 2 commits December 16, 2024 00:51

fix: web redirect

b193081

Signed-off-by: gfanton <[email protected]>

fix: minor fix

6ef3685

Signed-off-by: gfanton <[email protected]>

leohhhn self-requested a review December 16, 2024 10:02

leohhhn and others added 9 commits December 16, 2024 12:38

Merge branch 'master' into feat/dev/lazy-loader

08e9465

feat: add lazy proxy

0a1695c

Signed-off-by: gfanton <[email protected]>

Merge remote-tracking branch 'origin/master' into feat/dev/lazy-loader

e592fa9

Signed-off-by: gfanton <[email protected]>

fix: add missing files

2158026

Signed-off-by: gfanton <[email protected]>

fix: typo

d6315a1

Signed-off-by: gfanton <[email protected]>

fix: remote package load

8e931e2

Signed-off-by: gfanton <[email protected]>

fix: make custom resolvers override default resolver

a7a9a53

Signed-off-by: gfanton <[email protected]>

chore: cleanup errors

9f80c6f

Signed-off-by: gfanton <[email protected]>

Merge branch 'master' into feat/dev/lazy-loader

cc48898

fix: add content type ouput to gnoweb

fa4c5db

Signed-off-by: gfanton <[email protected]>

github-actions bot added the 📦 ⛰️ gno.land Issues or PRs gno.land package related label Jan 13, 2025

leohhhn and others added 19 commits January 20, 2025 01:33

Merge branch 'master' into feat/dev/lazy-loader

ab655b8

feat: add start time

afd6d89

Signed-off-by: gfanton <[email protected]>

chore: test & lint

8a4f615

Signed-off-by: gfanton <[email protected]>

Merge remote-tracking branch 'origin/master' into feat/dev/lazy-loader

20fde45

Signed-off-by: gfanton <[email protected]>

Merge remote-tracking branch 'origin/master' into feat/dev/lazy-loader

492bd82

feat: add packages test

981dc3e

Signed-off-by: gfanton <[email protected]>

chore: lint

eebb0bc

Signed-off-by: gfanton <[email protected]>

chore: missing file

3289bc3

Signed-off-by: gfanton <[email protected]>

chore: lint

d8ae1b7

Signed-off-by: gfanton <[email protected]>

fix: gnodev coverage test

da8c17b

Signed-off-by: gfanton <[email protected]>

Merge remote-tracking branch 'origin/master' into feat/dev/lazy-loader

4d48f3e

fix: packages modifiers

a622b89

Signed-off-by: gfanton <[email protected]>

feat: more more gnodev tests

c44b96f

Signed-off-by: gfanton <[email protected]>

chore: cleanup

c6dfac8

Signed-off-by: gfanton <[email protected]>

chore: fix test typo

2627a1d

Signed-off-by: gfanton <[email protected]>

feat: more test coverage

0c0b1ec

Signed-off-by: gfanton <[email protected]>

feat: improve proxy logic & tests

cb05e58

Signed-off-by: gfanton <[email protected]>

fix: empty paths

2b94133

Signed-off-by: gfanton <[email protected]>

Merge branch 'master' into feat/dev/lazy-loader

ec366b4

leohhhn reviewed Jan 27, 2025

View reviewed changes

leohhhn added 2 commits January 29, 2025 19:56

Merge branch 'master' into feat/dev/lazy-loader

f72fbbd

Merge branch 'master' into feat/dev/lazy-loader

46076f4

leohhhn approved these changes Jan 31, 2025

View reviewed changes

Kouteki removed the in focus label Feb 3, 2025

fix: improve gnodev starting logic:

2fdeda9

Signed-off-by: gfanton <[email protected]>

fix: typo

697e937

Signed-off-by: gfanton <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gnodev): lazy loading & staging support #3237

feat(gnodev): lazy loading & staging support #3237

gfanton commented Nov 28, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading

Gno2D2 commented Nov 30, 2024 •

edited

Loading

✅ Automated Checks (for Contributors):

☑️ Contributor Actions:

☑️ Reviewer Actions:

📚 Resources:

If

Then

If

Can be checked by

leohhhn Jan 27, 2025 •

edited

Loading

leohhhn Jan 27, 2025 •

edited

Loading

leohhhn Jan 27, 2025

gfanton Feb 4, 2025

leohhhn left a comment •

edited

Loading

gfanton commented Feb 4, 2025

	logger.Info("guessing directory path", "path", path, "dir", dir)
	logger.Info("getting package path", "pkgpath", path, "dir", dir)

feat(gnodev): lazy loading & staging support #3237

Are you sure you want to change the base?

feat(gnodev): lazy loading & staging support #3237

Conversation

gfanton commented Nov 28, 2024 • edited Loading

codecov bot commented Nov 28, 2024 • edited Loading

Codecov Report

Gno2D2 commented Nov 30, 2024 • edited Loading

🛠 PR Checks Summary

Manual Checks (for Reviewers):

✅ Automated Checks (for Contributors):

☑️ Contributor Actions:

☑️ Reviewer Actions:

📚 Resources:

If

Then

If

Can be checked by

leohhhn Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

leohhhn Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

leohhhn Jan 27, 2025

Choose a reason for hiding this comment

gfanton Feb 4, 2025

Choose a reason for hiding this comment

leohhhn left a comment • edited Loading

Choose a reason for hiding this comment

gfanton commented Feb 4, 2025

gfanton commented Nov 28, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading

Gno2D2 commented Nov 30, 2024 •

edited

Loading

leohhhn Jan 27, 2025 •

edited

Loading

leohhhn Jan 27, 2025 •

edited

Loading

leohhhn left a comment •

edited

Loading