[Vyper support] Add `VyperCheckedContract` in `lib-sourcify` and the new endpoint `/verify/vyper` in server #1760

marcocastignoli · 2024-11-28T09:55:08Z

This PR currently:

Implements VyperCheckedContract and SolidityCheckedContract. VyperCheckedContract generates a metadata file for Vyper contracts.
Implements useVyperCompiler, a function similar to useCompiler which handles dynamic download for Vyper versions (need some more work for the platform architectures), and vyper compilation.
verifyDeployed now takes as parameter CheckedContract, an abstract class extended by both VyperCheckedContract and SolidityCheckedContract. It skips metadata-related functions for VyperCheckedContract.
Stores match in all StorageServices (I still haven't checked everything in detail)

TODO:

- Review what's stored in the database
- Create a common folder for the compilers code used by both lib-sourcify and server (idk if it fits in this new feature, imo yes because it's a lot of duplicated code now that we have also vyper stuff)
- Rewrite platform architecture management (vyper versions for x86, arm, ...)
- Add documentation
- Handle vyper transformations (I still have to understand what we can do about it)
- Add more tests, and finalize lib-sourcify test
- Finalize /verify/vyper API errors + openapi responses
- Documentation, updating readme files

manuelwedler

I found it quite impressive how minimal the changes to lib-sourcify are for this feature.

Some general feedback:

Why do we need a metadata file mock for VyperCheckContract? I guess this is due to the verifyDeployed method which expects a metadata. This could be documented better in some way. Is it possible to get rid of it when we refactor the verification flow?
lib-sourcify README needs to be updated due to renaming of CheckedContract.

I also tried to verify a vyper contract on a local server instance. Unfortunately, it didn't work due to Vyper path not found. The server logs show that vyper was downloaded. But I cannot find the binary on my machine.

2024-12-03T11:19:24.889Z [debug] [Server] Downloaded vyper - version=0.4.0 | vyperPath=/home/manuel/.vyper-bin/vyper.0.4.0.linux | platform=linux - [traceId=db3e02b8-8701-47ad-9deb-c84b6af76241]
2024-12-03T11:19:24.889Z [info] [Server] safeHandler - errorMessage=Vyper path not found | errorStack=Error: Vyper path not found
    at useVyperCompiler (/home/manuel/Projects/sourcify/sourcify/services/server/dist/server/services/compiler/local/vyperCompiler.js:46:15)

packages/lib-sourcify/src/lib/AbstractCheckedContract.ts

services/server/src/server/services/compiler/local/vyperCompiler.ts

packages/lib-sourcify/src/lib/validation.ts

packages/lib-sourcify/src/lib/IVyperCompiler.ts

packages/lib-sourcify/src/lib/VyperCheckedContract.ts

services/server/src/server/controllers/verification/vyper/stateless/vyper.stateless.handlers.ts

services/server/src/server/controllers/verification/vyper/stateless/vyper.stateless.paths.yaml

manuelwedler · 2024-12-03T10:41:08Z

services/server/src/server/cli.ts

@@ -70,6 +72,11 @@ if (config.get("lambdaCompiler.enabled")) {

 export const solc = selectedSolidityCompiler;

+logger.info("Using local vyper compiler");
+const vyperRepoPath =
+  (config.get("vyperRepo") as string) || path.join("/tmp", "vyper-repo");


I think this needs to be documented in the server's README.

I don't see this fixed yet

services/server/test/integration/verification-handlers/verify.stateless.spec.ts

marcocastignoli · 2024-12-03T12:06:23Z

Why do we need a metadata file mock for VyperCheckContract? I guess this is due to the verifyDeployed method which expects a metadata. This could be documented better in some way. Is it possible to get rid of it when we refactor the verification flow?

The main reason why we still need the metadata here is because we want to store it in RepositoryV1/V2 to align the repository fs to solidity contracts. I think @kuzdogan had this idea first.

marcocastignoli · 2024-12-03T12:07:57Z

I also tried to verify a vyper contract on a local server instance. Unfortunately, it didn't work due to Vyper path not found. The server logs show that vyper was downloaded. But I cannot find the binary on my machine.

That's probably related to the fact that I still have to work on rewriting platform architecture management (vyper versions for x86, arm, ...)

kuzdogan · 2024-12-03T15:30:23Z

Why do we need a metadata file mock for VyperCheckContract? I guess this is due to the verifyDeployed method which expects a metadata. This could be documented better in some way. Is it possible to get rid of it when we refactor the verification flow?

The main reason why we still need the metadata here is because we want to store it in RepositoryV1/V2 to align the repository fs to solidity contracts. I think @kuzdogan had this idea first.

We discussed this during the call but to recap, the reason is all of the APIv1 is based on the existence of the metadata.json. Some examples:

People check if a contract is verified by GETing https://repo.sourcify.dev/contracts/partial_match/1/0x1F98431c8aD98523631AE4a59f267346ea31F984/metadata.json
The only way to get the ABI is the metadata.json file
Compilation information is only in metadata.json

So it's needed for backwards compatiility

manuelwedler

The additions until now look good. There are still some things missing from above.

services/server/src/server/controllers/verification/vyper/stateless/vyper.stateless.handlers.ts

packages/lib-sourcify/src/lib/types.ts

manuelwedler

lgtm

We'll address the last comments afterwards

…new endpoint `/verify/vyper` in server (#1760) * add VyperCheckedContract * rename CheckedContract.solidity to sources * refactor checkedcontract into an abstract class * refactor AbstractCheckedContract in server * use AbstractCheckedContract type in StorageServices * add /verify/vyper endpoint * successfully store is database and repository vyper match * dynamically download the vyper compiler * fix lib-sourcify-tests * enable all tests in server * improve comments for IVyperCompiler * fix pr suggestions * rename checkFiles to CheckFilesWithMetadata * rename useCompiler to useSolidityCompiler * fix pr suggestions

Adds vyper support, here the initial PR: #1760. Below the related tasks: - [x] Review what's stored in the database - [x] ~~Create a common folder for the compilers code used by both lib-sourcify and server (idk if it fits in this new feature, imo yes because it's a lot of duplicated code now that we have also vyper stuff)~~ postponed - [x] Rewrite platform architecture management (vyper versions for x86, arm, ...): #1776 - [x] Update READMEs #1786 - [x] Update documentation sourcifyeth/docs#27 - [x] Handle vyper transformations: #1783 - [x] Add more tests, and finalize lib-sourcify test: #1781 - [x] Finalize `/verify/vyper` API errors + openapi responses: #1774 - [x] Check Vyper immutable field to see if we can support immutable transformation: #1790 After merging: - publish new docs

marcocastignoli added 10 commits November 27, 2024 10:30

add VyperCheckedContract

ec29c19

rename CheckedContract.solidity to sources

6bd8ed9

refactor checkedcontract into an abstract class

dc28120

refactor AbstractCheckedContract in server

895409c

use AbstractCheckedContract type in StorageServices

d4170b0

add /verify/vyper endpoint

fe0abe4

successfully store is database and repository vyper match

41d16e9

dynamically download the vyper compiler

ac9df57

fix lib-sourcify-tests

35bb902

enable all tests in server

6548721

marcocastignoli mentioned this pull request Nov 28, 2024

[Vyper support] Add VyperCheckedContract in lib-sourcify and the new endpoint /verify/vyper in server #1759

Closed

marcocastignoli requested a review from manuelwedler November 28, 2024 09:56

improve comments for IVyperCompiler

c1eb47f

manuelwedler self-assigned this Nov 29, 2024

manuelwedler requested changes Dec 3, 2024

View reviewed changes

marcocastignoli added 3 commits December 4, 2024 16:40

fix pr suggestions

89e7a43

rename checkFiles to CheckFilesWithMetadata

5270444

rename useCompiler to useSolidityCompiler

6d7b689

manuelwedler reviewed Dec 5, 2024

View reviewed changes

services/server/src/server/controllers/verification/vyper/stateless/vyper.stateless.handlers.ts Outdated Show resolved Hide resolved

packages/lib-sourcify/src/lib/types.ts Outdated Show resolved Hide resolved

fix pr suggestions

fca3379

manuelwedler approved these changes Dec 9, 2024

View reviewed changes

marcocastignoli merged commit 71320f8 into vyper-verification-main Dec 9, 2024
6 checks passed

marcocastignoli mentioned this pull request Dec 12, 2024

Add Vyper support to Sourcify #1784

Merged

9 tasks

kuzdogan deleted the vyper-support branch January 2, 2025 18:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Vyper support] Add `VyperCheckedContract` in `lib-sourcify` and the new endpoint `/verify/vyper` in server #1760

[Vyper support] Add `VyperCheckedContract` in `lib-sourcify` and the new endpoint `/verify/vyper` in server #1760

marcocastignoli commented Nov 28, 2024 •

edited by manuelwedler

Loading

manuelwedler left a comment •

edited

Loading

manuelwedler Dec 3, 2024

manuelwedler Dec 5, 2024

marcocastignoli commented Dec 3, 2024

marcocastignoli commented Dec 3, 2024

kuzdogan commented Dec 3, 2024

manuelwedler left a comment

manuelwedler left a comment

[Vyper support] Add VyperCheckedContract in lib-sourcify and the new endpoint /verify/vyper in server #1760

[Vyper support] Add VyperCheckedContract in lib-sourcify and the new endpoint /verify/vyper in server #1760

Conversation

marcocastignoli commented Nov 28, 2024 • edited by manuelwedler Loading

manuelwedler left a comment • edited Loading

Choose a reason for hiding this comment

manuelwedler Dec 3, 2024

Choose a reason for hiding this comment

manuelwedler Dec 5, 2024

Choose a reason for hiding this comment

marcocastignoli commented Dec 3, 2024

marcocastignoli commented Dec 3, 2024

kuzdogan commented Dec 3, 2024

manuelwedler left a comment

Choose a reason for hiding this comment

manuelwedler left a comment

Choose a reason for hiding this comment

[Vyper support] Add `VyperCheckedContract` in `lib-sourcify` and the new endpoint `/verify/vyper` in server #1760

[Vyper support] Add `VyperCheckedContract` in `lib-sourcify` and the new endpoint `/verify/vyper` in server #1760

marcocastignoli commented Nov 28, 2024 •

edited by manuelwedler

Loading

manuelwedler left a comment •

edited

Loading