The process of upgrading the flow-go package has become more complicated

[Guitarheroua]

Defenition

The process of upgrading the flow-go package, especially for Access Node changes, has become more complicated after recent updates. It would be beneficial to establish a procedure or solution to make the process more intuitive and understandable.

Problem

Currently, flow-emulator uses flow-go version 0.37.10. Our team is working on a fork of the flow-go project. When a PR is created with changes to the Access API or related components that affect the flow-emulator, temporary replacements must be made in both flow-go and flow-emulator due to circular dependencies between these repositories (flow-go integration tests depend on flow-emulator, and flow-emulator depends on flow-go).

Previously, the process was as follows:

1. Introduce a replacement in flow-emulator pointing to the flow-go branch that contains the changes.
2. Introduce a replacement in flow-go pointing to the flow-emulator branch that contains changes.
3. Merge the flow-emulator changes with the replacement to the master branch.
4. Remove the replacement in flow-go pointing to the merged flow-emulator on master.
5. After the flow-go branch is merged, create another PR to remove the replacement in flow-emulator.

However, this process has now become even more complicated. It seems we need to follow the same pattern for the v.0.37 branch and then merge changes from v.0.37 to master, but I am not sure if this is correct.

We need to define a proper process for merging in such cases or find a solution to handle the circular dependencies because merging even small changes to the flow-emulator is currently very cumbersome.

[bluesign]

I dont think v0.37 will be needed anymore as mainnet migrated to Cadence 1.0

Though personally, I am not so happy with this circular dependency situation too. ( in year 2024 )
but also this is the only thing keeping emulator on par with flow-go (barely)

maybe someone better than me at go can have some better ideas for this process ( cc: @turbolent )

[peterargue]

v0.37 is the production branch of onflow/flow-go we're using that's deployed to mainnet/testnet. I don't think we're going to remove that.

One simple solution to the current issue (it wont make the previous process any better), is to just always track onflow/flow-go master. This will probably mean we can't use tagged versions, or at least we'd have to use pre-release versions.

We could also use a similar branching strategy that flow-go uses, where master tracks master and some release branch track the current live branch from flow-go.

[turbolent]

This circular dependency between the repositories has been a problem since the very start, and so far nobody has come up with any suggestions to improve it. I also don't like how cumbersome it is, but unfortunately also don't have any ideas on how to improve it.

The process is already cumbersome for core contributors who have push access, but is even worse for external contributors submitting contributions from a fork (need to using go mod replace statements). Have we discussed just giving push access to contributors to at least make the process a bit easier for them?

I'm not sure when or why this changed, mostly because I'm not keeping up with all changes in the tooling repos, but we used to track flow-go master in the Emulator -- when did that change to v0.37 and why? (not saying it is wrong, just trying to understand)

[peterargue]

I opened #751 to switch it over to master. we'll start there.

I'm also in favor of granting @Guitarheroua push access. I'll look into making that happen

[bluesign]

I think embedded a bit stripped down emulator to flow-go is the best option, I made a poc here: https://github.com/bluesign/flow-go/tree/withoutEmulator

[bluesign]

closing this as onflow/flow-go#6594 is merged