CLOUDP-289219: Create CLI commands for manipulating stream processors #3496

auddin431 · 2024-12-19T22:06:07Z

Proposed changes

This work adds the noun processors to the top level noun streams so that stream processors can be interacted with through the CLI. The processors noun has the following verbs:

create
delete
list
describe
start
stop

Jira ticket: CLOUDP-289219

Checklist

I have signed the MongoDB CLA
I have added tests that prove my fix is effective or that my feature works
I have added any necessary documentation in document requirements section listed in CONTRIBUTING.md (if appropriate)
I have addressed the @mongodb/docs-cloud-team comments (if appropriate)
I have updated test/README.md (if an e2e test has been added)
I have run make fmt and formatted my code

tcannon91

Next time, it might be a little easier to review if we break down the commits into each command or each command into their own PR :)

Dropping comments because I think I have gone through all of the create command. I haven't looked but I will try not to leave the same comments on the other commands if they have the same issues. I will continue looking at the other commands and try to only leave net new stuff

internal/store/streams.go

docs/command/atlas-streams-processors-create.txt

internal/cli/streams/processor/create.go

docs/command/atlas-streams-processors-create.txt

internal/cli/streams/processor/create_test.go

tcannon91

E2E tests reviewed

test/e2e/atlas/data/create_streams_connection_atlas.json

test/e2e/atlas/streams_with_clusters_test.go

tcannon91

Delete and Describe reviewed

internal/cli/streams/processor/delete.go

internal/cli/streams/processor/delete_test.go

internal/cli/streams/processor/describe.go

internal/cli/streams/processor/describe_test.go

tcannon91

Start and Stop reviewed

internal/cli/streams/processor/start.go

internal/cli/streams/processor/start_test.go

github-actions · 2025-01-20T00:04:30Z

This PR has gone 30 days without any activity and meets the project’s definition of "stale". This will be auto-closed if there is no new activity over the next 30 days. If the issue is still relevant and active, you can simply comment with a "bump" to keep it open, or add the label "not_stale". Thanks for keeping our repository healthy!

gssbzn · 2025-01-21T09:30:59Z

@auddin431 this is currently mark as stale by our bots, are you still working on this?

auddin431 · 2025-01-22T21:48:58Z

@auddin431 this is currently mark as stale by our bots, are you still working on this?

@gssbzn yep im still working on this, was out for a while during holidays and got occupied with other tickets. Will push out requested changes in the next day or two

auddin431 · 2025-01-27T16:14:55Z

evergreen keep-definitions

gssbzn · 2025-01-27T16:30:23Z

internal/cli/streams/processor/create.go

+		return nil, err
+	}
+
+	if processor.Name == nil {


Suggested change

if processor.Name == nil {

if len(processor.Name) == 0 {

you may want to validate empty strings

gssbzn · 2025-01-27T16:33:04Z

internal/cli/streams/processor/create.go

+}
+
+// atlas streams processor create [--projectId projectId].
+func CreateBuilder() *cobra.Command {


does this command need a --watch as in are the changes applied on create or does it take a while to be effective? see

mongodb-atlas-cli/internal/cli/clusters/create.go

Line 55 in 97d3e3b

cli.WatchOpts

for the use of watch on other commands

Hm, I know cluster creation can be a very lengthy process so I see why it's necessary for the --watch, but this command takes about 5-10 seconds. Is there a general cutoff as to when --watch is necessary?

I guess the main rule would be created vs ready for use, customers that script with CLI or other tools want to know when is ready for use and for that reason we provide --watch

gssbzn · 2025-01-27T16:40:35Z

internal/cli/streams/processor/delete.go

+}
+
+func (opts *DeleteOpts) Run() error {
+	return opts.Delete(opts.store.DeleteStreamProcessor, opts.ProjectID, opts.Instance)


Suggested change

return opts.Delete(opts.store.DeleteStreamProcessor, opts.ProjectID, opts.Instance)

return opts.Delete(opts.store.DeleteStreamProcessor, opts.ConfigProjectID(), opts.Instance)

here and everywhere else you do opts.ProjectID, project id is layered, it can come from config, env or flag, using th emethod ConfigProjectID() will make sure to use the correct one other wise you are looking only to the flag values

tcannon91

Looks pretty good

docs/command/atlas-streams-processors-create.txt

tcannon91

Looks good

gssbzn

I'm finding a bit confusing that some commands don't follow the general output logic of the CLI

gssbzn · 2025-01-29T13:21:45Z

internal/cli/streams/processor/create_test.go

+		if err := createOpts.Run(); err != nil {
+			t.Fatalf("Run() unexpected error: %v", err)
+		}


since using testify to keep it consistent

Suggested change

if err := createOpts.Run(); err != nil {

t.Fatalf("Run() unexpected error: %v", err)

}

require.NoError(createOpts.Run())

gssbzn · 2025-01-29T13:23:40Z

internal/cli/streams/processor/describe.go

+	opts.Output = jsonOutput
+	if opts.includeStats {
+		return opts.Print(r)
+	}


why overloading output options with your own option?

gssbzn · 2025-01-29T13:25:20Z

internal/cli/streams/processor/list.go

+	opts.Output = jsonOutput
+	if opts.includeStats {
+		return opts.Print(*r.Results)
+	}


same question? why is this a custom way to output instead of the standard --out|-o?

gssbzn · 2025-01-29T13:26:14Z

internal/cli/streams/processor/list.go

+		return opts.Print(*r.Results)
+	}
+
+	for _, res := range *r.Results {


don't use pointers, use getters https://github.com/mongodb/atlas-sdk-go/blob/main/docs/doc_5_best_practices.md#using-getters-instead-of-direct-field-access

Suggested change

for _, res := range *r.Results {

for _, res := range r.GetResults() {

gssbzn · 2025-01-29T13:27:02Z

internal/cli/streams/processor/list.go

+		return opts.Print(*r.Results)
+	}
+
+	for _, res := range *r.Results {


why are you iterating for the print? why don't you define a template that can handle the envelop object?

auddin431 requested review from a team as code owners December 19, 2024 22:06

auddin431 requested review from tcannon91 and jarjee December 19, 2024 22:06

auddin431 force-pushed the CLOUDP-289219 branch from 22ef83a to e916573 Compare December 19, 2024 22:42

tcannon91 requested changes Dec 20, 2024

View reviewed changes

tcannon91 reviewed Dec 20, 2024

View reviewed changes

test/e2e/atlas/data/create_streams_connection_atlas.json Outdated Show resolved Hide resolved

test/e2e/atlas/streams_with_clusters_test.go Show resolved Hide resolved

tcannon91 requested changes Dec 20, 2024

View reviewed changes

internal/cli/streams/processor/start.go Outdated Show resolved Hide resolved

internal/cli/streams/processor/start_test.go Outdated Show resolved Hide resolved

github-actions bot added the stale A stale issue or PR label Jan 20, 2025

github-actions bot removed the stale A stale issue or PR label Jan 22, 2025

auddin431 added 3 commits January 22, 2025 15:49

CLOUDP-289219: Create CLI commands for manipulating stream processors

c9c9cbe

rebase to latest commit

c437c96

fix function arguments

ebac9cc

pr comments

c9887a4

auddin431 force-pushed the CLOUDP-289219 branch from e916573 to c9887a4 Compare January 22, 2025 23:28

stronger mocking assertions and remove processorName from create

2031d6c

gssbzn reviewed Jan 27, 2025

View reviewed changes

auddin431 added 2 commits January 28, 2025 10:40

replace ProjectID with ConfigProjectID() and validate empty string

f729621

fix e2e test

c432ab2

auddin431 requested a review from tcannon91 January 28, 2025 17:49

tcannon91 reviewed Jan 28, 2025

View reviewed changes

docs/command/atlas-streams-processors-create.txt Show resolved Hide resolved

tcannon91 approved these changes Jan 28, 2025

View reviewed changes

gssbzn requested changes Jan 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLOUDP-289219: Create CLI commands for manipulating stream processors #3496

CLOUDP-289219: Create CLI commands for manipulating stream processors #3496

auddin431 commented Dec 19, 2024 •

edited

Loading

tcannon91 left a comment •

edited

Loading

tcannon91 left a comment

tcannon91 left a comment

tcannon91 left a comment

github-actions bot commented Jan 20, 2025

gssbzn commented Jan 21, 2025

auddin431 commented Jan 22, 2025

auddin431 commented Jan 27, 2025

gssbzn Jan 27, 2025

gssbzn Jan 27, 2025

auddin431 Jan 28, 2025

gssbzn Jan 29, 2025

gssbzn Jan 27, 2025

tcannon91 left a comment

tcannon91 left a comment

gssbzn left a comment

gssbzn Jan 29, 2025

gssbzn Jan 29, 2025

gssbzn Jan 29, 2025

gssbzn Jan 29, 2025

gssbzn Jan 29, 2025

	return opts.Delete(opts.store.DeleteStreamProcessor, opts.ProjectID, opts.Instance)
	return opts.Delete(opts.store.DeleteStreamProcessor, opts.ConfigProjectID(), opts.Instance)

	for _, res := range *r.Results {
	for _, res := range r.GetResults() {

CLOUDP-289219: Create CLI commands for manipulating stream processors #3496

Are you sure you want to change the base?

CLOUDP-289219: Create CLI commands for manipulating stream processors #3496

Conversation

auddin431 commented Dec 19, 2024 • edited Loading

Proposed changes

Checklist

tcannon91 left a comment • edited Loading

Choose a reason for hiding this comment

tcannon91 left a comment

Choose a reason for hiding this comment

tcannon91 left a comment

Choose a reason for hiding this comment

tcannon91 left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 20, 2025

gssbzn commented Jan 21, 2025

auddin431 commented Jan 22, 2025

auddin431 commented Jan 27, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tcannon91 left a comment

Choose a reason for hiding this comment

tcannon91 left a comment

Choose a reason for hiding this comment

gssbzn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

auddin431 commented Dec 19, 2024 •

edited

Loading

tcannon91 left a comment •

edited

Loading