Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(HMS-2933): Improve some HTTP error codes #752

Merged
merged 1 commit into from
Nov 22, 2023
Merged

fix(HMS-2933): Improve some HTTP error codes #752

merged 1 commit into from
Nov 22, 2023

Conversation

avitova
Copy link
Member

@avitova avitova commented Nov 14, 2023

Hopefully this should be okay.
I am not entirely sure, if ParseUint would prevent out of range error. Maybe sources use ParseInt, and I just added a bigger range for them.
Also, I think, that they validate with different regex: https://github.com/RedHatInsights/sources-api-go/blob/4a3475e59b8f515dbda70c423826b2a3e4d07cb6/middleware/id_validation.go#L14, and I do not think that this is correct tbh, so I did this my way.
This is still WIP, but I wanted you to be able to comment on this in advance.

@avitova avitova force-pushed the BugFix2933 branch 2 times, most recently from 6a50bd2 to a9850f9 Compare November 14, 2023 15:15
@ezr-ondrej
Copy link
Member

Nice, this is what we probably should do with all the id params. 🧡
Do we also want to correct the return codes on the sources non-200 response, or would that be another PR? :)

@avitova
Copy link
Member Author

avitova commented Nov 15, 2023

How to test this:
These requests returned 500:
GET: /api/provisioning/v1/sources/{ID}/launch_templates
GET: /api/provisioning/v1/sources/zj{{=9243*9806}}zj/launch_templates
/api/provisioning/v1/pubkeys (for example, add a new field in the request body, that should not be there) - on create/update

@avitova
Copy link
Member Author

avitova commented Nov 15, 2023

Nice, this is what we probably should do with all the id params. 🧡
Yeah, I discarded the "id must not be zero" logic, that is what sources does, but I think that for our purpose, this might be better?

Do we also want to correct the return codes on the sources non-200 response, or would that be another PR? :)

Depends on how much is this in a rush. I do not mind doing a followup, but I can include it in here as well.

@avitova avitova marked this pull request as ready for review November 15, 2023 16:34
@ezr-ondrej
Copy link
Member

I do not mind doing a followup, but I can include it in here as well.

No real rush ;) We are on target with SLO now, so we can do it properly and do follow-up :)

Copy link
Member

@lzap lzap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, tho is could be simplified a bit.

}

return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not assume that primary keys in databases are higher than 0, in fact, when a database reaches the maximum value it starts from -MAX back to zero. This is at least behavior of postgres. So I would drop this check only keeping ParseUint and checking for error. In addition, I would expect this function to actually error out when called with negative number anyway.

I would not bother too much about the error message, we have properly documented what is expected, just returning "invalid id" is enough.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I think about it, you should not use ParseUint because our primary keys are int64, just parse a regular 64 signed number and report errors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sorry. There was even an invalid string, this regex does not check against 0, but numbers overall. The second check is against numbers that might overflow.

The second check would not fail for negative numbers. So I think we can keep both.

internal/services/aws_iam_service.go Show resolved Hide resolved
@lzap
Copy link
Member

lzap commented Nov 20, 2023

Any updates @avitova ? Thanks.

internal/services/aws_iam_service.go Show resolved Hide resolved
Comment on lines -225 to +251
if resp.JSON404 != nil || resp.JSON400 != nil {
if resp.JSON404 != nil {
logger.Warn().Bytes("body", resp.Body).Int("status", resp.StatusCode()).Msg("Get authentication from sources returned 4xx")
return nil, fmt.Errorf("get source authentication call: %w", clients.ErrNotFound)
}
if resp.JSON400 != nil {
logger.Warn().Bytes("body", resp.Body).Int("status", resp.StatusCode()).Msg("Get authentication from sources returned 4xx")
return nil, fmt.Errorf("get source authentication call: %w", clients.ErrBadRequest)
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two would be the most common cases - sources return 404, 400 or 2xx payloads. I think any other case could be a larger failure that we might want to know about anyway.
Therefore, after calling GetAuthentication, I feel like we want to check for ErrBadRequest and ErrNotFound, and the rest, like ErrNoResponseData, ErrUnexpectedBackendResponse should be left as is. :)

@avitova
Copy link
Member Author

avitova commented Nov 21, 2023

/retest

@avitova
Copy link
Member Author

avitova commented Nov 21, 2023

payloads.NewClientError takes care of the errors codes pretty well, as I have found out.
I think that this covers the ticket pretty well.

}
if resp.JSON400 != nil {
logger.Warn().Bytes("body", resp.Body).Int("status", resp.StatusCode()).Msg("Get authentication from sources returned 4xx")
return nil, fmt.Errorf("get source authentication call: %w", clients.ErrBadRequest)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The missing return at this resp.JSON404 != nil || resp.JSON400 != nil branch was the cause for the invalid returns.
The next if resp.JSON200 == nil returned ErrUnexpectedBackendResponse.
We are now validating id, so this would not happen for invalid ids at all. For 404 and 400 we are now able to return the correct status.

@avitova avitova requested review from lzap and ezr-ondrej November 21, 2023 14:06
Copy link
Member

@ezr-ondrej ezr-ondrej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not tested this, but code looks exactly what I've imagined! 🧡

Copy link
Member

@lzap lzap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why the regular expression is needed?

internal/payloads/validation/id.go Show resolved Hide resolved

var (
idValidation = regexp.MustCompile("^[0-9]+$")
ErrInvalidId = fmt.Errorf("invalid id")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ErrInvalidId = fmt.Errorf("invalid id")
ErrInvalidId = errors.New("invalid id")

internal/payloads/validation/id.go Outdated Show resolved Hide resolved
internal/payloads/validation/id_test.go Show resolved Hide resolved
Copy link
Member

@ezr-ondrej ezr-ondrej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the log of the failure in JSON200 == nil that we have in Get authentication also in ListProvisioningSourcesByProvider and ListAllProvisioningSources for easier debugging in the future?

@avitova
Copy link
Member Author

avitova commented Nov 22, 2023

Yeah, and you have correctly reminded me that this has the exact handling, and we should return the error even for 404 and 400 at those two places as well. @ezr-ondrej

Copy link
Member

@lzap lzap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very extensive testing indeed :-) Nice.

I would put few empty lines between those if statements and the error message is about "id" while function is called DigitsOnly which is a slight inconsistency but
that is okay. Merging, thanks.

@lzap lzap merged commit a3c3201 into main Nov 22, 2023
6 checks passed
@lzap lzap deleted the BugFix2933 branch November 22, 2023 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants