Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GateIO: Fix GetFuturesContractDetails for Deliveries and minor other fixes #1766

Merged

Conversation

gbjk
Copy link
Collaborator

@gbjk gbjk commented Jan 4, 2025

GateIO

  • 🐛 🚨 Fix GetFuturesContractDetails for Deliveries
    Was returning the product of all the contracts, so 1444 instead of 38 contracts.
    Doesn't touch anything else around this; Futures asset split coming separately
  • Rename GetPersonalTradingHistory to GetMyTradingHistory
  • Remove duplicate DeliveryTradingHistory
  • Rename GetMyPersonalTradingHistory to GetMyFuturesTradingHistory
  • Rename GateIOGetPersonalTradingHistory to GetMySpotTradingHistory
  • Rename GetSingleContract and GetSingleDeliveryContracts
  • Fix GetOpenInterest returning asset.ErrNotEnabled

Linter

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How has this been tested

  • go test ./... -race
  • golangci-lint run
  • GetFuturesContractDetails

@gbjk gbjk added bug review me This pull request is ready for review labels Jan 4, 2025
@gbjk gbjk self-assigned this Jan 4, 2025
@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch from 282c54b to cb7b226 Compare January 4, 2025 09:28
Copy link

codecov bot commented Jan 4, 2025

Codecov Report

Attention: Patch coverage is 66.66667% with 14 lines in your changes missing coverage. Please review.

Project coverage is 37.40%. Comparing base (50448ec) to head (f21d447).
Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
exchanges/gateio/gateio.go 36.36% 7 Missing ⚠️
exchanges/gateio/gateio_wrapper.go 81.48% 5 Missing ⚠️
engine/rpcserver.go 0.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1766      +/-   ##
==========================================
+ Coverage   37.37%   37.40%   +0.03%     
==========================================
  Files         415      415              
  Lines      178627   178336     -291     
==========================================
- Hits        66760    66715      -45     
+ Misses     104089   103847     -242     
+ Partials     7778     7774       -4     
Files with missing lines Coverage Δ
currency/pairs.go 96.25% <ø> (ø)
exchanges/btse/btse_wrapper.go 43.77% <100.00%> (ø)
exchanges/gateio/gateio_types.go 65.71% <ø> (ø)
exchanges/subscription/template.go 97.70% <100.00%> (ø)
engine/rpcserver.go 39.81% <0.00%> (ø)
exchanges/gateio/gateio_wrapper.go 39.97% <81.48%> (-0.18%) ⬇️
exchanges/gateio/gateio.go 13.91% <36.36%> (ø)

... and 14 files with indirect coverage changes

@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch from dcf0c6c to 77569e5 Compare January 5, 2025 08:13
@gbjk gbjk changed the title GateIO: Fix GetFuturesContractDetails for Deliveries GateIO: Fix GetFuturesContractDetails for Deliveries and minor other fixes Jan 5, 2025
@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch 2 times, most recently from e0ceb88 to ba00163 Compare January 5, 2025 11:16
Copy link
Collaborator

@gloriousCode gloriousCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

exchanges/gateio/gateio_wrapper.go Show resolved Hide resolved
.golangci.yml Outdated Show resolved Hide resolved
.golangci.yml Outdated Show resolved Hide resolved
@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch from c19f66c to f21d447 Compare January 6, 2025 01:38
Copy link
Collaborator

@gloriousCode gloriousCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK!

@gbjk gbjk mentioned this pull request Jan 6, 2025
1 task
@gloriousCode gloriousCode added the gcrc GloriousCode Review Complete label Jan 6, 2025
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, Just some fields that have been left out:

diff --git a/exchanges/gateio/gateio_types.go b/exchanges/gateio/gateio_types.go
index 3bca9c0cc..1a91b5e24 100644
--- a/exchanges/gateio/gateio_types.go
+++ b/exchanges/gateio/gateio_types.go
@@ -679,16 +679,22 @@ type FuturesContract struct {
        OrdersLimit           int64        `json:"orders_limit"`
        TradeID               int64        `json:"trade_id"`
        OrderbookID           int64        `json:"orderbook_id"`
+       EnableBonus           bool         `json:"enable_bonus"`
+       EnableCredit          bool         `json:"enable_credit"`
+       CreateTime            types.Time   `json:"create_time"`
+       FundingCapRatio       types.Number `json:"funding_cap_ratio"`
+       VoucherLeverage       types.Number `json:"voucher_leverage"`
 }

 // TradingHistoryItem represents futures trading history item.
 type TradingHistoryItem struct {
-       ID         int64        `json:"id"`
-       CreateTime types.Time   `json:"create_time"`
-       Contract   string       `json:"contract"`
-       Text       string       `json:"text"`
-       Size       float64      `json:"size"`
-       Price      types.Number `json:"price"`
+       ID           int64        `json:"id"`
+       CreateTime   types.Time   `json:"create_time"`
+       CreateTimeMs types.Time   `json:"create_time_ms"`
+       Contract     string       `json:"contract"`
+       Text         string       `json:"text"`
+       Size         float64      `json:"size"`
+       Price        types.Number `json:"price"`
        // Added for Derived market trade history datas.
        Fee      types.Number `json:"fee"`
        PointFee types.Number `json:"point_fee"`

@gbjk
Copy link
Collaborator Author

gbjk commented Jan 11, 2025

Thanks for the update, Just some fields that have been left out:

Fixed gbjk@1565c773b

@gbjk gbjk requested a review from shazbert January 11, 2025 08:13
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clickity Clack Changes ACK.

@shazbert shazbert added the szrc shazbert review complete label Jan 12, 2025
Copy link
Collaborator

@thrasher- thrasher- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor nit and then LGTG

// Added for Derived market trade history datas.
ID int64 `json:"id"`
CreateTime types.Time `json:"create_time"`
CreateTimeMs types.Time `json:"create_time_ms"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CreateTimeMs types.Time `json:"create_time_ms"`
CreateTimeMS types.Time `json:"create_time_ms"`

Or CreateTimeMilliseconds; I generally prefer the latter to be more human friendly with fields even though someone can infer what it means

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't fix this one alone, without fixing all the others. :)
And if we're going to touch all of them, I think we should just ditch the non _ms field and rename the _ms field:

	CreateTime types.Time   `json:"create_time_ms"`

Unless you object real quick, that's coming in on runway 2 🛩️

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's two fields doing the same thing and one with less precision than the other, makes sense. We should always default to whatever gives us the most precision (milliseconds field in this case) 👍 Assuming that both fields are always populated as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one edge case I'm chasing down now - The V3 implementation under the V4 api url for the options websocket, and the response header not having time_ms.
Anyway, yeah. I'm on it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

@gbjk gbjk Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark referenced in commit message. Just realised that won't persist.
Here it is for posterity

Not committing because once validated, there's no reason to doubt this will regret.
It was just to ensure that the EachKeys we haven't used before is as performant as their godoc claimed.

@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch from 3459b22 to 7e730dd Compare January 13, 2025 06:33
@gbjk gbjk requested a review from thrasher- January 13, 2025 06:34
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look great, just 2 suggestions. Thanks heaps!

exchanges/gateio/gateio_types.go Show resolved Hide resolved
@@ -216,6 +217,45 @@ func (g *Gateio) WsHandleSpotData(_ context.Context, respRaw []byte) error {
return nil
}

func parseWSHeader(msg []byte) (r *WSResponse, errs error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely more performant than json I did my own benchmarks to validate

jsonparser:

// 1646160	       666.3 ns/op	     312 B/op	       7 allocs/op
func BenchmarkParseHeader(b *testing.B) {
	in := `{"time":1726121320,"time_ms":1726121320745,"id":1,"channel":"spot.tickers","event":"subscribe","result":{"status":"success"},"request_id":"a4"}`
	b.ReportAllocs()
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, err := parseWSHeader([]byte(in))
		if err != nil {
			b.Fatal(err)
		}
	}
}

normal json:

// 654258	      1810 ns/op	     408 B/op	      10 allocs/op
func BenchmarkJson(b *testing.B) {
	in := []byte(`{"time":1726121320,"time_ms":1726121320745,"id":1,"channel":"spot.tickers","event":"subscribe","result":{"status":"success"},"request_id":"a4"}`)
	b.ReportAllocs()
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		var resp WsResponse
		err := json.Unmarshal(in, &resp)
		if err != nil {
			b.Fatal(err)
		}
	}
}

Using the bench mark above I then switched branch #1623 to use sonic and that has comparative performance using go test --tags=sonic -bench=BenchmarkJson -v but has way less implementation complexity. I would suggest to reverse course just to reduce or limit that complexity and the ability for all websocket asset types pathways to achieve that relative boost without that toil.

BenchmarkJson-12         1775338               643.5 ns/op           361 B/op          7 allocs/op
BenchmarkJson-12         1857788               742.5 ns/op           362 B/op          7 allocs/op
BenchmarkJson-12         1890859               640.5 ns/op           361 B/op          7 allocs/op
BenchmarkJson-12         1871022               686.6 ns/op           362 B/op          7 allocs/op

Copy link
Collaborator Author

@gbjk gbjk Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard pass: I was already on the very very edge of acceptability when dealing with a bugfix.
This implementation is more performant than currently and not more complex for the callers of parseWSHeader.

I'm very excited to adopt sonic, but not in a bug fix
2) I'm wary because of the hit we take on Silicon chips if we go sonic over jsonparser.
That's why I haven't yet gone gungho on it.
In the example here, we don't gain any performance by using sonic, but we lose performance on M* chips.
That's not a debate for here, admittedly.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct, the current option is still a better approach and more sonic things can be done later when that gets merged, I wasn't suggesting to add sonic to this PR to be clear. In my benchmarks I didn't even account for the UnmarshalJSON edge case which adds further overhead. Silly billy.

Copy link
Collaborator

@thrasher- thrasher- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes ACK, thanks for making the WSHeader improvements and benching it!

Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK! Thanks

gbjk added 2 commits January 14, 2025 11:01
Was returning the product of all the contracts, so 1444 instead of 38
contracts.
Using wrong error for pair not enabled
gbjk added 4 commits January 14, 2025 11:01
Especially fixes GetSingleContract, which seems misleading to not say
Futures.
There's a load of `GetSingle*` here that should probably also be fixed,
but these two justified a dyno
@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch from 9d28eda to 9338a2c Compare January 14, 2025 04:02
gbjk added 5 commits January 14, 2025 11:03
It's been a year, and I'm still getting caught out by govet demanding I
don't shadow a var I was deliberately shadowing.
Made worse by an increase in clashes with stylecheck when they both want
opposite things on the same line.
This unifies handling for time_ms and time in response headers, since
options and delivery have only time, but spot has time_ms as well.
We use the better of the two results.

Also [improves performance 2x](https://gist.github.com/gbjk/7cacb63b9a256e745534bb05ca853c48)
Removes the deprecated _time json fields and populates our Time fields
with the time_ms values
@gbjk gbjk force-pushed the bugfix/gateio_GetFuturesContractDetails branch from 9338a2c to 193b255 Compare January 14, 2025 04:03
@thrasher- thrasher- merged commit 4c7f48a into thrasher-corp:master Jan 14, 2025
5 of 11 checks passed
@gbjk gbjk deleted the bugfix/gateio_GetFuturesContractDetails branch January 14, 2025 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug gcrc GloriousCode Review Complete review me This pull request is ready for review szrc shazbert review complete
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants