KCA assignment pull request (gsmadi) #4
Conversation
There was a problem hiding this comment.
Hi Gabriel, 🙂
This is a good submission. I can see that you went an extra step and decided to add unit tests for your code and used a linter. The strategy you chose is simple. Evaluating it based on a hit-or-miss ratio is an unusual approach, a data science-inspired one.
Will set up an interview.
- Good to see a "Getting started" part and linting used.
- Function/class structure is okay but could be further improved (commenting, using user-input values, avoiding loops).
- Would love to see more analysis of the KCA, alternative strategy ideas.
- Would be interesting to discuss the topic of data insufficiency here.
- With extra time put in, the strategy can be packaged in a function/class structure. And later assessed based on the generated equity curve.
- Extra points for adding unit tests.
|
|
||
| ## KCA Trading Algorithm Design | ||
|
|
||
| We fit our KCA trading algorithm with 360 days worth of data. We select a year worth of data given thats the resolution we have plus it captures four quarters worth of price movements. |
There was a problem hiding this comment.
About 252 days of data can be used here, as a rough number of trading days per year.
| slot of the tuple and a 1 or -1 on the second slot to indicate | ||
| a buy and sell signal, respectively. | ||
| """ | ||
| randq = random.randrange(1000, 5000) # Generate random seed for KCA q seed |
There was a problem hiding this comment.
This parameter can be user-input, or be calculated based on price_df, as it depends on the range of time series values, right?
There was a problem hiding this comment.
I think it was just a scalar to multiply against the time series. Agreed this could potentially be user input.
Q = q * np.eye(A.shape[0]) - Comment from paper q: Scalar that multiplies the seed states covariance
| "\n", | ||
| "The second anomaly can be seen a bit after year 2018. For now, we lack an explanation for such a deviation." |
There was a problem hiding this comment.
It would be interesting to explore what caused this performance in 2018.
| "source": [ | ||
| "Now that we have produced predictions and we know the actual values from our test set samples, lets see how well KCA fares. To see how well KCA performs we essentially see if it was right in direction in regard to the price movement and by how much.\n", | ||
| "\n", | ||
| "In the `prediction_delta` column we take the difference from the actual to the predicted value. Using the sign of this value and of the decision we create the `outcome` column. In this column, if the direction produced by KCA was correct we set a `1`, else a `0` for wrong.\n", |
There was a problem hiding this comment.
Using a hit-to-miss ratio is quite unusual to test a strategy performance, but it does, in general, tell us if the prediction rate is good. It would be nice to also see some adjustments to the simple strategy to see what performance can be obtained, maybe base it on velocity and/or acceleration?
There was a problem hiding this comment.
Yes agreed on both the ratio being unusual and trying out other strategies.
For hit and miss I think I just wanted a simple way to convey prediction rate as vetting the strategy with actual trading introduces other variants (perhaps position sizing, unwinding held positions, etc).
In terms of other strategies, I think an interesting one would be to see if we can use velocity or acceleration as perhaps leading indicators to position spikes.
| "\n", | ||
| "In the `prediction_delta` column we take the difference from the actual to the predicted value. Using the sign of this value and of the decision we create the `outcome` column. In this column, if the direction produced by KCA was correct we set a `1`, else a `0` for wrong.\n", | ||
| "\n", | ||
| "Computing a hit-to-miss ratio below, we see its not the greatest. Essentially, predicting 10 days worth of price movements, it only got 1 right. Now, lets highlight how little data we have to make conclusions on this." |
There was a problem hiding this comment.
"Now, lets highlight how little data we have to make conclusions on this."
How much data do you think would be sufficient to make conclusions regarding the performance of such a model?
There was a problem hiding this comment.
I believe we need to perform at least 10 experiments (for the same price path) and generate north of 100 predictions per experiment (~1000 predictions). Then that should perhaps suffice to obtain a sample mean and standard deviation for say things like prediction rate (hit-miss).
No description provided.