-
-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added Time Series Container and Block #199
Conversation
I am also changing so that we can read the data from |
This allows us to run the following code, currently only uni variate time series are supported. The block and container will work for multi variate too but the process to read the input would change somewhat. using FastAI
path = "/home/saksham/Desktop/GSOC/Adiac"
recipe = TimeSeries.TimeSeriesDatasetRecipe(file="Adiac_TRAIN.arff")
data, blocks = loadrecipe(recipe, path)
println(Datasets.testrecipe(recipe, data, blocks))
sample = getobs(data,5)
println(checkblock(blocks, sample)) which returns
|
This allows us to do the following path = datasetpath("adiac")
recipe = TimeSeries.TimeSeriesDatasetRecipe(file="Adiac_TRAIN.arff")
data, blocks = loadrecipe(recipe, path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking very good so far! I've left some comments. Unfortunately I am not familiar with time series using deep learning, so I don't know what the observations should look like and what encodings will be needed. Do you have a reference that gives an overview of these things?
Other than that, some tests would be great :) (see the rest of the library as reference for how to write these inline, and check ReTest.jl for how to run these interactively); dataset doesn't need a test since we don't want to download that on every CI run)
TimeSeriesRow{2,51}() # Multivariate time series with 2 variables and length 51. | ||
``` | ||
""" | ||
struct TimeSeriesRow{M,N} <: Block end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need these to be type parameters, i.e. are there places where we need to dispatch on them? See for example the dispatches on Image{2}
for constructing models as an example where being able to dispatch helps.
If not, I would suggest storing the sizes as fields, so something like:
struct TimeSeriesRow{M,N} <: Block end | |
struct TimeSeriesRow{N} <: Block | |
# number of observed variables | |
n::Int | |
# observation length | |
t::Int | |
end |
""" | ||
struct TimeSeriesRow{M,N} <: Block end | ||
|
||
function checkblock(::TimeSeriesRow{M,N}, obs::AbstractArray{T,2}) where {M,N,T<:Number} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the above suggestion, we would also not have to dispatch on ::TimeSeriesRow
as much here, and access the fields instead.
src/TimeSeries/recipes.jl
Outdated
labels = Array(df[!, recipe.targetcol]) | ||
rows = Matrix(select(df, Not(:target))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fine for now, but maybe there is a way to get an array view without copying everything. Not sure, though :)
Let me see if I can find a good reference for this. This has some (https://timeseriesai.github.io/tsai/data.preparation.html) which we can refer to for encoding. |
That's a good resource and probably a good reference for implementations here 👍 The best way to proceed is to take an example task from tsai, e.g. classification, and implement the components (incl. the model) used there. |
Hey sorry, I was busy with some school stuff. |
That seems like a great starting point!
No worries, it's normal that PRs take time and are punctured by some inactivity 😄 👍 |
I was previously using ARFFFiles.jl. It was not working with the multivariate time series, so I used this function from tsai with some modifications. We are able to now read .ts for both univariate and multivariate time series. |
Added Time Series Container and Block. Currently can only load univariate time series. This is work in progress for issue #155 .
I was planning to add
loaddataset
function for such datasets. Currently all datasets have the same root URL :- "https://s3.amazonaws.com/fast-ai-" . For time series datasets the root url is different so i think we can proceed by addroot_url
field in theFastAIDataset
structure.How does this sound ?