Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Provider Skeleton Mockup #75

Closed
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the //TODO comments from here because it's invalid json and was causing some framework tests to fail. We should probably explicitly say in the docs that users will need to edit this file once they write their config object.

If they try to configure anything other than data source id with this default schema, validation will fail. So hopefully that will force users to update this file even without the TODOs

"$id": "/resources/source_config_schemas/skeleton",
"type": "object",
"properties": {
"dataSource": {
"type": "string",
"const": "skeleton"
}
// TODO: add all other possible fields from your config object
},
"required": ["dataSource"], // TODO: add all other required fields here
"additionalProperties": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package config

import (
"testing"

"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/configs"
"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/sources/skeleton"
"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/utils"
"github.com/stretchr/testify/assert"
)

func TestValidSkeletonConfig(t *testing.T) {

// TODO: set this to a valid config string using a feed from your new source
validConfig := `{}`

config, err := configs.LoadConfigFromBytes([]byte(validConfig))
assert.NoError(t, err)

assert.Equal(t, 1, len(config.Sources))

sourceConfig := config.Sources[0]

dataSourceId, err := utils.GetDataSourceId(sourceConfig.Config)
assert.NoError(t, err)
assert.Equal(t, skeleton.SkeletonDataSourceId, dataSourceId)

sourceSpecificConfig, err := skeleton.GetSourceSpecificConfig(sourceConfig)
assert.NoError(t, err)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generate tests to make sure the config is valid and can be loaded, that we can extract the data source id from it, and that we can extract the SkeletonConfig.

The user just has to fill in a valid config string and write some asserts


// TODO: write some asserts to check that the fields on sourceSpecificConfig have the values you'd expect
panic("implement me")
}
8 changes: 8 additions & 0 deletions apps/lib/data_provider/sources/skeleton/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package skeleton

import "github.com/Stork-Oracle/stork-external/apps/lib/data_provider/types"

type SkeletonConfig struct {
DataSource types.DataSourceId `json:"dataSource"` // required for all Data Provider Sources
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All configs need to have a DataSource field

The user just needs to add configuration details specific to their dataSource

// TODO: Add any additional config parameters needed to pull a particular data feed
}
24 changes: 24 additions & 0 deletions apps/lib/data_provider/sources/skeleton/data_source.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package skeleton

import (
"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/types"
)

type skeletonDataSource struct {
// TODO: set any necessary parameters
}

func newSkeletonDataSource(sourceConfig types.DataProviderSourceConfig) *skeletonDataSource {
skeletonConfig, err := GetSourceSpecificConfig(sourceConfig)

Check failure on line 12 in apps/lib/data_provider/sources/skeleton/data_source.go

View workflow job for this annotation

GitHub Actions / build

skeletonConfig declared and not used

Check failure on line 12 in apps/lib/data_provider/sources/skeleton/data_source.go

View workflow job for this annotation

GitHub Actions / build

skeletonConfig declared and not used
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little opinionated, but it seems like any data source would need to be initialized using the data source's config.

Note that as written this skeleton code won't even compile, since this variable is unused. It might be a little nicer to have skeleton code which compiles cleanly but fails at runtime - this way seemed a little more readable to the developer though.

if err != nil {
panic("unable to decode config: " + err.Error())
}

// TODO: add any necessary initialization code
panic("implement me")
}

func (r skeletonDataSource) RunDataSource(updatesCh chan types.DataSourceUpdateMap) {
// TODO: Write all logic to fetch datapoints and add to updatesCh
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super open ended

panic("implement me")
}
9 changes: 9 additions & 0 deletions apps/lib/data_provider/sources/skeleton/data_source_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package skeleton

import (
"testing"
)

func TestSkeletonDataSource(t *testing.T) {
// TODO: write some unit tests for your data source
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very open ended - didn't want to auto-generate unit tests that actually run the integration in case the integration hits external services

}
30 changes: 30 additions & 0 deletions apps/lib/data_provider/sources/skeleton/init.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package skeleton
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is fully auto-generated - user doesn't need to touch it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can add a comment at the top of the file saying that it is autogenerated and to please not touch - similar to mockery

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I'm not sure if we want to actually stop people from changing this file if they want to - it wouldn't be crazy for some advanced user to make changes to the factory code.


import (
"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/sources"
"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/types"
"github.com/Stork-Oracle/stork-external/apps/lib/data_provider/utils"
"github.com/mitchellh/mapstructure"
)

var SkeletonDataSourceId types.DataSourceId = types.DataSourceId(utils.GetCurrentDirName())

type skeletonDataSourceFactory struct{}

func (f *skeletonDataSourceFactory) Build(sourceConfig types.DataProviderSourceConfig) types.DataSource {
return newSkeletonDataSource(sourceConfig)
}

func init() {
sources.RegisterDataSourceFactory(SkeletonDataSourceId, &skeletonDataSourceFactory{})
}

// assert we're satisfying our interfaces
var _ types.DataSource = (*skeletonDataSource)(nil)
var _ types.DataSourceFactory = (*skeletonDataSourceFactory)(nil)

func GetSourceSpecificConfig(sourceConfig types.DataProviderSourceConfig) (SkeletonConfig, error) {
var config SkeletonConfig
err := mapstructure.Decode(sourceConfig.Config, &config)
return config, err
}
Loading