Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add method to return the output as an xarray? #21

Open
nbren12 opened this issue Jul 8, 2023 · 1 comment
Open

Add method to return the output as an xarray? #21

nbren12 opened this issue Jul 8, 2023 · 1 comment

Comments

@nbren12
Copy link
Contributor

nbren12 commented Jul 8, 2023

Hi @spencerkclark. I've found myself using xpartition for scoring workloads. Usually there is a big data reduction involved so the output does fit in memory. I've been using xpartition like this

    import xpartition
    block_list = []
    n = 20
    for i in range(n):
        print(i)
        index = output.z500.partition.indexers(n, i, dims=['initial_time'])
        block = output.isel(index).compute()
        block_list.append(block)
    output = xr.concat(block_list, dim='initial_time')

Maybe it's worth adding some helper that does this.

@spencerkclark
Copy link
Owner

spencerkclark commented Jul 10, 2023

Neat, yeah, I'd be open to adding something like this if we can implement a version that handles partitions along n-dimensions. I've found myself in this situation before too. Keeping with accessors for the time being maybe we could spell it da.partition.compute(steps, dims)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants