Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compute residuals to the timeseries inversion, save to raster outputs #523

Merged
merged 16 commits into from
Jan 9, 2025

Conversation

scottstanie
Copy link
Collaborator

@scottstanie scottstanie commented Jan 8, 2025

To see whether the time series inversion ran smoothly (which encompasses detecting unwrapping errors, as well as several other upstream errors which may have snuck in), we'd like to see the residuals of

$$ \mathbf{r} := | A\mathbf{x} - \mathbf{b}| $$

where $A$ is the incidence matrix used during the inversion, $\mathbf{b}$ is the vector of unwrapped phases at a single pixel, and $\mathbf{x}$ is the vector of solved phases, one per secondary date.

Since there could be one single date (or a couple sparse dates) which had problems, we the same number of residual rasters as timeseries rasters by doing

$$ |A|^{T} \mathbf{r} $$

where $|A|^{T}$ is the elementwise absolute value of A.

Small test over tests on the walkthrough data:

image

A histogram of all the residuals for this area (with vertical lines marking multiple of pi)
image

reader = io.RasterStackReader.from_file_list( sorted(Path("work-walkthrough/timeseries/").glob("residu*")))
residuals = reader[:, :, :]
plt.hist(residuals.ravel(), log=True, bins=30)
for i in range(1, 8):
    plt.vlines(i * np.pi, 0, 1e6, color='k', label=f'{i}pi')

scottstanie and others added 13 commits October 31, 2024 16:57
When doing `GDAL_NUM_THREADS=2` and LZW, this came up
  File "/Users/staniewi/repos/dolphin/src/dolphin/_overviews.py", line 95, in create_image_overviews
    ds.BuildOverviews(resampling.value, levels)
  File "/Users/staniewi/miniforge3/envs/mapping-312/lib/python3.12/site-packages/osgeo/gdal.py", line 4378, in BuildOverviews
    return _gdal.Dataset_BuildOverviews(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: /vsimem/decompress_0x312935528.tif:Using code not yet in table

After turning the python multithreading off, and the GDAL_NUM_THREADS=1,
it was still giving this:

RuntimeError: 20210606_20210618.tif, band 1: IReadBlock failed at X offset 0, Y offset 2: TIFFReadEncodedTile() failed.
May be caused by: TIFFReadEncodedTile() failed.
May be caused by: /Users/staniewi/repos/dolphin/docs/notebooks/work-walkthrough/timeseries/20210606_20210618.tif:Using code not yet in table

This was on a Mac with Gdal 3.9.0
$ gdalinfo --version
GDAL 3.9.3, released 2024/10/07
@scottstanie scottstanie marked this pull request as ready for review January 9, 2025 03:25
@scottstanie scottstanie requested a review from mirzaees January 9, 2025 16:09
@scottstanie scottstanie merged commit ad0e4a4 into isce-framework:main Jan 9, 2025
5 checks passed
@scottstanie scottstanie deleted the add-ts-residuals-raster branch January 9, 2025 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant