add earthdata support

[mgovorcin]

Adding earthdata download support [placeholder at the moment]

The code seems to work for searching products on Earthdata, however when downloading with earthdataaccess getting errors:
Unauthorized for url: https://urs.earthdata.nasa.gov/oauth/authorize?response_type=code&client_id=BO_n7nTIlMljdvU6kRRB3g&redirect_uri=https://auth.asf.alaska.edu/login&state=https%3A%2F%2Fgrfn.asf.alaska.edu%2Fdoor%2Fdownload%2FS1-GUNW-A-R-014-tops-20200102_20191221-152853-00038E_00019N-PP-e9d0-v3_0_1.nc&app_type=401

but when clicking on the url link, download starts with not issues, ask ASF folks what might be the cause.

---

Description
Added

• query_earthaccess: that searches Earthdata and returns list of Earthaccess.Granule and Dataframe with all attributes:
  Index(['sensor', 'datasetName', 'flight_direction', 'look_direction', 'track_number', 'mode', 'reference_secondary', 'UTC_time', 'upper_left_latlon', 'orbits', 'system_tag', 'version', 'geometry', 'ASCENDING_DESCENDING', 'BEAM_MODE', 'POLARIZATION', 'PERPENDICULAR_BASELINE', 'VERSION', 'FRAME_NUMBER', 'PATH_NUMBER', 'TEMPORAL_BASELINE_DAYS', 'URL', 'PNG_URL', 'start', 'end', 'btemp'], dtype='object')
• download_earthdata using preselect Granules

Command:

     scenes, query_df = query_earthaccess((30,20,40,30), tracks_id=[14])

By using geopandas, we can vizualize the frames without downloading;

query_qdf = gpd.GeoDataFrame(query_df, geometry=query_df.geometry)
query_qdf.exterior.plot()

and probably also plot the network, as bperp and btemp are included

[jhkennedy]

@mgovorcin I'll have to look into this a little bit, but Earthaccess has a hard time with some of the ASF datasets; I thought I had fixed those issues already, but I've gotten a couple of reports now that it's not working (again?) with various datasets.

That said, is there any particular reason you're using Earthaccess since you already have asf_search as a dependency in this module and it would handle these searches and methods just fine?

[mgovorcin]

@mgovorcin I'll have to look into this a little bit, but Earthaccess has a hard time with some of the ASF datasets; I thought I had fixed those issues already, but I've gotten a couple of reports now that it's not working (again?) with various datasets.

That said, is there any particular reason you're using Earthaccess since you already have asf_search as a dependency in this module and it would handle these searches and methods just fine?

@jhkennedy, I am adding earthaccess as alternative option to asf_search, to assist one of our users in downloading ARIA-S1-GUNW version 3 products with ariaDownload.py, while waiting for these products to be visible with asf_search. They are eager to use the data, so we trying to make things easier for them. We can touch base on this on our regular Tuesday meetings

[dbekaert]

@jhkennedy is this an easy fix or should we defer to asf_search support for the new GUNW collection (@glshort)? If we persue this earthdata search then we should probably also work directly off the dev_refactor branch of @alexfore.

[jhkennedy]

@mgovorcin @dbekaert Earthaccess uses EDL tokens to access datasets, but the GRFN distribution app does not support tokens, so Earthaccess will not be able to download them at this time. @betolink and I may update Earthaccess to handle this dataset, or ASF my update the distribution app to support tokens -- all of those options are a ways out unfortunately.

That said, asf_search currently (v7.0.8) can access these products just fine. You can search for version 2 and 3 products by updating this the query_asf method to:

        dct_kw = dict(dataset=asf.DATASET.ARIA_S1_GUNW,
                      collections=["C2859376221-ASF", "C1261881077-ASF"],
                      relativeOrbit=tracks,
                      flightDirection=flight_direction,
                      intersectsWith=bbox)
        scenes = asf.geo_search(**dct_kw)

With the next release, which may only be a day or two away, you can drop the collection= line, but leaving it won't hurt either.

Or if you just want to search for version 3 products, it'd look like:

        dct_kw = dict(shortName='ARIA_S1_GUNW',
                      relativeOrbit=tracks,
                      flightDirection=flight_direction,
                      intersectsWith=bbox)
        scenes = asf.geo_search(**dct_kw)

---

You can also get to a GeoDataFrame from asf_search results with:

import geopandas as gpd
gdf = gpd.GeoDataFrame.from_features(scenes.geojson(), crs='EPSG:4326')

which will have all the properties as columns and a geometry column.
https://geopandas.org/en/stable/docs/reference/api/geopandas.GeoDataFrame.from_features.html

You can then add whatever columns you need from the file names or what have you.

[mgovorcin]

@jhkennedy thanks for checking on this, I updated asf_search to version v7.0.8 and can find product v3.

Here is my query:

dct_kw = dict(dataset=asf.DATASET.ARIA_S1_GUNW,
                collections=["C2859376221-ASF", "C1261881077-ASF"],
                relativeOrbit=[14],
                flightDirection=asf.FLIGHT_DIRECTION.ASCENDING,
                processingLevel=asf.constants.GUNW_STD,
                intersectsWith='POLYGON((34.9254 24.4438,38.8022 24.4438,38.8022 28.9871,34.9254 28.9871,34.9254 24.4438))')
scenes = asf.geo_search(**dct_kw)

Confirm that both query_earthaccess and asf_search give the same number of found v3 product: 3627

@jhkennedy minor thing to report that for the gdf = gpd.GeoDataFrame.from_features(scenes.geojson(), crs='EPSG:4326')
it appears that some of the fields are empty

where can retrieve the values for those [bperp, temporal_baseline] with query_earthaccess:

Tagging @bbuzz31 and closing this PR as we do not need Earthaccess support

[jhkennedy]

@mgovorcin @dbekaert @bbuzz31 asf_search v7.0.9 has been released so you shouldn't need the collections= argument anymore --v2 and v3 products should show up in searches like:

dct_kw = dict(dataset=asf.DATASET.ARIA_S1_GUNW,
                relativeOrbit=[14],
                flightDirection=asf.FLIGHT_DIRECTION.ASCENDING,
                processingLevel=asf.constants.GUNW_STD,
                intersectsWith='POLYGON((34.9254 24.4438,38.8022 24.4438,38.8022 28.9871,34.9254 28.9871,34.9254 24.4438))')
scenes = asf.geo_search(**dct_kw)

And finding only the v3 products remains the same:

  dct_kw = dict(shortName='ARIA_S1_GUNW',
                relativeOrbit=[14],
                flightDirection=asf.FLIGHT_DIRECTION.ASCENDING,
                processingLevel=asf.constants.GUNW_STD,
                intersectsWith='POLYGON((34.9254 24.4438,38.8022 24.4438,38.8022 28.9871,34.9254 28.9871,34.9254 24.4438))')
  scenes = asf.geo_search(**dct_kw)

Though when providing processing levels, you may need to also add collectionAlias=False, as "asf-search will aliases any known platforms/processing levels with their associated concept-ids in the background when searching CMR". That'd look like:

  dct_kw = dict(shortName='ARIA_S1_GUNW',
                relativeOrbit=[14],
                flightDirection=asf.FLIGHT_DIRECTION.ASCENDING,
                processingLevel=asf.constants.GUNW_STD,
                intersectsWith='POLYGON((34.9254 24.4438,38.8022 24.4438,38.8022 28.9871,34.9254 28.9871,34.9254 24.4438))',
                collectionAlias=False)
  scenes = asf.geo_search(**dct_kw)

[bbuzz31]

@jhkennedy I don't fully understand how important this collectionAlias flag may be; I don't think it's supported in geo_search (just search)

[jhkennedy]

@bbuzz31 ah, right, yes, you can't pass it directly to the "specialized" search methods, but you can provide it via the asf.SearchOptions class.

I don't know if you'll need it, so I'd try without it. But if you're seeing v2 products in your results without it, you can add it like:

import asf_search as asf

opts = asf.ASFSearchOptions(
    shortName='ARIA_S1_GUNW',
    relativeOrbit=[14],
    flightDirection=asf.FLIGHT_DIRECTION.ASCENDING,
    processingLevel=asf.constants.GUNW_STD,
    intersectsWith='POLYGON((34.9254 24.4438,38.8022 24.4438,38.8022 28.9871,34.9254 28.9871,34.9254 24.4438))',
    collectionAlias=False,
)
scenes = asf.geo_search(opts=opts)