205 add geopackage handling in incore services

[ywkim312]

This PR has the followings

• service recognizes the geopackage file
• dataset's format should be "geopackage"
• check if geopackage has guid
• check if geopackage has only one layer
• check if geopackage's data table name is the same as geopackage file name (this is necessary for posting it to geoserver)
• geoserver should be able to handle geoserver upload
• when uploading geoserver, it has to have dataset id as a name instead of database name

Here's how to test

• Before the test, please download guid_yes.zip and geopackages.zip file that are attached in the next comment
• You don't need to unzip guid_yes.zip but need to unzip geopackages.zip file since the github attachment doesn't allow to attach gpkg files.

Scenario 1 (should make an error) - upload shapefile to geopackage dataset

1. create a new dataset using the json
   {"format": "geopackage", "title": "test-delete", "description": "delete me", "dataType": "incore:addressPoints"}
2. using the dataset created above, attach the guid_yes.zip file (shpaefile) from the attachment
3. this should give the error saying the given file is not a geopackage file
4. this should be the same if you upload unzipped shapefile if you test uploading after unzip the zip file
5. after finishing the test, please remove the dataset created in step 1

Scenario 2 (should make an error) - upload geopackage file to shapefile dataset

1. create a new dataset using the json
   {"format": "shapefile", "title": "test-delete", "description": "delete me", "dataType": "incore:addressPoints"}
2. using the dataset created above, attach the 'guid_test.gpkg' file from the attachment
3. this should give the error saying the format and file doesn't match
4. after finishing the test, please remove the dataset created in step 1

Scenario 3 (should make an error) - upload geopakcage file that has no guid field

1. create a new dataset using the json
   {"format": "geopackage", "title": "test-delete", "description": "delete me", "dataType": "incore:addressPoints"}
2. using the dataset created above, attach the 'guid_no.gpkg' file from the attachment
3. this should give the error saying there is no guid field
4. do not delete the dataset created in step 1 since it will be used in the next scenario

Scenario 4 (should make an error) - upload geopackage file that has multiple layers in a single file

1. use the dataset that was created in Scenario 3
2. using the dataset, attach the 'epn_network.gpkg' file from the attachment
3. this should give the error saying the data is not a single layer
4. do not delete the dataset create in Scenario 3 yet since it will be used in the next scenario as well

Scenario 5 (working scenario) - upload geopackage file with guid and single layer

1. use the dataset that was created in Scenario 3
2. using the dataset, attach the 'guid_test.gpkg' file from the attachment
3. this should work without any error
4. the updated dataset has one entry in the fileDescriptor
5. the updated dataset should have bounding box values
6. please try to see the preview in the data viewer

• this will show only guid_test.gpkg but geoserver should have the dataset with correct id
• the frontend should be updated to show the geopakcage as well
• you can check the dev geoserver with dataset id to see if the dataset is there (optional)
• if you see the dataset, the renaming and uploading the geoserver is successful (optional)

7. please delete the dataset created in the test

[ywkim312]

here are the file for the test
guid_yes.zip
geopackages.zip

[ylyangtw]

I tested scenarios 1-3, but all seemed to work and didn't get errors.. Did I test it the right way?

[ylyangtw]

Tested 5 scenarios locally. Here is the result:

Scenario 1: Give file is not a geopackage file.
Scenario 2: The attached file is geopackage but dataset's format is not geopackage.
Scenario 3: Geopackage is not a single layer or layer name is not the same as file name.
Scenario 4: Geopackage is not a single layer or layer name is not the same as file name.
Scenario 5: Works without errors. It has boundingBox and fileDescriptors.

Test results make sense and code changes look good. Approving~

[longshuicy]

I'm not familiar with uploading geopackage to geoserver. Just curious, why is it different than posting tif or shp? specifically, any reason why it needs to rename the datastore?
also why does the layer name has to match the file name?

Maybe just a quick walk over would be greatly helpful.

[ywkim312]

I'm not familiar with uploading geopackage to geoserver. Just curious, why is it different than posting tif or shp? specifically, any reason why it needs to rename the datastore? also why does the layer name has to match the file name?

Maybe just a quick walk over would be greatly helpful.

It was because geopakcage is not just a simple file like shp or tiff. Geopakcage contains sqlite data in the file and it becomes the layer name when it gets uploaded to geoserver. We are making the geoserver's layer name as dataset id so the visualization of the map should be chained using this dataset id in frontend or pyincore-viz. However, if it is just database name that might be very genetic, we will not be able to connect the dataset and geoserver layer, unless the dataset entry keeps the geopackage database name, which is not happening currently. Shp and tiff are easily being posted with the dataset id as layer name, but geopcakge is not because the file wraps the geodatabase in it and it decides the layer name. By doing the method in this PR, I was able to create layer name as a dataset id. The previous method was renaming the database name inside it by duplicating it which could take very long if the data is big, but using the way of this PR, it doesn't require any additional time.

[longshuicy]

Code looks good. Approve.