All kg-obo index links are broken (but just on kghub.io)

[caufieldjh]

Describe the bug

All links on https://kghub.io/kg-obo/index.html are 404 (or rather they raise NoSuchKey errors because the paths don't exist in the S3 bucket).

To Reproduce

Visit https://kghub.io/kg-obo/zfa/ or any other subdirectory of https://kghub.io/kg-obo/index.html

Expected behavior

The above should resolve to https://kghub.io/kg-obo/zfa/index.html

Additional context

Links should all be hardcoded to https://kghub.io/kg-obo/ontology_name/index.html

[caufieldjh]

Note that links at https://kg-hub.berkeleybop.io/kg-obo/ work as expected, e.g. https://kg-hub.berkeleybop.io/kg-obo/zfa/ .
So I suspect it's the redirect that's causing this issue

[caufieldjh]

Should be able to do it with a redirection rule: https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-page-redirect.html
The examples in the docs don't quite give me enough detail to create a blanket rule to always redirect foo/ to foo/index.html so looking for more examples

[caufieldjh]

Related to #14

[kltm]

Okay, it is easy to get confused here, as a couple of red herrings are built into the system at this point, like Cloudfront being fully setup, but not activated (related to another item when we were leaving the berkeley proxy). If testing, it's good to keep in mind that in the indexes have links to "bad" locations #19 . As well, it seems that distro invalidations may have not been applied consistently (hard to tell now), which may have caused some confusion. (It did for me.)

So, in order, from bottom to top, it currently looks like:

• There is a public S3 bucket kg-hub-public-data, that is setup as a "website"
• This bucket has an "artificial" index.html (to simulate apache2, etc) generated by a script, and is uploaded to the bucket in each path
• This bucket has two Cloudfront distributions associated with it
  • d33btrxvoajrjl.cloudfront.net (alt kghub.io)
  • d14ydh29js64ek.cloudfront.net (alt kg-hub.berkeleybop.io)
• These distributions provide HTTPS support and domain coverage
• In Route 53, kghub.io is A record for d33btrxvoajrjl.cloudfront.net

The issue that we had here is that the first distro had a slightly incorrect setup for what we want to be doing now (possibly as part of an earlier attempt to incorporate Cloudflare, as was being required at LBL for other projects). Instead of the distro pointing to the "website" endpoint of the bucket, it pointed to the "naked" bucket contents (i.e. kg-hub-public-data.s3.us-east-1.amazonaws.com rather than kg-hub-public-data.s3-website-us-east-1.amazonaws.com). An additional factor may have also been that the bad distro also used more modern settings, rather than the "legacy" settings as found in the d14ydh29js64ek distro, which may have caused some confusion.

[kltm]

@caufieldjh Reports as working now. Can close?

[caufieldjh]

Thanks @kltm !

[caufieldjh]

test:

graph TD;
    Route53["Route 53"] --> A_Record["kghub.io A Record<br/>d33btrxvoajrjl.cloudfront.net"];
    S3_Bucket["S3 Bucket<br/>kg-hub-public-data"] --> Website["Website Configuration"];
    Website --> Index["artificial index.html"];
    Index --> Script["Script for Generation"];
    Script -->|Upload| S3_Bucket;
    S3_Bucket --> Cloudfront1["d33btrxvoajrjl.cloudfront.net<br/>(alt kghub.io)"];
    S3_Bucket --> Cloudfront2["d14ydh29js64ek.cloudfront.net<br/>(alt kg-hub.berkeleybop.io)"];
    Cloudfront1 --> HTTPS["HTTPS Support"];
    Cloudfront2 --> HTTPS;

Loading