Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hepa: add third party ozone support to automod #894

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

haileyok
Copy link
Contributor

@haileyok haileyok commented Jan 2, 2025

No description provided.

@bnewbold
Copy link
Collaborator

bnewbold commented Jan 2, 2025

I'm probably forgetting something, but can't folks who are self-hosting Ozone use an admin token?

The biggest friction I know of right now is doing full account hydration using the firehose, which will hit rate-limits doing identity resolution and appview fetches. One solution would be to add pre-filtering to the firehose, eg to specific record types, which would drop the event rate significantly. Another would be the option to selectively reduce hydration and verification. We could also look at bumping the rate limits, I guess, if the gap isn't too big.

@haileyok
Copy link
Contributor Author

haileyok commented Jan 2, 2025

I'm probably forgetting something, but can't folks who are self-hosting Ozone use an admin token?

I couldn't get this to work no matter what I tried. For what it's worth, imo this was the more "straightfoward" way to do configuration, though honestly it doesn't matter much if the admin token will actually work. Maybe I did something wrong, but even if I added the atproto-proxy header, the admin token value still wouldn't work for me...unless it's supposed to be some env var in Ozone itself which I couldn't find documented or in the code anywhere (I saw commits where we were actually removing some admin token stuff like last year...)

The biggest friction I know of right now is doing full account hydration using the firehose, which will hit rate-limits doing identity resolution and appview fetches.

Interestingly I wasn't able to hit (unless this fails silently which I doubt) any rate limits (though I didn't run it at peaks, maybe tomorrow I'll run it and see if I can hit it. In either case, I would like to actually have a way in here to support "refetching" metadata (there are some rules where having cached identities isn't great). It seems like something along those lines could work here as well. I.e.:

  • Rule checks post text
  • If it matches some requirements, fetch the identity.

The lookups are fast enough, and the necessity for speed is slightly less in most applications I can think of here (and in fact, in a lot of cases the identity might not even be required). Feels like that might be a decent avenue.

In either case, even if the rate limit gaps are not big right now (or even if its totally fine) I suspect we'd see people hit them during a "big wave", so having some sort of mechanism here no matter what would be nice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants