Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automod: rules engine framework #434

Merged
merged 35 commits into from
Nov 20, 2023
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
d98fd5d
automod: start sketching package API
bnewbold Nov 11, 2023
86161b4
hepa: initial skelton of daemon
bnewbold Nov 11, 2023
5541d2c
automod: building
bnewbold Nov 13, 2023
eb7f8b8
automod: significantly simplify firehose consumption, for now
bnewbold Nov 14, 2023
e4744a3
automod: basic set storage API
bnewbold Nov 14, 2023
85d4cd2
automod: rules sub-package, a bit of tests
bnewbold Nov 14, 2023
0eb79b7
automod: misleading URL and mention rules
bnewbold Nov 14, 2023
f9c1fe6
automod: helper to load sets from JSON file
bnewbold Nov 14, 2023
f9af94d
automod: persist mod actions
bnewbold Nov 14, 2023
1d120a5
automod: more persisting and logging
bnewbold Nov 14, 2023
79df1f0
hepa: wire up mod config
bnewbold Nov 14, 2023
b53bcd4
automod: fix trivial err
bnewbold Nov 14, 2023
fbec7f6
hepa: load set config from JSON file
bnewbold Nov 15, 2023
01ae2e9
automod: redis identity cache; early parallelism; fix rules
bnewbold Nov 15, 2023
b195b04
automod: fix label creation/action
bnewbold Nov 15, 2023
1045d05
automod: redis counters; some refactors
bnewbold Nov 16, 2023
d46ed60
rules: refactor with some helpers
bnewbold Nov 16, 2023
ef07c1e
automod: add generic caching, and hydrate some account meta
bnewbold Nov 16, 2023
6b1c597
automod: fix tests (no XRPC in testing)
bnewbold Nov 17, 2023
c0a1092
HACK: remove did doc from createSession to un-break it (temporarily)
bnewbold Nov 17, 2023
f67fb98
automod: private admin state hydration
bnewbold Nov 17, 2023
37042b1
hepa: persist cursor state in redis
bnewbold Nov 17, 2023
21b494d
syntax: fix AT-URI Path() impl
bnewbold Nov 17, 2023
cc9ed96
automod: process individual pre-existing records (by AT-URI)
bnewbold Nov 17, 2023
c54a5e4
hepa: dockerfile and github build actions
bnewbold Nov 18, 2023
aef1908
makefile and HACKING entries for hepa
bnewbold Nov 18, 2023
a70cd1c
automod brief docs/context
bnewbold Nov 18, 2023
499d1fa
automod: update docs
bnewbold Nov 20, 2023
6aee771
automod: refactor event types
bnewbold Nov 20, 2023
4e599db
automod: refactor out RelayClient, connect to account's PDS directly
bnewbold Nov 20, 2023
a2b29a8
automod: don't actually label, just flag
bnewbold Nov 20, 2023
3539c95
automod: tweak misleading URL processing
bnewbold Nov 20, 2023
695ed76
automod: more URL tweaks
bnewbold Nov 20, 2023
ce3afdd
automod: fix nil error on IdentityEvent logging
bnewbold Nov 20, 2023
c4aefa7
automod: yet more URL cleaning
bnewbold Nov 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
automod: more URL tweaks
  • Loading branch information
bnewbold committed Nov 20, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 695ed761ef5920fe2a2a4e60312602220831a910
7 changes: 6 additions & 1 deletion automod/rules/misleading.go
Original file line number Diff line number Diff line change
@@ -25,7 +25,12 @@ func MisleadingURLPostRule(evt *automod.RecordEvent, post *appbsky.FeedPost) err
continue
}

text := strings.TrimSpace(facet.Text)
// basic text string pre-cleanups
text := strings.TrimSuffix(strings.TrimSpace(facet.Text), "...")
// if really not a domain, just skipp
if !strings.Contains(text, ".") {
continue
}
// try to fix any missing method in the text
if !strings.Contains(text, "://") {
text = "https://" + text