Enable the use of Front and Signalwire for marketing messages

data/messages/templates/sms_compliance/help.en.sms.liquid

@@ -0,0 +1 @@
+Thanks! We'll be in touch soon. STOP to unsubscribe.

data/messages/templates/sms_compliance/optin.en.sms.liquid

@@ -0,0 +1 @@
+Thanks for signing up for suma! Msg&Data rates may apply. STOP to unsubscribe or HELP for assistance.

data/messages/templates/sms_compliance/optout.en.sms.liquid

@@ -0,0 +1 @@
+You have successfully unsubscribed from suma and will no longer receive SMS messages. START to resubscribe.

db/migrations/057_front_marketing.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+Sequel.migration do
+  change do
+    alter_table(:message_preferences) do
+      rename_column :account_updates_optout, :account_updates_sms_optout
+      add_column :account_updates_email_optout, :boolean, default: false, null: false
+      add_column :marketing_sms_optout, :boolean, default: false, null: false
+      add_column :marketing_email_optout, :boolean, default: false, null: false
+    end
+  end
+end

db/migrations/058_signalwire_unsubscribe.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+Sequel.migration do
+  up do
+    if ENV["RACK_ENV"] == "test"
+      run <<~SQL
+        CREATE TABLE signalwire_message_v1_fixture (
+          pk bigserial PRIMARY KEY,
+          signalwire_id text UNIQUE NOT NULL,
+          date_created timestamptz,
+          date_sent timestamptz,
+          date_updated timestamptz,
+          direction text,
+          "from" text,
+          status text,
+          "to" text,
+          data jsonb NOT NULL
+        );
+        CREATE INDEX IF NOT EXISTS svi_fixture_date_created_idx ON signalwire_message_v1_fixture (date_created);
+        CREATE INDEX IF NOT EXISTS svi_fixture_date_sent_idx ON signalwire_message_v1_fixture (date_sent);
+        CREATE INDEX IF NOT EXISTS svi_fixture_date_updated_idx ON signalwire_message_v1_fixture (date_updated);
+        CREATE INDEX IF NOT EXISTS svi_fixture_from_idx ON signalwire_message_v1_fixture ("from");
+        CREATE INDEX IF NOT EXISTS svi_fixture_to_idx ON signalwire_message_v1_fixture ("to");
+      SQL
+    end
+  end
+
+  down do
+    run("DROP TABLE signalwire_message_v1_fixture") if ENV["RACK_ENV"] == "test"
+  end
+end

docs/support.md

@@ -0,0 +1,60 @@
+# Customer Support and Marketing
+
+Suma is built with enhanced support to use [Front](https://front.com) for customer support and marketing.
+It is possible to not use Front, and manage support and marketing manually,
+but if you use Front you will get better features, including:
+
+- Automatic updating of Front Contacts based on Suma members
+- Automatic managing of a Front marketing List (subscribe/unsubscribe users)
+
+## Configuration
+
+Set `FRONTAPP_AUTH_TOKEN` to your team's auth token.
+
+See `Suma::Frontapp` for more configuration options.
+
+## Automatic Contact updating
+
+Whenever a `Suma::Member` is created or updated, we create or update the Front Contact.
+See `Suma::Async::FrontappUpsertContact` for more info.
+
+## Automatic marketing List management
+
+**NOTE: Using Front for marketing requires the use of custom channels. You MUST NOT send marketing messages
+directly through Front's email system.**
+
+Instead, you can hook up a [Front Channel like WebhookDB/Signalwire](https://docs.webhookdb.com/guides/front-channel-signalwire/)
+to send messages you compose in Front through Signalwire, and update SMS replies into Front messages.
+Ultimately how you send and sync messages is outside the scope of this document,
+but just don't use the default Front channel to do it.
+
+Aside from that, list management requires more configuration, since we have to manage subscribes and unsubscribes.
+
+Create two lists, one for SMS and another for Email:
+
+- Create a List in Front, from <https://app.frontapp.com/contacts-manager/contacts/lists/all>.
+  Call it something like "SMS Marketing".
+- Then, click on the list. In the URL will be an ID, like `19062177`.
+- Copy this ID, and set `FRONTAPP_MARKETING_SMS_LIST_ID` to it, to identify the list to store subscribed contacts.
+- Create another list called something like "Email Marketing", copy its ID, and set `FRONTAPP_MARKETING_EMAIL_LIST_ID` to it.
+
+Whenever fields on `Suma::Message::Preferences` are modified, Suma will update the lists the member's Front Contact is on.
+
+## Updating Suma Preferences from Email/SMS
+
+The last part of this is automatically updating a member's message preferences from external actions,
+like an email with 'Unsubscribe' or a "STOP" text.
+
+Here are the ways to keep preferences updated:
+
+### SMS via Signalwire and WebhookDB
+
+This requires a [Signalwire Messaging WebhookDB Integration](https://docs.webhookdb.com/integrations/signalwire_message_v1/).
+
+- Set `SIGNALWIRE_MARKETING_NUMBER` and `WEBHOOKDB_SIGNALWIRE_MESSAGES_TABLE`.
+- Whenever a text with a STOP keyword (`SIGNALWIRE_MESSAGE_MARKETING_SMS_UNSUBSCRIBE_KEYWORDS`) is received, opt-out is set.
+- Whenever a text with a START keyword (`SIGNALWIRE_MESSAGE_MARKETING_SMS_RESUBSCRIBE_KEYWORDS`) is received, opt-in is set.
+- Whenenever a text with a HELP keyword (`SIGNALWIRE_MESSAGE_MARKETING_SMS_HELP_KEYWORDS`) is received, no action is taken.
+  Instead, the Front channel will see the inbound message and create a Front conversation,
+  which should be handled by a support agent.
+

lib/suma/api/webhookdb.rb

@@ -2,6 +2,7 @@
 
 require "suma/api"
 
+require "suma/async/signalwire_process_optouts"
 require "suma/async/stripe_refunds_backfiller"
 
 class Suma::API::Webhookdb < Suma::API::V1
@@ -15,5 +16,13 @@ class Suma::API::Webhookdb < Suma::API::V1
       status 202
       present({o: "k"})
     end
+
+    post :signalwire_message_v1 do
+      h = env["HTTP_WHDB_WEBHOOK_SECRET"]
+      unauthenticated! unless h == Suma::Webhookdb.signalwire_messages_secret
+      Suma::Async::SignalwireProcessOptouts.perform_async
+      status 202
+      present({o: "k"})
+    end
   end
 end

lib/suma/async.rb

@@ -27,6 +27,8 @@ module Suma::Async
     "suma/async/deprecated_jobs",
     "suma/async/emailer",
     "suma/async/ensure_default_member_ledgers_on_create",
+    "suma/async/frontapp_list_sync",
+    "suma/async/frontapp_upsert_contact",
     "suma/async/funding_transaction_processor",
     "suma/async/member_onboarding_verified_dispatch",
     "suma/async/message_dispatched",
@@ -36,10 +38,10 @@ module Suma::Async
     "suma/async/process_anon_proxy_inbound_webhookdb_relays",
     "suma/async/reset_code_create_dispatch",
     "suma/async/reset_code_update_twilio",
+    "suma/async/signalwire_process_optouts",
     "suma/async/stripe_refunds_backfiller",
     "suma/async/sync_lime_free_bike_status_gbfs",
     "suma/async/sync_lime_geofencing_zones_gbfs",
-    "suma/async/upsert_frontapp_contact",
   ].freeze
 
   configurable(:async) do

lib/suma/async/deprecated_jobs.rb

@@ -12,4 +12,5 @@
   # Then it can be deleted later.
   "Async::AutomationTriggerRunner",
   "Async::TopicShim",
+  "Async::UpsertFrontappContact",
 )

lib/suma/async/frontapp_list_sync.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require "amigo/scheduled_job"
+require "suma/async"
+require "suma/frontapp/list_sync"
+
+class Suma::Async::FrontappListSync
+  extend Amigo::ScheduledJob
+
+  sidekiq_options(Suma::Async.cron_job_options)
+  cron "50 6 * * *"
+  splay 5
+
+  def _perform
+    return unless Suma::Frontapp.configured?
+    return unless Suma::Frontapp.list_sync_enabled
+    Suma::Frontapp::ListSync.new(now: Time.now).run
+  end
+end

lib/suma/async/upsert_frontapp_contact.rb → lib/suma/async/frontapp_upsert_contact.rb

@@ -2,7 +2,7 @@
 
 require "amigo/job"
 
-class Suma::Async::UpsertFrontappContact
+class Suma::Async::FrontappUpsertContact
   extend Amigo::Job
 
   on(/^suma\.member\.(created|updated)$/)
@@ -16,6 +16,4 @@ def _perform(event)
     end
     member.frontapp.upsert_contact
   end
-
-  Amigo.register_job(self)
 end

lib/suma/async/signalwire_process_optouts.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+require "amigo/scheduled_job"
+require "suma/async"
+require "suma/message/signalwire_webhookdb_optout_processor"
+
+class Suma::Async::SignalwireProcessOptouts
+  extend Amigo::ScheduledJob
+
+  sidekiq_options(Suma::Async.cron_job_options)
+  cron "*/30 * * * *"
+  splay 60
+
+  def _perform
+    Suma::Message::SignalwireWebhookdbOptoutProcessor.new(now: Time.now).run
+  end
+end

lib/suma/frontapp.rb

@@ -15,11 +15,12 @@ class << self
     # @return [Frontapp::Client]
     attr_accessor :client
 
-    def configured? = self.auth_token != UNCONFIGURED_AUTH_TOKEN
+    def configured? = self.auth_token != UNCONFIGURED_AUTH_TOKEN && self.auth_token.present?
   end
 
   configurable(:frontapp) do
     setting :auth_token, UNCONFIGURED_AUTH_TOKEN
+    setting :list_sync_enabled, false
 
     after_configured do
       self.client = Frontapp::Client.new(auth_token: self.auth_token, user_agent: Suma::Http.user_agent)

lib/suma/frontapp/list_sync.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+class Suma::Frontapp::ListSync
+  RECENTLY_UNVERIFIED_CUTOFF = 2.weeks
+
+  def initialize(now:)
+    @now = now
+  end
+
+  def run
+    specs = self.gather_list_specs
+    spec_names = specs.to_set(&:full_name)
+    groups = Suma::Frontapp.client.contact_groups
+    # The easiest way to bulk-replace all the contacts is to delete and recreate the group
+    groups_to_replace = groups.select { |g| spec_names.include?(g.fetch("name")) }
+    groups_to_replace.each do |group|
+      Suma::Frontapp.client.delete_contact_group!(group.fetch("id"))
+    end
+    specs_with_members = specs.reject { |sp| sp.dataset.empty? }
+    # Since create contact group does not return the ID, we create and then re-fetch
+    specs_with_members.each do |spec|
+      Suma::Frontapp.client.create_contact_group!(name: spec.full_name)
+    end
+    # Find the group we just created, and add all the contacts do it
+    groups = Suma::Frontapp.client.contact_groups
+    specs_with_members.each do |spec|
+      existing_group = groups.find { |g| g.fetch("name") == spec.full_name }
+      raise Suma::InvalidPostcondition, "cannot find the group we just created: #{spec.full_name}" if
+        existing_group.nil?
+      contact_ids = spec.dataset.select_map(:frontapp_contact_id)
+      next if contact_ids.empty?
+      Suma::Frontapp.client.add_contacts_to_contact_group!(existing_group.fetch("id"), {contact_ids:})
+    end
+  end
+
+  def gather_list_specs
+    result = []
+    result.concat(
+      ListSpec.for_languages(
+        name: "Marketing",
+        transport: :sms,
+        dataset: Suma::Member.where(preferences: Suma::Message::Preferences.where(marketing_sms_optout: false)),
+      ),
+    )
+    result.concat(
+      ListSpec.for_languages(
+        name: "Unverified",
+        transport: :sms,
+        dataset: Suma::Member.where { created_at > RECENTLY_UNVERIFIED_CUTOFF.ago }.where(onboarding_verified_at: nil),
+      ),
+    )
+    Suma::Organization.all.each do |org|
+      result.concat(
+        ListSpec.for_languages(
+          name: org.name,
+          transport: :sms,
+          dataset: Suma::Member.where(organization_memberships: org.memberships_dataset),
+        ),
+      )
+    end
+    return result
+  end
+
+  class ListSpec < Suma::TypedStruct
+    attr_reader :name, :transport, :language, :dataset
+
+    def self.for_languages(**kw)
+      return Suma::I18n::SUPPORTED_LOCALES.values.map do |locale|
+        self.new(language: locale.code, **kw)
+      end
+    end
+
+    def initialize(**kw)
+      super
+      preferences_ds = Suma::Message::Preferences.where(
+        "#{self.transport}_enabled": true, preferred_language: self.language,
+      )
+      @dataset = self.dataset.
+        not_soft_deleted.
+        exclude(frontapp_contact_id: "").
+        where(preferences: preferences_ds)
+    end
+
+    def full_name
+      lang = Suma::I18n::SUPPORTED_LOCALES.fetch(self.language).language
+      "#{self.name} - #{self.transport.to_s.upcase} - #{lang}"
+    end
+  end
+end

lib/suma/member/frontapp_attributes.rb

@@ -32,12 +32,10 @@ def _update_contact
   end
 
   def _contact_body
-    # In the future, we would look at things like organizations to add custom fields,
-    # like the housing partner they are a part of.
-    custom_fields = {}
     body = {
       links: [@member.admin_link],
-      custom_fields:,
+      # NOTE: Setting things like customFields or groupNames will REPLACE existing ones,
+      # so be very careful if we end up using them.
     }
     body[:name] = @member.name if @member.name.present?
     body

lib/suma/message/preferences.rb

@@ -4,6 +4,8 @@
 require "suma/postgres"
 require "suma/message"
 
+# TODO: We have some assumptions about SMS right now; when we add in email support into the UI,
+# stuff like subscription groups will need per-channel optin/out behavior.
 class Suma::Message::Preferences < Suma::Postgres::Model(:message_preferences)
   plugin :timestamps
 
@@ -45,12 +47,18 @@ def subscriptions
     groups = []
     groups << SubscriptionGroup.new(
       model: self,
-      optout_field: :account_updates_optout,
+      optout_field: :account_updates_sms_optout,
       key: :account_updates,
-      opted_in: !self.account_updates_optout,
+      opted_in: !self.account_updates_sms_optout,
+      editable_state: "on",
+    )
+    groups << SubscriptionGroup.new(
+      model: self,
+      optout_field: :marketing_sms_optout,
+      key: :marketing,
+      opted_in: !self.marketing_sms_optout,
       editable_state: "on",
     )
-    groups << SubscriptionGroup.new(key: :marketing, opted_in: false, editable_state: "hidden")
     groups << SubscriptionGroup.new(key: :security, opted_in: true, editable_state: "off")
     return groups
   end
@@ -67,6 +75,29 @@ def validate
     super
     self.validates_includes Suma::I18n.enabled_locale_codes, :preferred_language
   end
+
+  # @!attribute sms_enabled
+  # True if the member has a phone number and is okay receiving SMS.
+  # Note that if sms_enabled is false, it will override any 'true' settings on individual subscription groups.
+  # @return [TrueClass,FalseClass]
+
+  # @!attribute email_enabled
+  # True if the member has an email and can receive emails.
+  # Note that if email_enabled is false, it will override any 'true' settings on individual subscription groups.
+  # @return [TrueClass,FalseClass]
+
+  # @!attribute account_updates_sms_optout
+  # True if the user has opted out of receiving account update messages via SMS.
+  # @return [TrueClass,FalseClass]
+
+  # @!attribute account_updates_email_optout
+  # @return [TrueClass,FalseClass]
+
+  # @!attribute marketing_sms_optout
+  # @return [TrueClass,FalseClass]
+
+  # @!attribute marketing_email_optout
+  # @return [TrueClass,FalseClass]
 end
 
 # Table: message_preferences