Question: permantent / evergreen campaigns, AKA how to not destroy the membership at the last step?

[feliperaul]

Imagine you have an EvergreenCampaign that you subscribe your new users.

You want to keep adding content to the bottom of this campaign, and make so that all your current users also receive it.

However, subscribers memberships are removed when they reach the last step.

Since this is an EverGreen campaign (one that you probably subscribed with concurrent: true), how to make so that nobody gets unsubscribed at the end, being permanently in the state of "waiting for the next step" and, if one new step is created, existing users receive it?

[feliperaul]

@joshuap

If my use-case is currently not supported, what do you think about this hacky workaround, creating a blank custom step with a looong wait time at the end:

step :email_1
step :email_2
# Insert new steps here
step :do_not_unsubscribe, wait: 5.years { nil }

When desired, a new step email_3 can be inserted BEFORE the do_not_unsubscribe step, so when the Schedule run for the next time, users that were waiting with the last step in email_2 will get the email_3.

[joshuap]

Imagine you have an EvergreenCampaign that you subscribe your new users.

You want to keep adding content to the bottom of this campaign, and make so that all your current users also receive it.

However, subscribers memberships are removed when they reach the last step.

Since this is an EverGreen campaign (one that you probably subscribed with concurrent: true), how to make so that nobody gets unsubscribed at the end, being permanently in the state of "waiting for the next step" and, if one new step is created, existing users receive it?

Hi @feliperaul, I forget the exact reason why I didn't do this, but there was some implementation issue. It may have something to do with the campaign stacking feature (heya sends campaigns sequentially by default, unless you use the concurrent option). I'd love to go back and support leaving users in the campaign once they reach the end, but it's tricky.

Here's another possible workaround: you could re-add all users to the campaign when you add a new email to it. As long as you don't use the restart option, users who completed the campaign previously will start at the end.

@joshuap

If my use-case is currently not supported, what do you think about this hacky workaround, creating a blank custom step with a looong wait time at the end:

step :email_1
step :email_2
# Insert new steps here
step :do_not_unsubscribe, wait: 5.years { nil }

When desired, a new step email_3 can be inserted BEFORE the do_not_unsubscribe step, so when the Schedule run for the next time, users that were waiting with the last step in email_2 will get the email_3.

I think this approach could work, but you might want to test it locally first.

[feliperaul]

@joshuap Thanks for the quick reply Joshua!

I don't have much time this week so I'll probably go for one of these workarounds.

I'll try to test the 'very far away step' as well and I'll post back my findings! Cheers.

[feliperaul]

@joshuap Joshua, I'm coming back to this because I think i might have found a bug, but just want to double check expected behavior first.

Picture this:

# clean start
Heya::CampaignReceipt.all.each(&:destroy)
Heya::CampaignMembership.all.each(&:destroy)

class ContentMarketingCampaign < ApplicationCampaign

  default wait: 1.second

  step :step_1, subject: "step_1"
  step :step_2, subject: "step_2"
  step :step_3, subject: "step_3"
end

I then add my user to this campaign using ContentMarketingCampaign.add(user), and invoke Heya::Campaigns::Scheduler.new.run 3 times, making him receive all 3 e-mails and be removed from the campaign.

Then, we add a step_4 to the campaign, reload! the console, and add user back to it again by ContentMarketingCampaign.add(user) (notice I'm not using restart, because I don't want him getting step_1 and step_2 and step_3 again, only step_4).

Based on these two FAQs:

I was under the impression that when the scheduler was invoked for the next time (Heya::Campaigns::Scheduler.new.run), user would receive step_4 immediately, because it would check step_1 thru step_3 had already been sent (he had receipts for them), so they would be ignored, and since the next step is step_4, and the wait time is <= than the elapsed wait, he should get it immediately.

However, when Heya::Campaigns::Scheduler.new.run is invoked again, what actually happens is that no e-mail is sent, and the user's Heya::CampaignMembership gets updated with step_gid pointing to "gid://heya/Heya::Campaigns::Step/ContentMarketingCampaign%2Fstep_2".

Then Heya::Campaigns::Scheduler.new.run is invoked again, no e-mail is sent, and the user's Heya::CampaignMembership gets updated with step_gid pointing to "gid://heya/Heya::Campaigns::Step/ContentMarketingCampaign%2Fstep_3".

So, is this the expected behavior? Or I'm under the correct impression that this is a bug, and he should receive step_4 immediately?

PS: Even tough I contributed to the gem before, I tried debugging this, but the steps part uses an ActiveRecord extension, and I'm not familiar with that Arel SQL enough to help; it seems you are creating a somewhat virtual table with the steps dynamically generated, which is rad (had never seen this before and I want to study it more in the future).

[joshuap]

@feliperaul ahhh, yes, I see what's going on. I think this may be a bug, or at least unintended behavior.

Background: when a user is added to a campaign, we create a new CampaignMembership. The two attributes that control scheduling are step_gid (the current step we're waiting for) and last_sent_at (the time a step was last successfully processed). When a user matches a segment and an email is sent, we create a CampaignReceipt which stores a permanent record of that step being sent to that user. When a user exits the campaign, their CampaignMembership is destroyed, but the CampaignReceipt(s) stay. This is what enables Heya to skip steps if the user is re-added later.

I think the bug is that Heya isn't checking for existing CampaignReceipt(s) early enough, and is waiting to process those steps in the meantime. Basically they will still be skipped, but it's waiting to skip them until the scheduler tries to process them (which is after their wait time).

As you can see here, the user always starts a campaign on step 1. What it needs to do is find the first step for which a CampaignReceipt does not exist, and start there. If there is no CampaignReceipt for the user+campaign, then it should start on step 1 like it currently does.

That should fix most of this issue, but there is one other gotcha: last_sent_at defaults to the time the user was last added to the campaign. That means that if the user exits a campaign and then you add a step and re-add them a month from now, it will wait whatever the wait time is for that step. It would be better if last_sent_at would default to whatever it was when they previously exited the campaign, but of course, the previous CampaignMembership was destroyed. What we could do is again, look for the last prior CampaignReceipt for the user+campaign, and if one exists, do something like this: self.last_sent_at = last_campaign_receipt.created_at. That way it should be able to pick up right where it left off. 🤞

Does this all make sense?

I'm happy to look into both of these fixes, but I'd also welcome PRs if you're wanting to contribute. (No pressure either way!)

[joshuap]

What it needs to do is find the first step for which a CampaignReceipt does not exist, and start there. If there is no CampaignReceipt for the user+campaign, then it should start on step 1 like it currently does.

I may also want to do this when selecting the next step in the scheduler here—that way we don't have to wait to skip later steps (since you can add new steps anywhere in a campaign).

[leemcalilly]

FWIW - I'm still running into this problem. Here's the hack solution I tried last that did not work:

Adding a "do not unsubscribe" step way out into the future so that no one gets removed from the campaign.

campaigns/evergreen_campaign.rb

step :do_not_unsubscribe, wait: 10.years { nil }, subject: 'Your subscription has now ended'

Then, I set up a cron job on Render to basically remove then re-add every user once a day. This was the hack that we hoped would work, but once the user gets to the last step that's it. They'll receive the email then they won't receive any more if I add them to the campaign. That cron job is just a rake task I run that looks like this:

namespace :campaigns do
  desc "Re-adds everyone to the campaign"

  task :add_all_users => [ :environment ] do
    User.all.each do |user|
      # Why we're doing this can be found here:
      # https://github.com/honeybadger-io/heya/issues/186#issuecomment-1309630217
      EvergreenCampaign.remove(user)
      EvergreenCampaign.add(user, concurrent: true)
    end
  end
end

I'm not really sure how to best address this use-case, but I would love it if there was a way to set up an Evergreen Campaign that you could add steps to over time and have those new steps get delivered to anyone that hasn't received them yet. Basically let's say I started with 10 steps that sent out one email per week, then I can add emails to it overtime so that no user ever reaches the end. Rather than having to have all the steps they'll ever receive already set up when they first subscribe.

[joshuap]

@leemcalilly sorry it took me so long to get to this. I think I made some progress on the issue tonight. Check out #197 when you get the chance. I think that should fix your workaround.

I'd still like to figure out how to do true evergreen campaigns in the future (where you don't have to re-add users to campaigns).