PhoenixKitEcommerce.Workers.TranslationSweepWorker (PhoenixKitEcommerce v0.5.11)

Copy Markdown View Source

Self-rescheduling Oban worker for the AI-translation reconciliation sweep (design §4.3). Ticks itself roughly every TranslationSweepSettings.interval_minutes/0 minutes, without a cron entry or app-level heartbeat: no host config, and a changed interval takes effect on the very next reschedule (reschedule/0) rather than waiting for a static crontab edit.

The chain

unique: [period: :infinity, states: [:available, :scheduled]] (fixed, argument-free %{} args) keeps the queue holding at most one pending tick at a time. The state list is explicit rather than Oban's default because that default references :suspended, which is absent from the oban_job_state enum on hosts that upgraded the Oban library ahead of its migration — referencing it there raises 22P02 and would kill every insert (the exact reason PhoenixKitAI.TranslateWorker moved its own de-dup off Oban's unique:, see that module's docs).

perform/1 schedules its OWN successor as the very first action, before touching any settings or doing any work, via ensure_scheduled/0 — a tick that crashes after that point still leaves the chain alive. max_attempts: 1 follows from the same fact: retrying a failed tick is pointless when the next one is already queued.

ensure_scheduled/0 is unconditional — it does not check whether the sweep is enabled. The chain runs forever once started (design: "Тик выключенной сверки не прекращает цепочку, а выполняет пустую работу и планирует следующий" — accepted cost, ~24 no-op ticks/day at the default interval) so that a disabled-then-re-enabled sweep needs no manual kick to resume. It's called from three places designed to survive a broken chain regardless of cause (a server restart, Oban pruning, a run of failures): PhoenixKitEcommerce.enable_system/0, a settings save (reschedule/0 below), and the management page's mount/3 (next task). Because uniqueness does the actual de-duplication, calling it from all three concurrently is safe — see the moduledoc on unique: above; every racing call converges on at most one scheduled job.

What one tick does (design §4.3 step order)

  1. Schedule the next tick (ensure_scheduled/0).
  2. Stop, recording why, unless shop_translations_enabled is on AND AI is actually usable (PhoenixKitAI.Translations.available?/0 and a resolved default endpoint — available?/0 alone doesn't confirm the configured endpoint still exists and is enabled). The SCHEDULED tick (run_tick/0) additionally requires shop_translation_sweep_enabled — the manual "Запустить сверку" button (run_manual_tick/0) does not, by owner decision: that setting gates automatic scheduling only. Checking every toggle fresh on every tick (never cached in the job) means a state flipped by direct SQL is honoured immediately, not on the next code deploy.
  3. Stop, recording why (:sweep_stalled), if shop_translation_batch or shop_translation_max_in_flight is below 1 — no candidate can ever be selected under either (structurally_stalled?/3, Fix C). A ceiling below the number of configured target languages is the third arm of that predicate, but it does NOT stop the tick: a candidate whose own language gap fits the ceiling is still enqueued. It only decides the REASON recorded when a tick selects nothing — :sweep_stalled rather than :ok, since under that config the emptiness is permanent (take_within_budget/3 halts, never skips, so the first resource missing every target language blocks everything queued behind it), not the healthy idle 0 it is otherwise indistinguishable from. The operational panel (design §4.6) refuses to save a config this broken in the first place — this check is what stays honest about one that reached storage anyway (a pre-fix install, a hand-edited row).
  4. Stop, recording why, if the shop's incomplete TranslateWorker jobs (available/scheduled/executing/retryable — the same four states PhoenixKitAI.Translations dedups against; a snoozed job is scheduled and still counts) are already at or past shop_translation_max_in_flight. The ceiling counts JOBS, not resources — a resource with N stale languages contributes N.
  5. Select candidates: every stale/missing category first (no status filter — a hidden category would otherwise ship translated navigation before it's visible), then products filtered by shop_translation_statuses. Selection stops before the running job-count would exceed the remaining ceiling budget, AND before the resource count would exceed shop_translation_batch (take_within_budget/3) — two independent caps, combined across categories and products in one tick.
  6. Enqueue missing ∪ stale languages per selected resource via PhoenixKitAI.Translations.enqueue_all_missing/2. That call's own app-level de-dup means a resource already mid-translation (manual action, a previous tick's snoozed job) is skipped without this worker needing to check first.

Every stop — expected (disabled, AI down, ceiling) or a completed run — is recorded via finish/2 into the shop_translation_sweep_last_run setting, readable through last_run/0, so the management page (next task) can show "last tick: …" without re-deriving it live; status/0 bundles that with the live "when does the next one fire" instead.

Summary

Functions

Ensures a tick is scheduled (or already is — see the moduledoc on unique:). Unconditional: does not check sweep_enabled?/0, because the chain itself is meant to run forever once started (see moduledoc).

The persisted outcome of the most recent tick — nil before the first tick has ever run. Shape: %{"reason" => string, "at" => iso8601 string, ...}; the extra keys vary by reason (see run_tick/0's docs for the reason list). This is how the management page (next task) shows "why the sweep last stopped" without re-deriving it live — status/0 below is the live version, for "is it configured to run at all right now".

The scheduled_at of the pending tick, if any (design §4.5: "следующий тик в HH:MM").

Cancels whatever tick is currently scheduled (an available tick — one already due to run — is left alone; it will read the new interval when scheduling ITS successor) and schedules a fresh one at the currently-configured interval.

Manual twin of run_tick/0, for the management page's "Запустить сверку" button — called directly for immediate feedback, without disturbing the scheduled tick (an Oban-inserted immediate job would just collide with the same uniqueness that keeps the chain single-instance).

The tick's body, with the scheduling step removed — this is what perform/1 runs after scheduling its successor. This is the AUTOMATIC path: it stops (:sweep_disabled) when shop_translation_sweep_enabled is off, exactly as before. The management page's "Запустить сверку" button calls run_manual_tick/0 below instead, not this function.

Live scheduling status — next_tick_at/0 plus last_run/0 — bundled for the management page's sweep block (design §4.5).

Fix C: true when batch_size/max_in_flight, as currently configured, cannot be relied on to ever let take_within_budget/3 select a candidate — independent of any particular tick's in-flight count or candidate list.

Design §4.3: the limit is counted in JOBS (one per candidate language), not resources — job_budget — while resource_budget (shop_translation_batch) independently caps how many resources a single tick touches. Selection stops BEFORE either running total would be exceeded, never partially consuming a candidate's language list.

Functions

ensure_scheduled()

@spec ensure_scheduled() :: {:ok, Oban.Job.t()} | {:error, term()}

Ensures a tick is scheduled (or already is — see the moduledoc on unique:). Unconditional: does not check sweep_enabled?/0, because the chain itself is meant to run forever once started (see moduledoc).

Called on shop enable_system/0, after a sweep-settings save (reschedule/0), and on the management page's mount/3.

last_run()

@spec last_run() :: map() | nil

The persisted outcome of the most recent tick — nil before the first tick has ever run. Shape: %{"reason" => string, "at" => iso8601 string, ...}; the extra keys vary by reason (see run_tick/0's docs for the reason list). This is how the management page (next task) shows "why the sweep last stopped" without re-deriving it live — status/0 below is the live version, for "is it configured to run at all right now".

next_tick_at()

@spec next_tick_at() :: DateTime.t() | nil

The scheduled_at of the pending tick, if any (design §4.5: "следующий тик в HH:MM").

reschedule()

@spec reschedule() :: {:ok, Oban.Job.t()} | {:error, term()}

Cancels whatever tick is currently scheduled (an available tick — one already due to run — is left alone; it will read the new interval when scheduling ITS successor) and schedules a fresh one at the currently-configured interval.

Design §4.3: a settings save must call this, or a shortened interval (say 60 minutes down to 5) would not take effect until the stale 60-minute wait finished.

run_manual_tick()

@spec run_manual_tick() :: {atom(), map()}

Manual twin of run_tick/0, for the management page's "Запустить сверку" button — called directly for immediate feedback, without disturbing the scheduled tick (an Oban-inserted immediate job would just collide with the same uniqueness that keeps the chain single-instance).

Owner decision overriding design §4.5 as written: shop_translation_sweep_enabled gates AUTOMATIC scheduling only. An operator-initiated run performs the tick's work regardless of that setting — the documented manual-only mode (badge: "Automatic sweep: off (manual only)") would otherwise make its own "Run sweep" button refuse to run. Every OTHER gate (AI availability, the in-flight ceiling, target languages) still applies exactly as it does for the scheduled tick.

run_tick()

@spec run_tick() :: {atom(), map()}

The tick's body, with the scheduling step removed — this is what perform/1 runs after scheduling its successor. This is the AUTOMATIC path: it stops (:sweep_disabled) when shop_translation_sweep_enabled is off, exactly as before. The management page's "Запустить сверку" button calls run_manual_tick/0 below instead, not this function.

Returns {reason, info}reason is one of :translations_disabled, :product_source_unsupported (the shop reads products from the catalogue, so no shop adapter is registered with phoenix_kit_ai and every job would be discarded — see PhoenixKitEcommerce.translations_supported?/0), :sweep_disabled, :ai_unavailable, :sweep_stalled (Fix C: the tick selected nothing AND the configured batch/ceiling guarantees every future tick will do the same until they change — see structurally_stalled?/3), :ceiling_reached, :no_target_languages, or :ok (ran; info[:enqueued] may still be 0 if nothing needed translating or everything was already in flight — that 0 is never ambiguous with a structural stall, which always gets its own reason above). Every outcome is also persisted — see last_run/0.

status()

@spec status() :: %{next_tick_at: DateTime.t() | nil, last_run: map() | nil}

Live scheduling status — next_tick_at/0 plus last_run/0 — bundled for the management page's sweep block (design §4.5).

structurally_stalled?(batch_size, max_in_flight, target_langs)

@spec structurally_stalled?(non_neg_integer(), non_neg_integer(), [String.t()]) ::
  boolean()

Fix C: true when batch_size/max_in_flight, as currently configured, cannot be relied on to ever let take_within_budget/3 select a candidate — independent of any particular tick's in-flight count or candidate list.

take_within_budget/3 HALTS — never skips — on the first candidate whose language count exceeds the remaining job budget:

  • batch_size < 1 — its resource_budget starts at or below zero, so the very first check in its reduce_while halts before looking at any candidate at all. NOTHING is selectable, ever.
  • max_in_flight < 1 — the in-flight ceiling can never leave a positive job budget for any tick to spend, whatever is or isn't currently running. Again nothing is selectable, ever.
  • max_in_flight < length(target_langs) — a resource missing every configured target language (the worst case, and an entirely ordinary one: any newly added or bulk-imported resource starts there) needs length(target_langs) jobs; a ceiling below that can never admit it, and because the scan halts rather than skips, it blocks every cheaper candidate queued behind it too. Note the narrower claim: this arm does NOT make selection impossible — a candidate whose own language gap fits under the ceiling and sorts ahead of any such blocker is still enqueued, and run_tick/0 deliberately keeps enqueueing it rather than refusing to sweep.

So this predicate answers "is an empty tick under this config permanent rather than incidental?" — run_tick/0 records :sweep_stalled when it holds AND the tick selected nothing, so that emptiness stops reading as a healthy idle sweep. It is not, by itself, a licence to skip the work.

take_within_budget(candidates, resource_budget, job_budget)

@spec take_within_budget([map()], non_neg_integer(), non_neg_integer()) :: [map()]

Design §4.3: the limit is counted in JOBS (one per candidate language), not resources — job_budget — while resource_budget (shop_translation_batch) independently caps how many resources a single tick touches. Selection stops BEFORE either running total would be exceeded, never partially consuming a candidate's language list.