PhoenixKitEcommerce.Workers.ShopifyMediaSyncWorker (PhoenixKitEcommerce v0.5.11)

Copy Markdown View Source

Oban worker driving the three Block 7 catalogue writers as background jobs — Task 5 of docs/superpowers/plans/2026-09-06-block7-shopify- media-collections.md.

Job arguments

%{"kind" => "images" | "variants" | "collections", "actor_uuid" => uuid_or_nil}kind selects which writer runs; actor_uuid is the Storage file owner for "images" (ignored by the other two kinds, carried uniformly anyway so the sync page never has to special-case the enqueue call per button).

Products -> items: how "images"/"variants" find their item

Both kinds fetch every Shopify product once (opts[:client], default AdminClient.fetch_products/2) and, for each one, look up the catalogue item it belongs to: first by data["ecommerce"]["shopify"] ["product_id"] (stringified), then — because product_id is only backfilled onto an item the first time a regular field sync applies a change to it (Writer.update_from_shopify/3; see Task 1), so plenty of items carry only handle until that has happened — by data ["ecommerce"]["shopify"]["handle"].

A product already in the catalogue (matched either way) always syncs, no matter what — the sync scope below never gates an item the catalogue already has. Only an UNMATCHED product consults the scope: in scope, it's recorded as an error ("no_matching_item", as before — the operator should add it via "New in Shopify" on the sync page); out of scope, its absence is by design and is counted in the progress record's "skipped" field instead, with no error entry at all. Without this, a store that deliberately syncs a 665-product subset of a 2750-product catalog saw ~2215 "errors" on every run, drowning the handful of matched products that actually failed.

Sync scope (PhoenixKitEcommerce.Shopify.SyncScope)

Loaded ONCE per run (SyncScope.get/0, opts[:scope] overrides it — tests inject a fixed scope rather than round-tripping through phoenix_kit_shop_config), not once per product — same batch philosophy currency_verdict_for/2 already uses for the currency guard below. "collections" never consults it: CollectionSync's own "shopify_collections_filter" is a completely separate allowlist (which Shopify COLLECTIONS become categories at all), not this one (which unmatched Shopify PRODUCTS count as missing versus intentionally excluded).

Each product is independent: a Writer.sync_images/3 or sync_variants/2 failure on one product is recorded in the run's errors list and the loop moves on to the next product — one bad product must not stop the other ~664. This is CSVImportWorker's own per-row philosophy, not CollectionSync's (which halts on a write failure because collection membership assignment is one connected pass, not independent rows — save a catalogue refusing an item's category, which it logs and skips).

"variants" and the currency guard (per-domain-currency design §7.5)

Writer.sync_variants/2 writes per-option-value price modifiers straight from Shopify's variant "price" strings — a second price-writing path entirely outside Shopify.Sync.apply_change/3/ apply_changes/3, which this worker never calls. Without its own guard, a store-currency switch would leave a product's base price frozen in the old currency right next to option modifiers freshly computed in the new one — worse than either being wrong alone, since the two halves of one price would then disagree with nothing to reveal it. So a "variants" run calls Sync.currency_verdict/1 ONCE, before the per-product loop (not once per product — this is the same batch philosophy apply_changes/3 uses, reusing that exact function rather than a second implementation of its lookup/fail-open rules); on a mismatch every product in the run is skipped for "variants" with {:error, {:currency_mismatch, shop, base}} recorded in its own errors entry, and Writer.sync_variants/2 is never called at all. "images" and "collections" carry no money and are entirely unaffected — the lookup isn't even attempted for them.

"collections"

Delegates entirely to PhoenixKitEcommerce.Shopify.CollectionSync.run/1 — a single unit of work (total: 1), whose own {:ok, stats} map is kept as the progress record's "result" for the sync page to display ("last result"); actor_uuid plays no part here.

Progress record

One phoenix_kit_shop_config row PER KIND, key "shopify_media_sync:" <> kind — three independent rows, not the single "shopify_media_sync" row this worker used before per-kind storage existed. Running "variants" no longer erases "images"'s last result: an operator who ran images, then variants, could not previously tell whether images had run at all, let alone whether it succeeded or found nothing to do.

%{"kind" => "images" | "variants" | "collections",
  "total" => non_neg_integer(), "done" => non_neg_integer(),
  "skipped" => non_neg_integer(), "matched" => non_neg_integer(),
  "stats" => map(),
  "errors" => [%{"product" => String.t(), "reason" => String.t()}],
  "started_at" => iso8601, "finished_at" => iso8601 | nil,
  "result" => map() | nil}

"skipped" — unmatched Shopify products the sync scope excluded (see above); "matched" — products that found a catalogue item, whether or not that product's own sync had an error. "stats" aggregates each kind's own writer counts across the whole run: for "images", %{"downloaded" => n, "reused" => n, "attached" => n} summed from every Writer.sync_images/3 result; for "variants", %{"values_created" => n} summed from every Writer.sync_variants/2 result (%{} for "collections", which carries its own summary under "result" instead — see below). "total"/"done" still count every Shopify product this run looked at (matched + skipped + unmatched in-scope errors), same as before.

A job in flight has "finished_at" => nil; a caller reading this to decide whether to disable a button matches progress["kind"] against the button's own kind first — get_progress/1 already does this by construction (one row per kind). Every write also broadcasts on topic/0 (Manager.broadcast/2) so the sync page's LiveView can update live instead of polling — mirrors CSVImportWorker's own shop:import:* broadcasts; the broadcast payload is still one kind's progress map, unchanged, so handle_info/2 on the receiving end only needs to learn to file it under its own "kind".

get_progress/0/get_progress/1 fall back to the legacy single "shopify_media_sync" row for the one kind it names, so a stand that ran a sync before this change doesn't lose that last result the first time it reads progress under the new keys — nothing is ever written back to the legacy key again, only read from it as a fallback.

A no-op — {:error, :catalogue_source_inactive} — when ProductSource.current/0 isn't Catalogue (checked here too, even though every writer this dispatches to already self-gates: fetching Shopify products and building the item index first would be wasted work under the legacy source).

Summary

Functions

Reads every kind's current (or last) progress record — %{"images" => progress | nil, "variants" => ..., "collections" => ...}. See this module's moduledoc ("Progress record") for the legacy-key fallback.

Reads one kind's current (or last) progress record, nil if none exists yet.

Runs one sync kind directly — what perform/1 calls with production defaults. opts

The PubSub topic the sync page subscribes to for live progress.

Functions

get_progress()

@spec get_progress() :: %{required(String.t()) => map() | nil}

Reads every kind's current (or last) progress record — %{"images" => progress | nil, "variants" => ..., "collections" => ...}. See this module's moduledoc ("Progress record") for the legacy-key fallback.

One query for all four rows (the three per-kind keys plus the legacy single key) rather than get_progress/1 called three times — each of which would itself issue up to two queries (the per-kind key, then the legacy fallback) — which would mean up to 6 round-trips for a page that reads this once per mount/render.

get_progress(kind)

@spec get_progress(String.t()) :: map() | nil

Reads one kind's current (or last) progress record, nil if none exists yet.

run(kind, actor_uuid, opts \\ [])

@spec run(String.t(), String.t() | nil, keyword()) ::
  {:ok, map()} | {:error, :catalogue_source_inactive | term()}

Runs one sync kind directly — what perform/1 calls with production defaults. opts:

  • :client — module with fetch_products/2 (images/variants) and/or fetch_collections/1/fetch_collection_product_ids/2 (collections); defaults to AdminClient.
  • :downloader — forwarded to Writer.sync_images/3's own opts[:downloader].
  • :integration_uuid — skips resolving the shop's Shopify connection (tests inject this; production always resolves it).
  • :scope — a PhoenixKitEcommerce.Shopify.SyncScope.t() overriding the default SyncScope.get/0 lookup (tests inject a fixed scope); only consulted for "images"/"variants", never "collections" (see this module's moduledoc, "Sync scope").

Exists as a public function, separate from perform/1, so tests can exercise the real logic without going through Oban/HTTP.

topic()

@spec topic() :: String.t()

The PubSub topic the sync page subscribes to for live progress.