PhoenixKitEcommerce.TranslationFingerprint (PhoenixKitEcommerce v0.5.10)

Copy Markdown View Source

The staleness model shared by every AI-translation adapter (design doc §4.1/§4.3/§4.4): fingerprinting, the four-state computation, the per-field write-narrowing decision, and the hash-in-the-database candidate query. AITranslatable (products) and the category adapter both build on this rather than reimplementing it — the model has to stay identical across resource types or the sweep (design §4.3) can't treat them uniformly.

Storage

A fingerprint is sha256(String.trim(source_text)), hex-encoded, kept under metadata["_translation_fingerprints"][target_lang][field]metadata being the JSONB column every adapted schema already has. field is the SAME name used for the schema's own JSONB column ("title", "body_html", ...), not the AI-prompt vocabulary name, so the candidate SQL below can address both with one identifier.

No normalization beyond trim/1 — no HTML/whitespace canonicalization. Treating "the text changed" as "unchanged" is worse than an occasional extra translation (design §4.1).

The four states

Defined only where the source is non-empty — a field whose source is blank or missing has no state at all (nil), not stale. Without that carve-out stale never resolves for a deleted source (nothing to re-translate against), so a sweep would queue the same empty field forever (design §4.1's convergence argument; the candidate SQL below encodes the same rule).

  • :missing — source non-empty, no translation yet
  • :stale — translation exists, a fingerprint exists, and it no longer matches hash(source)
  • :unknown — translation exists, no fingerprint at all (pre-dates this scheme, or the reference was explicitly reset)
  • :fresh — translation exists and its fingerprint matches

Folding several field states into one resource-level state takes the worst: missing > stale > unknown > fresh (fold/1).

Write-narrowing

write_decision/3 is the lock this doubles as (design §4.1's "не только датчик... но и замок записи"): a field whose fingerprint still matches its stored translation is left alone even when a fresh AI response is sitting right there, so a manual edit an operator made after the last translation is never silently overwritten by a routine re-run. "Перевести заново" (design §4.4) is drop/3 — erasing the fingerprint is the only supported way to lift that protection.

Summary

Functions

Applies ONE round of write-time fingerprint updates for lang, as write_decision/3 produced them: a {field, hash} entry stamps that hash, a {field, nil} entry ERASES whatever fingerprint the field had.

Erases the fingerprints for every {lang, field} pair in langs × fields — the "reset the reference" action (design §4.4). Pure; the caller applies it under the same locked merge put_many/3 requires. An empty per-language map left behind is dropped, and an empty top-level key is dropped too, so a fully-reset resource's metadata never carries a dangling {}.

The state of one field: source is the CURRENT source text (nil or blank means no source — see the moduledoc), translation the stored translated value, fingerprint the stored hash for this field/lang. Returns nil when the source is blank — that field has no state and must be excluded from any fold, not counted as :missing.

Folds per-field states into one resource-level state: missing > stale > unknown > fresh. Fields with no state (nil — blank source) don't participate; if every field is nil the resource itself has no state (nil) — it never appears in a candidate set (design §4.1: "ресурс без единого непустого исходного поля состояния не имеет").

Reads the stored fingerprint for {lang, field} out of a resource's metadata map, or nil.

sha256(trim(value)), lowercase hex. The one hash function every fingerprint in this module is built from — source text, on both the write path and the read/candidate path, must go through exactly this so a value hashed at write time compares equal to itself hashed again at read time.

Merges field_hash_map (%{field => hash}) into metadata's stored fingerprints for lang, leaving every other language and every field not present in field_hash_map untouched. Returns the updated metadata map (the caller still owns writing it back under its own lock — this function is pure).

Schema-qualifies a bare table name with the configured config :phoenix_kit, :prefix — the SAME compile-time value every Ecto-schema-backed query in this package already carries via use PhoenixKit.SchemaPrefix (see Product, Category, ...). Raw SQL text never goes through Ecto's query builder, so it doesn't pick that prefix up automatically; select_candidates/2 below and the one-shot backfill mix task (build_sql/2) both call this on every table name they interpolate, so a named-schema install targets the schema the migrations actually installed into instead of raising "relation does not exist" against public.

Runs the hash-in-the-database candidate query for one resource table and returns only %{uuid:, languages: [...]} — never resource text (design §4.3: "наружу приезжают только uuid и список языков").

The character set to pass as btrim's second argument so a hash computed in Postgres equals hash/1 computed here — see the comment above its definition. Any SQL that recomputes a fingerprint MUST use it; select_candidates/2 and the one-shot backfill mix task both do.

Design §4.4's table, as a decision: given the source text this translation job actually read (nil when the caller didn't supply one — see below), the CURRENTLY stored translation, and the CURRENTLY stored fingerprint (both re-read under the adapter's row lock, not from a possibly-stale in-memory struct), decide whether to write.

Types

state()

@type state() :: :missing | :stale | :unknown | :fresh

Functions

apply_writes(metadata, lang, updates)

@spec apply_writes(map() | nil, String.t(), %{
  required(String.t()) => String.t() | nil
}) :: map()

Applies ONE round of write-time fingerprint updates for lang, as write_decision/3 produced them: a {field, hash} entry stamps that hash, a {field, nil} entry ERASES whatever fingerprint the field had.

The nil case is not a no-op, and that matters. write_decision/3 returns {:write, nil} when the caller supplied no source text for the field — the field is written from a source this module was never shown. Keeping the previous fingerprint would then have the metadata assert "this translation was made from source X" about a translation that replaced it, made from something else. Concretely, on a field that was :stale: the row keeps a fingerprint that still mismatches its source, so select_candidates/2 keeps returning it, every sweep tick pays for another model call, and the write never moves the fingerprint — a non-convergent loop of exactly the kind design §4.1's empty-source carve-out exists to prevent. Erasing instead lands the field in :unknown, which design §4.1 names as the state for "переводы, записанные в обход отпечатков" and which the sweep never picks up on its own; the operator sees it and decides.

Pure — the caller writes the result back under its own row lock.

drop(metadata, langs, fields)

@spec drop(map() | nil, [String.t()], [String.t()]) :: map()

Erases the fingerprints for every {lang, field} pair in langs × fields — the "reset the reference" action (design §4.4). Pure; the caller applies it under the same locked merge put_many/3 requires. An empty per-language map left behind is dropped, and an empty top-level key is dropped too, so a fully-reset resource's metadata never carries a dangling {}.

field_state(source, translation, fingerprint)

@spec field_state(String.t() | nil, String.t() | nil, String.t() | nil) ::
  state() | nil

The state of one field: source is the CURRENT source text (nil or blank means no source — see the moduledoc), translation the stored translated value, fingerprint the stored hash for this field/lang. Returns nil when the source is blank — that field has no state and must be excluded from any fold, not counted as :missing.

fold(states)

@spec fold([state() | nil]) :: state() | nil

Folds per-field states into one resource-level state: missing > stale > unknown > fresh. Fields with no state (nil — blank source) don't participate; if every field is nil the resource itself has no state (nil) — it never appears in a candidate set (design §4.1: "ресурс без единого непустого исходного поля состояния не имеет").

get(metadata, lang, field)

@spec get(map() | nil, String.t(), String.t()) :: String.t() | nil

Reads the stored fingerprint for {lang, field} out of a resource's metadata map, or nil.

hash(value)

@spec hash(String.t()) :: String.t()

sha256(trim(value)), lowercase hex. The one hash function every fingerprint in this module is built from — source text, on both the write path and the read/candidate path, must go through exactly this so a value hashed at write time compares equal to itself hashed again at read time.

put_many(metadata, lang, field_hash_map)

@spec put_many(map() | nil, String.t(), %{required(String.t()) => String.t()}) ::
  map()

Merges field_hash_map (%{field => hash}) into metadata's stored fingerprints for lang, leaving every other language and every field not present in field_hash_map untouched. Returns the updated metadata map (the caller still owns writing it back under its own lock — this function is pure).

qualify_table(table, prefix)

@spec qualify_table(String.t(), String.t() | nil) :: String.t()

Schema-qualifies a bare table name with the configured config :phoenix_kit, :prefix — the SAME compile-time value every Ecto-schema-backed query in this package already carries via use PhoenixKit.SchemaPrefix (see Product, Category, ...). Raw SQL text never goes through Ecto's query builder, so it doesn't pick that prefix up automatically; select_candidates/2 below and the one-shot backfill mix task (build_sql/2) both call this on every table name they interpolate, so a named-schema install targets the schema the migrations actually installed into instead of raising "relation does not exist" against public.

nil — the default, unprefixed public install — leaves table untouched, so behavior for every existing (unprefixed) install is unchanged.

select_candidates(repo, opts)

@spec select_candidates(Ecto.Repo.t(), keyword()) :: [
  %{uuid: String.t(), languages: [String.t()]}
]

Runs the hash-in-the-database candidate query for one resource table and returns only %{uuid:, languages: [...]} — never resource text (design §4.3: "наружу приезжают только uuid и список языков").

opts:

  • :table (required) — the table name, e.g. "phoenix_kit_shop_products".
  • :fields (required) — JSONB column names to check, e.g. ["title", "description", ...]. These come from this codebase's own fixed field maps, never external input, and are interpolated into the query text as identifiers — never pass user-controlled values here.
  • :source_lang, :target_langs (required).
  • :statuses — optional list to additionally require status = ANY(statuses); nil (default) applies no status filter. Design §4.3: categories never pass this.
  • :limit — optional row cap (rows are one per {uuid, lang}} candidate pair, not per resource).

A resource+language pair is a candidate the moment ANY field is missing or stale for it — matching design §4.4's decision that translation happens per-resource, not per-field (the field axis only narrows what gets written, never what gets queued).

sql_trim_chars()

@spec sql_trim_chars() :: String.t()

The character set to pass as btrim's second argument so a hash computed in Postgres equals hash/1 computed here — see the comment above its definition. Any SQL that recomputes a fingerprint MUST use it; select_candidates/2 and the one-shot backfill mix task both do.

write_decision(source, translation, fingerprint)

@spec write_decision(String.t() | nil, String.t() | nil, String.t() | nil) ::
  :skip | {:write, String.t() | nil}

Design §4.4's table, as a decision: given the source text this translation job actually read (nil when the caller didn't supply one — see below), the CURRENTLY stored translation, and the CURRENTLY stored fingerprint (both re-read under the adapter's row lock, not from a possibly-stale in-memory struct), decide whether to write.

  • no translation yet, no fingerprint yet, or the fingerprint no longer matches hash(source){:write, hash(source)}
  • fingerprint matches and a translation is already there → :skip (the manual-edit protection this whole model exists for)

source == nil is the one case outside that table — a caller that doesn't participate in fingerprinting at all (no :source_fields opt; every production caller after design §9.3 always supplies one). Without source text there is nothing to hash, so this falls back to the pre-fingerprint behavior — always write — rather than either guessing or blocking a legitimate write. The nil in {:write, nil} means "erase this field's fingerprint", NOT "leave it as it was": applied through apply_writes/3, that leaves the field in :unknown, the honest state, since this module was never told what the value it just accepted was translated from. See apply_writes/3 for why keeping the old fingerprint instead is not a harmless omission.