Handles custom alphabets — those other than the ones supported by
ExFPE.Codec.Builtin.
Each symbol is a single Unicode scalar (codepoint) that is guaranteed to stand on its own as one visual unit — a grapheme cluster. Alphabets are restricted at construction so this holds: codepoints that are unassigned, control/format/surrogate/private-use, combining marks, conjoining Hangul jamo, not in NFC form, or that would merge with an adjacent symbol (emoji modifiers, regional indicators, prepending marks, ...) are rejected.
Whitespace and separators (space, no-break space, line/paragraph separators, ...) are rejected too — not because they'd break the grapheme invariant (a space stands alone just fine), but on practical grounds: since encryption permutes over the whole domain, a separator can surface anywhere in the ciphertext, including as an invisible leading or trailing symbol.
This buys two guarantees of different strength:
- Round-tripping is ensured for any accepted alphabet, forever. Input is tokenized by codepoint and matched after NFC normalization. Per the Unicode Normalization Stability Policy and UAX #15, once a string contains only characters assigned in a given Unicode version, its NFC form is guaranteed to stay the same in all later versions — so ciphertext stays decryptable across Unicode/OTP upgrades.
- Visual-unit preservation (visual units out = visual units in) holds under any given Unicode version, because every symbol occupies exactly one grapheme cluster. Unlike normalization, grapheme-cluster segmentation (UAX #29) has no comparable stability guarantee in the Unicode Character Encoding Stability Policies; segmentation rules have changed between versions before (e.g. a boundary fix for several Brahmic scripts landed in Unicode 15.1), so a future version could in principle re-segment an exotic symbol. If that ever happened the data would still decrypt (the round-trip guarantee is independent of segmentation) — only the visual count could drift.
Case sensitivity
This codec is norm
insensitive, but
case sensitive (unlike ExFPE.Codec.Builtin). This is because:
- There are multiple, ways of making a string case agnostic;
- Case sensitivity makes sense in some cases (think base64) but not in others;
- What if an alphabet has multiple casings of the same symbol but single casings of others?
If you wish to handle case agnostically, you'll need to pick what best fits
your use case and normalize inputs yourself before invoking ExFPE
encryption and decryption functions.
Why alphabets must be in NFC, but not inputs
Encryption and decryption inputs are normalized toward the alphabet: an input symbol is NFC-normalized and then looked up among the alphabet's symbols. That lookup only works if the alphabet is itself in NFC.
ExFPE rejects a non-NFC alphabet rather than normalizing it, because
normalizing a symbol can silently change either its cardinality or its
identity, breaking invariants this codec depends on:
- One codepoint can become several.
U+0344(◌̈́, combining greek dialytika tonos) normalizes toU+0308 U+0301(◌̈ then ◌́, combining diaeresis then combining acute accent, shown on ◌ dotted-circle placeholders). A symbol is meant to be exactly one scalar (#codepoints = #symbols = #grapheme clusters); a two-codepoint symbol breaks splitting, length, and round-tripping. - One codepoint can become a different one — a Unicode singleton.
U+212B(Å, the Ångström sign — itself a starter) normalizes toU+00C5(Å, latin capital letter A with ring above).
Inputs are safe to normalize because there it's only a lookup. The alphabet, being the source of truth for the radix, symbol ordering, and symbol/amount bijection, is rejected loudly instead, so the caller declares a clean one.