ML Adapter

pyfe4ai.ml runs SIFE and MCFE schemes for encrypted linear inference and federated gradient aggregation behind one wrapper:

from pyfe4ai.ml import FESchemeWrapper, aggregate_gradients, encrypt_gradient

The functions and classes below are importable from pyfe4ai.ml; they are defined in its submodules wrapper (the wrapper; its schemes are those of pyfe4ai.core.registry), serialization (the encrypted containers), gradients (federated aggregation), threshold (the decryption servers of the threshold MCFE) and inference (encrypted linear inference).

Threshold MCFE

For mcfe/ring_lwe_threshold the wrapper also runs the decryption servers (one trust domain, for experiments). prepare_aggregation() and aggregate_gradients() take servers= (default: the first t of the public parameters’ lst_sid) and give each enrolled server its label key share for the round and, once per weights, its weight key share. The messages of a deployment are those of the separate steps: partial_decrypt_gradients() at each server and combine_partial_decryptions() at the combiner, exchanging ThresholdPartial objects:

prepared = prepare_aggregation(wrapper, weights, shape=(3, 7), label="round-a",
                               servers=["sid_1", "sid_2"])
received = {nid: EncryptedGradient.from_dict(d) for nid, d in client_messages.items()}
wire = {sid: partial_decrypt_gradients(received, wrapper, prepared, server=sid).to_dict()
        for sid in prepared.servers}
total = combine_partial_decryptions(
    {sid: ThresholdPartial.from_dict(d) for sid, d in wire.items()}, wrapper, prepared)

Every partial decryption carries the sharing_id of the server’s key share. Keep it when partials (or key shares) are stored or transmitted by other means: the combiner refuses partials of two sharings of one key (MixedSharingsError; prepare the round again), but partials without an id combine only with each other, so dropping it on every side would let two sharings be mixed unnoticed. examples/mcfe_ring_lwe_threshold_demo.py runs both the scheme classes and the adapter.

API

ML framework adapter layer for PyFE4AI.

Provides two high-level APIs that hide FE complexity from ML engineers:

FL aggregation (eta=1, ndarray helpers):

Encrypt per-element gradients from n clients, aggregate with weights.

>>> keygen_cfg = {"eta": 1, "n": 2, "lst_nid": ["c0", "c1"], "bound_x": 1000, "bound_y": 10}
>>> wrapper = FESchemeWrapper("mcfe", "lwe", keygen_cfg, precision=2)
>>> enc0 = encrypt_gradient(grad0, wrapper, nid="c0", label="round-1")
>>> enc1 = encrypt_gradient(grad1, wrapper, nid="c1", label="round-1")
>>> result = aggregate_gradients(
...     {"c0": enc0, "c1": enc1}, wrapper, {"c0": 1, "c1": 2}, label="round-1")

Extra entries of the crypto objects’ config go to crypto_options, e.g. FESchemeWrapper("mcfe", "ring_lwe", keygen_cfg, crypto_options={"ct_format": "binary"}).

Encrypted inference (eta=feature_dim, vector mode):

Encrypt a feature vector, compute ⟨x, w⟩ without revealing x.

>>> wrapper = FESchemeWrapper("sife", "lwe", {"eta": 3, "bound_x": 10, "bound_y": 10})
>>> enc = encrypt_features(feature_vec, wrapper)
>>> score = compute_linear(enc, wrapper, model_weights)

PyTorch is optional — functions accept both numpy.ndarray and torch.Tensor inputs.

Threshold MCFE (mcfe/ring_lwe_threshold): the wrapper also runs the s decryption servers. prepare_aggregation() (or aggregate_gradients() without prepared) gives each enrolled server its label key share of the round and, once per weights, its weight key share (the scheme’s split key shares, its default); aggregate_gradients() then has the servers decrypt partially and combines their answers. The two steps are also separate functions – partial_decrypt_gradients() per server, combine_partial_decryptions() at the combiner – whose ThresholdPartial serialises like EncryptedGradient (binary ciphertexts and partial decryptions, the scheme’s default format, as base64).

Single trust domain. A FESchemeWrapper runs the key generator itself, so it holds the master secret key and plays every party (clients, key authority, decryptor). It is meant for experiments and for code that already trusts one process with everything; a deployment keeps the master key away from the aggregator. The wrapper applies a default policy that a key holder would otherwise have to enforce: aggregate_gradients() needs a ciphertext and a non-zero weight for every enrolled client (pass allow_partial=True to relax this), each client encrypts at most once per label, and each label is aggregated with one weight vector only. These checks stop the obvious ways of isolating one client, not all of them: a single aggregate with very unequal weights (e.g. (W, 1) with W above twice the largest gradient entry) separates two clients’ gradients, as with any inner-product FE; choosing the weights is the key holder’s responsibility, and FESchemeWrapper.get_decryption_keys() issues keys for any weights.

The functions and classes are defined in the submodules wrapper, serialization, gradients, threshold and inference, and imported from here; pyfe4ai.ml.quantization holds the float / integer conversion.

class pyfe4ai.ml.FESchemeWrapper(scheme_type, variant, keygen_config, *, precision=0, qmode='decimal', crypto_options=None)[source]

Bases: object

Unified wrapper around any PyFE4AI scheme.

Manages the KeyGenerator lifecycle (setup + key distribution) and provides a consistent encrypt / decrypt surface regardless of the underlying scheme family. The wrapper runs the key generator itself and so forms a single trust domain (see pyfe4ai.ml).

Every registered (scheme_type, variant) pair is accepted: the schemes of pyfe4ai.core.registry (for example ("sife", "ddh"), ("mcfe", "lwe_threshold")), with their review status in status. The FL helpers are narrower: encrypt_gradient() and aggregate_gradients() refuse MIFE (no labels, so a key holder could combine a client’s ciphertexts across elements or rounds), and aggregate_gradients() and compute_linear() refuse the *_decentralized and *_threshold variants, whose decryption is a multi-party protocol; use the scheme classes directly for those. The exception is mcfe/ring_lwe_threshold (capability ml_servers), whose s decryption servers the wrapper runs (see the module docstring): aggregate_gradients() and prepare_aggregation() take it, with the enrolled servers.

A wrapper can be pickled and copied (multiprocessing, Ray): the copy carries the (nid, label) pairs used so far and refuses them as the original does. Copies do not learn of labels used later by the original or by other copies, so encrypt each client’s gradient for a label through one wrapper. A pickled wrapper carries everything it holds, the master secret key and every client’s key included: do not ship it to workers that should only encrypt. Give such a worker its client’s own crypto object instead – the scheme class built from wrapper.pp and wrapper.kg.get_private_keys(nid) (e.g. MCFERingLWE({"id": nid, "precision": p, "keys": {"pp": pp, "sk": sk}}) for mcfe/ring_lwe), whose encrypt_lst_ndarray the worker calls.

Parameters:

  • scheme_type (str) – "sife", "mife", "mcfe" or "quadratic".

  • variant (str) – Variant name within the family, e.g. "ddh", "lwe", "damgard", "paillier", "ring_lwe" or "lwe_threshold" (see pyfe4ai.list_schemes).

  • keygen_config (dict) – Configuration dict for the KeyGenerator (sec_param, eta, n, bound_x, security_profile, etc.). group="ristretto255" selects the elliptic-curve instantiation of the ddh variants (ddh_ristretto255).

  • precision (int, default: 0) – Quantization precision (0: no quantization).

  • qmode (str, default: 'decimal') – Quantization mode, "decimal" or "binary".

  • crypto_options (dict | None, default: None) – Extra entries of the crypto objects’ config, e.g. {"ct_format": "binary"} for the Ring-LWE MIFE / MCFE.

Raises:

ValueError – If (scheme_type, variant) is not registered.

status

Review status of the scheme (see pyfe4ai.core.status).

get_decryption_keys(sid, **credentials)[source]

Derive functional decryption keys.

Parameters:

sid (str) – Session / decryption-key identifier.

Return type:

dict

class pyfe4ai.ml.PreparedAggregation(label, sid, fusion_weight, weight_scale, n_chunks, dk, allow_partial, servers=None)[source]

Bases: object

A functional key derived before the round (prepare_aggregation()); for a threshold MCFE the key shares of the enrolled servers (dk = {sid: key share}) and sid the combiner.

dk is key material (a functional key, or the servers’ label key shares with their MAC and noise keys): keep it where the wrapper is, and do not send, log or store it elsewhere.

Parameters:
label: str | None
sid: str
fusion_weight: dict
weight_scale: int
n_chunks: int | None
dk: dict
allow_partial: bool
servers: list | None = None
class pyfe4ai.ml.EncryptedGradient(ciphertexts, shape, precision, mode, nid)[source]

Bases: object

Encrypted gradient array for FL aggregation.

Produced by encrypt_gradient().

Parameters:
ciphertexts: list

One entry per layer, an object array of per-element ciphertexts or, for the packed Ring-LWE MCFE schemes, a dict {"shape", "first_chunk", "chunks"} (see pyfe4ai.schemes.mcfe.ring_lwe.MCFERingLWE.encrypt_lst_ndarray()).

shape: tuple
precision: int
mode: str
nid: str
to_dict()[source]

Serialise for network transfer (e.g. federated learning).

A per-element layer is written as {"shape", "data"} with one entry per index, a packed layer as {"packed": {"shape", "first_chunk", "chunks"}}. Chunk fields holding bytes (binary ciphertexts, e.g. the default format of mcfe/ring_lwe_threshold) are written as base64 strings and named in the layer’s "base64" list; from_dict() decodes them.

Raises:

TypeError – If a binary field of one chunk is not bytes in another.

Return type:

dict

MAX_ELEMENTS = 10000000

Largest number of elements from_dict() accepts per array.

MODES = ('decimal', 'binary')

The quantization modes of a serialised gradient (QuantizationConfig).

classmethod from_dict(d)[source]

Deserialise from to_dict(), checking the field types and the layout (the scheme checks the ciphertexts when decrypting).

Raises:

ValueError – Unless d has exactly the keys nid (a client id: a str or an int, is_id(); a numpy integer, as an in-memory gradient may carry, is read as the int it equals), shape (a list of non-negative ints), precision (a non-negative int), mode ("decimal" or "binary") and ciphertexts (a non-empty list of layers); if an array’s shape is invalid or has more than MAX_ELEMENTS elements; if a per-element layer is not {"shape", "data"} with one ciphertext (a dict) under each index key to_dict writes, "(i, j, ...)", and nothing else; if a packed layer is not a valid {"shape", "first_chunk",     "chunks"} layout; or if packed and per-element layers are mixed.

Parameters:

d (dict)

Return type:

EncryptedGradient

class pyfe4ai.ml.EncryptedFeatures(ciphertext, dim, precision, mode)[source]

Bases: object

Encrypted feature vector for inference.

Produced by encrypt_features().

Parameters:
ciphertext: dict
dim: int
precision: int
mode: str
to_dict()[source]
Return type:

dict

classmethod from_dict(d)[source]

Deserialise from to_dict(), checking the field types (the scheme checks the ciphertext when decrypting).

Raises:

ValueError – Unless d has exactly the keys ciphertext (the scheme’s ciphertext, a dict), dim (a positive int), precision (a non-negative int) and mode ("decimal" or "binary").

Parameters:

d (dict)

Return type:

EncryptedFeatures

class pyfe4ai.ml.ThresholdPartial(sid, label, layers)[source]

Bases: object

One server’s partial decryptions of a round’s gradients (threshold MCFE), from partial_decrypt_gradients() for combine_partial_decryptions().

Parameters:
sid: str
label: str
layers: list

compute_lst_ndarray_ct() of the clients’ gradients: per layer {"shape", "parts"}, one partial decryption per chunk (w and c_y bytes for binary public parameters, lists of decimal strings for JSON ones; sharing_id that of the server’s key share, or None). to_dict() keeps the sharing_id; a transport of its own must keep it as well, or the combiner can no longer tell partials of two sharings apart.

to_dict()[source]

Serialise for transfer: {"sid", "label", "format", "layers"}, binary w and c_y (format = "binary") as base64 strings.

Raises:

TypeError – If the partial decryptions mix binary and JSON values.

Return type:

dict

classmethod from_dict(d)[source]

Deserialise from to_dict(), checking the layout (the combiner checks the values).

Raises:

ValueError – For another layout, a sid or lst_sid_enrolled entry that is not a server id (a str or an int, is_id(); a numpy integer is read as the int it equals), a partial decryption of another server, or w / c_y not of the format (strict base64 for "binary", lists of strings for "json").

Parameters:

d (dict)

Return type:

ThresholdPartial

pyfe4ai.ml.precompute_gradient(shape, wrapper, *, nid='nid_default', label)[source]

Offline phase of encrypt_gradient() for a gradient of shape.

For mcfe/ring_lwe (see precompute()): label pads, Gaussian samples and NTT products are computed before the gradient exists; encrypt_gradient(..., precomputed=...) then only adds the plaintext term. For mcfe/ddh, mcfe/damgard and mcfe/paillier (and their decentralized variants; pyfe4ai.schemes.mcfe.offline) the label masks of every element are computed. The result is secret and one-time; the ciphertexts are those of the plain call.

Raises:

ValueError – If the scheme has no offline phase, or nid already encrypted under label.

Parameters:
Return type:

Any

pyfe4ai.ml.encrypt_gradient(gradient, wrapper, *, nid='nid_default', label=None, precomputed=None)[source]

Encrypt a gradient array for federated aggregation.

Parameters:

  • gradient (ndarray | Any) – Gradient array (numpy array or torch.Tensor).

  • wrapper (FESchemeWrapper) – FE scheme wrapper instance.

  • nid (str, default: 'nid_default') – Node identifier.

  • label (str | None, default: None) – Round label; MCFE allows one ciphertext per client and label, so a label this wrapper already used for nid is refused.

  • precomputed (Any, default: None) – The result of precompute_gradient() for this client, label and shape (online phase).

Return type:

EncryptedGradient

pyfe4ai.ml.prepare_aggregation(wrapper, weights, *, shape, sid='sid_agg', label=None, allow_partial=False, servers=None)[source]

Derive the functional key of an aggregation before the ciphertexts arrive.

The key depends on the weights, the label and (for the packed Ring-LWE MCFE) the number of chunks of a gradient of shape, not on the ciphertexts; aggregate_gradients() with prepared= then only decrypts. The same policy checks apply, with the clients of weights as those whose ciphertexts will be aggregated.

For mcfe/ring_lwe_threshold the key shares of the enrolled servers (default: the first t of the public parameters’ lst_sid): their label key shares of the round, each server given its weight key share once per weights (again where a copy of the wrapper or the server’s store lacks it when the round is decrypted); sid is the combiner. The result holds key material (see PreparedAggregation). weights is keyed by client ids (numpy integer keys are read as ints; another key: ValueError).

Parameters:
Return type:

PreparedAggregation

pyfe4ai.ml.aggregate_gradients(encrypted, wrapper, weights=None, *, sid='sid_agg', label=None, allow_partial=False, prepared=None, servers=None)[source]

Decrypt and aggregate encrypted gradients from multiple clients.

Parameters:

  • encrypted (dict[str, EncryptedGradient]) – Encrypted data container.

  • wrapper (FESchemeWrapper) – FE scheme wrapper instance.

  • weights (dict[str, float | int | list] | None, default: None) – Weight per client (a number or a one-element list); fractional weights are quantised with the wrapper’s precision and the result is scaled back.

  • sid (str, default: 'sid_agg') – Session / decryption-key identifier.

  • label (str | None, default: None) – Round label (MCFE); each label is aggregated with one weight vector only.

  • allow_partial (bool, default: False) – Allow zero weights and a subset of the enrolled clients. Off by default: the wrapper holds the master key, and such weights would decrypt a single client’s gradient (so would very unequal weights; see pyfe4ai.ml).

  • prepared (PreparedAggregation | None, default: None) – A key from prepare_aggregation() (online phase); weights, if given, and label must match it.

  • servers (list | None, default: None) – mcfe/ring_lwe_threshold: the servers whose partial decryptions are combined (default: those of prepared, or the first t of lst_sid); the wrapper runs them (partial_decrypt_gradients(), combine_partial_decryptions()) and sid is the combiner. If their label key shares turn out to be of different sharings (MixedSharingsError), all servers fetch them again and decrypt once more.

Raises:

ValueError – If the policy above is violated, SIFE is given more than one client, encrypted or weights is keyed by a value that is no client id (numpy integers are read as ints; see pyfe4ai.core.ids), or a gradient is filed under another client than its nid.

Return type:

ndarray

pyfe4ai.ml.partial_decrypt_gradients(encrypted, wrapper, prepared, *, server)[source]

Server server’s partial decryptions of the clients’ gradients for a prepared threshold aggregation (mcfe/ring_lwe_threshold; the wrapper runs the server, with its key share of prepared, see prepare_aggregation()).

Raises:

ValueError – If server is not enrolled in prepared, the clients are not those of its weights, encrypted is keyed by a value that is no client id (numpy integers are read as ints), or a gradient is filed under another client than its nid.

Parameters:
Return type:

ThresholdPartial

pyfe4ai.ml.combine_partial_decryptions(partials, wrapper, prepared)[source]

Combine the enrolled servers’ partial decryptions (partial_decrypt_gradients()) into the weighted sum of the gradients, as aggregate_gradients() returns it.

Raises:
Parameters:
Return type:

ndarray

pyfe4ai.ml.encrypt_features(features, wrapper, *, nid='nid_default')[source]

Encrypt a feature vector for privacy-preserving inference.

Parameters:

  • features (ndarray | Any) – Feature vector (numpy array or torch.Tensor), flattened and quantized with the wrapper’s precision.

  • wrapper (FESchemeWrapper) – FE scheme wrapper instance.

  • nid (str, default: 'nid_default') – Node identifier.

Return type:

EncryptedFeatures

pyfe4ai.ml.compute_linear(encrypted, wrapper, weights, *, sid='sid_infer')[source]

Compute ⟨features, weights⟩ on encrypted features.

Parameters:

  • encrypted (EncryptedFeatures) – Encrypted data container.

  • wrapper (FESchemeWrapper) – FE scheme wrapper instance.

  • weights (ndarray | Any) – Weight vector or dict.

  • sid (str, default: 'sid_infer') – Session / decryption-key identifier.

Return type:

float

pyfe4ai.ml.tensor_to_ndarray(t)[source]

Convert a PyTorch tensor to a numpy array (detach + cpu + numpy).

Parameters:

t (Any)

Return type:

ndarray

pyfe4ai.ml.ndarray_to_tensor(arr)[source]

Convert a numpy array to a PyTorch tensor.

Parameters:

arr (ndarray)

Return type:

Any

Quantization

Precision management toolkit for functional encryption on real-valued data.

FE schemes operate on integers, but AI/ML workloads use floats. This module provides three tools to bridge the gap:

class pyfe4ai.ml.quantization.QuantizationConfig(precision=3, mode='decimal')[source]

Bases: object

Immutable quantization strategy for float↔int conversion.

Parameters:

  • precision (int, default: 3) – Number of fractional digits (decimal) or bits (binary) to preserve.

  • mode (str, default: 'decimal') – "decimal" scales by 10**precision (compatible with existing PyFE4AI ndarray helpers). "binary" scales by 2**precision (avoids base-10 rounding artefacts, better for ML weights).

Raises:

ValueError – If precision is negative or mode is unknown.

precision: int = 3
mode: str = 'decimal'
property scale_factor: int

Multiplicative factor used to shift floats into integer domain.

quantize(value)[source]

Convert a single float to an integer via rounding.

Parameters:

value (float)

Return type:

int

dequantize(value, order=1)[source]

Convert an integer back to float.

Parameters:

  • value (int) – Input value.

  • order (int, default: 1) – Power of the scale factor to divide by (2 for a product of two quantized values).

Return type:

float

quantize_array(arr)[source]

Quantize an entire array in one vectorised operation.

Uses np.round (banker’s rounding) instead of truncation to minimise systematic bias. Returns int64 array.

Parameters:

arr (ndarray)

Return type:

ndarray

dequantize_array(arr, order=1)[source]

De-quantize an integer array back to float64.

Parameters:

  • arr (ndarray) – Input array.

  • order (int, default: 1) – Power of the scale factor to divide by (2 for a product of two quantized values).

Return type:

ndarray

pyfe4ai.ml.quantization.estimate_bounds(x_data, y_weights, precision, eta=None, mode='decimal')[source]

Auto-compute safe FE bounds from sample data.

Parameters:

  • x_data (ndarray) – Sample plaintext data (floats).

  • y_weights (ndarray) – Sample weights (floats).

  • precision (int) – Quantization precision.

  • eta (int | None, default: None) – Inner-product vector dimension (default: the smaller sample size).

  • mode (str, default: 'decimal') – Quantization mode, "decimal" or "binary".

Return type:

dict

pyfe4ai.ml.quantization.validate_bounds(bound_x, bound_y, eta, precision)[source]

Check whether configured bounds are safe for FE operations.

Parameters:

  • bound_x (int) – Bound on plaintext values.

  • bound_y (int) – Bound on weight values.

  • eta (int) – Inner-product vector dimension.

  • precision (int) – Quantization precision.

Return type:

list[str]

pyfe4ai.ml.quantization.analyze_precision_loss(original, precision, mode='decimal')[source]

Report quantization error metrics for the given data and precision.

Parameters:

  • original (ndarray) – Float data to quantize and restore.

  • precision (int) – Quantization precision.

  • mode (str, default: 'decimal') – Quantization mode, "decimal" or "binary".

Return type:

dict