Probability Distributions for Bayesian and Generative Models

clorch.distributions offers data-first constructors, shape-aware sampling, log-prob, and closed-form moments for normal, categorical, gamma, and more.

The clorch.distributions namespace implements a probability distribution library with a data-first API. Each distribution is a plain Clojure map, which means it can be stored in atoms, serialized with EDN, and passed through any Clojure data-manipulation function without special wrappers. The four core operations — sample, log-prob, mean, and variance — accept these maps and dispatch on the :dist key.

(require '[clorch.distributions :as dist]
         '[clorch.torch :as t])

---

Distribution Constructors

Every constructor returns a map. Parameters can be Clojure numbers or Clorch tensors.

Location–Scale Distributions

;; Gaussian: mean 0, std 1
(dist/normal 0.0 1.0)

;; Log-normal: underlying Gaussian with loc=0, scale=0.5
(dist/log-normal 0.0 0.5)

;; Cauchy: heavy-tailed, no finite mean or variance
(dist/cauchy 0.0 1.0)

;; Gumbel: extreme value distribution
(dist/gumbel 0.0 1.0)

;; Laplace: double-exponential
(dist/laplace 0.0 1.0)

;; Logistic distribution
(dist/logistic 0.0 1.0)

Discrete Distributions

;; Bernoulli: accepts a tensor of probabilities
(dist/bernoulli (t/tensor [0.2 0.8]))

;; Categorical: one-hot selection from probability vector
(dist/categorical (t/tensor [0.1 0.2 0.7]))

;; Binomial: n trials, probability p
(dist/binomial 10.0 0.3)

;; Geometric: number of trials until first success
(dist/geometric 0.4)

;; Negative binomial
(dist/negative-binomial 5.0 0.3)

Rate and Count Distributions

;; Uniform: continuous on [low, high]
(dist/uniform -1.0 2.0)

;; Poisson: count distribution with given rate
(dist/poisson 3.0)

;; Exponential: waiting time with given rate
(dist/exponential 2.0)

Shape Distributions

;; Gamma: concentration (shape) and rate
(dist/gamma 2.0 1.0)

;; Beta: concentration1 and concentration0
(dist/beta 2.0 5.0)

;; Dirichlet: concentration vector (tensor or number)
(dist/dirichlet (t/tensor [1.0 2.0 3.0]))

;; Weibull: scale and concentration
(dist/weibull 1.0 2.0)

;; Pareto: scale and alpha
(dist/pareto 1.0 3.0)

Statistical Test Distributions

;; Student-t: df, loc, scale
(dist/student-t 5.0 0.0 1.0)

;; Chi-squared: degrees of freedom
(dist/chi2 4.0)

;; Fisher F: two degrees of freedom
(dist/fisher-f 3.0 10.0)

---

Core Operations

Sampling

dist/sample accepts a distribution map and an optional sample-shape vector. The returned value is always a Clorch tensor.

(def d (dist/normal 0.0 1.0))

;; Single sample (shape [1])
(dist/sample d)

;; Batch of 4 rows × 3 columns
(dist/sample d [4 3])

;; Large sample for Monte Carlo estimates
(dist/sample d [10000])

sample-shape is supported for all numeric-parameter distributions. Distributions parameterized by tensors (e.g., bernoulli with a tensor probability) currently support single samples; multi-dimensional sample-shape is available for the common cases where parameters are scalars.

Categorical sampling delegates to torch/multinomial internally. The integer sample-shape specifies the number of draws; results are returned as an integer tensor of category indices.

Log-Probability

dist/log-prob evaluates the analytical log-density or log-mass at a given value. It accepts both scalar numbers and tensors.

(def d (dist/normal 0.0 1.0))

;; Scalar input
(dist/log-prob d 0.0)    ;; → tensor(-0.9189...)

;; Tensor input — returns element-wise log-prob
(dist/log-prob d (t/tensor [-1.0 0.0 1.0]))

;; Log-likelihood over a batch
(def data (dist/sample d [100]))
(t/sum (dist/log-prob d data))

Mean and Variance

Closed-form moments are computed analytically when they exist. Distributions whose moments are undefined (e.g., Cauchy, categorical) return nil.

(dist/mean     (dist/normal 5.0 2.0))      ;; → 5.0
(dist/variance (dist/normal 5.0 2.0))      ;; → 4.0

(dist/mean     (dist/exponential 2.0))     ;; → 0.5
(dist/variance (dist/exponential 2.0))     ;; → 0.25

(dist/mean     (dist/uniform -1.0 3.0))    ;; → 1.0
(dist/variance (dist/uniform -1.0 3.0))    ;; → 1.333...

;; Cauchy has no finite mean
(dist/mean     (dist/cauchy 0.0 1.0))      ;; → nil

---

Example Use Cases

Sampling from a Posterior

;; Sample from a normal posterior given a conjugate update
(let [prior-mean  0.0
      prior-std   1.0
      likelihood-std 0.5
      observation 2.0
      ;; Conjugate Gaussian update
      post-var (/ 1.0 (+ (/ 1.0 (* prior-std prior-std))
                         (/ 1.0 (* likelihood-std likelihood-std))))
      post-mean (* post-var (+ (/ prior-mean (* prior-std prior-std))
                               (/ observation (* likelihood-std likelihood-std))))
      posterior (dist/normal post-mean (Math/sqrt post-var))]
  (dist/sample posterior [1000]))

Computing Log-Likelihood of Data

;; Evaluate how well a Gaussian fits some observed data
(defn gaussian-log-likelihood [data mu sigma]
  (let [d (dist/normal mu sigma)]
    (t/item-float (t/sum (dist/log-prob d data)))))

(def observations (t/tensor [1.2 0.8 1.5 1.1 0.9]))
(gaussian-log-likelihood observations 1.1 0.3)

Reparameterized Sampling for Variational Inference

;; Reparameterize: z = mu + sigma * eps, eps ~ N(0,1)
(defn reparam-sample [mu log-var]
  (let [std (t/exp (t/mul 0.5 log-var))
        eps (dist/sample (dist/normal 0.0 1.0) (t/size mu))]
    (t/add mu (t/mul std eps))))

Categorical Sampling for Token Generation

;; Sample the next token from a softmax distribution
(defn sample-token [logits]
  (let [probs (clorch.nn.functional/softmax logits -1)
        cat   (dist/categorical probs)]
    (t/item-float (dist/sample cat))))

---

Implementation Notes

Distribution implementation details

Distribution	Sampling backend	Notes
normal	torch/randn scaled and shifted	Tensor params use torch/normal
log-normal	torch/log_normal	Falls back to exp(normal) for tensor params
bernoulli	torch/bernoulli	Tensor probs only
categorical	torch/multinomial	Returns category indices as integers
binomial	torch/binomial	Both params must be scalar for sample-shape
uniform	torch/rand scaled
poisson	torch/poisson	Rate must be scalar for sample-shape
exponential	Inverse CDF of uniform	Uses -log(U) / rate
cauchy	Inverse CDF via tan(π(U-0.5))
geometric	torch/geometric (scalar p)
gumbel	Inverse CDF via -log(-log(U))
laplace	Inverse CDF via sign decomposition
gamma, beta, weibull, pareto	Apache Commons Math	Scalar params only
student-t, chi2, fisher-f	Apache Commons Math	Scalar df
dirichlet	Gamma normalization	Supports tensor concentration
logistic	Inverse CDF via log(U/(1-U))
negative-binomial	Gamma-Poisson mixture	Scalar params only