Stateless Operations with the Clorch Functional API

Reference for clorch.nn.functional: stateless convolutions, pooling, normalization, dropout, padding, attention, and when to prefer F/ over nn/ modules.

The clorch.nn.functional namespace — conventionally aliased as F — contains the stateless counterpart to every module in clorch.nn. Unlike modules, functional operations hold no parameters of their own: weights and biases must be supplied by the caller on every invocation. This makes the functional API ideal for custom architectures where you want explicit control over how parameters flow, or for operations that genuinely have no learnable state (activations, pooling, dropout).

(require '[clorch.nn.functional :as F])

---

Core Operations

Linear

Applies a linear transformation xW^T + b. Pass nil as the bias to omit it.

(F/linear input weight bias)

Convolution

All spatial parameters accept either a scalar (applied uniformly) or a sequence (one value per spatial dimension).

;; 1-D convolution
(F/conv1d input weight :bias bias :stride 1 :padding 0 :dilation 1 :groups 1)

;; 2-D convolution with an explicit weight tensor and optional bias
(F/conv2d input weight :bias bias :stride 1 :padding 0 :dilation 1 :groups 1)

;; 3-D convolution
(F/conv3d input weight :bias bias :stride 1 :padding 0 :dilation 1 :groups 1)

Batch Normalization

Applies batch normalization over a mini-batch of inputs. Requires pre-computed running statistics.

(F/batch-norm input running-mean running-var
              :weight weight :bias bias
              :training false :momentum 0.1 :eps 1e-5)

---

Pooling

;; Sliding-window max pooling
(F/max-pool2d input 2)

;; Average pooling
(F/avg-pool2d input 2)

;; Adaptive average pooling — squeezes any spatial size to [7 7]
(F/adaptive-avg-pool2d input [7 7])

---

Normalization

Both functions accept optional weight and bias tensors so you can wire in your own learnable scale/shift parameters.

;; Layer Normalization
(F/layer-norm input normalized-shape :weight weight :bias bias :eps 1e-5)

;; Group Normalization
(F/group-norm input num-groups :weight weight :bias bias :eps 1e-5)

---

Activations

Functional activations take a tensor as their first argument. See the Activations page for the full list.

(F/relu x)
(F/gelu x)
(F/silu x)
(F/softmax logits -1)    ;; dim is the second positional argument

---

Regularization

Dropout

;; Standard dropout: zeroes each element with probability p
(F/dropout x 0.5 :training? true)

;; Spatial (channel-wise) dropout for 2D feature maps
(F/dropout2d x 0.5 :training? true)

Always pass :training? true during training and :training? false (or omit it) during inference. The module form nn/dropout handles mode-switching automatically via nn/train; the functional form does not.

---

Interpolation and Padding

Interpolation

Resize spatial tensors to a target size or by a scale factor.

;; Nearest-neighbour upsampling to 224×224
(F/interpolate input :size [224 224] :mode :nearest)

;; Scale by 2× using nearest interpolation
(F/interpolate input :scale-factor 2 :mode :nearest)

Padding

The padding argument is a vector ordered [left right top bottom ...] — reversed from the spatial dimensions, following PyTorch convention.

;; Constant zero padding: 1 pixel on every side
(F/pad input [1 1 1 1])

;; Reflection padding with explicit mode
(F/pad input [1 1 1 1] :mode :reflect)

;; Replication padding
(F/pad input [2 2 2 2] :mode :replicate)

---

Pixel Shuffle

Rearranges elements in a tensor of shape [N C*r^2 H W] into [N C H*r W*r], and vice versa. Used in sub-pixel convolution super-resolution models.

;; Upscale by factor 2
(F/pixel-shuffle input 2)

;; Downscale by factor 2
(F/pixel-unshuffle input 2)

---

Loss Functions

Loss functions live in F/ alongside other stateless operations. See the Loss Functions page for full coverage.

;; Quick reference
(F/mse-loss input target)
(F/cross-entropy logits targets)

---

Scaled Dot-Product Attention

F/scaled-dot-product-attention wraps LibTorch’s fused SDPA dispatcher. On CUDA it automatically selects Flash Attention, memory-efficient attention, or the math kernel based on dtype, tensor shape, and hardware support.

(F/scaled-dot-product-attention
  query
  attention-key
  value
  :attention-mask mask   ;; optional boolean or float mask tensor
  :dropout-p      0.1   ;; applied only during training
  :causal?        true  ;; enables causal (lower-triangular) masking
  :scale          nil   ;; defaults to 1/sqrt(head_dim)
  :enable-gqa?    false ;; set true for Grouped Query Attention
  )

:causal? true is equivalent to passing a causal mask, but it is handled entirely inside LibTorch’s C++ layer, avoiding the cost of materializing a large boolean matrix on the JVM side.

---

When to Use Functional vs Modules

Use nn/ Modules

Choose the stateful module form when:

• The layer has learnable parameters (weights, biases, scale, shift).
• You want automatic parameter registration — nn/parameters, nn/state-dict, and nn/to work out of the box.
• You need training/eval mode switching (dropout, batchnorm).
• You are composing layers inside defmodel or nn/sequential.

;; Modules manage their own parameters
(def fc (nn/linear 128 64))
(nn/forward fc x)

Use F/ Functional

Choose the functional form when:

• The operation is stateless (activations, pooling, interpolation).
• You need total manual control over which weight tensors are used — for example, in meta-learning loops, weight sharing, or hypernetworks.
• You are implementing a custom defmodel forward body that calls into lower-level primitives.

;; Functional: caller supplies weights explicitly
(F/linear x my-weight my-bias)
(F/relu   x)

Most real models mix both styles: defmodel fields hold the learnable modules while the forward body calls F/relu, F/dropout, and F/softmax directly — no sub-module instantiation required.