> ## Documentation Index
> Fetch the complete documentation index at: https://antlobach-clorch-182a83cb.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Stateless Operations with the Clorch Functional API

> Reference for clorch.nn.functional: stateless convolutions, pooling, normalization, dropout, padding, attention, and when to prefer F/ over nn/ modules.

The `clorch.nn.functional` namespace — conventionally aliased as `F` — contains the stateless counterpart to every module in `clorch.nn`. Unlike modules, functional operations hold no parameters of their own: weights and biases must be supplied by the caller on every invocation. This makes the functional API ideal for custom architectures where you want explicit control over how parameters flow, or for operations that genuinely have no learnable state (activations, pooling, dropout).

```clojure theme={null}
(require '[clorch.nn.functional :as F])
```

***

## Core Operations

### Linear

Applies a linear transformation `xW^T + b`. Pass `nil` as the bias to omit it.

```clojure theme={null}
(F/linear input weight bias)
```

### Convolution

All spatial parameters accept either a scalar (applied uniformly) or a sequence (one value per spatial dimension).

```clojure theme={null}
;; 1-D convolution
(F/conv1d input weight :bias bias :stride 1 :padding 0 :dilation 1 :groups 1)

;; 2-D convolution with an explicit weight tensor and optional bias
(F/conv2d input weight :bias bias :stride 1 :padding 0 :dilation 1 :groups 1)

;; 3-D convolution
(F/conv3d input weight :bias bias :stride 1 :padding 0 :dilation 1 :groups 1)
```

### Batch Normalization

Applies batch normalization over a mini-batch of inputs. Requires pre-computed running statistics.

```clojure theme={null}
(F/batch-norm input running-mean running-var
              :weight weight :bias bias
              :training false :momentum 0.1 :eps 1e-5)
```

***

## Pooling

```clojure theme={null}
;; Sliding-window max pooling
(F/max-pool2d input 2)

;; Average pooling
(F/avg-pool2d input 2)

;; Adaptive average pooling — squeezes any spatial size to [7 7]
(F/adaptive-avg-pool2d input [7 7])
```

***

## Normalization

Both functions accept optional `weight` and `bias` tensors so you can wire in your own learnable scale/shift parameters.

```clojure theme={null}
;; Layer Normalization
(F/layer-norm input normalized-shape :weight weight :bias bias :eps 1e-5)

;; Group Normalization
(F/group-norm input num-groups :weight weight :bias bias :eps 1e-5)
```

***

## Activations

Functional activations take a tensor as their first argument. See the [Activations](/models/activations) page for the full list.

```clojure theme={null}
(F/relu x)
(F/gelu x)
(F/silu x)
(F/softmax logits -1)    ;; dim is the second positional argument
```

***

## Regularization

### Dropout

```clojure theme={null}
;; Standard dropout: zeroes each element with probability p
(F/dropout x 0.5 :training? true)

;; Spatial (channel-wise) dropout for 2D feature maps
(F/dropout2d x 0.5 :training? true)
```

<Warning>
  Always pass `:training? true` during training and `:training? false` (or omit it) during inference. The module form `nn/dropout` handles mode-switching automatically via `nn/train`; the functional form does not.
</Warning>

***

## Interpolation and Padding

### Interpolation

Resize spatial tensors to a target size or by a scale factor.

```clojure theme={null}
;; Nearest-neighbour upsampling to 224×224
(F/interpolate input :size [224 224] :mode :nearest)

;; Scale by 2× using nearest interpolation
(F/interpolate input :scale-factor 2 :mode :nearest)
```

### Padding

The `padding` argument is a vector ordered `[left right top bottom ...]` — reversed from the spatial dimensions, following PyTorch convention.

```clojure theme={null}
;; Constant zero padding: 1 pixel on every side
(F/pad input [1 1 1 1])

;; Reflection padding with explicit mode
(F/pad input [1 1 1 1] :mode :reflect)

;; Replication padding
(F/pad input [2 2 2 2] :mode :replicate)
```

***

## Pixel Shuffle

Rearranges elements in a tensor of shape `[N C*r^2 H W]` into `[N C H*r W*r]`, and vice versa. Used in sub-pixel convolution super-resolution models.

```clojure theme={null}
;; Upscale by factor 2
(F/pixel-shuffle input 2)

;; Downscale by factor 2
(F/pixel-unshuffle input 2)
```

***

## Loss Functions

Loss functions live in `F/` alongside other stateless operations. See the [Loss Functions](/models/losses) page for full coverage.

```clojure theme={null}
;; Quick reference
(F/mse-loss input target)
(F/cross-entropy logits targets)
```

***

## Scaled Dot-Product Attention

`F/scaled-dot-product-attention` wraps LibTorch's fused SDPA dispatcher. On CUDA it automatically selects Flash Attention, memory-efficient attention, or the math kernel based on dtype, tensor shape, and hardware support.

```clojure theme={null}
(F/scaled-dot-product-attention
  query
  attention-key
  value
  :attention-mask mask   ;; optional boolean or float mask tensor
  :dropout-p      0.1   ;; applied only during training
  :causal?        true  ;; enables causal (lower-triangular) masking
  :scale          nil   ;; defaults to 1/sqrt(head_dim)
  :enable-gqa?    false ;; set true for Grouped Query Attention
  )
```

<Note>
  `:causal? true` is equivalent to passing a causal mask, but it is handled entirely inside LibTorch's C++ layer, avoiding the cost of materializing a large boolean matrix on the JVM side.
</Note>

***

## When to Use Functional vs Modules

<Tabs>
  <Tab title="Use nn/ Modules">
    Choose the stateful module form when:

    * The layer has **learnable parameters** (weights, biases, scale, shift).
    * You want **automatic parameter registration** — `nn/parameters`, `nn/state-dict`, and `nn/to` work out of the box.
    * You need **training/eval mode switching** (dropout, batchnorm).
    * You are composing layers inside `defmodel` or `nn/sequential`.

    ```clojure theme={null}
    ;; Modules manage their own parameters
    (def fc (nn/linear 128 64))
    (nn/forward fc x)
    ```
  </Tab>

  <Tab title="Use F/ Functional">
    Choose the functional form when:

    * The operation is **stateless** (activations, pooling, interpolation).
    * You need **total manual control** over which weight tensors are used — for example, in meta-learning loops, weight sharing, or hypernetworks.
    * You are implementing a **custom `defmodel`** forward body that calls into lower-level primitives.

    ```clojure theme={null}
    ;; Functional: caller supplies weights explicitly
    (F/linear x my-weight my-bias)
    (F/relu   x)
    ```
  </Tab>
</Tabs>

<Tip>
  Most real models mix both styles: `defmodel` fields hold the learnable modules while the `forward` body calls `F/relu`, `F/dropout`, and `F/softmax` directly — no sub-module instantiation required.
</Tip>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.