> ## Documentation Index
> Fetch the complete documentation index at: https://antlobach-clorch-182a83cb.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Loss Functions for Regression and Classification Tasks

> Reference for Clorch loss functions — MSE, L1, Huber, cross-entropy, NLL, BCE — with autograd notes and the zero-grad → backward → step training pattern.

Loss functions measure the discrepancy between a model's predictions and the ground-truth targets, producing a scalar tensor whose gradient drives parameter updates. All Clorch loss functions live in `clorch.nn.functional` and return regular LibTorch tensors that participate in the autograd graph.

```clojure theme={null}
(require '[clorch.nn.functional :as F])
```

***

## Regression Losses

### Mean Squared Error

The standard choice for continuous regression targets. Penalises large errors quadratically.

```clojure theme={null}
(F/mse-loss input target)
```

### L1 Loss (Mean Absolute Error)

Sums absolute differences rather than squared differences. More robust to outliers than MSE.

```clojure theme={null}
(F/l1-loss input target)
```

### Smooth L1 Loss (Huber Loss)

Combines the best of MSE and L1: behaves like MSE for small errors (smooth gradient near zero) and like L1 for large errors (bounded gradient, resistant to outliers). Commonly used in object detection.

```clojure theme={null}
(F/smooth-l1-loss input target)
```

<Tip>
  When you are unsure whether your dataset contains outliers, prefer `smooth-l1-loss` over plain `mse-loss`. It converges reliably in both clean and noisy settings.
</Tip>

***

## Classification Losses

### Cross Entropy

Combines `log-softmax` and negative log-likelihood in a single numerically stable operation. Expects raw logits (before any softmax), not probabilities.

```clojure theme={null}
;; logits: [batch, num-classes]   targets: [batch] int64
(F/cross-entropy logits targets)
```

### NLL Loss — Negative Log Likelihood

Expects **log-probabilities** as input (e.g. the output of `F/log-softmax`). Useful when you want to decouple the softmax step from the loss.

```clojure theme={null}
(F/nll-loss (F/log-softmax logits -1) targets)
```

### Binary Cross Entropy

For binary classification where each output is an independent probability in `(0, 1)`. Input must already be passed through a sigmoid.

```clojure theme={null}
;; input: probabilities in (0, 1) — apply sigmoid first
(F/bce-loss (F/sigmoid logits) target)
```

### BCE with Logits

Numerically more stable than applying sigmoid then BCE separately. Accepts raw logits directly.

```clojure theme={null}
;; input: raw logits — sigmoid + BCE fused in one kernel
(F/bce-with-logits-loss logits target)
```

<Note>
  Always prefer `F/bce-with-logits-loss` over `(F/bce-loss (F/sigmoid x) target)`. The fused version avoids floating-point overflow that can occur when exponentiating very large logits.
</Note>

***

## Usage in a Training Loop

Loss tensors retain the full autograd graph until you call `autograd/backward`. The canonical pattern is:

```clojure theme={null}
(require '[clorch.nn :as nn]
         '[clorch.nn.functional :as F]
         '[clorch.optim :as optim]
         '[clorch.autograd :as autograd]
         '[clorch.torch :as t])

(defn train-step [model optimizer x y]
  ;; 1. Clear gradients from the previous iteration
  (optim/zero-grad optimizer)
  ;; 2. Forward pass
  (let [pred (nn/forward model x)
        ;; 3. Compute loss
        loss (F/cross-entropy pred y)]
    ;; 4. Backpropagate
    (autograd/backward loss)
    ;; 5. Update parameters
    (optim/step optimizer)
    ;; 6. Extract a plain JVM float before the scope closes
    (t/item-float loss)))
```

<Warning>
  Loss tensors — and all intermediate activations — hold native memory. Wrap each training iteration in `t/with-torch` (or an equivalent scope) so that non-root tensors are released after the optimizer step. See the [Memory Management](/concepts/memory) page for the canonical training loop pattern.
</Warning>

***

## Autograd Graph and Retention

Every call to a loss function produces a tensor whose `.requires_grad` is `true` as long as at least one input tensor requires gradients. Calling `autograd/backward` on that tensor computes and accumulates gradients into every leaf parameter.

<Accordion title="Multiple backward passes">
  By default LibTorch frees the intermediate computation graph after a single `backward` call. If you need to call `backward` more than once (e.g. for second-order gradients or gradient accumulation with manual graph retention), pass the `:retain-graph true` option:

  ```clojure theme={null}
  (autograd/backward loss :retain-graph true)
  ```

  Retaining the graph keeps all intermediate activations in native memory until you release them explicitly, so use this only when necessary.
</Accordion>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.