> ## Documentation Index
> Fetch the complete documentation index at: https://antlobach-clorch-182a83cb.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Clorch Quickstart: Tensors, Models, and Training Loops

> Build your first Clorch program: create tensors, compute gradients, define neural networks with defmodel, run a training loop, and move models to CUDA.

This guide walks you from a single tensor all the way to a complete training loop that runs on either CPU or CUDA. Each step builds on the previous one, so by the end you will have seen every core Clorch concept in a working context. All examples run against the real library — nothing is mocked or simplified.

<Steps>
  ### Tensors and Autograd

  Start by requiring the two core namespaces. `clorch.torch` provides tensor factory functions and arithmetic; `clorch.autograd` provides gradient computation.

  ```clojure theme={null}
  (require '[clorch.torch :as t]
           '[clorch.autograd :as autograd])
  ```

  Create a scalar tensor, raise it to the third power, and compute the gradient. Because `x = 2.0`, the derivative of `x³` is `3x² = 12.0`:

  ```clojure theme={null}
  (def x (t/tensor [2.0] {:requires-grad true}))
  (def y (t/pow x 3))

  (autograd/backward y)
  (autograd/grad x)
  ;; => [12.0]
  ```

  You can also use multi-dimensional tensors with `:requires-grad`:

  ```clojure theme={null}
  (def w (t/randn [3 4] {:requires-grad true}))
  (def z (t/sum (t/mul w w)))
  (autograd/backward z)
  ;; Gradient shape matches w: [3 4]
  (t/size (autograd/grad w))
  ;; => [3 4]
  ```

  <Note>
    Autograd records every operation on tensors whose `:requires-grad` is `true`. Call `autograd/backward` once on the scalar loss, then read each parameter's gradient with `autograd/grad`. Call `optim/zero-grad` (or `nn/zero-grad`) before the next backward pass to clear accumulated gradients.
  </Note>

  ### Neural Networks with `nn/sequential`

  `nn/sequential` composes layers into a forward pipeline. The result is a plain Clojure vector, so it is trivially inspectable and composable:

  ```clojure theme={null}
  (require '[clorch.nn :as nn])

  (def model
    (nn/sequential
      (nn/linear 10 20)
      (nn/relu)
      (nn/linear 20 1)))

  ;; Run a batch of 4 samples through the model
  (t/size (nn/forward model (t/randn [4 10])))
  ;; => [4 1]
  ```

  Standard layers available out of the box include `nn/linear`, `nn/conv1d`, `nn/conv2d`, `nn/lstm`, `nn/gru`, `nn/embedding`, `nn/batchnorm1d`, `nn/layernorm`, `nn/rmsnorm`, `nn/dropout`, and many activation functions (`nn/relu`, `nn/gelu`, `nn/silu`, `nn/tanh`, `nn/sigmoid`, and more).

  ### Custom Models with `nn/defmodel`

  `nn/defmodel` defines a named model as a Clojure record. The constructor arguments configure the model, the binding vector registers modules or parameters, and the `forward` form defines execution. Registered fields automatically participate in `nn/parameters`, `nn/to`, and state dictionaries.

  ```clojure theme={null}
  (require '[clorch.nn.functional :as F])

  (nn/defmodel CustomMLP [in hidden out]
    [l1 (nn/linear in hidden)
     l2 (nn/linear hidden out)]
    (forward [x]
      (nn/forward l2 (F/relu (nn/forward l1 x)))))

  (def custom-model (CustomMLP 10 32 1))
  (t/size (nn/forward custom-model (t/randn [4 10])))
  ;; => [4 1]
  ```

  Inspect layer shapes and parameter counts with `nn/summary`. Pass the model and an input shape vector:

  ```clojure theme={null}
  (nn/summary custom-model [4 10])
  ```

  ```text theme={null}
  ----------------------------------------------------------------
  Layer (type)                   Output Shape         Param #
  ================================================================
  Linear                         [4 32]               352
  Linear                         [4 1]                33
  CustomMLPRecord                [4 1]                385
  ================================================================
  Total params: 385
  Trainable params: 385
  Non-trainable params: 0
  ----------------------------------------------------------------
  ```

  Here is a slightly larger example from the `simple.clj` example that shows a two-layer `defmodel` inside a complete forward pass:

  ```clojure theme={null}
  (nn/defmodel MyModel [in out]
    [l1 (nn/linear in 16)
     l2 (nn/linear 16 out)]
    (forward [x]
      (let [h (nn/forward l1 x)
            h (F/relu h)]
        (nn/forward l2 h))))

  (def model (MyModel 10 3))
  (t/size (nn/forward model (t/randn [4 10])))
  ;; => [4 3]
  ```

  ### Full Training Loop

  A complete supervised training loop ties together an optimizer, a loss function, a backward pass, and an optimizer step. Wrap each allocating batch step in `t/with-torch` to release intermediate tensors immediately:

  ```clojure theme={null}
  (require '[clorch.optim :as optim])

  (def model
    (nn/sequential
      (nn/linear 10 4)
      (nn/relu)
      (nn/linear 4 3)))

  (def optimizer (optim/sgd (nn/parameters model) :lr 0.01))

  (nn/train model true)

  (dotimes [epoch 3]
    (let [losses (atom [])]
      ;; Simulate a mini-batch loop
      (dotimes [_ 5]
        (t/with-torch
          (let [x      (t/randn [8 10])
                target (t/rand-int 0 3 [8] {:dtype :int64})
                logits (nn/forward model x)
                loss   (F/cross-entropy logits target)]
            (optim/zero-grad optimizer)
            (autograd/backward loss)
            (optim/step optimizer)
            (swap! losses conj (t/item-float loss)))))
      (printf "Epoch %d | Avg loss: %.4f%n"
              (inc epoch)
              (/ (apply + @losses) (count @losses)))))
  ```

  The `synthetic.clj` example shows the same pattern with a real `tensor-dataset` and `data/dataloader`:

  ```clojure theme={null}
  (require '[clorch.data :as data])

  (def train-ds (data/tensor-dataset x-train y-train))
  (def train-loader (data/dataloader train-ds :batch-size 16 :shuffle true))

  (doseq [epoch (range 5)]
    (doseq [{:keys [data target]} train-loader]
      (t/with-torch
        (let [logits (nn/forward model data)
              loss   (F/cross-entropy logits target)]
          (optim/zero-grad optimizer)
          (autograd/backward loss)
          (optim/step optimizer)))))
  ```

  <Tip>
    Always return a JVM scalar (e.g., the result of `t/item-float`) or `nil` from a `t/with-torch` block. Returning a raw tensor is safe only when you immediately bind it outside the scope with `t/retain!`. Keeping models and optimizers alive outside `with-torch` is the normal pattern — only the per-batch intermediate tensors need scoped cleanup.
  </Tip>

  ### Device Placement: CPU and CUDA

  Resolve a device once at startup, then place models and input tensors explicitly. Device selection does not happen automatically — you choose a device and place everything on it:

  ```clojure theme={null}
  (require '[clorch.cuda :as cuda])

  ;; Resolve device at startup
  (def device (if (cuda/available?) :cuda :cpu))

  ;; Move model to device
  (def model-on-device (nn/to model device))

  ;; Create input directly on the target device
  (def input (t/randn [32 10] {:device device}))

  ;; Forward pass — model and input are on the same device
  (t/size (nn/forward model-on-device input))
  ;; => [32 1]
  ```

  `nn/to` recursively moves every parameter and buffer in the model tree, including nested `defmodel` records and `nn/sequential` vectors. `t/randn`, `t/tensor`, `t/zeros`, `t/ones`, and all other factory functions accept a `:device` option.

  <Info>
    `cuda/available?` returns `false` on CPU-only builds regardless of hardware. Use `CLORCH_FORCE_GPU=1` to request the CUDA backend explicitly when the automatic detection does not select it. See the [Installation](/installation) page for system-package requirements and the full environment-variable reference.
  </Info>

  A self-contained device-aware training snippet that mirrors the `pytorch_basics_tutorial.clj` example:

  ```clojure theme={null}
  (def device (if (cuda/available?) :cuda :cpu))

  (def model
    (nn/to
      (nn/sequential
        (nn/linear (* 28 28) 512)
        (nn/relu)
        (nn/linear 512 512)
        (nn/relu)
        (nn/linear 512 10))
      device))

  (def optimizer (optim/sgd (nn/parameters model) :lr 1e-3))

  (defn train-step! [data target]
    (t/with-torch
      (let [data   (nn/to data device)
            target (nn/to target device)
            pred   (nn/forward model data)
            loss   (F/cross-entropy pred target)]
        (optim/zero-grad optimizer)
        (autograd/backward loss)
        (optim/step optimizer)
        (t/item-float loss))))
  ```
</Steps>

## Next Steps

<CardGroup cols={2}>
  <Card title="Tensors & Operations" icon="table" href="https://antlobach.github.io/clorch/docs/tensors">
    Full reference for tensor creation, dtypes, arithmetic, reductions, slicing with `ix`, and shape manipulation.
  </Card>

  <Card title="Neural Networks" icon="network-wired" href="https://antlobach.github.io/clorch/docs/nn">
    Complete layer reference, custom models with `defmodel`, summaries, and state-dict checkpointing.
  </Card>

  <Card title="Optimizers" icon="sliders" href="https://antlobach.github.io/clorch/docs/optimizers">
    SGD, Adam, AdamW, RMSprop, and Adagrad with learning-rate scheduling.
  </Card>

  <Card title="Distributed Training" icon="diagram-project" href="https://antlobach.github.io/clorch/docs/distributed">
    Launch NCCL worker processes, wrap models in DDP, use AMP and gradient accumulation, and write rank-zero checkpoints.
  </Card>

  <Card title="Memory Management" icon="memory" href="https://antlobach.github.io/clorch/docs/memory">
    Understand native allocation scopes, `with-torch`, `retain!`, `release!`, and how to avoid memory leaks in long REPL sessions.
  </Card>

  <Card title="Examples" icon="code" href="https://github.com/antlobach/clorch/tree/main/examples">
    Browse the full example directory: PyTorch basics, autograd tutorial, synthetic training, modern Llama, NanoChat, and distributed CUDA training.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.