> ## Documentation Index
> Fetch the complete documentation index at: https://antlobach-clorch-182a83cb.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Performance Profiling and Memory Diagnostics in Clorch

> Maximize Clorch throughput: prefer vectorized ops over Clojure loops, use no-grad at inference, monitor native RSS for leaks, and batch JNI calls.

Clorch executes tensor operations in C++ through LibTorch, bypassing the JVM for all heavy numerical compute. The JVM is responsible for orchestration — building computation graphs, dispatching operations, and managing control flow — while actual matrix multiplications, convolutions, and reductions run entirely in native code. Understanding where the JVM boundary sits is the key to writing fast Clorch programs and diagnosing unexpected slowdowns or memory growth.

***

## Performance Design

### Native C++ Execution

When you call `(t/add tensor 1.0)`, Clorch invokes a native LibTorch operation through JavaCPP's JNI bridge. The JVM does not touch the tensor data; the operation executes in the same C++ runtime that backs PyTorch. This means:

* Matrix multiplications on CPU use BLAS/MKL/OpenBLAS (whichever LibTorch is linked against).
* CUDA operations execute asynchronously on GPU kernels.
* Autograd graph construction is handled in C++.

The JVM overhead per operation is a small, fixed JNI call cost — negligible when operations involve large tensors.

***

## Vectorized Operations vs Clojure Loops

The most common performance mistake is iterating over tensor elements in Clojure instead of using vectorized tensor operations. Each iteration of a Clojure loop that touches individual tensor elements crosses the JNI boundary, defeating the purpose of native execution.

```clojure theme={null}
;; FAST: a single C++ call, regardless of tensor size
(t/add tensor 1.0)

;; SLOW: crosses the JNI boundary for every element
(mapv #(+ % 1.0) (t/tseq tensor))
```

For a tensor with one million elements, the vectorized form is typically thousands of times faster. Always express computation as tensor operations rather than element-wise Clojure transformations.

<Tip>
  When you find yourself writing `mapv`, `reduce`, or `doseq` over `t/tseq`, look for a tensor operation that achieves the same result. The `clorch.torch` and `clorch.nn.functional` namespaces cover arithmetic, reductions, activations, pooling, normalization, and most common operations.
</Tip>

***

## Contiguous Tensors and `view`

Some shape manipulation operations like `t/view` require the tensor to be contiguous in memory. After operations like `t/transpose` or `t/ix` (slicing), a tensor may become non-contiguous — its elements are valid but laid out with non-unit strides.

```clojure theme={null}
;; This may fail after a transpose
(t/view (t/transpose tensor 0 1) [batch-size -1])

;; Fix: make contiguous first
(-> tensor
    (t/transpose 0 1)
    (.contiguous)
    (t/view [batch-size -1]))
```

If you encounter an error like `"input tensor must be contiguous"`, add `.contiguous` before the `view` call.

***

## Using `autograd/no-grad` for Inference Speed

During inference, the autograd engine builds a computation graph to support `backward`. When you only need forward-pass results, disable graph construction with `autograd/no-grad`. This eliminates significant memory allocation and overhead.

```clojure theme={null}
(require '[clorch.autograd :as autograd])

;; SLOW for inference: builds autograd graph unnecessarily
(nn/forward model x)

;; FAST for inference: no graph, lower memory, higher throughput
(autograd/no-grad
  (nn/forward model x))
```

`nn/generate` wraps its entire generation loop in `autograd/no-grad` automatically. Wrap your own evaluation and generation code similarly.

***

## JNI Overhead Guidance

Each JNI call has a small but non-zero fixed cost (typically tens of microseconds). This cost is negligible when amortized over large tensors, but it compounds when you make many small calls in a tight loop.

### Batch Your Operations

```clojure theme={null}
;; High overhead: N separate JNI calls
(doseq [i (range n)]
  (t/add (t/ix tensor i) 1.0))

;; Low overhead: one JNI call for the whole tensor
(t/add tensor 1.0)
```

### Keep Long-Lived Objects Outside Loops

Models, optimizers, and large tensors that persist across iterations should be created once outside the training loop. Creating them inside `with-torch` scopes or loop bodies triggers unnecessary native allocation and deallocation.

```clojure theme={null}
;; Correct: model created once
(def model (nn/to my-model :cuda))
(def optimizer (optim/adamw (nn/parameters model) :lr 3e-4))

(doseq [batch dataloader]
  (t/with-torch
    ;; Only per-batch tensors are allocated here
    (train-step model optimizer batch)))
```

***

## Memory Profiling

LibTorch allocates tensors in native (non-JVM) memory. The JVM garbage collector cannot see or apply pressure to this memory. RSS (Resident Set Size) is the correct metric for tracking total process memory — it includes both JVM heap and native allocations.

### Using the Built-in Profiler Script

Clorch includes a profiling script that monitors JVM heap and native RSS simultaneously:

```bash theme={null}
clojure -M test/profiler.clj
```

This script runs a workload and prints heap and RSS metrics at each step, making it easy to spot trends.

### Manual RSS Monitoring

Use `ps` to track the RSS of your Clojure process:

```bash theme={null}
# Get PID of the running process
pgrep -f "clojure"

# Poll RSS every second
watch -n 1 "ps -o rss -p <PID>"
```

Alternatively, use `/proc/<PID>/status` for more detail:

```bash theme={null}
grep VmRSS /proc/<PID>/status
```

***

## Diagnosing RSS Growth

If RSS grows steadily while the JVM heap remains stable, you likely have a native memory leak. The two most common causes:

### Missing `with-torch` Scope

Tensors allocated inside a training loop accumulate if no `with-torch` scope releases them. Wrap each batch's allocations:

```clojure theme={null}
;; Each iteration releases intermediate tensors when the scope exits
(doseq [batch dataloader]
  (let [loss-value
        (t/with-torch
          (let [loss (train-step model optimizer batch)]
            (t/item-float loss)))]
    (println "Loss:" loss-value)))
```

Return only JVM scalars (numbers, booleans, strings) from `with-torch`. If a tensor must escape the scope, call `t/retain!` on it explicitly before the scope closes, then `t/release!` when you are done with it.

### Leaked Native Pointers

Long-lived REPL sessions can accumulate tensors bound to Vars or atoms that are never released. Periodically check that your atoms and `def` values do not hold references to tensors that should have been freed.

<Warning>
  If RSS grows indefinitely across training epochs while the JVM heap remains flat, the most likely cause is a missing `with-torch` scope around per-batch allocation. Add the scope and verify that RSS stabilizes after the first epoch.
</Warning>

***

## Performance Checklist

<Accordion title="Quick performance review">
  | Concern | Check |
  | - | - |
  | Vectorization | Are you using tensor ops instead of element-wise Clojure loops? |
  | No-grad at inference | Is every evaluation loop wrapped in `autograd/no-grad`? |
  | Contiguous memory | Do you call `.contiguous` before `view` after transpose/slice? |
  | Long-lived objects | Are models and optimizers defined outside the training loop? |
  | Batch operations | Are you grouping small tensor calls rather than calling one at a time? |
  | Memory scopes | Is each batch's allocation wrapped in `t/with-torch`? |
  | RSS stability | Does RSS stabilize after the first epoch in a multi-epoch run? |
</Accordion>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.