Tensors and Operations: The Complete Clorch Reference

Create and transform LibTorch tensors from Clojure: factory methods, dtypes, element-wise math, shape operations, search utilities, and JIT save/load.

Tensors are the foundation of every Clorch computation. Each tensor is a multidimensional array backed by a native LibTorch object allocated outside the JVM heap. Clorch exposes the full LibTorch surface through the clorch.torch namespace, so every operation you know from PyTorch maps directly onto an idiomatic Clojure call. Tensor storage can live on CPU or CUDA VRAM, and any tensor whose :requires-grad flag is set participates automatically in Clorch’s autodiff graph.

(require '[clorch.torch :as t])

---

Creation

From Clojure Data

tensor accepts scalars, flat vectors, and nested vectors. An optional map configures the dtype, device, and gradient tracking.

;; 1-D tensor (defaults to :float32)
(t/tensor [1 2 3])

;; 2-D tensor
(t/tensor [[1 2] [3 4]])

;; Explicit dtype
(t/tensor [1 2 3] {:dtype :float64})

;; Gradient-tracked leaf
(t/tensor [1.0 2.0] {:requires-grad true})

;; Device and dtype together
(t/tensor [[1 2] [3 4]] {:dtype :float32 :device :cuda :requires-grad true})

tensor throws IllegalArgumentException if data is nil or an empty collection. Every element is cast to the target dtype at construction time, so passing integers with {:dtype :float32} is safe.

Standard Initializers

Zeros / Ones / Eye / Full / Empty

(t/zeros [3 3])       ;; all-zero matrix
(t/ones  [2 5])       ;; all-one matrix
(t/eye   3)           ;; 3×3 identity matrix
(t/full  [2 2] 3.14)  ;; constant-fill
(t/empty [4 4])       ;; uninitialized — values undefined

Range / Spacing

(t/arange 10)               ;; [0 1 2 … 9]
(t/arange 1 11)             ;; [1 2 3 … 10]
(t/arange 0 1 0.1)          ;; [0.0 0.1 … 0.9]

(t/linspace 0 10 5)         ;; [0.0 2.5 5.0 7.5 10.0]
(t/logspace 0 2 3)          ;; [1.0 10.0 100.0] (base 10)
(t/logspace 0 10 5 :base 2) ;; base-2 log spacing

Random

(t/randn   [3 3])          ;; N(0,1) normal
(t/rand    [3 3])          ;; U[0,1) uniform
(t/rand-int 0 10 [5])      ;; integers in [0, 10)
(t/randperm 5)             ;; random permutation of [0..4]
(t/bernoulli (t/full [5] 0.5)) ;; Bernoulli draws from probability tensor

(t/manual-seed 42)         ;; reproducible seed for all devices

;; On CUDA
(t/randn [32 128] {:device :cuda})

Supported Dtypes

Every factory and conversion function accepts a dtype keyword from this set:

Keyword	LibTorch scalar type	Typical use
:float32	kFloat	Default; most model weights
:float64	kDouble	High-precision numerics
:float16	kHalf	Mixed-precision (AMP)
:bfloat16	kBFloat16	Modern LLM training
:int32	kInt	Indices, masks
:int64	kLong	Token ids, labels
:int8	kChar	Quantised weights
:uint8	kByte	Image data
:bool	kBool	Boolean masks

Complex dtypes (:complex64, :complex128) and quantised dtypes (:qint8, :quint8, :qint32) are also exposed in dtype-map for advanced use.

;; dtype inspection
(t/dtype (t/ones [2 2]))               ;; :float32
(t/dtype (t/tensor [1] {:dtype :int64})) ;; :int64

---

Basic Operations

Element-wise Math

All four arithmetic operators broadcast scalars automatically:

(def a (t/tensor [1 2 3]))
(def b (t/tensor [4 5 6]))

(t/add a b)   ;; → [5 7 9]
(t/sub a b)   ;; → [-3 -3 -3]
(t/mul a b)   ;; → [4 10 18]
(t/div a b)   ;; → [0.25 0.4 0.5]

;; Scalar broadcasting
(t/add a 10)  ;; → [11 12 13]
(t/mul b 2.0) ;; → [8.0 10.0 12.0]

Math Functions

Clorch exposes the complete set of LibTorch pointwise math functions. Below is the full reference — each function takes a tensor and returns a new tensor of the same shape.

;; Powers and roots
(t/pow   (t/tensor [2.0 3.0]) 2)  ;; [4.0 9.0]
(t/sqrt  (t/tensor [4.0 9.0]))    ;; [2.0 3.0]
(t/rsqrt (t/tensor [4.0 9.0]))    ;; [0.5 0.333]

;; Sign and magnitude
(t/abs         (t/tensor [-1 2 -3]))   ;; [1 2 3]
(t/neg         (t/tensor [1 -2 3]))    ;; [-1 2 -3]
(t/sign        (t/tensor [-2 0 3]))    ;; [-1 0 1]
(t/reciprocal  (t/tensor [2.0 4.0]))   ;; [0.5 0.25]

;; Exponential / logarithm family
(t/exp   (t/tensor [0.0 1.0]))    ;; [1.0 2.718]
(t/exp2  (t/tensor [0.0 3.0]))    ;; [1.0 8.0]
(t/expm1 (t/tensor [0.0 1.0]))    ;; [0.0 1.718]
(t/log   (t/tensor [1.0 2.718]))  ;; [0.0 1.0]
(t/log1p (t/tensor [0.0 1.0]))    ;; [0.0 0.693]
(t/log2  (t/tensor [1.0 8.0]))    ;; [0.0 3.0]
(t/log10 (t/tensor [1.0 100.0]))  ;; [0.0 2.0]

;; Rounding
(t/floor (t/tensor [1.7 -1.3]))   ;; [1.0 -2.0]
(t/ceil  (t/tensor [1.2 -1.8]))   ;; [2.0 -1.0]
(t/round (t/tensor [1.5 2.5]))    ;; [2.0 2.0]  (round-half-to-even)
(t/trunc (t/tensor [1.7 -1.7]))   ;; [1.0 -1.0]
(t/frac  (t/tensor [1.7 -1.3]))   ;; [0.7 -0.3]

;; Modulo
(t/fmod      (t/tensor [5 7]) (t/tensor [3 3])) ;; [2 1]
(t/remainder (t/tensor [5 7]) (t/tensor [3 3])) ;; [2 1]  (Python-style)

;; Trigonometry
(t/sin  (t/tensor [0.0 1.5708]))  ;; [0.0 1.0]
(t/cos  (t/tensor [0.0 3.1416]))  ;; [1.0 -1.0]
(t/tan  (t/tensor [0.0 0.7854]))  ;; [0.0 1.0]
(t/asin (t/tensor [0.0 1.0]))     ;; [0.0 1.5708]
(t/acos (t/tensor [1.0 0.0]))     ;; [0.0 1.5708]
(t/atan (t/tensor [0.0 1.0]))     ;; [0.0 0.7854]
(t/atan2 (t/tensor [1.0]) (t/tensor [1.0])) ;; [0.7854]

;; Hyperbolic
(t/sinh  (t/tensor [0.0 1.0]))    ;; [0.0 1.175]
(t/cosh  (t/tensor [0.0 1.0]))    ;; [1.0 1.543]
(t/tanh  (t/tensor [0.0 1.0]))    ;; [0.0 0.762]
(t/asinh (t/tensor [0.0 1.175]))  ;; [0.0 1.0]
(t/acosh (t/tensor [1.0 1.543]))  ;; [0.0 1.0]
(t/atanh (t/tensor [0.0 0.762]))  ;; [0.0 1.0]

;; Special functions
(t/erf    (t/tensor [0.0 1.0]))   ;; [0.0 0.843]
(t/erfc   (t/tensor [0.0 1.0]))   ;; [1.0 0.157]
(t/erfinv (t/tensor [0.0 0.5]))   ;; [0.0 0.477]
(t/digamma (t/tensor [1.0 2.0]))  ;; [-0.577 0.423]
(t/lgamma  (t/tensor [1.0 2.0]))  ;; [0.0 0.0]

;; Neural-network utilities
(t/softmax (t/tensor [1.0 2.0 3.0]) 0) ;; [0.09 0.245 0.665]
(t/clamp   (t/tensor [-1 2 5]) -1 3)   ;; [-1 2 3]
(t/clip    (t/tensor [-1 2 5]) -1 3)   ;; [-1 2 3]  (alias for clamp)

Linear Algebra

(def m1 (t/randn [3 5]))
(def m2 (t/randn [5 2]))

(t/matmul m1 m2) ;; [3×2] — supports batched / broadcast
(t/mm     m1 m2) ;; 2-D matrix multiply (strict alias for matmul)
(t/bmm (t/randn [4 3 5]) (t/randn [4 5 2])) ;; [4×3×2] batched

(def v1 (t/tensor [1 2 3]))
(def v2 (t/tensor [4 5 6]))
(t/dot   v1 v2)  ;; 32  — 1-D dot product
(t/vdot  v1 v2)  ;; 32  — conjugate dot (same for real)
(t/inner v1 v2)  ;; 32  — generalised inner product
(t/outer v1 v2)  ;; [3×3] outer product

Reductions

(def x (t/tensor [[1 2] [3 4]]))

;; Global reductions
(t/sum  x)  ;; 10.0
(t/mean x)  ;; 2.5
(t/var  x)  ;; 1.667 (unbiased)
(t/max  x)  ;; 4.0
(t/min  x)  ;; 1.0

;; Along a dimension
(t/sum  x 0)               ;; [4 6]   — reduce rows
(t/sum  x 1)               ;; [3 7]   — reduce cols
(t/sum  x 1 :keepdim true) ;; [[3] [7]]
(t/mean x 0 :keepdim true) ;; [[2.0 3.0]]

;; Cumulative
(t/cumsum    (t/tensor [1 2 3 4]) 0) ;; [1 3 6 10]
(t/logsumexp (t/tensor [1.0 2.0 3.0]) 0) ;; 3.408

;; Index-based
(t/argmax x)        ;; flat index of max value
(t/argmin x)        ;; flat index of min value
(t/argmax x 0)      ;; per-column argmax
(t/argsort (t/tensor [3.0 1.0 2.0]) 0) ;; [1 2 0]

;; Top-k: returns [values indices]
(let [[vals idxs] (t/topk (t/tensor [0.1 0.8 0.4 0.3]) 2)]
  (println vals idxs)) ;; [0.8 0.4]  [1 2]

;; Sort: returns [values indices]
(let [[sorted idxs] (t/sort (t/tensor [3.0 1.0 2.0]) :descending true)]
  (println sorted))    ;; [3.0 2.0 1.0]

;; Gather values at specified indices along a dimension
(t/gather (t/tensor [[1 2] [3 4]]) 1 (t/tensor [[0 0] [1 0]] {:dtype :int64}))

;; Boolean reductions
(t/all (t/tensor [true true true]))  ;; true
(t/any (t/tensor [false true false])) ;; true

;; Zero-element queries
(t/nonzero       (t/tensor [1 0 2 0])) ;; [[0] [2]]
(t/count-nonzero (t/tensor [1 0 2 0])) ;; 2

---

Shape Management

Reshape and View

reshape may return a copy; view is strictly zero-copy and requires a contiguous tensor. Both support -1 for dimension inference.

(def x (t/arange 12))

(t/reshape x [3 4])   ;; [3×4]
(t/reshape x [2 -1])  ;; [2×6] — -1 inferred as 6
(t/view    x [4 3])   ;; [4×3] zero-copy view

Flatten and Unflatten

(def img (t/randn [1 3 4 4]))

(t/flatten img 1)               ;; [1×48] — flatten spatial+channel dims
(t/flatten img 1 2)             ;; [1×12×4] — partial flatten

(def y (t/randn [10 12]))
(t/unflatten y 1 [3 4])         ;; [10×3×4]

Unsqueeze and Expand

(def v (t/tensor [1 2 3]))

(t/unsqueeze v 0)               ;; [1×3]
(t/unsqueeze v 1)               ;; [3×1]

;; Expand a [1×3] to [4×3] without allocating new memory
(t/expand (t/unsqueeze v 0) [4 3])

;; Repeat tiles the data (allocates new memory)
(t/tile   v [3])                ;; [1 2 3 1 2 3 1 2 3]
(t/repeat v [2])                ;; repeat elements

Concat, Stack, Split, and Chunk

(def a (t/ones [2 1]))
(def b (t/ones [2 1]))

(t/cat   [a b] 1)    ;; [2×2] — concatenate along existing dim
(t/stack [a b] 0)    ;; [2×2×1] — concatenate along a new dim

(def x (t/arange 10))
(t/split x 3 0)      ;; [[0 1 2] [3 4 5] [6 7 8] [9]] — equal-size splits
(t/chunk x 4 0)      ;; 4 even chunks
(t/unbind x 0)       ;; sequence of scalar tensors along dim 0

Stacking Variants

(def xs [(t/tensor [1 2]) (t/tensor [3 4])])

(t/vstack xs)        ;; [[1 2] [3 4]] — vertical stack (alias: row-stack)
(t/hstack xs)        ;; [1 2 3 4]     — horizontal stack
(t/dstack xs)        ;; [[[1 3] [2 4]]] — depth stack

(t/row-stack    xs)  ;; alias for vstack
(t/column-stack xs)  ;; stack as columns

Transpose, T, Permute, and Swap

(def m (t/randn [2 3]))

(t/transpose m 0 1)          ;; [3×2] — swap two dims
(t/T m)                      ;; [3×2] — shorthand for last two dims
(t/swapaxes m 0 1)           ;; [3×2] — alias for transpose

(def cube (t/randn [2 3 4]))
(t/permute cube [2 0 1])     ;; [4×2×3] — arbitrary reordering
(t/movedim cube 0 2)         ;; [3×4×2] — move dim 0 to position 2

Other Shape Helpers

(t/tril (t/ones [3 3]))      ;; lower triangular (main diagonal)
(t/tril (t/ones [3 3]) -1)   ;; lower triangular (below main diagonal)
(t/triu (t/ones [3 3]))      ;; upper triangular

(t/diag     (t/tensor [1 2 3]))   ;; diagonal matrix from vector
(t/diagonal (t/randn [3 3]))      ;; extract main diagonal as 1-D tensor
(t/trace    (t/randn [3 3]))      ;; sum of diagonal elements

(t/flip (t/arange 5) [0])         ;; [4 3 2 1 0]
(t/roll (t/arange 5) 2 0)         ;; [3 4 0 1 2]
(t/rot90 (t/randn [2 2]) 1 [0 1]) ;; 90° rotation in the 0-1 plane

(t/cross (t/tensor [1 0 0]) (t/tensor [0 1 0])) ;; [0 0 1] cross product
(t/meshgrid [(t/arange 3) (t/arange 4)])         ;; coordinate grids

(t/broadcast-to (t/ones [1 3]) [4 3])            ;; expand to [4×3]
(t/broadcast-tensors [(t/ones [1 3]) (t/ones [4 1])]) ;; pair of [4×3]

---

Slicing and Indexing

ix is Clorch’s ergonomic indexer that mirrors Python’s [] syntax. It accepts integers, Clojure ranges, :_ for “all elements”, keyword :... for ellipsis, and nested vectors for advanced indexing. Full slicing documentation is on the Advanced Slicing page.

(def m (t/arange 12))
(def M (t/reshape m [3 4]))

;; Basic integer indexing
(t/ix M 0)           ;; first row → [0 1 2 3]
(t/ix M 1 2)         ;; element at row 1, col 2 → 6

;; Range slices: [start end] (exclusive end)
(t/ix M :_ [1 3])    ;; all rows, cols 1..2

;; select: pick a single index along a dimension
(t/select M 0 1)     ;; row 1

;; index-select: pick multiple indices along a dimension
(t/index-select M 0 (t/tensor [0 2] {:dtype :int64})) ;; rows 0 and 2

;; masked-fill: fill where a boolean mask is true
(t/masked-fill M (t/eq M 5) -1.0) ;; replace 5 with -1

;; take-along-dim: gather values using an index tensor
(t/take-along-dim M (t/tensor [[0 1 0 1]] {:dtype :int64}) 0)

;; scatter-reduce: reduce-scatter into a target tensor
(t/scatter-reduce (t/zeros [3 4]) 0
                  (t/tensor [[0 1 2 0]] {:dtype :int64})
                  (t/ones [1 4])
                  "sum")

---

Linear Algebra (linalg- functions)

The clorch.torch namespace exposes the full torch.linalg surface as linalg- prefixed functions.

(def A (t/randn [3 3]))

;; Decompositions
(t/linalg-cholesky A)          ;; lower-triangular Cholesky factor L
(t/linalg-svd      A)          ;; [U S Vh] — full singular value decomp
(t/linalg-qr       A)          ;; [Q R]
(t/linalg-eig      A)          ;; [eigenvalues eigenvectors]
(t/linalg-eigh     A)          ;; [eigenvalues eigenvectors] (symmetric/Hermitian)

;; Inverses and determinants
(t/linalg-inv A)               ;; matrix inverse
(t/linalg-det A)               ;; determinant
(t/linalg-pinv A)              ;; Moore-Penrose pseudo-inverse

;; Solving linear systems
(def b (t/randn [3 1]))
(t/linalg-solve  A b)          ;; solve Ax = b
(t/linalg-lstsq  A b)          ;; least-squares solution

;; Norms
(t/linalg-norm A)              ;; Frobenius norm (default)
(t/linalg-norm A 2)            ;; spectral norm

;; Matrix functions
(t/matrix-exp   A)             ;; matrix exponential
(t/linalg-matrix-power A 3)    ;; integer matrix power

---

Utility Methods

Type Conversion

(def x (t/tensor [1 2 3]))

(t/to-float x)          ;; cast to :float32
(t/to-long  x)          ;; cast to :int64
(t/to       x :float64) ;; explicit keyword
(t/to       x :bfloat16)

Device Placement

Clorch picks up the best available LibTorch backend when clorch.torch loads. Move tensors and models to a device explicitly:

(require '[clorch.cuda :as cuda]
         '[clorch.nn   :as nn])

(def device (if (cuda/available?) :cuda :cpu))

;; Create directly on device
(def x (t/randn [32 128] {:device device}))

;; Move an existing tensor
(def y (t/to existing-tensor device))

;; Move a model (all parameters migrate atomically)
(nn/to model device)

A model and its input tensors must reside on the same device. Passing a CUDA tensor to a CPU model — or vice versa — throws a runtime error from LibTorch.

Inspection

(t/size  (t/randn [3 4]))    ;; [3 4]
(t/size  (t/randn [3 4]) 0)  ;; 3 (single dim)
(t/dtype (t/ones  [2]))      ;; :float32

;; Extract a JVM scalar from a single-element tensor
(t/item-float (t/tensor [3.14])) ;; 3.14 (Float)

Printing

tprint is a minimalist REPL printer that shows shape, dtype, and values without flooding the output for large tensors:

(t/tprint (t/randn [3 3]))

tensor-string returns the same representation as a Clojure String, useful for logging:

(println (t/tensor-string (t/ones [2 2])))

Tensors also implement print-method, so they render sensibly via println and pr.

---

Logic and Search

Comparisons

All comparison functions return a :bool tensor of the same shape:

(def a (t/tensor [1 2 3]))
(def b (t/tensor [3 2 1]))

(t/eq a b)   ;; [false true false]
(t/ne a b)   ;; [true false true]
(t/gt a b)   ;; [false false true]
(t/lt a b)   ;; [true false false]
(t/ge a 2)   ;; [false true true]
(t/le a 2)   ;; [true true false]

Floating-point Predicates

(t/isnan    (t/tensor [1.0 Float/NaN]))   ;; [false true]
(t/isinf    (t/tensor [1.0 t/inf]))       ;; [false true]
(t/isfinite (t/tensor [1.0 t/inf]))       ;; [true false]
(t/isclose  (t/tensor [1.0]) (t/tensor [1.0001]) :atol 1e-3) ;; [true]
(t/allclose (t/tensor [1.0]) (t/tensor [1.0001]) :atol 1e-3) ;; true (scalar bool)

Logical Operations

(def p (t/tensor [true  false true]))
(def q (t/tensor [true  true  false]))

(t/logical-and p q)  ;; [true false false]
(t/logical-or  p q)  ;; [true true  true]
(t/logical-xor p q)  ;; [false true  true]
(t/logical-not p)    ;; [false true false]

Conditional Selection with where

;; Select from x where condition is true, else from y
(t/where (t/gt (t/tensor [1.0 -2.0 3.0]) 0)
         (t/tensor [1.0 -2.0 3.0])
         (t/zeros [3]))
;; → [1.0 0.0 3.0]

Searching

(def scores (t/tensor [0.1 0.8 0.4 0.3]))

(t/argmax scores)            ;; tensor(1) — flat index of max
(t/argmin scores)            ;; tensor(0)

(t/nonzero (t/tensor [1 0 2 0])) ;; [[0] [2]] — indices of non-zero elements

;; Top-k: returns [values indices]
(let [[vals idxs] (t/topk scores 2)]
  (println vals idxs))       ;; [0.8 0.4]  [1 2]

;; Sort: returns [values indices]
(let [[sorted idxs] (t/sort scores :descending true)]
  (println sorted))          ;; [0.8 0.4 0.3 0.1]

---

Sampling

Multinomial

Sample one or more indices according to a probability distribution:

;; Draw 1 token from a probability vector
(t/multinomial (t/tensor [0.1 0.8 0.1]) 1)

;; Draw 3 samples with replacement from a batch row
(t/multinomial (t/tensor [[0.2 0.5 0.3]]) 3)

Nucleus (Top-p) Sampling

top-p implements the nucleus sampling algorithm used in autoregressive text generation. It applies softmax, accumulates sorted probabilities, masks out tokens beyond the probability threshold p, renormalises, and draws one sample:

;; logits tensor of shape [batch seq] or [1 vocab]
(t/top-p (t/tensor [[0.1 0.8 0.05 0.05]]) 0.9)
;; → [[1]] — index of the sampled token

top-p expects raw logits (pre-softmax). The function applies softmax internally before computing the cumulative distribution.

---

Rotary Position Embeddings (RoPE)

Clorch provides first-class RoPE support for transformer models. precompute-rope-freqs generates the cosine and sine embedding tables once, then apply-rope rotates query and key tensors in-place.

;; Precompute embedding tables for dim=64, seq-len=2048
(def [cos-emb sin-emb] (t/precompute-rope-freqs 64 2048))
;; optionally pass :theta for a custom base frequency:
;; (t/precompute-rope-freqs 64 2048 :theta 500000.0)

;; Apply to query/key tensors of shape [batch seq heads dim]
(def q (t/randn [1 16 8 64]))
(def q-rotated (t/apply-rope q cos-emb sin-emb))

---

JIT Save and Load

Clorch supports both eager-mode checkpoint save/load and TorchScript JIT modules.

Eager checkpoint: save and load

(require '[clorch.torch :as t])

;; Persist a tensor
(t/save (t/ones [2 2]) "weights.pt")

;; Reload into an existing tensor
(def w (t/ones [2 2]))
(t/load w "weights.pt")

save and load also accept nn/Module and optimizer objects.

TorchScript JIT: jit-save, jit-load, jit-forward

;; Load a TorchScript module exported from Python
(def jit-mod (t/jit-load "traced_model.pt"))

;; Load onto CUDA
(def jit-mod-gpu (t/jit-load "traced_model.pt" {:device :cuda}))

;; Run inference
(def output (t/jit-forward jit-mod [(t/ones [1 3 224 224])]))

;; Re-save the module
(t/jit-save jit-mod "traced_model_copy.pt")

jit-forward wraps the input tensors in IValueVector automatically and unwraps the returned IValue back to a tensor.

jit-trace and jit-script are not available at the Clojure level — tracing and scripting must be done in Python and the resulting .pt file loaded into Clorch with jit-load.

---

Memory Management

The following functions belong to clorch.torch and are central to native memory control. For a complete explanation see the Memory Management page.

Function	Effect
with-torch	Opens a native pointer scope; releases block-local intermediates on exit
retain!	Removes a pointer from its current scope, handing ownership to the GC
rescue-pointers!	Alias for retain!
release!	Immediately and unconditionally deallocates owned pointers
start-session!	Opens a long-lived thread-local scope for interactive REPL use
stop-session!	Closes the current thread’s interactive scope and releases all its pointers
gc!	Requests JVM GC and finalisation; diagnostic use only