Recap on ML

A structured look at Transformer trade-offs, examining quadratic time/space complexity, O(1) path length, parallelization advantages, and the role of transfer learning and pre-training.

A comparison of Python's matmul operator and NumPy/PyTorch einsum for tensor contraction, covering theoretical background and practical case studies relevant to ML workloads.

An outline of covariance shift and inductive priors in machine learning, touching on regularization techniques including dropout, label smoothing, and the distinction between regularization and normalization.

A concise reference on Python iterators and generators — the protocol-based lazy-evaluation primitives underpinning memory-efficient iteration in Python.

Part two of the Transformer deep-dive, covering regularization, the distinction between self-attention and cross-attention, and the choice between adding and concatenating positional encodings.

The comprehension techiques for the data types `list`, `dict` and `set` append values to new data objects. They can be used to replace loops with `for` and functions like `map` and `filter`.