Transformer distiled, Part 2 of 2
Regularization
Self- vs Cross-Attention
Adding vs concatenating positional encoding
Read More
Python Comprehension vs map()
The comprehension techiques for the data types `list`, `dict` and `set` append values to new data objects. They can be used to replace loops with `for` and functions like `map` and `filter`.
Read More
Static-sites on GH-Pages
Recap on ML
Github Pages
Jekyll small themes
As easy as adding remote_theme
inside _config.yml
, e.g. from Cayman theme:
remote_theme: pages-themes/cayman@v0.2.0
plugins:
- jekyll-remote-theme
Jekyll full themes
Gatsby
Sphinx
ReadTheDocs
Read More
Python Closures and Decorators
Functions are treated as objects and can be used as arguments or return values for other higher functions. This paradigm is called first-class functions. A built-in example in python would be map() which takes a function as an argument.
Read More
Transformer distiled, Part 1 of 2
Scaled dot-product
Softmax and multi-head attention
Linear layers
Learned Embeddings
Read More