Skip to main content

Estimated reading time: 7 minute(s).

Links of Interest

  1. “No inventions; no innovations," a History of US Steel: https://www.construction-physics.com/p/no-inventions-no-innovations-a-history
  2. Why are Apple silicon VMs so different?: https://eclecticlight.co/2023/12/29/why-are-apple-silicon-vms-so-different/
  3. How Arm conquered the chip market without making a single chip: https://www.theverge.com/23373371/arm-chips-chip-shortage-ceo-rene-haas-tech-intel-apple-decoder
  4. The Random Transformer: https://osanseviero.github.io/hackerllama/blog/posts/random_transformer/
  5. You don't need analytics on your blog: https://blog.yossarian.net/2023/12/24/You-dont-need-analytics-on-your-blog
  6. On building a semantic search: https://vickiboykis.com/2024/01/05/retro-on-viberary/
  7. Some good pointers on lasting web pages: https://jeffhuang.com/designed_to_last/
  8. Simple lasts longer: https://newsletter.pnote.eu/p/simple-lasts-longer
  9. Guide to Self-Attention: https://twiecki.io/blog/2024/01/04/
  10. Building a Container from Scratch in Rust: https://brianshih1.github.io/mini-container/
  11. Nice looking blog w/ contents: https://matthewsanabria.dev/posts/no-shell-for-you-container/#difficulties-of-minimal-container-images
  12. Setting up Windows 11 w/o Microsoft Account: https://www.tomshardware.com/how-to/install-windows-11-without-microsoft-account
  13. Follow RSS across the web: https://openrss.org/
  14. (Machine Learning Engineering Open Book ): https://github.com/stas00/ml-engineering
  15. Why is machine learning hard?: https://ai.stanford.edu/~zayd/why-is-machine-learning-hard.html
  16. Machine learning is still too hard for software engineers: https://news.ycombinator.com/item?id=39109469
  17. Build an LLM (from Scratch): https://github.com/rasbt/LLMs-from-scratch
  18. Goodhart's law: https://en.wikipedia.org/wiki/Goodhart%27s_law ("When a measure becomes a target, it ceases to be a good measure.")
  19. Technology is the problem )includes mention of "doped silicon user interfaces"): https://www.shyamsankar.com/p/technology-is-the-problem
  20. (I just like the heading fonts): https://joshstrange.com/2019/09/26/my-mac-apps/
  21. This site's auto-generated sitemap: https://zenosmosis.com/sitemap.xml
  22. Running Local LLMs and VLMs on the Raspberri Pi: https://towardsdatascience.com/running-local-llms-and-vlms-on-the-raspberry-pi-57bd0059c41a
  23. Adding full-text search to a static site: https://www.markusdosch.com/2022/05/adding-full-text-search-to-a-static-site-no-backend-needed/
  24. AI-driven search engine: https://www.perplexity.ai/
  25. Building container images using no tools: https://ravichaganti.com/blog/2022-11-28-building-container-images-using-no-tools/
  26. (A search engine in 80 lines of Python [give or take])
  27. Aligning an LLM with human preferences (I like the "trainer" API design, at first glance): https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/aligning.html
  28. Self-balancing cube: https://willempennings.nl/balancing-cube/
  29. StatQuest: An epic journey through statistics and machine learning: https://statquest.org/
  30. Fractals in Neural Network Hyperparameter Tuning: https://www.linkedin.com/posts/liorsinclair_this-is-incredible-fractal-patterns-were-ugcPost-7162848575834501125-TuDV?utm_source=share&utm_medium=member_desktop (related paper: https://arxiv.org/abs/2402.06184 )
  31. AdaBelief Optimizer: https://arxiv.org/abs/2010.07468 (source code: https://github.com/juntang-zhuang/Adabelief-Optimizer )
  32. [Mostly non-optimized] Machine Learning Algorithms in Python: https://github.com/rushter/MLAlgorithms
  33. Fix loose git object corruption: https://accio.github.io/programming/2021/06/16/fix-loose-objects-in-git.html
  34. GPT in 500 lines of SQL: https://explainextended.com/2023/12/31/happy-new-year-15/
  35. GPT in 60 lines of NumPy (Hacker News): https://news.ycombinator.com/item?id=34726115
  36. Emergent mind (monitors social mdedia for discussions about recently-published arXiv papers): https://www.emergentmind.com/
  37. Accelerating Generative AI with PyTorch II: GPT, Fast: https://pytorch.org/blog/accelerating-generative-ai-2/
  38. Keras (MLX backend): https://github.com/keras-team/keras/pull/18962
  39. Kaggle Finance Dataset (search): https://www.kaggle.com/search?q=finance+dataset
  40. Eloquent JavaScript 4th Edition (2024): https://news.ycombinator.com/item?id=39629044
  41. Famous algorithmic patterns and their everyday usage: https://www.linkedin.com/posts/arslanahmad_systemdesign-softwarearchitecture-softwaredevelopment-activity-7170661917621936128-qwFf?utm_source=share&utm_medium=member_desktop
  42. Big-O cheat sheet: https://www.linkedin.com/posts/arslanahmad_codinginterview-timecomplexity-activity-7167956367599763456-4H4t?utm_source=share&utm_medium=member_desktop
  43. Data structures worth knowing: https://www.linkedin.com/posts/arslanahmad_systemdesign-coding-interviewtips-activity-7171064213992402944-0qSv?utm_source=share&utm_medium=member_desktop
  44. Exponential smoothing animation tricks (with code examples): https://lisyarus.github.io/blog/programming/2023/02/21/exponential-smoothing.html
  45. I Was a Statistics Professor. I Used Sports Betting to Retire at 43: https://www.newsweek.com/i-was-statistics-professor-i-used-sports-betting-retire-43-1858974
  46. Linear warmup with cosine annealing (machine learning): https://github.com/rasbt/LLMs-from-scratch/blob/main/appendix-D/01_main-chapter-code/appendix-D.ipynb
  47. An invention to silence reggaeton with artificial intelligence: https://english.elpais.com/technology/2024-03-17/an-invention-to-silence-reggaeton-with-artificial-intelligence.html
  48. Sam's Journey (new NES game): https://news.ycombinator.com/item?id=39730787
  49. Grok-1 (xAI model): https://github.com/xai-org/grok-1
  50. Spleeter (audio source separation library with pretrained models; stem separation; vocals / drums / bass / piano / other separation): https://github.com/deezer/spleeter/wiki/2.-Getting-started#using-docker-image
  51. Free Spark Resources: https://www.linkedin.com/posts/shubhamwadekar_spark-data-dataengineering-activity-7178246825999556609-CjSI/?utm_source=share&utm_medium=member_android
  52. AssemblyScript: A TypeScript-like language for WebAssembly: https://www.assemblyscript.org/
  53. Radios, how do they work? (A brief introduction to antennas, superheterodyne receivers, and signal modulation schemes): https://lcamtuf.substack.com/p/radios-how-do-they- (Hacker News discussion: https://news.ycombinator.com/item?id=39813679 )
  54. Modern AI Discourse is Talentless Business Students Trying to Give Young Engineers PTSD: https://www.timokats.xyz/?content=writings/aiblog
  55. Reading and Writing WAV Files in Python: https://realpython.com/python-wav-files/
  56. Conversation as an Interface (2016): https://annjose.com/post/conversation-as-interface/
  57. Svelte parses HTML all wrong: https://github.com/sveltejs/svelte/issues/11052
  58. Find Median from Data Stream: https://leetcode.com/problems/find-median-from-data-stream/ (maybe it could be useful for implementation of RobustScaler partial fit [open-source the solution?]; related StackOverflow discussion: https://stackoverflow.com/questions/57291876/robustscaler-partial-fit-similar-to-minmaxscaler-or-standardscaler )
  59. LeetCode (25 questions to cover the most important patterns): https://www.linkedin.com/posts/alexandre-zajac_softwareengineering-coding-programming-activity-7181538559999238144-lwW8?utm_source=share&utm_medium=member_desktop
  60. JetMoE: Reaching LLaMA2 Performance with 0.1M Dollars: https://research.myshell.ai/jetmoe (Hacker News discussion: https://news.ycombinator.com/item?id=39933076 )
  61. Books for Learning Math for Machine Learning: https://mltechniques.com/2022/06/13/math-for-machine-learning-12-must-read-books/
  62. llm.c (Karpathy; LLM training in simple, pure C/CUDA): https://github.com/karpathy/llm.c
  63. ETF Dictionary: https://etfdb.com/etfs/
  64. Pick great stocks if you are a developer: https://medium.com/@Sanji_vals/pick-great-stocks-if-you-are-a-developer-detailed-guidance-on-stock-selection-for-developers-c1629b3a8eed
  65. ML big or small data and how it affects model architecture decisions: https://www.linkedin.com/posts/damienbenveniste_your-data-can-tell-you-a-lot-about-the-type-activity-7186390432434577409-xYI5
  66. JSR (The open-source package registry for modern JavaScript and TypeScript): https://jsr.io/
  67. How Vector Databases Work: https://www.linkedin.com/pulse/understanding-how-vector-databases-work-damien-benveniste-g6htc/
  68. Multihead Attention Layer with a Kolmogorov–Arnold Networks (KAN): https://www.linkedin.com/posts/damienbenveniste_here-is-my-implementation-of-a-multihead-activity-7192940670192394240-xq05
  69. Sparse Multihead Attention (implementation): https://www.linkedin.com/posts/damienbenveniste_here-is-how-you-can-create-a-multihead-self-attention-activity-7195828872121102337-OsPg/
  70. AI-powered robots are finding the flaws in ‘D’ grade U.S. infrastructure, from commuter bridges to military hardware: https://www.cnbc.com/2024/05/15/these-wall-climbing-robots-are-finding-flaws-in-d-grade-infrastructure.html
  71. MoE multi-head attention (PyTorch example): https://www.linkedin.com/posts/damienbenveniste_can-we-mix-the-concepts-of-multihead-attentions-activity-7199089073381072897-nNfF
  72. Here’s what’s really going on inside an LLM’s neural network: https://arstechnica.com/ai/2024/05/heres-whats-really-going-on-inside-an-llms-neural-network/
  73. Economic theory: https://en.wikipedia.org/wiki/Economics#Theoretical_research
  74. GuruFocus (investing): https://www.gurufocus.com/
  75. Financial Statement Analysis with Large Language Models: https://www.newsletter.datadrivenvc.io/p/financial-statement-analysis-with (LinkedIn thread: https://www.linkedin.com/posts/andreretterath_financial-statement-analysis-with-large-language-activity-7199289952910667776-14At )
  76. Vector Indexing all of Wikipedia, on a laptop: https://foojay.io/today/indexing-all-of-wikipedia-on-a-laptop/ (Hacker News discussion: https://news.ycombinator.com/item?id=40514266 )
  77. De-Googling: https://blog.nradk.com/posts/degoogling/
  78. Don't be stupid about trading: https://www.linkedin.com/posts/alfonso-peccatiello-72156a6a_trading-is-hard-and-the-rule-1-not-to-activity-7203750640576086017-KDKA
  79. Interesting CSS-separator generator (different shapes + demo): https://github.com/wwebdev/separator-generator (demo: https://wweb.dev/resources/css-separator-generator)
  80. Google Search console: https://search.google.com/search-console
  81. Apple MusicKit ("what am I listening to?"): https://developer.apple.com/documentation/musickitjs
  82. Immersive Linear Algebra (interactive book): https://immersivemath.com/ila/index.html
  83. Extracting Concepts from GPT-4: https://openai.com/index/extracting-concepts-from-gpt-4/
  84. Polars-Cookbook (Jupyter notebook examples for Python Polars): https://github.com/PacktPublishing/Polars-Cookbook
  85. Naked JSX: https://nakedjsx.org/
  86. Why Google Sheets ported its calculation worker from JavaScript to WasmGC: https://web.dev/case-studies/google-sheets-wasmgc
  87. Rust learning resources:
  88. Active Strategies Are Looking Good, But Don’t Abandon Your ETFs: https://www.barrons.com/articles/active-strategies-are-looking-good-but-dont-abandon-your-etfs-2057c60d
  89. Eureka Labs (Andrej Karpathy education project): https://eurekalabs.ai/
  90. GitHub Status: https://www.githubstatus.com/
  91. CrowdStrike Technical Details on Today’s [July 19, 2024] Outage: https://www.crowdstrike.com/blog/technical-details-on-todays-outage/
  92. Branchless Programming: https://www.linkedin.com/posts/heriklima_cplusplus-branchlessprogramming-optimization-activity-7218766293506674688-RQu5
  93. HATEOAS (Hypermedia as the engine of application state): https://en.wikipedia.org/wiki/HATEOAS (Roy Fielding: https://en.wikipedia.org/wiki/Roy_Fielding )
  94. Argo tunnels that live forever: https://blog.cloudflare.com/argo-tunnels-that-live-forever
  95. The Standard (A collection of decades of experience in the engineering industry): https://github.com/hassanhabib/The-Standard
  96. How Postgres stores data on disk: https://drew.silcock.dev/blog/how-postgres-stores-data-on-disk/ (Hacker News thread: https://news.ycombinator.com/ )
  97. This Week in Rust (Aug. 7, 2024): https://this-week-in-rust.org/blog/2024/08/07/this-week-in-rust-559/
  98. Distributed Filesystem written in Rust: https://medium.com/@xorio42/list/distributed-filesystem-written-in-rust-317d40f38304
  99. Thoughts on vector-database searches: https://www.linkedin.com/posts/damienbenveniste_we-have-recently-seen-a-surge-in-vector-databases-activity-7226984916687708160-K-1Z/
  100. e18e (initiative to improve JavaScript ecosystem performance): https://e18e.dev/
  101. Information bottleneck method: https://en.wikipedia.org/wiki/Information_bottleneck_method
  102. Winsorizing (transformation of statistics by limiting extreme values in the statistical data to reduce the effect of possibly spurious outliers): https://en.wikipedia.org/wiki/Winsorizing
  103. How to Make Your Machine Learning Models Robust to Outliers: https://www.comet.com/site/blog/how-to-make-your-machine-learning-models-robust-to-outliers/
  104. Apache Parquet vs. CSV: https://www.databricks.com/glossary/what-is-parquet
  105. Ask HN: What are your favorite index ETFs for Investing? https://news.ycombinator.com/item?id=40947524
  106. DFS as a filesystem (use DNS for a filesystem): https://www.linkedin.com/posts/laurie-kirk_dns-can-be-used-as-a-filesystem-yes-you-activity-7234610957002399744-Lqjv
  107. "Microsoft's Phi 3.5 vision model is really good at OCR/text extraction": https://x.com/dylfreed/status/1828132226523131931
  108. ETF Research Center: https://www.etfrc.com/portfolios/builder.php
  109. Information about passive investing (inspired by the philosophy of Vanguard founder John C. Bogle): https://bogleheads.org/
  110. Multithreading in Node.js: Using Atomics for Safe Shared Memory Operations: https://pavel-romanov.com/multithreading-in-nodejs-using-atomics-for-safe-shared-memory-operations
  111. Windows NT vs. Unix - A design comparison: https://blogsystem5.substack.com/p/windows-nt-vs-unix-design
  112. DARPA Translating All C to Rust (TRACTOR): https://www.darpa.mil/program/translating-all-c-to-rust (related Hacker News thread: https://news.ycombinator.com/item?id=41110269 )
  113. Moodist - Ambient sounds for focus and calm: https://moodist.app/
  114. Apple Mobile Processors Are Now Made in America (By TSMC): https://timculpan.substack.com/p/apple-mobile-processors-are-now-made
  115. The 2 ETFs That Track Congressional Stock Trades: https://www.morningstar.com/funds/2-etfs-that-track-congressional-stock-trades
  116. MLOps-Basics ("Understand the basics of MLOps like model building, monitoring, configurations, testing, packaging, deployment, cicd, etc."): https://github.com/graviraja/MLOps-Basics
  117. LLMs in Finance: https://github.com/hananedupouy/LLMs-in-Finance/tree/main (Related LinkedIn post: https://www.linkedin.com/posts/hanane-d-algo-trader_anthropic-agent-with-llamaindex-financial-activity-7241080939131392000-5sJa )
  118. Google Illuminate ("Transform your content into engaging AI‑generated audio discussions"): https://illuminate.google.com/
  119. ML - How to optimally sample Imbalanced Data: https://www.linkedin.com/feed/update/urn:li:activity:7247530677594718209
  120. JavaScript Structs proposal: https://github.com/tc39/proposal-structs
  121. "Whenever I lack motivation, I watch this": https://www.linkedin.com/posts/linasbeliunas_whenever-i-lack-motivation-i-watch-this-activity-7253121679277641728-koQo
  122. The pros and cons for investors of nonstop trading as NYSE looks to go 22 hours a day: https://www.cnbc.com/2024/10/28/the-pros-and-cons-for-investors-of-nonstop-trading-as-nyse-looks-to-go-22-hours-a-day-.html
  123. Send Web Push notifications from a Node.js server: https://www.bocoup.com/blog/full-stack-web-push-api-guide (also: https://www.npmjs.com/package/web-push )
  124. The Rust eBookshelf (The Rust Language & Ecosystem): https://dieterplex.github.io/rust-ebookshelf/ ( https://github.com/dieterplex/rust-ebookshelf )
  125. Your Hacker News (personalized, newspaper-like Hacker News): https://yourhackernews.com/
  126. You can work at McDonald’s and still become a millionaire: https://www.cnbc.com/2024/11/09/how-to-work-at-mcdonalds-and-still-become-a-millionaire.html
  127. Draw graphing trees with OpenAI: https://www.linkedin.com/posts/damienbenveniste_can-we-draw-trees-with-llms-you-bet-you-activity-7267594686788182016-CLQi/
  128. Rust WebAssembly OCR experiments: https://hugopeixoto.net/articles/rust-wasm-ocr-experiments.html

Papers of Interest

  1. Fractals in Neural Network Hyperparameter Tuning: https://arxiv.org/abs/2402.06184
  2. Deep Reinforcement Learning for Quantitative Trading: https://arxiv.org/abs/2312.15730v1
  3. Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping: https://arxiv.org/html/2402.14083v1
  4. [Apple's multi-modal, foundational model] MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training: https://arxiv.org/abs/2403.09611 (relevant Hacker News discussion: https://news.ycombinator.com/item?id=39722498 )
  5. Can Large Language Models Reason and Plan? (PDF): https://arxiv.org/pdf/2403.04121.pdf

Books of Interest

  1. Build a Large Language Model (From Scratch): https://www.manning.com/books/build-a-large-language-model-from-scratch?utm_source=raschka&utm_medium=affiliate&utm_campaign=book_raschka_build_12_12_23&a_aid=raschka&a_bid=4c2437a0&chan=mm_github
  2. WebGPU Unleashed: A Practical Tutorial (free book on WebGPU programming): https://shi-yan.github.io/webgpuunleashed/ (Hacker News thread: https://news.ycombinator.com/item?id=41156872 )

Videos of Interest