NLP for Work

On why language looks the way it does, how humans process and use language, and a perspective on LLMs from linguistics.

Neural models greatly outperform grammar-based models across many tasks in modern computational linguistics. This raises the question of whether linguistic principles, such as the Principle of Compositionality, still have value as modeling tools. We review…

This special issue focuses on measuring language complexity. The contributions address methodological challenges, discuss implications for theoretical research, and use complexity measurements for testing theoretical claims. In this introductory article,…

Scaling, emergence, and reasoning in large language models Jason Wei Twitter: @_jasonwei Mistakes & opinions my own, and not of my employer.

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), ga...

Semantics II (Graduate, Rutgers, Spring 2021) Unit 1: Compositionality and Binding Part I: Compositionality, Heim & Kratzer theory of pronoun binding Readings: Heim & Kratzer (1998: Chapter…

Some thoughts on non-AR language models, and what it might take to dethrone autoregression: https://sander.ai/2023/01/09/diffusion-language.html

ChatGPT出现后惊喜或惊醒了很多人。惊喜是因为没想到大型语言模型（LLM,Large Language Model）效果能好成这样；惊醒是顿悟到我们对LLM的认知及发展理念，距离世界最先进的想法，差得有点远。我属于既惊喜又惊醒的…

An often-expressed sentiment is that deep learning (and machine learning in general) is “simply statistics,” in the sense that it uses different words to describe the same concepts statisticians ha…

The last decade has yielded dramatic and quite surprising breakthroughs in natural language processing through the use of simple artificial neural network computations, replicated on a very large scale and trained over exceedingly large amounts of data. The…

On going beyond message-passing based graph neural networks with physics-inspired “continuous” learning models

Xiang Hu, Haitao Mi, Zujie Wen, Yafang Wang, Yi Su, Jing Zheng, Gerard de Melo. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long…