104: 3700次预训练寻找非共识，MiniMax-01开发者讲述4年线性注意力之旅 - 晚点聊 LateTalk
上期，聊了稀疏注意力。这期，聊注意力机制的另一大改进方向：线性注意力。嘉宾是 MiniMax 高级研究总监，负责 MiniMax-01 模型网络架构的钟怡然。他开发了 1 月中旬开源的 MiniMax-01 中使用的线性注意力架构。
怡然在 2021 年线性注意力还是“美好的泡泡”时就开始关注它的实现。这我一个 AI 研究者关注与投入小众方向的过程。
（本期因录音 bug，音质不太好，有波动，不时出现“变声期”，请见谅。）
BY AI探索指南
The campaign, which security firm Check Point has named Rampant Kitten, comprises two main components, one for Windows and the other for Android. Rampant Kitten’s objective is to steal Telegram messages, passwords, and two-factor authentication codes sent by SMS and then also take screenshots and record sounds within earshot of an infected phone, the researchers said in a post published on Friday.
A spike in interest rates since the start of the year has accelerated a rotation out of high-growth technology stocks and into value stocks poised to benefit from a reopening of the economy. The Nasdaq has fallen more than 10% over the past month as the Dow has soared to record highs, with a spike in the 10-year US Treasury yield acting as the main catalyst. It recently surged to a cycle high of more than 1.60% after starting the year below 1%. But according to Jim Paulsen, the Leuthold Group's chief investment strategist, rising interest rates do not represent a long-term threat to the stock market. Paulsen expects the 10-year yield to cross 2% by the end of the year. A spike in interest rates and its impact on the stock market depends on the economic backdrop, according to Paulsen. Rising interest rates amid a strengthening economy "may prove no challenge at all for stocks," Paulsen said.
telegram from us