Xiaomi MiMo releases RL environments and training code
On September 26, 2026, a Xiaomi MiMo post pointed to reinforcement learning environments, training code, and a MiMo-V2.6 RL dataset for exploring language-model workflows.
Read articleKnowledge you can apply
Radar
Daily bookmarks: relevant news, facts and discoveries.
Memory Attention proposes token memory
Published on September 25, 2026, the post describes an alternative to attention value projection: token-indexed memory specific to each layer, combined with contextual keys.
Read articleMemory Attention adds token-indexed memory
A post published on September 25, 2026 describes a change to Transformer attention: replacing the learned value projection with a sum of contextual keys and token-indexed memory vectors specific to each layer. The memory can reside on the CPU.
Read articleNVIDIA Model Optimizer gathers model compression techniques
NVIDIA library with quantization, distillation, pruning, neural architecture search and speculative decoding, with checkpoints exportable to inference runtimes.
Read articleSelf-play explores pretraining without real data
A study investigates scalable pretraining from random initialization using a program generator and a learner, without training on real data.
Read articleSmolDataEnvs collects 5,000 verifiable RL tasks
Launched on September 25, 2026, the open-source collection includes tasks, environments, evaluations, and training to optimize small models for programming and data science.
Read articleInference caching: evaluate performance and isolation
Efficient memory can improve service capacity. The design must also prevent one session's context from appearing in another session.
Read articleTest-time training layers for sequence modeling
An article published on September 24, 2026 presents linear-complexity layers with expressive hidden states, updated through self-supervised learning.
Read articleBacking up high-value secret keys
A September 24, 2026 bookmark points to an article about backing up valuable secret keys. The record notes the security concern but gives no specific methods or results.
Read articleDifferential Transformer subtracts attention maps
A September 24, 2026 publication describes an attention variant that subtracts two maps to reduce noise and highlight signal. It notes potential interest for engineers exploring alternatives to standard attention.
Read articleFrom REINFORCE to GRPO: policy-gradient methods for LLMs
A publication dated September 24, 2026, outlines policy-gradient methods used to train LLMs, from vanilla policy gradient and REINFORCE to PPO, GRPO, and GRPO variants.
Read articleWeight spectra signal memorization in LLMs
A preliminary analysis reports that the attention matrix K exponent α tracks memorization of random labels during training, with variation across layers.
Read articleFlashAttention-3 aims to speed up attention on Hopper GPUs
The paper presents techniques for speeding up attention on Hopper GPUs, exploring asynchronous execution, low-precision computation, memory traffic, and hardware utilization.
Read articleLoRA+ proposes distinct learning rates for adapter matrices
Dated September 24, 2026, the summary describes an analysis of equal learning rates for LoRA matrices A and B and a proposal to use different rates in wide models.
Read articleMamba and Transformers through state-space duality
Published on September 24, 2026, the article explores theoretical links between state-space models such as Mamba and attention variants, using structured semiseparable matrix decompositions.
Read articleMegalodon explores sequence modeling with unlimited context
An architecture designed for efficient pretraining and inference on long sequences aims to address Transformer limitations.
Read articleMemory Attention proposes an alternative to value projections
Published on September 24, 2026, the summary introduces Memory Attention, a proposal replacing the Transformer's learned value projection and examined across several attention variants.
Read article
Document RAG: process data before composing the answer
Retrieval can combine information from several documents. Privacy must follow retrieval, context assembly and the answer.
Read articleRetNet combines parallel training and recurrent inference
Published on September 24, 2026, the article presents the Retentive Network, a language-model architecture proposing parallel training alongside recurrent and chunkwise recurrent inference paradigms.
Read articleEncryption and privacy: secure transit and reduce content
An encrypted connection protects transit. The recipient still receives what you send, so policy must act before delivery.
Read articleRota Nacional
Bring privacy into your workflow.
30 days, no card, with a starting quota. After that, Pix credit from R$ 10,00.