LLM Compressor gathers compression techniques for vLLM
A briefing published on August 16, 2024, presents a library for applying GPTQ, SmoothQuant, and SparseGPT to models used with vLLM.
Read articleKnowledge you can apply
Radar
Daily bookmarks: relevant news, facts and discoveries.
Parler-TTS Mini and Large v1 are released
Announced on August 8, 2024, the speech synthesis models were trained on 45,000 hours of speech and let users control voice characteristics with text prompts.
Read articleSGLang and vLLM: throughput on A6000 GPUs
Published on August 4, 2024, the report compares two serving engines with Mistral Large 2 on four A6000 GPUs: about 1,100 tokens/s with SGLang and under 750 with vLLM; performance was similar at smaller batch sizes.
Read articlesqlite-vec brings vector search to SQLite
A publication dated August 2, 2024 describes a dependency-free C extension that adds vector search to SQLite and lists distance metrics, quantization, integrations, and SDKs.
Read articlePEER retrieves over one million experts
Published on August 1, 2024, this summary describes a Mixture of Experts architecture that uses product keys for sparse expert retrieval.
Read articleLoRA example for a 5-million-parameter MLP
Published on July 30, 2024, this example applies LoRA fine-tuning to a simple classification MLP and compares performance with adapters enabled and disabled.
Read articleA self-reasoning framework for RAG
Published on July 30, 2024, the summary describes a framework that evaluates retrieved documents, selects evidence, and examines reasoning steps before answering.
Read articlePyTorch presents torchchat for local inference
Published on July 30, 2024, the post introduces torchchat, a PyTorch library for running Llama 3, 3.1 and other language models on laptops, desktops and mobile devices.
Read articleNotebook transfers domain knowledge to Llama 3 8B
Published on July 29, 2024, an open-source Colab notebook fine-tunes Llama 3 8B with domain knowledge generated by Llama 3 405B.
Read articleAgentic components for Llama Stack APIs
A July 24, 2024 publication presents a GitHub repository of agentic components for Llama Stack APIs and highlights using tools and functions with Llama 3.1 models.
Read articleProject explores a 64-bit RISC-V computer on FPGA
Published on July 10, 2024, the entry describes an open-source project aiming to implement a 64-bit RISC-V computer on an FPGA, able to run Linux and host its own development.
Read articleESFT selects task-relevant experts in sparse LLMs
Published on July 5, 2024, the paper examines parameter-efficient fine-tuning for Mixture-of-Experts models. ESFT trains selected experts based on their relevance to customization tasks.
Read articleXEUS: multilingual speech encoder announced
On July 1, 2024, WavLab announced XEUS, a self-supervised speech encoder trained on more than 1 million hours of audio and covering more than 4,000 languages.
Read articleRadar: comparing speech-to-text models
Published on June 24, 2024, this item describes a comparison of speech-to-text models by word error rate, speed, and price.
Read articleGradient descent as attention over training examples
A February 6, 2023 article interprets gradient-trained linear layers as memories of examples and initial weights, relating predictions to a form of attention.
Read articleRota Nacional
Bring privacy into your workflow.
30 days, no card, with a starting quota. After that, Pix credit from R$ 10,00.