Jose Mora

🔍 Curious about optimizer performance? Benjamin Marie's latest article delves into a comparative analysis of AdamW-32bit, its 8-bit variant, and paged AdamW optimizers. Discover their effects on memory consumption, learning curves, and training time. #MachineLearning #Programming

13

This article by Pere Martra explores a structured pruning technique for state-of-the-art models, that uses a GLU architecture, enabling the creation of smaller and more efficient large language models. #Llama #LLM

5

Are you interested in vector search, indexing, or database efficiency? Then you won't want to miss this talk by Etienne Dilocker from Weaviate at Carnegie Mellon University's Database Group with Andy Pavlo! Etienne breaks down key challenges and innovations that are powering the future of search: ✨ Bit-Sliced Indexing - tackling complex range queries like price filters with innovative indexing. ✨ BM25 Hybrid Search - combining keyword and vector search to handle nuanced queries. ✨ Graph-Based Indexing (HNSW) - optimizing for high update rates and recalls without latency drops. ✨ Overcoming the Minority Filter Problem - enhancing search relevance through in-place filtering, based on the ACORN paper. ✨ Weaviate's Roadmap - performance, enterprise readiness, and pioneering "Recommendations as a Service." Check out the full talk for an in-depth look at how Weaviate is setting new standards in search technology: https://lnkd.in/gavAZRrn

33

2 Comments

#mdpiforecasting #openaccess Article "Spatial Dependence of Average Prices for Product Categories and Its Change over Time: Evidence from Daily Data" https://lnkd.in/edQMeFsj #prices #spatialvariation #spatialautocorrelation

1

Learn the basics of nonlinear dimensionality reduction, kernel PCA, and multidimensional scaling by following along Biman Chakraborty's clear, hands-on Python tutorial.

18

Explore José Antonio Siqueira de Cerqueira's efforts in enhancing language models for greater safety and reliability. Dive into this article to discover his significant contributions to the field - https://lnkd.in/dA-EWMu2! 📚👨‍💻 #research #languagemodels

4

We recently held the BCI and WBE for Safe AI Workshop. Diogo Di Lucena's (AE Studio) talk reviewed 'Self Modeling in Neural Networks.' Di Lucena discussed how he and his collaborators implement self-modeling in neural networks, inspired by Graziano's attention schema theory, to reduce model complexity and foster prosocial behavior in multi-agent systems. By creating simplified models of one's attention and the exact mechanisms used to model others' attention, the team demonstrates that self-modeling reduces complexity, making it easier for agents to model the system and potentially enhancing prosocial behavior. Evaluating their method on various tasks and models, the authors find consistent results in complexity reduction, suggesting that their approach, grounded in neuroscience, is distinct from other regularization methods and applicable across diverse contexts to improve model complexity and prosocial behavior in multi-agent settings. Learn more: https://lnkd.in/gFtdec-m

2

👨‍🔬 🚀 Thrilled to announce the new paper, “Softening Ontological Reasoning with Large Language Models” co-authored by our scientists! 🔗 https://lnkd.in/drX-vKyQ 🔍 Why It Matters: Prometheux data platform enables highly optimised big data queries to fragmented data sources and schemas, showing exactly how it obtains results. 💡This work will advance Prometheux integration with LLMs, further complementing the precision of structured data-processing with the flexibility of language models, empowering organisations to make real-time, fully auditable decisions driven by both structured and unstructured data. #ExplainableAI #knowledgegraphs #AI #neurosymbolic #LLMs

41

5 Comments

Mistral AI Launches Codestral Mamba 7B: A Revolutionary Code LLM Achieving 75% on HumanEval for Python Coding In a notable tribute to Cleopatra, Mistral AI has announced the release of Codestral Mamba 7B, a cutting-edge language model (LLM) specialized in code generation. Based on the Mamba2 architecture, this new model marks a significant milestone in AI and coding technology. Released under the Apache 2.0 license, Codestral Mamba 7B is available for free use, modification, and distribution, promising to open new avenues in AI architecture research. The release of Codestral Mamba 7B follows Mistral AI’s earlier success with the Mixtral family, underscoring the company’s commitment to pioneering new AI architectures. Codestral Mamba 7B distinguishes itself from traditional Transformer models by offering linear time inference and the theoretical capability to model sequences of infinite length. This unique feature allows users to engage extensively with the model, receiving quick responses regardless of the input length. Such efficiency is particularly valuable for coding applications, making Codestral Mamba 7B a powerful tool for enhancing code productivity. Codestral Mamba 7B is engineered to excel in advanced code and reasoning tasks. The model’s performance is on par with state-of-the-art (SOTA) Transformer-based models, making it a competitive option for developers. Mistral AI has rigorously tested Codestral Mamba 7B’s in-context retrieval capabilities, which can handle up to 256k tokens, positioning it as an excellent local code assistant. Article: https://lnkd.in/gvS73iK3 Check out the model: https://lnkd.in/d7rinrNY Mistral AI

20

Are you facing difficulties in deploying LLMs? Garvan D., a Technical Staff Member at Anthropic, has crafted an insightful guide on deploying Large Language Models (LLMs). This guide addresses the challenges and solutions in LLM deployment. In this walkthrough he builds a simple classifier and shares prompt engineering techniques to improve the model's accuracy.

1

NeuralForecast 1.7.4 Released: Nixtla’s Advanced Library Revolutionizes Neural Forecasting with Usability and Robustness In a significant development for the forecasting community, Nixtla has announced the release of NeuralForecast, an advanced library designed to offer a robust and user-friendly collection of neural forecasting models. This library aims to bridge the gap between complex neural networks and their practical application, addressing the persistent challenges faced by forecasters in terms of usability, accuracy, and computational efficiency. NeuralForecast is positioned as a comprehensive toolkit that includes a variety of neural network architectures such as Multi-Layer Perceptrons (MLP), Recurrent Neural Networks (RNNs), Temporal Convolutional Networks (TCNs), and more sophisticated models like NBEATS, NHITS, Temporal Fusion Transformer (TFT), and Informer. This wide range of models ensures users can access state-of-the-art techniques for diverse forecasting needs.... Full read: https://lnkd.in/gVsTDTXd Code: https://lnkd.in/g6ThGxfP Docs: https://lnkd.in/gP2DSZWj

19

📢Remote Sensed and/or Global Datasets for Distributed #HydrologicalModelling: A Review by Muhammad Haris Ali, Ioana Popescu, Andreja Jonoski and Dimitri P. Solomatine https://lnkd.in/e9FEM4a3 #hydrology

32

2 Comments

SparseEmbed. This work combines the strengths of sparse (SPLADE) and dense (ColBERT) representations for first-stage retrieval. TL;DR: - For a given sentence, sparse vectors are generated (similar to SPLADE) - For the same sentence, contextualized embeddings are generated for each token (similar to ColBERT). However, the attention layer only uses the logits of the sparse vectors (including token expansions) to generate the embeddings. - This results in faster training and smaller contextualized embeddings for each token. - Retrieval is hybrid. First, keyword matching matches the query to document expanded tokens. Then, scoring is computed based on the dot products between query and document contextual embeddings.

47

🧠 What’s a Semantic Cache in RAG? Here’s What We’ve Learned Semantic cache is a time and cost saving method for Retrieval Augmented Generation (RAG). Instead of always running a full vector search, a semantic cache stores embeddings of frequently asked queries and their responses. This speeds up your RAG system by skipping redundant searches for similar queries. How do you implement it? Step 1: Store history of answered questions, their embeddings and the LLM response in a semantic cache table. Step 2: When a new query comes in, compare it against the cache table using vector similarity. Step 3: If there’s a match with score over a certain threshold, serve the cached response. If not, run a fresh retrieval and add it to the cache. Now, let’s talk vector search methods: 1️⃣ Cosine Similarity: Measures angle between vectors, works well for normalized data. 2️⃣ Euclidean Distance: Measures straight-line distance but can be skewed by vector magnitude. 3️⃣ Dot Product: Our favorite—it factors in vector size and performs best for our use cases. Want to see how semantic caching powers a RAG system in real life? Check out our demo at https://lnkd.in/g2jFuHTT. 🚀

2