Daily report — 2026-09-29

Deep learning problems rarely involve objectives that are equal in importance. A primary objective defines the goal, whilst secondary objectives, such as sparsity, compression, or robustness constrain the solution.

Claimed: none extracted

Evidence: + cheap/closed-form ("closed-form"); + toy/synthetic experiments ("synthetic"); + small/synthetic data ("synthetic"); + concrete method ("closed-form"); + formal statement ("convergence"); - system/framework (many moving parts) ("framework"); + code: https://github.com/DaraVaram/priority-constrained-descent; + 1 HF upvotes

heuristic

We study the problem of learning directed acyclic graphs (DAGs) with nonnegative edge weights from observational data. We propose the Nonnegative and Concomitant (NoCo) DAG estimator, which jointly recovers the weighted graph structure and the exogenous noise variances in the linear structural equation model for the observations.

Claimed: none extracted

Evidence: + cheap/closed-form ("convex"); + toy/synthetic experiments ("Simulated"); + small/synthetic data ("Simulated"); + explicit algorithm/pseudocode ("Algorithm 1"); + concrete method ("closed-form"); no public code found

heuristic

Model merging integrates the capabilities of multiple experts without joint retraining, but standard expert training optimizes task loss alone and does not guarantee good performance after merging. Merge-aware training (MAT) aims to improve merged performance, but existing methods do not fully account for common merging operations and add training cost.

Claimed: none extracted

Evidence: + toy/synthetic experiments ("simulated"); + small/synthetic data ("simulated"); + formal statement ("bound"); + claims simplicity ("Simple"); - survey/position (no method) ("perspective"); + code: https://github.com/egangu/smat; + keywords: sampling, kernel; + 1 HF upvotes

heuristic

Multimodal language models rerank visual document retrieval results accurately, but scoring every candidate page at full cost makes them slow. Some methods that compress these rerankers need relevance labels to regain accuracy, and they rank by the reranker score alone.

Claimed: NDCG@5 1.2

Evidence: + cheap/closed-form ("closed-form"); + explicit algorithm/pseudocode ("Algorithm 1"); + concrete method ("closed-form"); + formal statement ("bound"); no public code found; + keywords: retrieval, ranking

heuristic

Attention often concentrates on a small subset of tokens in the context, but which subset matters changes from one query to the next. To exploit this changing structure, we introduce SANTA++, a training-free stochastic attention method that uses representative keys for memory-efficient selection without scanning the entire key-value (KV) cache.

Claimed: none extracted

Evidence: + formal statement ("bound"); + code: https://github.com/OPUSLab/santapp-kernel-demo.git; + keywords: sampling, retrieval, kernel

heuristic

Moving ML-mediated decision making onto privacy-preserving clients decentralises the economic decision along with the inference. Shared budget constraints then depend on information that cannot be globally current, creating an information-structure failure that conventional pacing is not designed to solve.

Claimed: none extracted

Evidence: + toy/synthetic experiments ("simulation"); + formal statement ("bound"); + code: https://github.com/sarkar-dipankar/on-device-auction-audit; + keywords: ranking

heuristic

Federated Bayesian Optimization (FBO) enables distributed agents to collaboratively optimize expensive black-box objectives without sharing raw local observations. However, effective knowledge transfer remains challenging under communication constraints and task heterogeneity.

Claimed: none extracted

Evidence: + toy/synthetic experiments ("synthetic"); + small/synthetic data ("synthetic"); + public benchmarks ("benchmarks"); + concrete method ("closed-form"); + formal statement ("bound"); no public code found

heuristic

Automatic relevance determination (ARD), the default tool for variable selection in Gaussian-process (GP) regression, ranks inputs by inverse lengthscales -- which measure how fast a function varies, not how much an input contributes to prediction -- and offers no calibrated rule for deciding which inputs to keep. The prediction-centred alternative, the derivative sensitivity $ν_j = \mathbb{E}[(\p

Claimed: none extracted

Evidence: + cheap/closed-form ("closed form"); + concrete method ("closed form"); + formal statement ("We prove"); no public code found; + keywords: kernel, ranking

heuristic

Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched parameters or per-token FLOPs.

Claimed: accuracy 53.0%; accuracy 3.4

Evidence: + public benchmarks ("benchmarks"); + code: https://github.com/thu-nics/TaH; + 40 HF upvotes

heuristic

Large language models are increasingly deployed as stateful assistants that retain information across interactions and use tools to read, modify, and create persistent artifacts. As these artifacts are shared between users, they form an indirect communication channel between otherwise independent assistants.

Claimed: none extracted

Evidence: - LLM-based ("LLM"); + toy/synthetic experiments ("simulated"); + small/synthetic data ("simulated"); + code: https://github.com/psidharth567/Share-Borne-Virus

heuristic