r/deeplearning • • 22h ago

Title: Scaling Laws Be Gone: With the Rise of Agentic AI, Is the New Bottleneck at Inference-time Compute?

Thumbnail
0 Upvotes

r/deeplearning • • 19h ago

Looking for ImageCLEF 2021 DrawnUI Wireframe Dataset

0 Upvotes

Hello everyone!
We are a group of female Computer Science students working on our graduation project.
We are having difficulty finding suitable datasets to train a YOLO model.
We found a dataset called **ImageCLEF 2021 DrawnUI Wireframe Datase https://www.aicrowd.com/challenges/imageclef-2021-drawnui-wireframe?utm\\_source=chatgpt.com\*\*
but unfortunately, it is no longer available for download
**Does anyone have access to this dataset or know how we can obtain it?**
We need it for research purposes only.
Thank you so much to anyone who can help us! ❤️


r/deeplearning • • 17h ago

I Finished This Book In April. The Story Didn't Wait

0 Upvotes

A Forbes analysis published this week found that 79% of organizations are running AI agents in production. Only 2% have identity security tooling to govern them.

That 77-point gap is not a planning failure. It is the normal operational state of enterprise AI right now. Most of those agents carry no declared identity, no defined behavioral scope, and no path to revoke access when something goes wrong. They are already running on production systems that nobody officially registered them to touch.

Shadow IT has always existed. But a misconfigured server sits still. An agent with live access to production data does not.

The discovery problem is upstream of everything else. You cannot scope, audit, or revoke what you have not cataloged. Security teams are already stretched before you add the question of finding agents that were never surfaced through any approval process.

For those of you actually working on this: how are you finding the unsanctioned ones? IAM log analysis, network traffic inspection, developer surveys, something else entirely? And how confident are you that your current method catches agents that were spun up outside the normal provisioning path?


r/deeplearning • • 1h ago

The idea behind Adam Optimizer

Thumbnail youtube.com
• Upvotes

Hi all! I'm AI engineer. This is the first video on my channel. I tried to build Adam from scratch, motivating each step by a concrete problem of the previous one: plain GD, Nesterov, AdaGrad, RMSProp ...

All animations are made with Manim. I'd really appreciate feedback, especially on anything that's mathematically imprecise or unclear.


r/deeplearning • • 1h ago

[R] SANATAN ATTENTION: No Growing KV cache (Standard Attention issue), no context decay (Mamba Issue) binary gate hiding inside i² = −1

Post image
• Upvotes

The idea in one paragraph: over 2x2 real matrices, i² = −1 has two orthogonal solutions, rotation by +90 and −90 degrees:

i₊ = [[0, −1], [1, 0]], i₋ = [[0, 1], [−1, 0]]

Square either one and you get [[−1, 0], [0, −1]] = −I, so both are genuine square roots of minus one. Their combination is

S(θ) = cosθ · i₊ + sinθ · i₋ = [[0, −(cosθ − sinθ)], [cosθ − sinθ, 0]]

whose off-diagonal entry is

±(cosθ − sinθ)

That ± in front is a binary object sitting inside a smooth formula. SANATAN promotes it into an explicit soft variable:

r = tanh( γ ( cos(θ+ω) − sin(θ+ω) ) ) ≈ ±1

Each token projection u becomes an angle θ = π·u / (1 + |u|), every feature dimension softly picks a root, and the six-channel feature map

Φ(x) = [ 1, cosθ, sinθ, r, r·cosθ, r·sinθ ]

makes the attention score factorize exactly:

⟨Φ(q), Φ(k)⟩ = (1 + r_q r_k)(1 + cos Δθ)

a per-dimension parity gate (0 or 2 for hard roots) times an angular similarity. The GIF above walks through it step by step.

Because the score is an inner product, the p=1 case runs as an exact recurrence with a constant-size state, no KV cache and no decay:

S_t = S_{t−1} + Φ(k_t) v_tᵀ, z_t = z_{t−1} + Φ(k_t), out = ⟨Φ(q), S⟩ / ⟨Φ(q), z⟩

What I measured (184M params, single seeded held-out protocol):

- The recurrence is exact in practice, not just on paper: parallel and recurrent computation of the trained model agree within 1e-4 nats (float32). There is a demo notebook that reproduces this on CPU in about two minutes on a small model (agreement ~4e-6 per logit).

- Quality survives the constant state: 3.951 held-out loss recurrent vs 3.966 for its full parallel parent, via a two-stage recipe (sharpened pretraining, brief p=1 adaptation). Deploying p>1-trained models through the recurrence costs ~5 nats, so that path is dead and the paper says so.

- Decode: fixed 14.4 MB state vs a KV cache that hits 2.4 GB at 32K context; 0.153 ms/token flat vs 0.49 to 7.3 ms/token for softmax decoding (3.2x to 48x from 1K to 32K).

- Ablation: the root-parity gate is worth +0.53 nats over the same angular kernel without it.

Paper (PDF): https://github.com/bhargavpatel431997/sanatan-attention/blob/main/paper/sanatan_v2.pdf

Code + runnable demo notebook: https://github.com/bhargavpatel431997/sanatan-attention

Visual explainer of the idea: https://github.com/bhargavpatel431997/sanatan-attention/blob/main/docs/MOTIVATION.md

Independent researcher, single GPU/TPU budget, so critiques and pointers are genuinely welcome, especially from anyone who has tried interaction-level gating inside linear attention before.

Thanks for reading,

Bhargav Patel

https://www.linkedin.com/in/bhargav-patel-63bb27121/


r/deeplearning • • 20h ago

Neural Networks Are Already Clean -- A Paradigm Shift in Interpretability Research (with interactive visualizations)

Thumbnail already-clean.goodfire.com
2 Upvotes

r/deeplearning • • 21h ago

We trained a chest X-ray TB model on a single RTX 3060. It topped Google's published result on one public benchmark and matched it on the other.

Post image
1 Upvotes

We're Mn AI, a two-person startup in Bahrain. No cluster, no big team. Just one RTX 3060 (12 GB) on a desk.

For comparison, Google's published model for this task was built by a Google Health research team on 248,445 X-rays.

We tested ours on the two standard public TB benchmarks from the US National Library of Medicine, the same ones Google reported on:

| Benchmark | Ours (AUC) | Google (AUC) |

|---|---|---|

| Montgomery (138 X-rays) | **0.998** | 0.97 |

| Shenzhen (662 X-rays) | **0.951** | 0.95 |

How we kept it honest:

- Zero benchmark images in training (we ran a duplicate check to make sure)

- Decision threshold locked before testing, on a separate set

- Every image used, no cherry-picking

The benchmarks are free and public. If you want to check our numbers yourself, DM us and we'll give you a free API key to run all 800 images.

Happy to answer anything about the setup, training on a 3060, or the evaluation.


r/deeplearning • • 20h ago

Mamba2-8B compression research series: 27.18 MiB recurrent cache or 3.141 GB stored model data

Thumbnail
3 Upvotes