r/deeplearning • • 11h ago

Reinforcement learning explained visually: how AI learns through rewards, mistakes, and trial and error

6 Upvotes

I made a beginner-friendly visual explanation of reinforcement learning—the type of machine learning behind game-playing AI, robotics, and parts of chatbot training.

Instead of starting with equations, it uses a robot-in-a-maze example to explain:

  • Agents, environments, actions, and rewards

  • How trial and error improves performance over episodes

  • Exploration vs exploitation

  • Delayed rewards and policies

  • Reward hacking and other risks

Watch here: https://youtu.be/a_MSTPB5MyA

If you like the explanation, please like, share, and subscribe to my channel for more visual AI and machine learning videos.


r/deeplearning • • 14h ago

The Glass Transfomer - My attempt at trying to understand the transformer better.

Thumbnail drive.google.com
4 Upvotes

Hi , so recently iv been experimenting with designing transformer alternatives, however i kept hitting a wall on how well they would proform against the Transformer.

So iv decided to try and track tokens from input to output. As models are normally very large and can take months to train, i designed my own training routine to try and get around the problem so its 11 minutes on kaggle for free and afew hours work. The data deffinetly isnt gold standard, but it sould give me more of a view into the mechanisms being used.

Maybe someone else can find it useful , i wouldnt read all 1000+ pages , maybe the first few pages then its quicker to plug it in the a LLM and get it too sort and analyse the data.

Kind regards.... Rick


r/deeplearning • • 7h ago

Could hierarchical latent representations be a practical alternative to next-token generation?

2 Upvotes

Been looking into Next-Concept Modeling (NCM).

The approach has two main stages:

  1. A multiscale VQ-VAE learns coarse-to-fine representations of text.
  2. A transformer predicts each level of discrete codes in parallel, conditioned on previous levels.

Once the hierarchy is complete, a decoder reconstructs the text.

What interests me is the trade-off between parallel generation and preserving exact token-level information.

The paper reports promising efficiency gains, but its tokenizer and generator are trained separately.

Would end-to-end training improve this approach?

Paper: https://arxiv.org/abs/2610.14162


r/deeplearning • • 8h ago

How should we evaluate AI-generated research summaries?

2 Upvotes

AI can make it easier to organize information about complex scientific topics, but producing a readable summary doesn't necessarily mean the underlying research is accurate.

I came across Autumn AI's research roundup on the 2026 Nobel Prize winners, which brings together information about their work and achievements. It got me thinking about how we should evaluate systems that synthesize information from multiple sources.

For an AI-assisted research workflow, I would look at three things:

Source traceability: Can each factual claim be checked against its original source?

Factual consistency: Does the summary preserve the meaning and context of the original information?

Uncertainty handling: Does the system distinguish verified facts from incomplete or conflicting information?

These seem like important evaluation criteria, especially as AI systems become more capable of processing large amounts of information.

For those working with deep learning, what evaluation methods would you use to measure the factual reliability of a research summarization system?


r/deeplearning • • 9h ago

Automatic Model-Agnostic Compression algorithm

1 Upvotes

I made [Sigularty](https://github.com/DewanshShah/Sigularty), an algorithm that combines multiple compression techniques, takes in any model and chooses which techniques to use, and also finds hyperparameters for a few of the techniques. This is one of my first proper projects, and I would love feedback. It is completely free and open source, and these are a few results the algorithm achieved by itself, without any human fine-tuning. You can easily try out some models through the model registry and experiment with it.

|Model|Acc Drop|Compression Ratio|CQI|Speedup|Size (MB)|Accuracy (%)|Latency (ms)|

|:-|:-|:-|:-|:-|:-|:-|:-|

|`Resnet50`|**2.20%**|**8.00x**|**9.897**|**1.28x**|90.66 -> 11.34|66.60 -> 64.40|6.977 -> 5.451|

|`vit_b_16`|**0.00%**|**4.51×**|**4.526**|**1.00x**|327.59 -> 72.67|77.00 -> 77.00|14.96 -> 14.90|

|`efficientnet_b0`|**2.80%**|**2.61x**|**2.284**|**0.90x**|15.95 -> 6.11|91.80 -> 89.00|8.238 -> 9.124|

|`bert_base`|**4.20%**|**6.61x**|**6.449**|**1.02x**|417.66 -> 67.17|86.20 -> 82.00|10.393 -> 10.209|

|`distilbert`|**5.60%**|**9.55x**|**7.049**|**0.78x**|255.42 -> 26.75|82.40 -> 76.80|5.441 -> 6.957|

|`distilgpt2`|**0.00%**|**2.00x**|**2.903**|**1.45x**|312.48 -> 156.24|87.20 -> 87.20|8.559 -> 5.896|


r/deeplearning • • 10h ago

How does VLLM and SGLANG work from first principles? An Armature Video

1 Upvotes

r/deeplearning • • 13h ago

[Tutorial] Extending AgentCore Agents with Custom Tools and MCP Servers

1 Upvotes

Extending AgentCore Agents with Custom Tools and MCP Servers

https://debuggercafe.com/extending-agentcore-agents-with-custom-tools-and-mcp-servers/

In the previous article on DebuggerCafe, we covered how to bootstrap an agent using Amazon Bedrock AgentCore, deploy it, and test it with different methods. However, the agent had access to only the default tools and MCPs. For most use cases, this is not enough. This article takes it to the next step, where we will be extending AgentCore Agents with custom tools and MCP servers.


r/deeplearning • • 16h ago

Introducing Agent Compute Engine and expanding agent governance | Google Cloud Blog

1 Upvotes

Google's Agent Compute Engine announcement this week buried the most important detail: they explicitly called out the need for verifiable agent identity and per-call policy enforcement at runtime. This isn't a Google-specific gap. It's the same gap that showed up in the Wiz research last year on compromised cloud service accounts and in the Anthropic model card disclosures about tool-call amplification.

The hard number that keeps coming up in incident post-mortems: the window between a rogue agent's first unauthorized action and the second is under 50ms. That's the blast radius window. By the time a human is paged, the agent has already made 3–12 downstream tool calls, each inheriting the same compromised credential or misconfigured permission scope.

Most teams are still treating this as an identity provisioning problem — give the agent a service account, scope it tightly, rotate it on a schedule. But the incidents keep happening because the enforcement gap isn't at provisioning time. It's at call time, when the agent is live and making decisions no static policy anticipated.

How are practitioners here actually handling real-time per-call enforcement for agents that have legitimate broad access by design — things like orchestrator agents or agents with delegated user permissions? Are you enforcing at the infra layer, the SDK layer, or somewhere else?


r/deeplearning • • 17h ago

SteerTTS-edge: a 12 MB on-device TTS (int4) with 200 voices, 50 style tags and 12 numeric prosody controls, no cloning

1 Upvotes

Few days back I shared SteerTTS v1, a small English TTS you steer with numbers (https://huggingface.co/WhissleAI/steer-tts). Today I am sharing the edge variant

Size: everything an app has to ship (weights, voices, text front end and Python runtime) totals 11.84 MB.

How:

  • Acoustic model (9.25M params, attention + bidirectional Mamba-3) is quantization-aware trained to int4
  • 6.2M-param Vocos-style vocoder is distilled from charactr/vocos-mel-24khz, also int4 QAT
  • Text front end is pure numpy, with no NLTK, g2p_en, torchaudio or num2words at runtime

Quality (Open TTS Leaderboard protocol, run locally, not an official submission):

Seed-TTS WER / UTMOS CV3-Eval WER / UTMOS
v1 fp32 (no refiner) 0.96% / 4.20
edge 0.96% / 4.11

Intelligibility is essentially unchanged. UTMOS drops about 0.1, almost all of it from the small int4 vocoder.

Steering survives quantization. Every one of the 12 controls (rate, loudness, pitch, pitch range, final rise, pauses, tilt, breathiness, and more) keeps at least 95% of its fp32 effect. You also get 50 style tags and inline markup:

Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow> <whisper>Don't tell anyone.</whisper>

Speed (CPU only, a 2018 Threadripper under load): RTF 0.053 with 4 threads (~19x realtime), ~120 ms to first audio, with sentence-level streaming.

Bash

pip install .
steertts-edge "Hello there, nice to meet you." -o hello.wav

Limitations:

  • English only, and 200 voices (a subset of the full model’s 2,825)
  • No voice cloning, free-text prompts or flow refiner on device. Those stay in the full model.
  • Weights are dequantized to fp32 at load, so the win is size, not speed. Unless we write new kernels for edge devices.
  • Emotion tags are mostly style switches, and fast/anger are weak

I need feedback, especially on the vocoder quality gap and any edge targets you’d like to see (ONNX, mobile, WASM).

Model card:

https://huggingface.co/WhissleAI/steer-tts-edge

Some examples here:

https://whissleai.github.io/steer-tts/


r/deeplearning • • 19h ago

Fresher applying for ML / CV / AI engineer roles. Is this resume good enough?

1 Upvotes

Hey all, I finished my B.Tech in AI/ML this year (8.67 GPA) and I'm looking for fresher roles in ML, CV or AI engineering. Resume attached, personal details removed

Be honest, would this get you to call me for an interview? What would you fix first? Also not sure if I should cut the smaller projects (like the MNIST one) so the main project gets more space Thanks!


r/deeplearning • • 21h ago

I’m building an open-source collection of ML research paper implementations from scratch, looking for people who want to learn and contribute together!

Thumbnail
1 Upvotes

r/deeplearning • • 21h ago

Memorizer automaton

Thumbnail
1 Upvotes

r/deeplearning • • 22h ago

Sparse paged GQA decode attention with FP8 KV cache on SM89: CUDA vs Triton vs CuTe DSL vs Mojo vs eager PyTorch

Thumbnail github.com
1 Upvotes

r/deeplearning • • 23h ago

Sparse paged GQA decode attention with FP8 KV cache on SM89: CUDA vs Triton vs CuTe DSL vs Mojo vs eager PyTorch

Thumbnail github.com
1 Upvotes

r/deeplearning • • 23h ago

olá estou tentando criar um modelo multi-task mas ele só alcança 0.40 de acurácia

1 Upvotes

alguém sabe como aumentar a acurácia será que e o tamanho ou falta de pré-treino

link do dataset link https://kaggle.com/datasets/ca3ba9a1f48f6bac94b7f592f53d96954e7bf52ac0e4e25f5e1422b2f06bc71c

link do notebook https://github.com/WELLINGTON455/modelo-30m

link do modelo https://www.kaggle.com/models/wellingtonneresjose/model-30m1


r/deeplearning • • 2h ago

olá estou tentando criar um modelo multi-task mas ele só alcança 0.40 de acurácia

0 Upvotes

r/deeplearning • • 2h ago

I made Transformer Architecture from scratch [P]

Thumbnail github.com
0 Upvotes

I implemented the 2017 transformer architecture using pytorch, the model right now doesn't predict very good due to the fact that I am using on 2 layers of encoder and decoder with just 1000 words to train it , due to the fact that it takes a lot of time to train my implementation. Is there a way that I can reduce the time it takes for training or maybe improve its results?