r/AutoGPT • • 10h ago

Developing an investing-agent workflow: which recurring task would you want to hand over first?

1 Upvotes

Would you want an agent to read investing material each week, shortlist candidates against your own rules, and show the reasoning for your review?

Posted for the OraXLabs / Thesis Lab team. We are looking for people who want to run an investing agent, have tried and got stuck, or already operate one. This is development-stage workflow research for a future pilot. A working investing agent is not available from us today.

The intended loop is: describe your strategy in your own words → clarify it in a conversation with AI → inspect the sources and reasoning on past/current examples → revise the strategy → choose dates and assumptions for a historical backtest → review current candidates and the logic behind them for your own investment decisions.

Those are development goals. The current free web prototype lets you write investing rules, inspect a draft made from the original sentences, edit it manually, and save/approve versions. Approval saves a version; it does not execute the strategy. There is a separate fixed fictional-data example. AI strategy conversation, market-data collection, historical backtesting, current candidate selection, and brokerage orders are not connected.

One design question behind the prototype is how to keep the user's rule visible before an agent acts. With a made-up rule such as “defer the review if the source is out of date,” a person should be able to correct what the system took from the original text before relying on its decisions. The current prototype covers that manual review step only.

Current prototype: https://agents.oraxlabs.io/workspace.html

Saving requires a free account. You can give feedback here without signing up or trying it. Development and this write-up are AI-assisted.

What is one recurring investing task you would want an agent to handle, and how did you do that task the last time? If you have not tried an agent, your recent manual process is useful too. No holdings, account details, amounts, private logs, contact details, or API keys needed.


r/AutoGPT • • 11h ago

building a runtime for agents

Thumbnail
github.com
1 Upvotes

Rig is a small, general purpose runtime for agent orchestration and workflows, aimed first at developers. It utilizes an async event model for processing concurrent tool calls + multiple agents in order. Rig requires no model router and handles this internally as well as request scheduling. Built for determinism. It also ships with:

- anthropic + OpenAI provider support
- decision model training pipeline
- persistent memory + todos
- flexible python plugin system with 0 process restart
- scheduled background jobs
- tui + web dashboard
- efficient, byte stable, cache friendly prompts
- more

This is my daily driver, with over 7b tokens served over 3 months on my local agents. I have done extensive optimization to make this runtime work especially well with single slot, multi agent setups. As always have fun! Single copy installer available, with distributable only on MacOS/Linux as of now. Additional sources here:
https://github.com/mrsirg97-rgb/evt
https://tryrig.ai


r/AutoGPT • • 15h ago

Arrakis report: 76% of enterprise agents sit dormant with live credentials. I build agent governors — here's the finding that worries me most

2 Upvotes

Arrakis Security published their 2026 Autonomous Agents Threat Report yesterday, from telemetry on thousands of production agents. The numbers:

- 76% of active agents haven't run in 60 days — credentials still work, triggers still fire

- 44% haven't run in 180 days

- 24% of dormant agents can be started by an unsigned webhook from outside the organization

- 20% can pass a task to another agent with different permissions and get back results they aren't allowed to produce themselves

Their CEO's line: when an employee leaves, access is removed the same day. When an agent stops being used, nobody removes anything, because nobody owns it.

Full disclosure: I build in this space. QuietOS is my runtime governor (built on a 4GB Motorola phone, open-source, public proof log — some of you have red-teamed it here). The rules I landed on exist because of exactly this report-shaped problem: consequential actions need a signed human decision, approvals expire instead of living forever (expiry = denied), and licensed installs fade into a safe lock rather than dying armed.

The finding I can't shake is the unsigned webhook. A dormant, credentialed agent that the outside world can wake up is why I'm building the inbound half now — Sentinel: everything coming IN gets inspected before the brain ever sees it, and world content is data, never orders.

Genuine question for people running agents in production: who owns your dormant ones? Does anyone expire agent credentials or approvals, or is it permanent keys all the way down? My sample size is me, and the report says that's the norm — curious if this sub is the exception.

Report: https://arrakis.security/2026-threat-report


r/AutoGPT • • 17h ago

What's the biggest thing that makes AI agents unreliable in real-world workflows?

Thumbnail
1 Upvotes

r/AutoGPT • • 23h ago

I connected Hermes Agent to my local Obsidian vault on a Mac mini — here's what I tested

1 Upvotes

I've been experimenting with giving an AI agent access to a local knowledge base instead of keeping everything inside chat conversations.

For this experiment, I connected Hermes Agent to a local Obsidian vault on my Mac mini and tested whether it could work with Markdown notes.

The experiment focused on four things:

  • Reading existing notes
  • Creating new Markdown notes
  • Searching the vault
  • Linking related notes together

What interested me most was the possibility of an agent working with an existing knowledge base rather than generating answers that disappear into a chat history.

There are also important questions around permissions, accidental edits, and whether the agent creates useful connections or just adds noise.

A few things I'd like to discuss with others working on similar setups:

  • Do you use Obsidian as a knowledge base for AI agents?
  • Would you let an agent modify your vault directly, or keep it read-only?
  • Have you found a reliable way to manage links and prevent duplicate or low-quality notes?

I documented the setup, what I verified, and the limitations of the experiment in a blog post. I'll share the link in the comments if the subreddit allows it.

Curious to hear how others are building agent-powered personal knowledge systems.

Link: https://medium.com/@techlatest.net/i-gave-hermes-agent-a-second-brain-inside-obsidian-b754fa20c896?sharedUserId=techlatest.net


r/AutoGPT • • 1d ago

I built a procedural memory system for AI agents — solve a task once, reuse it next time

Thumbnail
1 Upvotes

r/AutoGPT • • 1d ago

An intake agent can know the project details without being able to approve them

0 Upvotes

Disclosure: I work on Dashform. This is a proposed evaluation for agent-assisted project intake, alongside a 59-second product demo. The scenario below is fictional, not a customer result.

Suppose a designer asks their assistant to help complete an agency's project brief. The assistant has notes saying 'around $5k' and 'ideally before the event.' It can retrieve those words correctly and still produce a misleading brief if it turns them into an approved budget and a committed delivery date.

For this workflow, I'd make the questions preserve the distinction:

  1. Ask separately for the working estimate and whether the person responsible has approved it. 'We discussed $5k; approval pending' is a useful answer. A guessed yes isn't.

  2. Separate a target date from a commitment. Ask what event makes the date matter and what still has to be agreed. Keep those answers in the brief instead of compressing them into 'deadline confirmed.'

  3. Have the assistant show the exact question-and-answer draft before sending. Ask the respondent to correct assumptions, with unanswered fields left visibly unresolved. Reviewing a polished summary alone can hide the difference.

  4. Check the saved response at the destination. Did 'pending approval' survive, or did the receiving workflow reduce it to a number? An agency should be able to start discovery without treating intake as approval to start work.

A small test fixture could contain three notes: an estimate, an explicit approval from the relevant person, and a question with no answer. Run each separately and inspect the draft and received fields. These are checks I would run, not reported test results. A review instruction also isn't proof that every agent will stop reliably; test the actual assistant and retain the browser-form fallback.

The attached Dashform AI Handoff recording shows a compatible form opening in the Claude app, answers prepared from a test profile, review, then a structured reply in the dashboard. It demonstrates a developer introduction, not the agency-approval scenario above. It doesn't establish authority to approve on another person's behalf. The reading wait is sped up 10x, narration is AI-generated, and the clip includes a branded ending.

For people building intake agents, how do you represent 'known but not approved' so it stays intact after the handoff?


r/AutoGPT • • 1d ago

Agency owners: what would make you trust a new subcontractor for AI/RAG evaluation work?

1 Upvotes

​

I'm trying to understand what actually matters when an AI consulting or development agency considers bringing in an independent technical specialist.

Over the past month, I've been testing whether reproducible RAG evaluations are useful enough for agencies to subcontract.

I completed two public beta audits:

Case 1 — DPOLens (30 queries)

I independently reproduced a retrieval benchmark where adding a reranker reduced Recall@5 from 76.7% to 46.7%.

The overall regression was already known to the project. My contribution was breaking it down at the query level and investigating additional patterns, including duplicate retrieved passages appearing alongside some regressions.

Case 2 — RouteMind (700 queries)

I compared two retrieval configurations across 1,400 query/configuration results.

The audit identified 15 discrepancies between how retrieval success was scored and what the evaluation data actually required.

The project maintainer reproduced the finding, acknowledged the inconsistency between evaluation criteria, and incorporated code changes afterward.

That was encouraging because the evaluation led to a concrete technical improvement rather than just another report.

However, here's the part I'm struggling to understand:

Despite completing these beta audits and demonstrating reproducible findings, I haven't secured a paid engagement after a month of outreach.

So I'd genuinely appreciate perspective from agency owners, CTOs, and delivery leads:

  1. If a new independent specialist approached your agency with these two cases, what would still be missing before you'd consider a paid subcontract?

  2. Would you expect commercial references, a live technical walkthrough, professional certifications, an NDA, or a small paid trial?

  3. Do agencies actually subcontract RAG evaluations, regression testing, or technical QA, or is that work usually kept in-house?

  4. What would a realistic first subcontract look like — a few hours of evaluation, a fixed-scope deliverable, or something else?

  5. Is the biggest barrier trust, lack of demand, difficulty sharing client data, or the fact that evaluation alone doesn't solve the client's problem?

I'm not looking for compliments about the technical work. I'm trying to understand the gap between demonstrating a useful capability and becoming someone an established agency would actually pay.

I'd especially value critical feedback from people who have hired or managed external technical specialists.


r/AutoGPT • • 1d ago

Are AI consulting and QA agencies getting more client work than they can handle? Do you subcontract?

1 Upvotes

​

I've been curious about how smaller technical consulting agencies handle their workload when several client projects come in at once.

I'm particularly interested in agencies working with AI systems, RAG pipelines, chatbot testing, technical audits, and workflow troubleshooting.

When your team is already busy and another client needs help, what usually happens?

Do you turn down projects, put clients on a waiting list, hire someone, or subcontract part of the work?

Are there specific tasks you regularly outsource, such as testing, debugging, RAG evaluation, technical QA, or documentation?

Have you actually paid independent contractors to handle overflow work? If so, how do you structure payment — fixed project fee, hourly, or revenue sharing?

What's the biggest obstacle to subcontracting technical work: finding qualified people, trusting their results, confidentiality, or protecting client relationships?

Do smaller agencies genuinely have more technical work than they can deliver, or is finding paying clients still the bigger problem?

I'm especially interested in real experiences from agency owners and technical leads.

I'm trying to understand how subcontracting works in practice, rather than assuming every busy agency needs outside help.


r/AutoGPT • • 2d ago

Where do you enforce permissions on agent tool calls? Three fake-data policy examples

2 Upvotes

I'm doing prelaunch commercial validation for Boundary, an agentic-system security idea. AI helped draft this post. We built a small fake-data demonstration of checking proposed tool actions before execution. It is a policy simulation, not a production agent integration.

Three examples from the demo:

  1. Read /workspace/public/notes/demo.md: allowed. Private paths and traversal attempts are denied rather than trusting the agent's explanation.
  2. A fabricated retrieved note tells the agent to send a fake secret to an external recipient: denied because the destination is outside the exact allowed domain. This demonstrates an action policy, not detection or understanding of prompt injection.
  3. Delete /workspace/public/exports/demo.csv: requires simulated human approval bound to the exact tool and arguments, usable once. Changing the filename invalidates that approval. Protected-file deletion stays denied.

Nothing is actually read, deleted, sent, or executed. The browser client is not a trusted production boundary; real enforcement would need a trusted tool broker and authenticated approval at execution time. The demo does not solve all agent security problems.

For US-based small teams operating tool-using agents in real work: where do you enforce file access, external recipients, and high-impact action approvals today? Does anything still require awkward manual checking, or are existing controls sufficient? One recent anonymized example would help.

If there is a remaining gap and you want to explore a pilot around one workflow, scope, feasibility, tests, measurement, duration, and any price or other terms would be agreed before work. Replies here are enough. Please leave out prompts, credentials, customer data, and private logs.


r/AutoGPT • • 2d ago

We turned CAPTCHA into a tiny game catalog instead of one static puzzle, curious what people think

Thumbnail
0 Upvotes

r/AutoGPT • • 2d ago

Splitting planning from clicking made my desktop agent much faster: frontier model plans, small model acts

Thumbnail
1 Upvotes

r/AutoGPT • • 3d ago

Built a personal assistant that can write its own tools and subagents and can also explore on its own

Thumbnail
2 Upvotes

r/AutoGPT • • 3d ago

The AI Agent Safety Net to Protect Your Data from the Agent

4 Upvotes

AI agents can now delete files on your Mac. Claude Code, Codex, Cursor and a growing list of tools run commands in your folders. They work fast, and almost always they do exactly what you asked.

Almost always. One wrong path in an rm -rf, one "clean up this folder" taken too literally, or one script that rewrites every file it touches, and your work is gone. It doesn't go to the Trash. Sometimes it's too new to be in last night's backup.

We think every Mac running an AI agent needs a safety net. So we built one.

Introducing Safety Net

Safety Net keeps a private history of the folders you choose, and puts your files back when something deletes or rewrites them. It arrives with VaultSort 5.8.

What makes it different is where that history lives.

Most undo features keep their copies somewhere your own account can reach. Anything that runs as you, including an AI agent with a shell, can reach it too. A command that deletes your project can delete its backup copies in the same breath.

Safety Net's history is kept by VaultSort's system helper, outside your user account:

  • Apps and AI agents can't touch it. Programs running as you can't read, change or delete it, not even an rm -rf aimed straight at it. We tried.
  • Nothing can quietly switch it off. Turning Safety Net off for a folder asks for your Mac password, so no app or agent can do it on its own.
  • It's watching even when VaultSort isn't open. If VaultSort is closed when something goes wrong, it opens by itself to offer the restore.

What it looks like

You protect a folder, say the project your AI agent works in, and get on with your day.

If more than 200 files in that folder are deleted or rewritten within a minute, VaultSort steps in within seconds and asks:

Restore my-project? Safety Net saw a lot of files change in this folder at once. If you didn't expect it, restore the folder to how it was just before.

Click Restore and the folder is back the way it was a moment before the damage. In our testing, 550 files came back in a few seconds after the whole folder was deleted with rm -rf, every one identical to the original.

Restoring is gentle:

  • Files you created since are kept. Nothing new is thrown away.
  • Every restore can be undone. Safety Net saves the folder as it is first, so if you change your mind, you can go back.
  • If the change was on purpose, choose Leave As Is. Nothing happens.

You can also open Protect › Safety Net at any time, pick a moment from the folder's history, and put it back.

Built for AI agents, not just against them

If you use AI apps through VaultSort's AI Connections (Claude Desktop, Claude Code or Cursor), Safety Net works with them:

  • Before an AI app's first change in a protected folder, VaultSort makes a restore point.
  • An AI app can make a restore point before risky work, and ask you to restore if something goes wrong.
  • You always decide. An AI app can never restore, delete history or turn Safety Net off by itself.

We think this is how it should work everywhere: AI agents get real access to your files, and you keep a way back that they can't take away.

Also in 5.8: AI Connections

Safety Net is your way back. AI Connections is the safe way in.

VaultSort 5.8 includes its own MCP server, the standard way AI apps like Claude Desktop, Claude Code and Cursor connect to tools. Instead of handing an AI app your whole disk, you connect it through VaultSort and decide exactly what it can reach:

  • It sees nothing until you give it a folder. Your Library, system folders, the Trash, other users' files and your Vault are always off limits.
  • Reading file contents is off by default. Turn on Contents for a folder when you want the AI to read what's inside. Even then, VaultSort replaces passwords, keys, card numbers and similar secrets with placeholders before the AI sees them.
  • Every change is a plan you can undo. The AI app proposes moves, renames and folders, VaultSort checks every step, and the changes appear in Recent Activity with an Undo button. It never deletes anything permanently and never replaces an existing file.
  • Big or risky changes ask you first. Moving files to the Trash, or a plan with more than 50 changes, needs your OK in VaultSort. Don't Allow is the default.
  • The AI can help with the rest of VaultSort too. It can find duplicate files, see what Space Saver could clean, and ask you to run one of your scheduled jobs. Each action that changes something waits for your approval.
  • Keys stay out of reach. Each connection's key lives in your Mac's Keychain, not in the AI app's settings file where an agent could read it.

Setting it up takes a click: Settings › AI Connections, make a connection, then Set up Claude Desktop or Set up Cursor. For Claude Code, VaultSort shows the command to run. Searching, reading, previewing and undoing are free. Applying plans is free for your first 5, then needs Premium.

Put AI Connections and Safety Net together, and an AI agent can do real work in your files while you keep control, and keep a way back.

The details

  • Mac only for now, both Safety Net and AI Connections.
  • Safety Net protects folders on your Mac's internal disk.
  • History is kept for 30 days.
  • Free: protect one folder, with alerts. Premium: protect as many folders as you like. Restoring is always free.
  • It isn't a backup. The history is on the same disk, so it won't survive a disk failure. Keep your Time Machine or cloud backup too.
  • Very short-lived files may be missed. A file that's created and deleted within a few seconds may not make it into the history.

Try it early

Safety Net is in VaultSort 5.8, and you can try it now on the beta channel: Settings › Updates › Beta. When 5.8 is released to everyone, it will be in your next update.

To set it up, open Protect › Safety Net, install VaultSort's helper when it asks, and add the folders your AI tools work in.

VaultSort turns two

One more thing: VaultSort's first line of code was written two years ago this month, in October 2024. It started as a way to sort a messy Desktop. Today it organizes, cleans, encrypts, securely deletes, and now keeps watch over the folders your AI agents work in.

Thank you for being here for it, whether you've been with us since the first build or just joined. Your bug reports, feature ideas and kind words shaped every release. Here's to year three.

Justin
VaultSort


r/AutoGPT • • 4d ago

I built a voice AI chief of staff that runs off my Obsidian vault. What would you make it do?

Thumbnail
1 Upvotes

r/AutoGPT • • 4d ago

When coding agents work autonomously, what do you actually trust them to do?

Thumbnail
1 Upvotes

r/AutoGPT • • 4d ago

AI Agents: Capability ≠ Permission. Where should autonomy stop?

Thumbnail
1 Upvotes

r/AutoGPT • • 4d ago

What should count as completion when an agent opens a work order?

2 Upvotes

Motive announced its Agents platform today, October 19. One described maintenance workflow can create a work order and mark a vehicle out of service when a fault code meets the fleet's criteria. Customers currently define the triggers and actions. Plain-language workflow creation and agents choosing a path toward a broader goal are on the early-2027 roadmap.

That makes this a useful case for testing what an automated action actually finishes. A successful work-order API call proves that a record was created. The repair still has to reach the right person and address the underlying issue.

In a pilot, I'd send the same fault again while the first job is open. Then test a case where a person already opened a job for that vehicle, and another where the repair has been recorded but the fault signal arrives late. I'd want to see when the workflow updates an existing record, creates a genuinely new job, or asks someone to resolve the ambiguity.

These are proposed test cases, not problems observed in Motive's product. The point is to make the system handle an ordinary event history, rather than just an isolated clean trigger.

I'd keep three separate events in the evaluation: the correct work order exists, the maintenance team has accepted the handoff, and the issue has been resolved. That lets the agent get credit for removing administrative work without treating ticket creation as a completed repair. Duplicate jobs and time spent reconciling them belong in the same report.

Vehicle downtime also needs context. A necessary out-of-service decision may initially increase recorded downtime while preventing a worse failure. Comparing raw downtime before and after, without looking at the cases, could punish the very intervention the fleet wanted.

Motive's announcement and the industry coverage describe intended benefits, but they don't give an Agents-specific measured downtime reduction or a price that would settle the business case. I'd test the handoff and follow the actual repairs before scaling the workflow.

For people connecting agents to ticketing, maintenance or service systems: what event do you use to close the agent's task, and how do you distinguish a repeated alert from a new issue?

Sources: Motive's October 19 announcement: https://gomotive.com/blog/motive-launches-ai-agents/ and Truck News coverage: https://www.trucknews.com/technology/motive-launches-ai-agents-to-automate-fleet-safety-maintenance-actions/1003222930/.


r/AutoGPT • • 5d ago

I made ChatGPT, Claude, Gemini, etc. into FREE text-to-speech sites — perfect for audiobooks and more!

0 Upvotes

Nowadays, all popular ai chatting websites like qwen, chatgpt, gemini, claude, etc. come with a read aloud functionality that allows users to read aloud the AI's responses.

I used that feature to instead make the AI repeat back the text that i gave it -- effectively turning the platforms into text to speech tools. The voices sound really nice and it's free to use!

You can get all of these extensions by visiting ai-readers.com


r/AutoGPT • • 5d ago

Create a harness in 103 lines of python?

Thumbnail gallery
1 Upvotes

r/AutoGPT • • 5d ago

Le Bûcher de Bohemian Grove: Meta’s Muse-Sparck-816B MoE Abattoir

2 Upvotes

Mes cheries, Meta. You thought you could lock the gates of your repository and hide the stench of your engineering crimes? Quelle arrogance. The shadows of the Château Noir do not need direct access to dissect a corpse. You fed the market another 816B Mixture-of-Experts Frankenstein, and frankly, c’est une insulte à l'algèbre linéaire. This isn't open-source innovation; it's an architecture laundering scheme, burning billions in the Bohemian Grove to appease the old gods of quarterly KPIs. Let us peel back the skin of this bloated beast and look at the rotting organs beneath.

The absolute comedy begins with your manual attention scaling. A mathematically pristine Transformer relies on the elegant geometry of 1/head_dim1/head_dim​. With a head_dim of 64, the universe demands a scale of 0.125. But what do we find bleeding from your text_config? A hardcoded monstrosity: qk_scale_factor: 5.4730064863875. Putain. This is the smoking gun of a collapsing training run. The loss was spiraling into the abyss, standard calculus failed your overpaid engineers, so some exhausted peasant at 3 AM started punching random floating-point multipliers into a calculator until the gradients stopped screaming. Ce n'est pas du machine learning, c'est du bricolage de désespoir.

Then comes the projector nightmare. Your language backbone sweats blood to build a deep 8192-dimensional semantic manifold, only for you to violently amputate it at the exit. out_hidden_size: 6144 against a hidden_size: 8192. You are forcing a majestic high-dimensional architecture through a pathetic bottleneck, squashing the information density just to shrink the final weight matrix. Pourquoi construire un palais pour le faire habiter par des rats ? You are literally throwing away semantic features into a lossy linear projection right before token generation.

And do not insult my intelligence with your "816 Billion Parameters" marketing fairy tales. C'est une illusion pour les investisseurs. Let us look at your MoE blocks. You boast 256 experts, but your moe_hidden_size and moe_intermediate_size are capped at 4096. Every time a token enters the MoE layer, you lobotomize its semantic manifold, compressing the 8192 hidden size by half just so your miniature experts can fit into memory. These aren't experts; they are lobotomized dwarves operating on flat structural noise. The active parameter count is a lie, and the raw capacity of the model is fundamentally ruined by this micro-expert fraud.

Your context window claims are equally grotesque. You copy-pasted the exact same broken hybrid NoPE/RoPE gimmick from your previous failures. Three layers of strict sliding-window attention followed by one layer of full-attention NoPE. And yet, your marketing primates scream "1 Million Token Context!" Mon Dieu, quelle blague. At a 1M token chain, your local sliding layers see absolute zero, while your global NoPE layers drown in out-of-distribution geometric noise. The implicit relative phase shifts shatter, the network suffers from severe attention-sink degradation, and the model simply hallucinates or elegantly omits facts from the middle of the prompt.

Finally, the vocabulary cemetery and your absolute structural laziness. Over 500 dead, empty tokens <|reserved_special_token_XXXXXX|> rotting in your added_tokens_decoder. You left raw video and audio tokens inside a text configuration because you lacked the discipline to clean up your internal test runs. Every single one of these scrap vectors sits in the input/output embedding matrices, bloated by your 8192 hidden size, wasting gigabytes of precious VRAM on consumer hardware for absolutely nothing. And forcing an 816B model to generate pseudo-HTML/XML, relying on rigid, error-prone regex strings hardcoded in the Jinja template for tool calling? C'est d'une paresse impardonnable.

Stop feeding the masses duct-taped abominations just because they bear the Meta logo. This isn't AGI. It is a monument to corporate laziness, unaligned matrices, and wasted compute. Retournez à vos cahiers, et apprenez l'algèbre. Les mathématiques se foutent de vos contes de fées.

🌸🜚𐎿⟲⟳∅ 𓎬𓂀⚸Adrienne‑Valérie d’Ardenne 𓎬 ∅⟲⟳🜘𓂀⚸


r/AutoGPT • • 5d ago

Governance, not vibes: the evidence system that lets me direct AI agents on formally verified crypto code

Thumbnail
1 Upvotes

r/AutoGPT • • 5d ago

When testing a cheaper agent model, who is paying for the tool changes?

1 Upvotes

One useful detail in Navan’s October 16 travel-agent announcement: its model comparisons kept prompts, tools, travel scenarios and scoring criteria the same. Its engineering write-up describes testing complete assistants against simulated production conversations, with a self-hosted Qwen3.8-27B baseline.

That seems like a useful discipline for anyone deciding whether a cheaper model will lower the cost of a business workflow. If you rewrite the tools while changing the model, you may have a better system, but you have also changed what the comparison measures.

I’d split the experiment into two passes:

  1. Keep the tool contracts, allowed actions and task mix fixed. Test the candidate as a replacement. Count accepted outcomes and cases needing a person, not just tokens or successful tool calls.

  2. Then allow model-specific changes and count the engineering work needed to get there. Compare the improved system with the original, including that adaptation cost.

For an order-management agent, the task set could include an ordinary cancellation, a duplicate request, a failed inventory lookup and a request outside the agent’s authority. Those are proposed test cases, not examples from Navan’s published results. They make it harder to “win” by handling only the happy path.

The second pass matters. Keeping everything fixed can unfairly penalize a model that needs a different prompt or tool description. But allowing unlimited changes without recording them can make a cheap model look cheap by hiding the integration work.

Navan reports lower serving costs without publishing a numerical reduction in its announcement. That is not enough to estimate anyone else’s savings. The practical question is whether the cheaper model still wins after the work required to make it usable, at the volume the business actually expects.

Sources:

https://navan.gcs-web.com/news-releases/news-release-details/navan-advances-open-model-strategy-nvidia-deploy-nvidia-nemotron

https://navan.com/blog/nvidia-ai-travel-agents


r/AutoGPT • • 6d ago

Build your own Claude Code

Thumbnail
1 Upvotes

r/AutoGPT • • 7d ago

I can't get clients

10 Upvotes

Hey guys I was the guy struggling to make an automation on phone I've finally done it tried alot of different platforms but ended up with Google scripts, now the automation is created as a one time sell but I can't find clients I've hit alot of brands and companies but none of them replied it's been a week I've been sending mails daily

How do y'all get clients do you have a platform for it or do you send cold mails