r/devops • • 18h ago

Discussion How do you get your first DevOps job when every position requires experience?

29 Upvotes

Hey everyone!

I'm struggling to land my first DevOps opportunity. I've been sending my CV to companies for junior positions and internships, but I'm barely getting any responses.

I've been working on hands-on projects involving Docker, GitHub Actions, CI/CD, Linux, Nginx, PostgreSQL, and Prometheus to build my practical skills.

For those who have been in a similar situation, how did you land your first job? Should I focus on networking, contributing to open-source projects, or something else?

I'm also open to junior opportunities, internships, or teams willing to give a motivated beginner a chance, including remote positions.

Any advice, leads, or personal experiences would be greatly appreciated!

ps: i know devops is jot an entry level role, i already worked as odoo developer nd DBA so i have a good foundation.


r/devops • • 36m ago

Discussion Who else has a "no deploys on Friday" rule, and what's the story behind it?

• Upvotes

Every team I've been on has the rule. No deploys on Friday. Nobody remembers who made it, but everyone remembers the incident that caused it.

DORA's research keeps finding that elite teams deploy on demand, multiple times a day, with change failure rates under 15%. The Friday rule was never about Friday. It's about not trusting your own pipeline.

So who broke the rule, and what happened?


r/devops • • 8h ago

Discussion Jr Software Engineer Here! Was wondering, how do you manage, monitor, and stress-test your production VPS setups (like DigitalOcean)?

0 Upvotes

Hey everyone,

I'm trying to level up my devops knowledge. Most of my projects are currently hosted on Railway, but as costs are scaling up, I am planning to migrate them over to DigitalOcean Droplets to reduce our monthly spend.

My past experience with managing bare VPS instances is pretty basic. I know how to spin up a Droplet, use Nginx to configure domains and SSL certificates, and use PM2 for process management to keep web clients and Node APIs alive. (but to be honest I'm a bit rusty now, been almost 2 years now since I last setup one during my college days)

Railway handles so much for me, moving to host my own VPS feels like a big jump in terms of responsibility. I want to make sure I am doing this right and not accidentally make prod unavailable due to a dns config lol

So If you are running production workloads on VPS instances, I would love to get your insight on a few things:

  1. Daily Checks: Do you actually log in/ssh in and check things manually every day, or do you rely entirely on automated alerts? Beyond basic Nginx and PM2 logs, what are your go-to health metrics?, Opensource options?
  2. Uptime and Alerting: What does a reliable, budget-friendly uptime monitoring stack look like when you are managing your own VPS?
  3. Triage and Debugging: When an API slows down or a Droplet spikes to 100% CPU, what is your immediate workflow to figure out what is wrong?
  4. Stress Testing: When you want to find your system's limits, do you stress-test a lower-spec instance to see where it breaks, or do you spin up an exact clone of your production spec?
  5. Lastly, spec selection. How do you know if you need more than 2gb of ram for instance? K6 testing on local before selecting? idk tbh.

Any advice, tool recommendations, or things you wish you knew when managing your first production servers would be incredibly helpful. Thanks!


r/devops • • 1d ago

Discussion I abandoned the open-source database anonymizer. Should I kill it?

21 Upvotes

Before that, a bit of story.

Our dev team was located in Ukraine, but we mostly worked on New Zealand & Australian projects. We needed a couple of hours to download staging DB dumps because the download speed was 200-400 kb/s.

So we set up a small proxy on our European server, and via cron these dumps were uploaded to that server, and we were able to download them faster.

After the team scaled, we added more protection, like only developers who worked on the project having access to these dumps; later, we integrated it into our local scripts that set up projects locally and even then added anonymization & faker to replace real data or populate it with synthetic data.

I didn't find a comprehensive solution for that, but it was a really useful tool: developers could quickly update local projects with new safe DB dumps.

It was an internal tool; it worked, but we thought maybe it made sense to create such a service: admins can easily configure rules via the UI, and developers using a small local CLI can easily download it.

We developed it. We tested. Then we abandoned it... We were an outsourcing company without sales or contribution skills.

After some time, the company fell apart, and I got permission from our CEO to deploy this tool to GitHub, and the story ended.

Yesterday I remembered this tool and thought maybe it might even be useful in the era of AI to anonymize data for AI agents. I dug a bit; there are some MCPs for that, and basically tools for anonymization, but I still didn't see tools like anonymize and share safely.

I'm wondering whether it makes sense to try to revive this repo or just kill it and forget it (it requires a security audit and fixes, updating dependencies, etc.)

The repo: https://github.com/veildb/veildb


r/devops • • 1d ago

Discussion Feels like we're about to enter the next fad skill that gets you hired

Post image
42 Upvotes

Just like coding boot camps and such before, it feels like agent boot camps are going to be the next big thing in hiring. Feels like these are going to kick off and go hard until the next inevitable downturn


r/devops • • 18h ago

Vendor / market research on-call routing/escalation in small teams, what do you actually use?

0 Upvotes

a pattern that seems really common: alerts go to a slack channel and whoever sees it first deals with it. fine during the day, but at night nobody's watching the channel.

so how do you handle it? rotation calendar? the service owner gets paged? and if the person doesn't pick up, does something escalate automatically or is it down to luck?

pagerduty and opsgenie get mentioned a lot, but per-seat pricing adds up fast for a small team. grafana oncall comes up too. anyone running something homegrown or self-hosted?

mostly interested in small team setups, not what google does


r/devops • • 1d ago

Discussion Why are we still reviewing massive PRs in alphabetical file order?

30 Upvotes

I was looking into ways to improve how teams review pull requests now that engineers are using agents to generate large multi-file changes. But I noticed an issue with how fast mental models break down when reading a standard diff. 

When an agent puts together a 20-file change, git diff lists those files alphabetically. You end up reading utility helpers or UI components before you even reach the underlying database schema or core interfaces that dictate how they behave. You would spend half the review reconstructing the call graph in your head instead of evaluating the actual logic. 

Dependency-aware reading orders (grouping files by architectural hierarchy rather than folder names) is like the correct thing to do, and it made me wonder why review interfaces still treat diffs like flat alphabetical lists.

How do you read a 20-file diff? Do you have a specific order you follow?


r/devops • • 17h ago

Discussion Why do dumb sites take whole team?

0 Upvotes

Hey,

So I'm a self-taught DevOps guy and been running a SaaS with a large scale cloud infra. I recently started looking for a job to move on and I had an interview with InBev for there BeerDraft team.

What I don't understand is BeerDraft is literally just a small eCommerce site and yet they need a team of 5 all on 100K a year so that is half a million a year to run a tiny eCommerce site that mostly likely never has more than 10 concurrent users. As a guy that does allot solo I'm left thinking wtf do they do all day?


r/devops • • 1d ago

Discussion For those of you who use open-spec, what's your method for tracking work alongside proposed changes to the spec?

6 Upvotes

I've been trying to determine a pattern where proposed changes get made to the open-spec config, and a PR is reviewed, approved, and merged with those proposed changes. At that point then, team-members can address those proposed features. But it's a little cumbersome. Just looking for the most elegant way to do this if that's possible.


r/devops • • 18h ago

DevOps is cooked.

Thumbnail
0 Upvotes

r/devops • • 3d ago

Troubleshooting That moment when you realize you're in production

Post image
1.6k Upvotes

No errors. No warnings. Just 8.3 million reasons to update your résumé.


r/devops • • 2d ago

Discussion Anyone noticed AI PRs that ignore infrastructure and DevOps realities?

106 Upvotes

I work with some very talented developers (I'm also a full-stack developer in addition to handling DevOps), but since the rise of AI, I've noticed more pull requests that include pipeline changes or introduce new services that require configuration outside of the codebase.

I have safeguards in place that automatically add me as a reviewer whenever these areas are affected. As a result, I end up rejecting a significant number of these PRs because many of the proposed changes simply won't work in our environment and this has been stirring up drama with people asking “who made you king”….

The code itself may look reasonable and it works on the devs pc (actually sometimes not between devs since AI might have added some packages or services to original devs pc), but the developers overlooks the infrastructure, deployment, networking, security, and operational requirements needed to make those changes work correctly in production.

What surprises me most is that some of them genuinely don't seem to understand why it won't work simply because it worked on their PC. They fail to realize that a production environment involves far more than just writing code that runs locally.
sometimes they give me some copy-pasted answers that it will work if you just do X.

Is anyone else experiencing this or is it just me?


r/devops • • 1d ago

Vendor / market research Any good arguments for getting AI agents approved by security?

0 Upvotes

Working with a team where engineers really want to use Claude Code on servers, not just for local dev. Security and audit keep blocking it.
I get their concerns. Who approved the command, how do you prove what it did, that kind of thing. But the push from the engineers isn’t going away.
Has anyone found an argument or approach that actually got security on board?
Full disclosure, I work in this space so it’s not purely random curiosity. But I’m not here to pitch anything. I just want to know how teams usually solve this in practice.


r/devops • • 2d ago

Ops / Incidents am i in the right job!?

29 Upvotes

its morning & our Databricks jobs were all red. Everything from last night had failed. Started digging through the logs with my colleague, found a bunch of Azure WebJobs had failed too, dug further and it turned out our integrated ERP had been rate limiting us.

Next thing I know I’m in one of famous calls, 10 people firefighting, the US team woke up early to check, we got more, someone got dragged back from vacation because he’d approved a weird PR, and everyone’s jumping between App Insights, running queries and dropping bits of info in chat trying to figure out wtf is going on. Boss is mad, management is asking for resolve time, I didn’t get lunch, and I was at my desk from 9 to 4 until we finally got everything resolved.

It was chaos and stressful, but damn, the dopamine! It was actually fun. I dont want this everyday, but now and then it makes you feel alive

Anyone else enjoy incidents a little?

btw it was again our damn event stream from outlook integration! It flooded the pipeline, everything downstream piled onto the ERP, and it rate limited. yeah we are going to put exponential backoff with retries and a circuit breaker and other stuff tmr, or next week hopefully


r/devops • • 2d ago

Ops / Incidents Title: What does your deployment actually verify?

0 Upvotes

Disclosure: I work on the platform this happened on. No links and nothing to sell, I just don't have a good answer to the question.

We deployed an app last week, and every signal said it had worked. The deploy was marked as running, the health check was green, and every tool in the pipeline reported success.

Except the app was incomplete. Eight of its 27 starter records had never been created. The API rejected them one by one with permission errors while accepting the records around them.

The setup step actually caught this. It printed a warning saying the app was incomplete and shouldn't be treated as a successful deployment. Then it exited with code zero, so everything downstream happily marked the deploy as successful.

We had a health check proving the process was alive, but nothing proving that the application matched what we thought we had deployed.

I have been trying to work out what the right check should be, and none of the obvious answers feels complete.

A health endpoint returning 200 only tells you the process is running and the port is open. It says nothing about whether the expected data or configuration exists.

A smoke test against a few routes is better, but it only covers whatever somebody remembered to include. In this case, the missing records weren't part of that test.

Comparing the declared state with the actual state feels like the right answer. But that requires the deployment to produce a manifest of everything it intended to create, and most deployment systems don't seem to do that.

You could also fail on every warning, but then harmless warnings become blockers, people add skip flags, and eventually you are back where you started.

The two cases I am still stuck on are partial failures, where each individual rejection might be valid but the overall pattern is clearly wrong, and tools that print serious warnings but still exit zero. That convention isn't going away anytime soon.

And before anyone says readiness probes: readiness tells you whether the application can serve traffic. It doesn’t tell you whether it's the application you intended to deploy.

What does your deployment actually verify beyond "the process is running"? And what has shipped half-finished before you noticed?


r/devops • • 2d ago

Discussion How to manage access for 50 EKS cluster with AWS IAM?

0 Upvotes

When you have 50 EKS clusters, how do you manage and configure which developers have access to specific clusters using RBAC and AWS IAM? Isn’t it too much to manually configure IAM with k8 mappings on every single cluster?


r/devops • • 3d ago

Discussion Is Trunk Based Development possible if an integration test takes too long to run?

11 Upvotes

I'm proposing my team move from long-lived dev/stg/prd branches to trunk-based development, with PRs straight to main.

I'm fairly new to this, so forgive me if I'm missing something obvious.

The sticking point is testing before merge.

Our current integration test runs the entire DAG end-to-end. This is fine as a single check before merging our test branch (which will contain all commits from developers over the sprint) to main. But it's slow and costly if it were to run as part of every PR. Today we run it manually, check the outputs, then merge.

I am wondering if it’s bad practice to leave this integration test until the end of the sprint. As in, developers merge to main (with some automated checks running as part of each PR before merge), and then when are team want to deploy the sprint changes, first run a single integration test before creating a new release tag In GitHub.

This does mean that main can’t be considered prod ready at all times.

Edit: we’d be using release tags, so for rollbacks we’d rollback to last stable tag. For hotfixes, we would cut from last release, the merge changes to main and create a new release with target set to hotfix branch.


r/devops • • 3d ago

Discussion Do you use AWS Systems Manager?

15 Upvotes

And how do you utilize it?


r/devops • • 2d ago

Discussion GitHub is making CI/CD workflows more predictable with workflow dependency lockfiles…

0 Upvotes

…but securing your pipeline is only half the battle.
What happens between a successful build and production?

Who reviewed the changes? Who approved the release? Can you trace exactly what shipped? And if something breaks, can you roll back quickly?

As teams adopt AI coding agents and ship code faster, controlling the release process becomes just as important as writing the code itself.


r/devops • • 4d ago

Discussion If you're into Cybersecurity, Cloud, DevOps, or Linux, let's connect and learn together

81 Upvotes

I'm a Cybersecurity undergrad with a strong interest in Cloud and DevOps. Honestly, it's hard to find likeminded people around me who are equally passionate about tech. I'm looking for people who want to learn together, build projects, participate in online hackathons, and explore new technologies. If you're into Cloud, DevOps, Linux, or Cybersecurity, let's connect and grow together


r/devops • • 3d ago

Discussion syllabus for a undergrad devops course

8 Upvotes

I was invited to teach MLOps and DevOps on a local university next semester and I'm trying to plan the course. Which topics would you guys like to learn if you enrolled on a DevOps/Platform Engineering course today?

The university is a big deal on the regional scene but not known globally.

EDIT: This college never had a formal DevOps course, students focus on CyberSec (college board includes big names from that industry) and AI (college board wants to look cool). They are never formally taught containerization, CI/CD, monitoring, infrastructure as code, they are expected to learn that by osmosis.

They had a single experience teaching MLOps, but it was so bad that the teacher was dismissed and the staff won't mention that course.


r/devops • • 4d ago

Discussion Meme from Russian DevOps channel (autrotranslated to English) AKA "List of DevOps interview questions for October 2026"

460 Upvotes
  • Do you have experience literally extinguishing a production environment?
  • What is the difference between Disaster Recovery and Damage Assessment?
  • What is the RTO for a data center after a direct drone hit?
  • Can you debug infrastructure using satellite imagery?
  • What would you do if your Kubernetes cluster suddenly became "bare metal" in the literal sense?
  • How do you configure HPA so that pods can outrun drones?
  • What would you do if not just a node, but the entire data center went down?
  • Explain the difference between a Blue-Green Deployment and a "Red" air raid alert.
  • How do you interpret the term "hot standby" if the backup data center is still on fire?
  • What Prometheus alert would you set up for smoke coming from the server room?
  • Have you ever migrated infrastructure using kubectl evacuate?
  • What do you do if terraform destroy was executed without Terraform?
  • Can a drone strike on the server room be considered a successful Chaos Engineering experiment?
  • How many nines of availability remain for the service after losing two data centers?
  • How do you distinguish a DDoS attack from an attack that renders a DDoS attack unnecessary?
  • What do you do if your disaster recovery plan begins with the words "find a surviving server"?
  • How do you explain to management that an Availability Zone is no longer available?
  • Which metric do you choose for your SLO: uptime, MTTR, or the number of surviving racks?
  • How do you ensure geo-redundancy when geography itself has become the primary risk factor?
  • What do you do if you have an Active-Active setup, but both sites have switched to an Inactive-Inactive state?
  • How do you configure fencing in a cluster when the fencing has already occurred physically?
  • What do you do if your Witness in the third data center is the first to stop showing signs of life?
  • Can a storage system be considered fault-tolerant if the data outlived the hardware, but the hardware didn't survive its operational lifespan?
  • How do you calculate the "blast radius" when it’s no longer just a metaphor from the documentation?
  • What is the point of synchronous replication if both data centers cease to exist synchronously?
  • How do you implement automatic failover when the decision to switch over is made by the Ministry of Emergency Situations?
  • Discuss what status should you set in Grafana: CRITICAL, DOWN, and NO LONGER EXISTS?
  • How do you convince the CFO to approve a third data center when the first two have already been written off as CAPEX?
  • What backup retention period do you recommend when the main issue is the lifespan of the storage hardware itself?
  • How do you achieve "five nines" of availability when the only nines in the project are the digits in the recovery cost?

r/devops • • 3d ago

Discussion How are you giving AI agents access to AWS in production?

0 Upvotes

I run multi-account AWS (Control Tower, Identity Center) and I'm trying to understand how other teams handle agents that need to do more than read.
After the PocketOS and Replit incidents, our default is read-only roles plus changes through PRs. That works, but it breaks down the moment an agent needs to apply a plan or fix an incident at 3am.
Curious what you do today: read-only only? Long-lived keys with tight policies? Some approval step? What has actually bitten you?
I'm exploring short-lived credentials issued per approved action (plan shown in Slack, a key valid 15 minutes for that plan only). Happy to be told it's a bad idea.


r/devops • • 3d ago

Troubleshooting GitHub Actions self-hosted runner gets stuck downloading Java 17 with setup-java — curl: (18) Transferred a partial file

6 Upvotes

I'm learning GitHub Actions and have set up a self-hosted runner inside an Ubuntu Docker container on my Windows machine.

My workflow uses actions/setup-java to install Java 17:

- name: Set up JDK 17
  uses: actions/setup-java@v4
  with:
    java-version: '17'
    distribution: 'temurin'
    cache: maven

However, the job gets stuck at the Set up JDK 17 step. The logs stop at:

Resolved latest version as 17.0.20+101
JDK cache is not found for Temurin-Hotspot 17.0.20+101
Trying to download...
Downloading Java 17.0.20+101 (Temurin-Hotspot) ...

I tested the download URL manually from inside the container using curl. The download repeatedly stops before completion and returns:

curl: (18) Transferred a partial file

I also tried resuming the download with curl -C - and retry options, but it failed again. A basic connectivity test to https://github.com returns HTTP 200.

The runner itself appears to be connected and running jobs correctly. I haven't yet confirmed whether the issue is related to the network, the GitHub release download redirect, a proxy, or something else.

My questions:

  1. What could cause curl: (18) Transferred a partial file to occur repeatedly when downloading a large release asset?
  2. How can I identify whether the problem is caused by my network, a redirect destination, a proxy, or the download server?
  3. Is there a recommended way to configure actions/setup-java on a Docker-based self-hosted runner to avoid this issue?

I'd appreciate any troubleshooting steps or suggestions. Thanks!


r/devops • • 4d ago

Tools soft recommendation to visualize Service/Dependency Map

5 Upvotes

I'm looking for something similar to Datadog's "Service Map", but to include external services as well and without dynamic health statuses. I think my top requirements are:

  1. clear visualization for design reviews and planning changes, also for onboarding of new hires to provide a simple overview and context.

  2. text-based source to track changes in git.

  3. be able to annotate services to make the map more interactive (this is why Figma is not enough anymore ;)

Right now I need to scope from 50 to 100 services which I still believe can be nicely presented. I'll add an example to the comments.

At the current scale I can't justify any "enterprise" soft, but fine to pay for the tool as well. Also I'm not a big fan of automated discovery that requires installing agents in our network to collect and map services.

Any ideas or recommendations on what to try?