DevOps and other issues by Yurii Rochniak (@grem1in) - SRE @ Preply && Maksym Vlasov (@MaxymVlasov) - Engineer @ Star. Opinions on our own. We do not post ads including event announcements. Please, do not bother us with such requests!
You probably heard of Figma's AWS bill already. If not, tldr is that they've listed for IPO, which means that they had to disclose their financial information. One of the findings there was that Figma spends roughly $300k a day on AWS, and many people freaked out.
In this article, Corey Quinn puts these spendings into perspective: this astonishing sum is just about 12% of their rolling revenue or $0.7 per active user, which is not that bad.
It's a short, funny read, since Corey has his own style of presenting his opinions:
HackerNews commenters claimed they could cut Figma’s costs by “at least 30%, often more than half.” Sure, Steven; that seems credible. I’m certain your experience running a Minecraft server uniquely qualifies you to architect infrastructure for 95% of Fortune 500 companies.
Continuing the topic of AI: CTOs Reveal How AI Changed Software Developer Hiring in 2025 is a collection of testimonies from C-levels and technical leaders on what they pay attention to when hiring people in the AI-age.
There are many interesting and important points there. It's true, that writing code was never the biggest issue. Understanding what you're doing or what the code produced by AI does is crucial today. Especially, when you're hiring people. I don't know about you, but we already have a couple of cases, when candidates were seemingly using an AI assistant at interviews.
One interesting take away is the evolution of the coding challenge. It's not stated in this article explicitly, but we could all agree that take-home assignments are dead. However, some respondents are using code reviews as a challenge: they ask candidates to review the AI-generated code for bugs and possible improvements. This is a very interesting approach that could reveal, how people think about a problem.
So yeah, these two quotes summarize this article quite good:
everyone's debating whether AI will replace developers, the people actually hiring them are looking for the opposite of what you'd expect.
They don't want prompt engineers or AI evangelists. They want the developers who can clean up the mess AI creates.
you're worried about AI taking your job, stop.
My wife is helping to raise funds for the repair shop for the "Omega Wings" unit. They have a unit that takes care of ground drones, and while there are the drones themselves, those require routine maintenance and occasional repairs.
So, these funds are aimed for the equipment for their repair shop: both tools and materials.
The main goal is 300k UAH, this helper jar is 20k.
https://send.monobank.ua/jar/5XD1eci1Ac
I would greatly appreciate your help! 2/3 of the helper jar is already there, so we can easily close it today!
#donations #Monday
A new issue of the CatOps Newsletter is out!
https://newsletter.catops.dev/p/catops-digest-2025-07-03
#digest #newsletter
For today’s Donations Monday, I’d like to remind you about the fundraiser that DOU has started for the 3rd Assault Brigade.
They are raising 20M UAH (~€410k) for reconnaissance drones for the 3rd Assault Brigade.
Here's the direct link to the Monobank Jar:
https://send.monobank.ua/jar/AGK8qiQwQX
There is also a raffle for donations of more than 300UAH. If you know to know more about the raffle, as well as about the fundraiser itself, check out the dedicated page (information is in Ukrainian):
https://dou.ua/triyka/
#donations #Ukraine
How We Load Test Argo CD at Scale: 1,000 vClusters with GitOps on Kubernetes.
An interesting benchmark of ArgoCD. While the setup is somewhat too specific (vCluster), this benchmark provides some insights into the limitations of ArgoCD at scale. There are also manifests available, so you can reproduce the experiment (using some YAML generation with Bash :D)
#kubernetes #gitops #argocd
Denys Vasyliev shares the challenges AI brings to contemporary SRE practices.
Not in terms of some autogenerated code breaking production, although that also happens, but in terms of how can we access the reliability of AI interfaces, and what the word “reliability” even means in the age of AI.
Also, make sure to subscribe to his Telegram channel (in Ukrainian), if you haven’t already.
#ai #sre
A quick explainer for the caching strategies. It's concise, but it would be useful, if you're preparing for a system design interview, or thinking of implementing caching in your app.
#system_design
I'm experimenting with the day of digest. So, here's the new issue:
https://newsletter.catops.dev/p/catops-digest-2025-06-17
#digest #newsletter
Who would win: a null pointer crash loop or a multi-billion dollar corporation?
And here we have a postmortem of the recent Google’s outage.
P.S. Kudos to Google for releasing this postmortem so quickly!
#postmortem
This article is quite old, but it's interesting nonetheless, since it describes an approach rather than a specific technology.
Moreover, it describes a phenomenon that was identified long time ago. However, here Slack shows how they used it to adopt (or discard) software within the company. Sure, such an approach would work better in larger organizations, but it's still interesting to read about.
#culture
To analyze the data, one has to collect it first. So, I'd like to invite you to participate in two ongoing surveys:
- 2025 Stack Overflow Developer Survey - an annual survey from a very important (albeit not so popular anymore) engineering resource (in English).
- DOU Salary survey - an annual survey of the Ukrainian community (in Ukrainian).
#random
All talks today are about AI: models, agents, RAGs, MCPs, editors, etc.
In this article, Arseniy Zinchenko explains what is an MCP (model context protocol) with an example.
And in the follow-up article, he expands the example by writing a basic MCP for Victoria Logs.
BTW, if you're still not subscribed to his Substack, make sure to subscribe! Arseniy posts some great technical content there and makes it quite regularly.
#ai
Let’s close the last week’s fundraiser today for good! There’s not that much left.
A friend of mine's recon team is getting a Shark complex, but they need a trailer to move it!
This powerful UAV needs a two-axle trailer for transport. Let's help them get it.
Donate to get us closer to giving them the mobility they need:
- Monobank jar: https://send.monobank.ua/jar/9hNbCnoiN1
- Card: 4441 1111 2429 2776
#donations #Ukraine
I think, I first encountered this tool in Den Vasyliev's channel. Kubeshark - a network observability tool for Kubernetes.
Network observability comes handy at times. So, here are some other tools and articles one can use to capture packets in your sustem.
- ksniff - a Kubectl plugin to capture traffic
- Hubble - an observability tool for Cilium
- How to use debug containers to capture the traffic - basically running tcpdump
inside a pod
- A hands on lab on how to run tcpdump
in a pod
Happy capturing!
#kubernetes #networking
LinkedIn is at it again!
Apparently, the Kafka's creators have overgrown its capabilities, so they decided to create a replacement called Northguard.
Northguard uses different protocol, replicates the data in the different way, and has different client-side. So, it's not an evolution of Kafka - it's its replacement.
Still, Northguard is not open source yet, so we do not have all the information to evaluate it - only some rumors from within LinkedIn. They do plan to open source it eventually, though.
Would it replace Kafka outside LinkedIn as well? I doubt it, Kafka (and more importantly its protocol) has too wide adoption to be thrown away that easily. But still, if we, as humanity, never reinvented the wheel, we would still travel in wooden carts of something...
#data #kafka
A small article about software development in the time of AI -
Writing Code Was Never The Bottleneck.
As the name suggests, this article is about the fact that although many vendors try to sell their AI coding tools as a "replacement for the developers", the blockers that those tools remove were never the biggest ones.
There are companies that understand that and encourage their employees to do "assisted" development, which in turn can yield great results, if people are not afraid of being fired. What a surprise...
#ai #programming
Security researches from WIZ have a fun capture the flag challenge for you!
It has monthly challenges and a leaderboard. Feel free to participate!
#security
Under daily routine, it's easy sometimes to forget, why we've gotten into the industry in the first place.
This is where pet-projects come in handy. Also, they are great for learning! Unfortunately, the "pet-project" term got a commercialized a bit. However, this article gave me a new term - "toy software"! I'm gonna use it from now on.
There are a few examples of such toy software that the author wrote through the years. Almost all those examples look super-complex to me. However, one need to understand that each of us has different expertise, and thus different things look easy or hard to each of us. I can easily think of a couple of examples of such platform-related toy software one could build. For example, Cost Exporter really took maybe a week, if I don't account for all the procrastination time.
#programming
Ktea - pronounced "K-tea" - is a TUI client for Kafka that allows you to see the topics and consumer groups, produce and consume messages, etc.
It's a bit rough and you may need to configure the Meta
button for your terminal (can be a bit of a PITA on MacOS), but otherwise it may be useful!
Another way of working with Kafka in more or less visual way is kafka-ui from Provectus. It's a web GUI, but it can be spun up locally in Docker.
#kafka
My friend is raising 300k UAH for a pickup truck for the 28th brigade.
Your help is greatly appreciated!
https://send.monobank.ua/jar/6dbYCchqSh
#donations #Ukraine
Time to update your passwords, or at least check if you've been pwned. Also, make sure to have MFA configured for your important accouts.
16 billion passwords exposed in record-breaking data breach, opening
access to Facebook, Google, Apple, and any other service imaginable
#security
Apple has apparently realized that the big chunk of their customers are tech companies.
So, we are closer to get a native Linux container implementation, or rather the official implementation, in macOS 26 (Tahoe).
An interesting thing about this implementation is that instead of spinning a single big VM for all the containers, which is pretty much what all the existing implementations do; Apple chose to have micro-VMs for each container. In any case, I don't think this implementation detail would impact the UX a lot.
#apple #containers
For today's Donations Monday, I want to share with you a fundraiser by DOU.ua.
They are raising 20M UAH (~€42k) for reconnaissance drones for the 3rd Assault Brigade.
Here's the direct link to the Monobank Jar:
https://send.monobank.ua/jar/AGK8qiQwQX
There is also a raffle for donations of more than 300UAH. If you know to know more about the raffle, as well as about the fundraiser itself, check out the dedicated page (information is in Ukrainian):
https://dou.ua/triyka/
#donations #Ukraine
Yet another bundle of programming books on HumbleBundle. This one is from No Starch Press, who usually have good books.
#books
Figma runs in Kubernetes. How can I be sure? By reading their blog post How we migrated onto K8s in less than 12 months.
This blog post doesn't dive deep into technical details, but it provides a glimpse of what technologies are used by Figma to manage their infrastructure.
What I liked about this article is that they have "in less than 12 months" right in the title! I think, more articles should provide realistic timelines, especially when talking about production systems under load. "Kubernetes up & running in 30 minutes" have its own merit, but not in prod.
#kubernetes
A very nice overview (with examples) of Wardley maps & Pace Layering - methods that can help you building a technical strategy at your company, and reason about planning and budgeting.
#leadership #strategy
A bundle of book bundles for you today. There were a couple of bundles released recently, so I just grouped them together.
1. ML/AI books by O'Reilly
2. Learn to program by Pearson
3. Cybersecurity and forensics by Pearson
Just keep in mind that often Humble Bundle shares reoccurring bundles. So, always check your library before the purchase :)
#books #ml #ai #security #programming
A new issue of the CatOps Newsletter is here!
https://newsletter.catops.dev/p/catops-digest-2025-06-01
#newsletter #digest
On Describing Not Explaining is a neat life-story that unveils a way of reasoning about incident investigations.
The gist is that instead of guessing what could possibly happen (an instinctive approach), you try to describe what exactly happened and in what order. Just saying this out loud can help you to cut off many unlikely causes, and also may help you to remember some less obvious recent changes.
#sre #incidents