DevOps and other issues by Yurii Rochniak (@grem1in) - SRE @ Preply && Maksym Vlasov (@MaxymVlasov) - Engineer @ Star. Opinions on our own. We do not post ads including event announcements. Please, do not bother us with such requests!
A sneak peek into the database design based on two incredible books: "Database Internals" by Alex Petrov and "Designing Data-Intensive Applications" by Martin Kleppmann. Mostly on the first one, though.
This article touches topics of ACID, underlaying data structures (B- & LSM-trees), and distributed systems. Just like the aforementioned books, which I also highly recommend!
A random quote:
> Going distributed should be a last mile resort, introducing it to a system adds a ton of complexity, as we will soon learn. Please avoid using distributed systems when non distributed solutions suffice.
#databases
Python is incredibly popular programming language among DevOps / Site Reliability / Platform specialists. It’s also used as the language of choice to build backend in many companies.
So, here is a book bundle by Packt to enhance your Python skills:
https://www.humblebundle.com/books/python-from-beginner-to-advanced-packt-books
P.S. I know that many folks are skeptical about Packt books, so keep this in mind. I still think it worth sharing, though.
There are two types of folks: those who pin their dependencies to (often) mutable tags and other pointers, and those who already pin to hash sums.
This is true for GitHub Actions as well. For example, most tags in the tj-actions/changed-files repository were repointed to compromised versions by an attacker, potentially affecting 23,000 repositories until GitHub shut down the entire repository.
Manually pinning dependencies can be tedious, so it's better to utilize tools like Renovate. The SpotOnInc/renovate-config preset does exactly that. Note that you can pin it to a hash sum too, just in case ;)
P.S. If you used that GitHub Action, here is a list of possible mitigation steps: antonbabenko/pre-commit-terraform#837.
#security #github #gha
Git is one of those things, where you can discover new things even after decades of using it.
This article provides some tricks you could do with Git. If you knew those already - that's cool. If not, you're welcome :)
#git
I do not have small fundraisers for you today, so let's donate to Serhii Sternenko for FPV drones:
https://send.monobank.ua/jar/2JbpBYkhMv
#donations #Ukraine
A book bundle on various computer science topics by No Starch Press.
https://www.humblebundle.com/books/computer-science-fun-way-no-starch-books
#books #bundle
One of the interesting questions when it comes to SLOs is how to define them, in other words, how to set up boundaries to make the SLOs meaningful.
This article by Honeycomb provides some ideas on how to organize your SLOs, so on one hand they represent the user's needs, but on another hand do not cover things that your team cannot control.
#observability #slo #sre
Let’s help Seva to close this fundraising round for a night vision device!
We could close it right away, if only every subscriber of this channel sent ₴6 there.
Monobank jar:
https://send.monobank.ua/jar/vxr5ywPjM
#donations #Ukraine
For my DevEx specialists out there: What's Next in Measuring Dev Productivity?.
tl;dr: this is kinda a very basic description, but at least it has a table with metrics and some justification behind them. This is why I decided to share it.
#culture #devex
What is common between Helm and the article about the abuse of the curl | bash
setup to load bad payload?
Year of initial release. But if the article is not more accessible outside the Web Archive, Helm still provides that as a valid installation option.
Anyway, the article is old but gold, worth reading to understand how tricky attakers can be.
#security #bash
CatOps Digest 2024-02-23 is here!
https://newsletter.catops.dev/p/catops-digest-2024-02-23
#digest #newsletter
How hard could it be to write a calculator app?
This article tells the story of the Android’s calculator app and shows that building a calculator is not as trivial as it may seem.
Honestly, I love articles like this one! They truly remind me why I like engineering.
#programming
Have you ever needed to check what's going on with an OSS project, who uses it, and so on, or compare two of them?
OSSInsight.io can do both!
I'm still not sure how it should help you compare two tools like Terraform vs OpenTofu and it couldn't help me choose between a 4-year-old dead project and its feature-rich, well-supported fork (which is now a standalone repo) - check this.
But it is definitely worth getting insights about repos - just look at these wonderful stats! I've been maintainer of this repo for a few years and I love to check different stats from time to time, but I didn't know about half of these insights until today - that's how good it is.
OSSInsight provides statistics on:
- Who starred the repository, open PRs and issues, their locations, and companies (when available)
- Basic GitHub stats like commit and push history, lines of code changed, presented in a simpler view
- Issue first response time
- Time taken for pull requests from submission to merging
- Issue and pull request histories, including PR sizes
- Overall repository stats for the last 28 days
- Detection of "star fraud" by analyzing whether stars grew organically or had sudden spikes (sometimes caused by viral posts, but often indicating fake stars)
P.S. Thanks Valerii Tatarin for sharing OSSInsight with us. If you'd like to share something with community too, feel free rich @MaxymVlasov or @grem1in.
#opensource
I wrote an article about working with `.terraform.lock.hcl` two years ago and had mostly forgotten about it. However, DOU recently reminded me by publishing a translation of it into Ukrainian. Interestingly, the content is still relevant today. The AWS provider (and many others) can still break your tfstate
, as seen with the S3 lifecycle rules issue in v5.86.0.
For this specific edge case, you need integration tests via Terratest. However, there a good practice to pin your dependencies and test all updates, even with a simple tf plan
. Unfortunately, this practice is still not widely adopted.
#terraform #hashicorp
For today’s donations Monday I would like to share with you a standing jar of a friend of mine who serves in the 130th battalion of the Territorial Defense.
This money is used for car repairs 🛻
🎯 The goal is 100 000.00 ₴, but it’s a standing jar.
🔗Monobank Jar
https://send.monobank.ua/jar/A7HftuxgZx
💳 Card number
5375 4112 2037 9825
#donations #Ukraine
Today, I'd like to share with you a nice YouTube channel Polylog, that does short explainer videos about the computer science concepts.
PolylogCS" rel="nofollow">https://www.youtube.com/@PolylogCS
#programming #cs #youtube
The highest priority after completing the task is to preserve the life and health of the personnel. So, let's help raise money for tactical medicine!
TL;DR: Donate to Monobank jar till EOW and I will double the amount raised[1].
The Hospitallers Battalion is a volunteer formation known for its professionalism and dedication. When I say "voluntarily," I mean that you can join for a rotation, for example, for 2 weeks, and then return to civilian life for a few months before repeating the process. This flexibility allows specialists who, for various reasons, do not want or cannot join AFU still contribute to defensive operations. However, since the Hospitallers are a volunteer formation, their funding relies on donations from ordinary people.
Currently, the Hospitallers are raising 7.5 million UAH (~$180k) for this spring.
Let's aim to raise at least 10k UAH from CatOps by the end of the week, and I will double the amount raised[1].
Monobank jar: https://send.monobank.ua/jar/6eEHjgDTGq
Btw, the Hospitallers offer courses on first aid and tactical medicine that are available to civilians. I recently took the TCCC ASM (3-day) course, and it was excellent. You can check out their courses at Hospitallers Courses site and track new dates on their Telegram channel.
#donations #Ukraine
[1] But not more than 20k UAH/month from me until the fund campaign is closed or until I have doubled the amount, whichever comes first.
Yet another article from the old archives.
This one is about building a multicluster setup with Kubernetes using Kubernetes tools only. Moreover, there are more tools these days that can help you accomplish the architecture outlined in this article.
For example, you could use ClusterAPI instead of Crossplane for leaf-clusters and so on.
The only sad thing is that Kubernetes doesn’t support federation. So, any viable multi-cluster setup always boils down to two distinct approaches: a bunch of independent clusters, or a single leader with leaves.
#kubernetes
How to Scale Elasticsearch to Solve Your Scalability Issues is a neat guide for areas where you could optimize your ElasticSearch clusters.
Ties article doesn’t provide any concrete numbers, but gives plenty of suggestions. It makes sense, since any optimization heavily depends on one’s use case.
#elasticsearch
The first Spring issue of the newsletter!
https://newsletter.catops.dev/p/catops-digest-2025-03-09
#digest #newsletter
Another article from the old stockpiles, and again it's from Julia Evans.
This one is about the ways DNS can break.
It's been a couple of years since this article was published, so some things may be irrelevant today. For example, the issue with DNS over TCP was fixed in musl
to my knowledge.
In any case though, this is a nice article that deserves your attention.
#dns #networking
How Do Websockets Work is a short explainer of websockets by the System Design Newsletter.
BTW, that is a good newsletter, I can totally recommend!
#web
From time to time I’m revisiting old articles that I have saved somewhere or those that just laying around, etc.
Today, I’d like to share with you two articles by Julia Evans on how to learn new skills, which is a skill on its own.
The second article is actually a presentation from the Strangeloop conference on how to reason about things and make hard or confusing things easier to understand.
#education
Terraform Patterns, Observed. Part 5: Logic in Terraform is a nice article with observations of how people incorporate logic in their Terraform code, even though TF’s logic is somewhat limited.
BTW, as you can see, this is a part 5. Robert has other articles with Terraform observations on his Medium. I highly recommend checking them out as well!
P.S. This article is brought to you (and me) by weekly.tf - a newsletter about the Terraform ecosystem by Anton Babenko. This is a great place to keep up-to-date with Terraform, even if you don’t work with it a lot.
#terraform
On Saturday, I gave a talk at the FW Days DevOps conference. I will share the videos with you once they are available.
During the conference, they were raising money for a night vision device PVS-14 for the 4th battalion of the 46th separate brigade.
The goal is to raise 220k UAH, and almost 150k was raised during the conference. Let's help them to close this goal!
Monobank Jar: https://send.monobank.ua/jar/vxr5ywPjM
#donations #Ukraine
Here are two somewhat related articles. Related, because they come from the same author and touch the same topic of the incident management. The articles are not in order and they highlight different aspect of how we deal with incidents.
- The danger of overreaction is about how good intentions of preventing incidents (usually taken after a high-impact incident occurs) can lead to worse results. In German language there are words "die Verschlimmbesserng" and "zu verschlimmbessern" respectively, that mean exactly that: make something worse by trying to make it better.
- You’re missing your near misses about the importance of tracking situations when incidents nearly happen. This intertwines a bit with an article I shared earlier. Yet, this one is less science and more common wisdom.
#sre
The Theory Behind Understanding Failure is a great article that provides a viewpoint that any system has more than just a technical aspect of it, as well as any failure (incident) is not a single isolated event. It uses some known and less known examples to illustrate it.
I also appreciated that it has references to the scientific works behind it. So, you can dive deeper, if you want to. The author did, however, misinterpret Nietzsche, but everybody does, so it's Ok.
#culture #theory
I tried a hat of ChatGPT and summarized some talks from FOSDEM and CfgMgmt Camp for you.
Here are my notes:
- On Substack
- In my blog
P.S. it’s the same article in both places.
#event #fosdem
A great article about PostgreSQL monitoring by Setevoy.
In this article, he takes a deep dive into what can cause "slow queries" on a PostgreSQL RDS, what metric to check, and where to find those metrics when you're using AWS RDS, so the observability is a bit limited.
Also, make sure to subscribe to his Substack - he has a lot of great in-depth articles, and by subscribing, you will get them right away. Besides, make sure to subscribe to his Telegram channel (in Ukrainian). You can get updates there as well, if you prefer Telegram to email.
#databases #postgresq #aws
The International AI Safety Report is the world’s first comprehensive synthesis of current literature of the risks and capabilities of advanced AI systems. Chaired by Turing-award winning computer scientist, Yoshua Bengio, it is the culmination of work by 100 AI experts to advance a shared international understanding of the risks of advanced Artificial Intelligence (AI).
https://assets.publishing.service.gov.uk/media/679a0c48a77d250007d313ee/International_AI_Safety_Report_2025_accessible_f.pdf