Apoligies, this month’s update was quite delayed. March was not an easy time for me, between travel and other engagements. I was also pre-occupied with interviews and I couldn’t find the right amount of time to write extensively.

So what did I get done in March ?

I finally got the time to start working on porting Nsync to Rust !

Thanks to Krishna Vishal, I started working on porting the Nsync library to Rust. This would be my first BIG Rust project. The going is not easy as there is so many things one has to learn about Rust’s ownership system especially when dealing with building fundamental concurrency primitives while also making the code safe for consumers. There might be easier, simpler projects for improving my Rust skills, but hey, YOLO right ? Besides, it feels nice to fail gloriously, than to be humbled by a simple webdev project.

Speaking of webdev, I shifted gears to temporarily work with the Kubernetes team to offer Hosted Kubernetes solutions to other teams in the company. This is quite a jump from the product engineering work I was doing previously and the job has been having me trying hard to navigate the (in)famous CNFC landscape of tools, plugins and whatnot ?

What motived this change of teams and engineering track ?

Burnout and getting the ick

Honestly, I got burnt-out out of web dev. Building APIs, figuring out SQL queries and writing business logic feel stale, boring and tiring. I have been working on webdev since my first full time job in 2015 and now that it has been 10 years, I want to try something different. The other reason is that the previous project I was working wasn’t gaining any traction and the lack of user feedback, features that went nowhere, having no interesting demos to show my fellow team members or org colleagues was having a serious negative impact on my self-esteem and my career progress. I felt a failure, of having no useful skills whatsoever and was afraid of losing my job and not being able to find a new one. Was I overthinking it ? Perhaps. I did get an excellent rating in my previous performance cycle, but I still felt like I was not doing enough. Also mindlessly doing what I was doing before wouldn’t have set me on a path to get a promo and move up to the next level. As important it is to have skills, learning how to choose good projects (or being lucky enough to land one) is equally important in the long arc of a having a good career.

I decided that it was time to branch out and try something different. My first impulsive act, as I had done many times previously, was to try interview at other companies (if you do have an interesting position for me, please do reach out to me), but the market being what it is in 2025, the bar is both extremely high and the opportunities too few. In the meanwhile, a casual chat with an engineering manager from within the org turned out to be a new opportunity: the manager was looking for engineers to join the Kubernetes platform and having heard good thing about me from my manager, decided to offer me a chance to work with the team. Having been on the user side of Kubernetes before and constantly cursing it for its complexity, almost outright dismissing the need for Kubernetes (why not just run everything on an EC2 VM or a bare metal server ?), I took this opportunity to see what benefits Kubernetes can offer to engineering teams.

Put yourself in the perspective of a Platform Engineer: there are so many concerns that must be addressed: are teams using secure containers. What about logs ? Is a team sending too much of it causing us to burn through our quotas and end up overpaying on top of already expensive contracts ? What about networking ? One of the security teams had flagged an issue with the DNS setup for a team that allowed DNS takeovers of endpoints owned by the company. This is a dangerous situation, as hackers can hijack authentication information by simply crafting links to subdomains in their control. Access controls ? Are teams using correct RBAC for their resources ? Are secrets being rotated on a periodic basis ? What about resource usage ? The Devops model of Software Development puts a heavy load on individual engineering teams to configure and rightsize their clusters. One of the common patterns that I have noticed in multiple companies is that very few teams have the right mix of experience and talent to skillfully do both product development and manage infrastructure. The number of experienced infrastructure engineers are far and few. As a result, it makes sense for large companies to build platforms on top of solutions like AKS or GKE to reduce the burden and provide a more simplified path for engineering teams to deploy their software. I hope to get a better understanding of these issues by working on the platform side of Kubernetes and help reduce the burden on engineering teams to setup and operate their software. Will I find this challenging and interesting in the long term and find satisfaction ? Stay tuned for upcoming episodes of my What I learnt series !

Ok, enough depression talk, back to the regular programming !

Go-like defers are not new to C !

Golang mainstreamed the defer pattern. essentially you create some resource and add a

defer resource.Cleanup()

after the resource is created (file handles, sockets etc) and the defer will be executed when the function returns. This solves a major pain point with languages like C where resources have to be cleaned up manually while handling errors and is a major source of bugs. But was Go really the first mainstream language to have this feature ? The ansiwer is No !

Turns out GNU and Clang support a __cleanup__ attribute in C that can execute a fn at the end of variable’s scope. Here is a guide on how to use this attribute. In 15 years of having played with C since my University days, I have never come across a mention of this attribute until now. Resource de-allocation is a big pain-point of programming in C and I am surprised that the cleanup attribute wasn’t popularized by programmers and yes, even critical projects like SystemD and the Linux Kernel use it. One of the reasons it isn’t popular could be that the cleanup attribute is a GCC specific extension (that is also supported by Clang) and most people still writing C care about cross compiler/platform compabitility and thus do not use it or use it minimally (if you know of any other reasons why __cleanup__ isn’t more popular, please let me know)

Location Databases and K-D trees

I was going through System Design Interview prep and was watching a video walkthrough of a system design for Uber. What impressed me was not just the design, but the way the engineer drove the system design walkthrough. The step by step process of

1. setting up requirements
2. describing the core entities in the system
3. API
4. High level design
5. Deep Dives is an excellent way of convincing the interviewer on how meticulous and thorough you are. I regrettably admit, that my disdain for system design interview prep courses and channels have held me back from trying to truly improve myself. Interviews are a performance, kind of like a dance on the stage. Sure, your day to day work might be very different and coding and designing might only be 10% of it, but if you need a role in the movie, you absolutely have to nail the audition. We might disagree with the process of Leetcode and System Design interviews, but it is far easier to change our expectations, buckled down and do the needful than to try to change the world, especially when you have bills to pay and bellies to fill.

In the system design, the engineer mentions how senior and staff level engineers can distinguish themselves by deep-diving into some aspects of the system. I was intrigued by the location database part of it : given a ride from a source to a destination, how can I find the drivers closest to the source (and thus the customer) and send them a notification that a ride is available ? One of the possibilities is a K-d (or K-dimensional Tree) structure that holds the locations of all of the drivers and thus performing a N nearest neighbors search on it, or a Geohashing system that turns every point co-ordinates into a unique string. The interesting aspect of Geohashing seems to be that the hashes of locations close by on a 2-D grid are layout close to each other, so we can possibly use a B-Tree to answer nearest neighbor queries.

It was however the K-D Tree design that caught my eye because of an elegant trick (dare I say hack ?) that allows a Binary tree to answer location based queries.

Let us start with a 1-D layout and extend it to 2-D

We can think of each point in this line segment as splitting the segment into left and right halves, with points between > 30 in the left half of 30 and points > 30 in its right half.

We can turn this into a Binary search tree to answer point queries (searching for a point) or answer nearest neighbot queries by maintaining a min heap of size k and inserting points into the heap until we have k points in the heap (and or if the distance between the points exceed a certain threshold)

This scheme works for one dimensional data, but how can we extend it to 2-D ? We can use the X co-ordinate of each point to insert points to the left and right of a node, but will that be right ?

Take for example, a Binary tree of 2-d co-ordinates with the following points:

When 2 points have the same x co-ordinate, we use the y-cordinate as a tie-breaker. For points A (100, 50) and B (100, 60), we will place B as the right child of A as B.y > A.y

However this can lead to an imbalanced tree, where the subtrees turn into an almost linear chain, thus turning our average O(log n) search into an almost linear search.

The trick a K-d tree uses is elegant. At each level i of the tree, we alternate between using the x-axis and the y-axis as the axis to compare against. For example, if we use the point A (250, 150) as the root node, then all the points with x < 250 will go to the left side of the node and points with x > 250 will go to the right side. For points A (250, 150), B (100, 160), C (350, 150), we will have

A.left = B
A.right = C

When we are inserting children into B, however, we alternate to using the y-axis for comparison. Let us say we are inserting points E (100, 280) and F (150, 150)

F.y < B.y, hence B.left = F
E.y > B.y, hence B.right = E

Similarly when inserting to the left and right subtrees of E, we will alternate to using the x-axis, thus alternating between the X and y-axes

The K-D tree tends to be more balanced. However, when reading through the implementation, it looks like a fundamental limitation of the K-D tree is that it isn’t good when the dataset is dynamic. Deletes aren’t easy at all with K-D Trees having to be rebalanced and it might not be the right choice for many applictions.

Quadtrees seem to be more commonly used, especially in Games. A point based Quadtree stores points inside a bounding box such as (0,0) to (50, 50) in a leaf node upto a capacity X. When a new point is inserted and the capacity exceeds X, the leaf node is split into 4 sub-nodes, one for each quadrant of the original box and the points are redistributed accordingly. I shall try to cover Quadtrees in more detail in a subsequent blog post, but for now, the automatic splitting and re-distributing of points in nodes seems like a good idea to self-balance the height of trees, especially in very dense areas

I got curious and dug a bit deeper and found out how PostGIS uses an R-Tree to create spatial indexes. An R-Tree spatial index kind of draws rectangles around objects it stores and finding out geo features close by an existing feature reduces to a problem of finding interesting rectangles. So many interesting rabbit holes, eh ?

What next ?

Having changed teams is a breath of fresh air. Ramping up is always a struggle, but it also offers a chance to learn totally new things and prove to more people that you have skills. While people might pretend otherwise, the first impressions are almost always the best impressions and I am trying to do my best to shine in my first few weeks with the new team. On the side project side, I have more time for porting Nsync to Rust and my goal is to make significant progress in the next month or so. Stay tuned for more updates on Nsync and my platform engineering journey !