Putting this talk together for AlterConf Portland has been an interesting journey. I’ll admit that for the two weeks before the conference I felt a lot of anxiety about giving this talk. Not only does the presentation touch on a few subjects that are taboo, at least in white middle-class society (race, wealth, class privilege), but it also required some public vulnerability from me around my class and race privilege. I also really wanted to do justice to these huge topics – which felt challenging since I am still learning so much myself.
Since I am a white person speaking about race, and am someone with an upper-middle-class childhood speaking about giving away money, I wouldn’t be surprised if there are moments when some people in the audience feel uncomfortable. It is my hope that the discomfort is not due to any personal harm caused by my words, but is instead the GOOD kind of discomfort, like the discomfort of entertaining new and different thoughts.
Today my new manager told me he believes measurable goals are overrated.
Ever since I started working as a software engineer, I have been fearful of falling behind. Of not knowing enough. Of not being intelligent enough. I set tons of goals for myself as soon as I started my job, hoping to power through the uncertainty and the fear. At first these goals were hugely unhelpful. Not only were my lists of goals paralyzingly long – multiple pages in a Google doc – but they demanded time-consuming data-tracking just to know whether or not I was succeeding.
When I worked for Causes.com organizing student organizers, I learned about SMART goals – goals that are Specific, Measurable, Achievable, Realistic, and Timely. I thought SMART goals sounded so, well, smart. Having recently come from college organizing myself, I thought that a good campus campaign should be concrete, have an end date in sight, and convey a clear story about results – all things that SMART goals spoke to.
Later, as a software engineer, I assumed that my personal goals should also be SMART. I spent a lot of time trying to think of ways to measure my knowledge and learning. By calculating sheer time spent? By counting the number of bonus study sessions per week? By recording a thing I learned every day? By counting numbers of pull requests, pairing sessions, projects completed on my own? I asked my most goal-oriented coworkers to help me craft my goals and strategies, I tried to institute new practices, I kept a daily log of things I’d learned or accomplished.
But in the end, my goals didn’t act as the solid guideposts I was hoping for. They were worthwhile, for sure, because they forced me to clarify my challenges and intentions. But the act of setting those goals didn’t get me much closer to the knowledge and skills – or the feelings of safety and progress – that I was after.
Measurable goals are overrated
At first, when my manager told me that measurable goals are overrated, I felt resistant. Goals should be measurable, I thought to myself, dismayed. How else can you know that you’re meeting them? Especially considering the goals he was suggesting for me:
acquiring mastery of UI development (by pairing closely with my new coworkers)
Mastery was an exciting, thrilling word to read in the context of goals. But it made me feel nervous too. How would I know if I was on track for something as giant-sounding as “mastery”?
When I asked that question, my manager said that forcing nuanced goals to meet requirements for objective measurability squashes out the richness of the overall goal. “Mastery of UI development” is a very rich goal – it has many aspects to it, none of which are easily measurable. In fact, it would require a good amount of mental contortion to find opportunities for objectivity.
Hearing his arguments against measurability, I started to feel a sense of relief. This way, I wouldn’t set myself up for another failure on improperly-calibrated, falsely-measurable goals. I wouldn’t have to add a bunch of tasks to my daily to-do list just to try and track progress. I wouldn’t be handing myself yet another yardstick with which to measure myself and find myself lacking.
The part that finally sold me: my manager asserted that the point of having these goals is to feel excited about coming to work each day. Thinking about UI mastery (as unmeasurable as it is) is already doing the trick.
An interesting aspect of my manager’s goals philosophy is that I can meet my goals simply by going about my regular business: by collaborating on code each day with good conscience and good intentions, and by talking with my manager each week in our regular check-in to resolve challenges as they come up.
This aspect reminded me of a term I’ve started hearing recently in activist circles, something called relational organizing. In relational organizing, as I understand it, human-to-human relationships are the basis upon which community activism and power is built.
Transferring this idea to my goals at work, the relationship is the building block. Not numbers of facts learned in a day, not PR counts. My goals will be met via relationships bewteen me and my coworkers and between myself and my manager. Goals will be achieved not by the individual studying solo and on the side, but by the individual being incorporated into the flow of others' work.
I think I’m going to really enjoy coming to work each day with this in mind.
P.S. Apologies to SONG if I’m misusing the concept of relational organizing!
According to Greek legend, Cassandra was a prophet. She foresaw terrible, tragic futures — but as if that wasn’t enough, she was also cursed to never be believed.
I had always wondered why on earth Apache Cassandra was named after a prophet who was never believed. Doesn’t give you the greatest confidence in their software, no?
But yesterday, I learned a ton about Cassandra from a knowledgeable coworker — and now, the reference to Greek legend makes sense.
What is Cassandra?
Cassandra is a distributed data management system built for handling large volumes of I/O with no single point of failure. With Cassandra, your database is just a file system, with files spread out over a number of nodes that are arranged in a cluster.
How is it different from MySQL?
Relational databases like MySQL and Postgres use a binary tree structure to store information. Binary trees are great for reading information quickly — they’re a data structure that accommodates binary search, dividing predictable/sortable information in half and search the new half for the desired row. However, adding new rows to a binary tree is kind of ugly. You add the new information as randomly-inserted leaf nodes in the tree and now your binary tree is out of order.
Cassandra solves this problem by getting rid of the binary tree and replacing it with a token ring. In a token ring, you start with a key (like “start time” or “id”) and generate a unique hashed key, an unsigned Long with a value somewhere between -264 and 263. The ring is divided into chunks, each of which have a node assigned to it. For example, you might have a chunk of hashed keys assigned to Node 1, another chunk to Node 3, another chunk to Node 2, another chunk to Node 1, another chunk to Node 3, etc. The chunks are distributed in a somewhat random way to try to fairly balance the load — you wouldn’t want a bunch of traffic going to Node 1 and overwhelming it with requests.
But, just like the prophet from Greek legend, never believe Cassandra.
Can’t trust Cassandra
Any one node by itself can’t be trusted. In fact, the token ring doesn’t just assign one node to each chunk of hash keys, but a list of nodes in the order in which to try to write the data. For example: chunk 256 => [2,3,1]. This will try to write to Node 2 first, then 3, then 1. The data will be stored on multiple nodes eventually.
Every cluster has a replication factor, a value that determines how many nodes to repeat the data on. For example, if our 3-node cluster has a replication factor (RF) of 3, then data will be stored on all 3 of the nodes. My coworker strongly believes that 3 is the minimum sensible RF — because if any of your nodes ever goes down, then you’d rely on a backup node, and you’d need 1 more node to double-check to verify the data is good. (3 nodes - 1 node = 2 nodes. 1 node for requesting the data, 1 to double-check that the first one is right.)
One thing that’s amazing about Cassandra is how easy it is to add a new node and therefore add capacity to a cluster. You add the provisioned server to the cluster by simply starting Cassandra on the server (it’s a Java app), then adding its IP address to the list of known nodes. The Cassandra node gets the schema information from the “seed” nodes (the seed nodes are just the nodes, listed by IP address, found in the Cassandra config under “seeds”), the token ring breaks and re-forms to include the new node, and the new node begins streaming data from older nodes. Once it’s joined the cluster and owns all the information it’s expected to own, then it’s ready to receive traffic, and you’ve just added a terabyte of available storage space to your database.
There are no masters, there are no replicas. Just nodes.
It’s a cool system. The co-op nerd side of me loves the hierarchy-free model based on negotiation, shared ownership, and consent among nodes.
However, there are some serious drawbacks to using Cassandra.
What’s wrong with Cassandra?
The biggest downside of Cassandra that has come up so far in conversation: it was designed to solve the problem of slow writes. When MySQL or Postgres write something new to a binary tree, that information gets spread out over the binary tree – it’s not all in one place. Cassandra, on the other hand, writes new information all together. This is convenient when your model of storage is a spinning hard disk drive, where you’d want all the writes to be together.
But with the advent of solid-state drives, this problem might feel a little irrelevant. And given that Cassandra requires a lot of specialized knowledge outside the comfort zone of people who understand SQL really well, there might not be enough of an incentive to make the switch – or to even stop using Cassandra after having started.
Sometimes you just need to say things out loud to someone else to know that you understand something. That’s what I did with my coworker the other day – just described out loud to him how I thought our internal service worked. It really helped. I got to put my vague thoughts into words, and he offered corrections as needed.
Turns out that, like most things, our service works because of the magic that is the Internet. HTTP requests, DNS lookup, IP addresses, CORS, etc, are all at the core of how it functions. Trying to explain how the service worked reminded me of the code interview prep question I practiced when I was trying to get my first software job: “Explain how the Internet works at a high level.”
The /etc/hosts hack
DNS (Domain Name System) lookup is hierarchical. When you make a request for a domain like google.com, the request will travel up through a series of DNS servers until it finds an entry for google.com that points to a specific IP address.
From your laptop, the very first place your computer looks in that hierarchy chain is a file called /etc/hosts. It contains a list of domain names and IP addresses, just like any other DNS server. And so, if you put an entry like this:
then from now on, when you try to load google.com in your browser (unless your browser has a cached version), you will actually be directed to 127.0.0.1 – your own machine.
This is useful if you want to simulate, say, hitting an internal service that proxies your request to another app. Put the IP address of the internal service with the name of your app in your /etc/hosts file, and your computer will map the domain of the app to the real live internal service, located at that IP address, that receives the request and proxies it elsewhere.
# the internal service IP the app
Who controls DNS?
So besides just putting it in your /etc/hosts file, how do IP addresses end up with registered domain names?
when you buy a domain (like from Namecheap or Godaddy), those vendors work with IANA to add your DNS entry – that’s the department of ICANN that controls IP/DNS stuff. Yes, deep down in there, there’s a bureaucracy (no offense, IANA) sitting inside the Internet, pulling the strings.
Companies may have their own DNS servers, too. Internal apps and services that don’t need to be accessed by the public Internet can have IP addresses that don’t need to be registered through IANA.
Domain Name VS. Host VS. IP
55.5.555.55 mydata.com # domain name
55.5.555.55 wwww.mydata.com # sub-domain
It used to be the case that subdomains usually had their own unique IP address. Then we learned about Reduce, Reuse, Recycle. Now, it’s common for any one IP address to have many subdomains. So how do we know where exactly to send the request?
Every HTTP request comes with a Host header. The Host header identifies the location – the actual host machine – to where we’re sending packets. The host is the domain name of the server (as well as the port if a nonstandard port is being used). Getting close to the metal here!
Knowing the hostname and port, we can now send the request to the correct host at the IP address we looked up.
Let’s say you’re making an internal service who receives requests from one lover, inserts some headers into the request, and proxies the request to its ultimate destination – then hands back the response. A go-between romantic messenger service.
Let’s pull in some Shakespeare: your service is Wall, and your lovers are Pyramus and Thisbe.
The Wall receives the request (because it’s at 22.214.171.124, the IP address that matches the domain), sees the host header, and passes the request on to Pyramus. Success! (Of course, there will be a response back to Thisbe too, but it’s too saccharine to print here.)
Almost a year and a half ago, on March 2, 2015, I started work as a software engineer. I was new to Portland, new to programming, and frankly, new to having any kind of foreseeable career path.
The first year was a rush: trying to learn as quickly as possible, trying to do well, trying not to fall behind. I set lots of goals for myself. Since I was working on a public REST API, my manager suggested that I learn as much about APIs as possible. Not only did I take that advice, but I started to veer my long-term career goals in the direction of becoming an API master. APIs are neat. They are symbolic of the Internet itself, showing how interconnection enables complexity and creativity. Pursuing that domain-specific knowledge, I focused on having a specialty – on being special.
From there it was easy to start playing the career ladder game. Can I get a promotion six months from now? What do I have to do to get there? I focused on promotion as a sign I was doing well, a sign that I was on the right track. I felt hugely validated (and, of course, sure that there was some mistake) when I was promoted to Software Engineer II a year in. I couldn’t wait to jump right in and start wading towards III, then IV, then senior. I joined a small group of women specifically working on “leveling up” and kept adding to my list of goals.
Over the last several months, though, I got a wake-up call. Part of the wake-up call was that I changed jobs. The API team (me) was merged into another team, one that owned a bunch of internal services. I stopped driving towards API knowledge and Ruby/Grape expertise and scrambled to pick up Java. Now that I was no longer working full-time on a REST API, my goals around API mastery loosened their grip. When I was asked where I was trying to go long-term, I found that my desire to climb to the level of senior engineer as quickly as possible had also disappeared. Instead, I started wondering: why am I here? What motivates me? And I remembered promises I’d made to myself back when I first embarked on the software engineering path. I remembered why I chose a tech career and how it fits into my life goals, not just my career plans.
It’s been an interesting process to re-envision my nearer-term goals with respect to these larger life values. I’m surprised by what has taken root so strongly: to really pursue work as a full-stack engineer. To be somebody who makes a point of being a generalist. I’ve been a generalist all my life. Why switch to being a specialist now? The heck with “Jack of all trades, master of none” and the associated stigma. I’ll be Jack. I like variety. Variety is useful.
I’m starting this blog (in earnest this time!) to capture what I learn in the process of becoming a Jack of all trades. If I’m going to gobble up knowledge involving React, Angular, Java, Kafka, Rails, Docker, Jenkins, and so much more, then I’ll need a place to recompile and condense that information – and that’s here.