Spifftastic,

 

Thoughts from 2016

This isn’t going to be much in the way of a coherent post. I’m not an authority on anything, and it’s tedious to write when you’re exquisitely aware you don’t know enough and don’t know what you don’t know. Anyway, I’ll try to headline each bit as follows.

Programming

For most of 2016, I’ve been less active in open source projects. I’ll contribute issues and patches when I’m working on something, but I won’t patrol someone’s issue list and try to fix those unless I’m affected (such as the tailcall issue in Shopify’s go-lua library). So, there’s not a lot to say there.

Rusalka 2

I began work on a new implementation of the Rusalka VM, this time in Go. This isn’t formally named Rusalka 2, and in fact is just rvm as far as the codebase is concerned. However, seeing as it’s the second implementation, I’ve been informally calling it R2. rvm, after all, isn’t a useful name: it’s ambiguous with several Ruby-land things already.

The new Rusalka represents an effort to trim the instruction set down and, in particular, encode it more effectively. A lot of this borrows from Lua’s ISA, where instructions are all encoded in 32 bits. Rusalka’s new ISA is sort of variable-width — instructions are either 32- or 64-bit. 32-bit instructions make up the bulk of all instructions, with a handful of extended instructions made available as 64-bit. The extended instructions effectively increase the addressable range of an instruction, either by widening the range of stack or constant indices available, or by simply providing more room to encode complex instructions.

This makes the new Rusalka a little weird, because the goal is to make it RISC-like, but the extended ISA makes it possible to represent many more instructions. I’m fairly content with this, even if it ends up making the ISA strange. The upside is that you can decode the ISA as entirely 32-bit instructions and simply read the next instruction if the extended bit is set on the current instruction. However, this means jumps have to hop over two instructions instead of one, so there’s a small compiler/linker detail there that can’t be ignored.

Go

I’ve tried to not write about Go for a while, but I realize I haven’t mentioned Go in any detail on my blog before today. My resume mentions Go and that I introduced it to the dev team at Kochava. Prior to that, the languages of choice were either PHP or Javascript (in node.js). We still use PHP for UI and some of the larger existing codebases, we still maintain systems on node.js, but new development for core systems is in Go.

Note: I can’t speak for every developer, so what follows is entirely my opinion of Go and how I use it, and in particular why I pushed for it at Kochava.

Go came about as a reaction to PHP and node.js, and in particular legacy PHP and regular Javascript written for node.js. As anyone who’s used them knows, both PHP and Javascript are dynamically typed scripting languages — phrased differently, the language does nothing to aid you in understanding the code beyond basic logic and arithmetic. The arithmetic part falls apart a little more in Javascript due to several operators being overloaded (i.e., + for both addition and string concatenation, where you may see x+'' to implicitly convert x to a string). These are also languages where implicit conversions are common and data structures tend to be fairly loose.

What this ultimately means is that there aren’t a lot of guarantees you get just be having source code. You have fewer guarantees than most due to the lack of a compiler to check your code for validity. At best, you might have linting, and the linting might not even tell you about serious errors (e.g., undefined variable access — which is not an error in PHP but a notice, as the runtime treats undefined variables as null and undefined constants as bareword strings).

So, in contrast, Go is a static strongly-typed language. I can look at a function in Go and immediately know its input and output types. It’s harder to guarantee the body of the function is entirely known, since there may be an thing := functionCall() somewhere and I’d need to look at functionCall to know what thing is if its usage wasn’t obvious (e.g., thing + 5 means it’s a kind of number, thing + int(5) means it’s an int, and so on).

In addition, there’s the whole concurrency thing, which is fairly important. This is more where node.js comes in, what with the event loop and async insanity that node.js promoted. Where node.js is effectively single-threaded and gets around concurrency through callbacks and other additions to the Javascript language, Go avoids it altogether through goroutines and channels, effectively allowing you to run concurrent routines in a program and communicate with them, safely, over channels. You can read about all that on Go’s website, since my goal isn’t to teach you anything about Go here.

More than concurrency, though, I appreciate that Go encourages direct, readable code. There are no not-really-DSLs as in Ruby, there isn’t a heavy emphasis on unusual design and callbacks as in node.js, there isn’t the unbearable quirkiness of PHP. Error handling is immediate and direct, as exceptions do not exist, and the standard library is robust without forcing all programs into a specific mould (as with all the awful frameworks for things out there). Most of all, there isn’t the ton of bootstrapping needed to guarantee that an int is an int in a particular scope or even having to check that a function isn’t just null. It makes programming fun again: I don’t need to guard against the language itself, I can write code that does something.

Other General Notes

Bullet-points follow:

  • Five months out of this year at work were spent effectively hunting down / fixing bugs in legacy systems and other folks’ code, so that was an interesting use of my time. This has helped a lot at learning how everything flows through our systems, since it’s fairly difficult these days for one person to know it all.
  • I’ve been at the same job for something like two and a half years and it’s still fun? I think that’s an accomplishment, but I’m not sure how much of that is me. Still, it’s nice being able to list off a couple huge things that greatly changed how we work. First, GitLab was a huge change for us. We previously used a mix of Subversion and Mercurial, neither of which had UIs tailored for collaboration and rapid development. GitLab CI’s also made it easier to test and build products as we go. Go, as well: I think it helped us become more productive overall (again, you’re writing code that does something instead of spending a ton of time on boilerplate). I like to credit myself with the death of node.js at the office, but that was probably natural causes: nobody really liked node.js and working in it was a pain.
  • I increased throughput of our outgoing postback systems by about 6x per instance (this was fairly significant). Postbacks are a node.js system that I inherited maintainer status on after helping get them into production about a year and a half ago. So, increasing our throughput by 6x per instance allowed us to reduce the number of machines running and save some money.
  • Aerospike sucks and you shouldn’t use it.
  • The Snow engine now builds using CMake. This is actually the most recent accomplishment, as I did this just hours before the new year. I figured I needed to do something regrettable before the year ended. The transition from premake4 to CMake was tedious, but mostly due to dependencies used by Snow — everything the engine uses is now included in its CMake file as an external project, ensuring that almost every dependency does not change from system to system.
  • Despite my absence in public repositories, I’ve still made more contributions to open source projects this year than last.

Closing

Fuck you, 2016. Maybe I’ll post more in 2017.