I gave a talk at Google Cloud NEXT, highlighting our use of BigQuery and Bigtable for building out our Personas architecture. In this session, I share a little bit about Personas, and dig into the Dremel and Bigtable papers to share why they work for our use case.
I recently gave a talk at our annual Synapse conference about ctlstore (control store), a new database that we’ve been developing internally at Segment. It’s designed to give good consistency guarantees to the writer, while providing good availability guarantees for readers.
Over the past five years of being on-call, seeing hundreds of production outages, and working with a growing SRE team, I've started to see patterns when it comes to making teams work effectively.
Everyone wants to build the next “rocketship.” We all want to start businesses with an incredible growth trajectory and massive impact.
“What’s the biggest thing you’ve learned while building Segment?”
Terraform is a wonderful tool for describing infrastructure as code. It’s fast, flexible, automatically resolves dependencies, and is rapidly improving.But in some ways, Terraform is flexible like AWS is flexible. You can do pretty much anything, but it’s also easy to shoot yourself in the foot if you aren’t…
Recently, there’s been a lot of commotion on twitter and in #node.js about the new streams2 API. The official stream docs leave a lot to be desired, which has lead to general confusion. That’s too bad, because using new streams can really simplify your code once you understand how they work. Let me take you there…
A little over a year ago, I tore my ACL playing ultimate frisbee. I made a quick cut to lose my defender, and as I planted my foot to change directions, I felt a sudden pop in my knee. Next thing I know, I’m on the ground because my knee had given out.
At Segment.io, we deal with a lot of important user data on a daily basis. Consequently, our top priorities are that we don’t lose your visitor’s data and that our incoming API stays available at all times.
When my co-founders and I first started on our startup a little over a year ago, we asked other startups about what to database they were using. Nine out of ten people all had the same response: “Just go with Mongo.”
At Segment.io, we’ve been using node.js with the express framework for about 8 months now. We switched over from using the java play! framework and I’ve found express to be much more elegant. It’s simple, doesn’t prescribe too much, and is way less verbose than the same java code.
One of the best parts about mongo is its entirely schemaless nature. We can store any kind of data we’d like without having to worry about where that data comes from or how it should be laid out.
Well, once again there is too much other stuff to work on and my last blog project didn't turn out quite the way I hoped, so there hasn't been a recent blog post. Look for a few concerning my latest project in the coming weeks, but for now I'll be putting up a few shorter or less technical ones about various subjects…
Well folks, I currently have 2 or 3 significant blog entries whizzing around my head, and I have to post up one of them even though I have already started writing the other or else I might explode . I guess this can be a little tidbit to keep the posts going until some of my more major endeavors arrive.So the…
Those of you rolling around the internet back in the day might remember some images that displayed some seemingly "personal" information to you. Granted I'm not exactly THAT much of an old timer that I programmed my own Ethernet protocol in assembly or anything- so you too might remember something like the image at…
All right, as promised last week today's topic is LZW compression. As we talked about in our networking class, LZW compression is about as close as you can get to an optimal compression rate for large compression objects. The single biggest advantage of LZW is that unlike the traditional Huffman Encoding that was…
I'd like to talk a little about and give an implementation of B-Trees, perhaps the best example of an algorithmic data structure designed specifically 'real computers' rather than the theory of academia. Given that, it took me a while to figure out the why on earth B-Trees are used. It's sort of like the Binary tree's…
