I really wish they turned it into a dependency or database framework where users could define their own business logic to swap out the double entry accounting, while reusing all the system architecture and networking features, consensus etc.
Sort of like a new paradigm where opinionated custom databases could be created with arbitrary entry logic built on this stack.
Joran from TigerBeetle here! I created TB. Happy to answer questions!
Hey Joran, awesome stuff. I see a TB post every now and then and it seems like such an interesting problem space to work in. I'm all the way at the other end of the stack, most software we write day-to-day is in JVM-based languages where you don't have to think about these things at all (we get by with ms latencies instead of ns). Reading this post inspires me explore low-level engineering more and I figure zig or rust would be a good place to start.
What are some interesting problems or things you can think of to work on that would give someone new a nice amount of exposure to this kind of programming?
If one has a TigerBeetle cluster with high inter node latency, are there any easy wins to lower the latency of the whole cluster left? My head hurts when I think of latency in large clusters, so your work this year with latency was inspiring.
EDIT: I guess part of the question is about the problems with clusters with >130ms latency and if there are challenges you consider easy.
Hey, very inspiring article, Redis engineer here. How do you work with static allocation on variable query structure, and row counts that can explode depending on the data shape?
And isn't there a benefit for small allocations on advanced memory allocations that you can't leverage if all is working in big page allocations? Do you implement memory allocations from scratch or leveraging existing allocator on top of these memory blocks strategy somehow?
Thanks! We use streaming data structures.
For example, if you take a look at our LSM compaction, regardless of the table size, we compact at the 512 KiB block granularity, and everything is streaming.
The same principle applies everywhere.
In our experience writing TigerStyle (and for all our internal code and tooling, not only TB as DBMS), we’ve never had a scenario where static allocation was not applicable or didn’t produce a better design.
You also tend to become more memory efficient, not less. Again, since you’re streaming. (You’re not allocating a massive buffer, just because a file is multi-GiB.)
Their simulation is fantastic:
Amazing writing, very accessible too.
So batching requests is always something I think should increase performance by a lot, but most server implementations make this pretty difficult, but the thing I struggle the most to understand is how to keep the latency down if you have multiple clients request all batched together? The total amount of latency for all clients is always the latency for the slowest.
I think you design in layers, frontends that work as clients to TigerBeetle for work in batches (as mentioned by the sibling comment), but the whole idea of removing latency differences by removing unpredictability means that you don't get the jitter of latency differences that can cause backing up in normal scenarios.
If you give the TigerBeetle client a single transfer, it sends it off immediately to the cluster. There's no delay. No Nagle!
But if your application then creates another transfer against the client, and another, while the first request is inflight, then the client will autobatch under the hood and send these off as a batch when the first request returns.
You get this sweetspot then between latency and throughput. And your latency is not spiking as your load increases, since your throughput is now able to keep up.
Emphasis on: opinionated custom databases. One database might be SQL-based for periodic report generation. Another database might be a noSQL key-val store designed to effortless grow with the number of end-users. General-purpose programming languages already cater to the 'own business logic' part - it's their whole job. What's left? System architecture, networking features, consensus. That's Kafka.
This was always the plan, and if you look closer at VSR and the state machine interface you’ll see it’s already pluggable. We just haven’t packaged it. (We’re dogfooding our first few internal “CustomBeetles” before we package and document.)
That's cool! I will be then probably taking a closer look!
Thanks! Watch IronBeetle too on Twitch if you’d like to go really deep.
llvm for databases?
I'm pretty sure they manage to get that level of performance and reliability, since they have a very limited schema.
I'm pretty sure you can't just do a precise 128 byte align, if one of the element is an image blob or varchar(1000).
No, the LSM in TB is generalizable at comptime to any combination of power of two sized key/value tuples.