User stories

“Data streaming will change the way we think about reporting”

Alasdair Brown

Alasdair Brown

Developer Relations @ Tinybird

Your job title sounds very fancy, what does Developer Relations do exactly do?

We build our product squarely for developers. That means we need to understand how developers work, what they are trying to accomplish, and what they might want to do in the future. My developer relations (”DevRel”) team has two goals: to talk to developers and to listen to developers. It’s really easy to do the first one, you can get on Twitter or LinkedIn and tell people about all the amazing things you can build with our product and how the sky is the limit. But it’s the ‘listening’ that many companies get wrong.

You can get on Twitter or LinkedIn and tell people about all the amazing things you can build with our product and how the sky is the limit. But it’s the ‘listening’ that many companies get wrong.

Everyone in the DevRel team is a Developer first and foremost. We have all been boots-on-the-ground engineers. It’s a highly technical role. But the other side of it’s that we love to communicate. We love to talk with the folks using our product, to understand what they are doing, why they are doing it, and how they’re going about it. Learning about everything they do and don’t like, and educating them on better ways to accomplish something.

Then, we can digest all of this information, and turn it into language that other teams in the company understand; we can break customer problems down into individual technical issues that our product teams can solve, and translate solutions into a higher-level language to help our marketing teams understand what needs to be published.

Your product focuses on making data streaming easier. Why has this become so important?

Currently, data is mainly used for automated reporting to replace the reports we used to do manually. We aren’t using this data to tell us what to do on a daily basis, it just tells us about what happened in the past. For example, a shop wants to know how their business is performing. To find this out, they gather all of the transactions that have happened in the past and total up how much each customer spent. A lot of effort is spent digitizing processes, capturing data, and sending it all to a big database, only to sit there getting stale until a report runs.

Data is mainly used for automated reporting to replace the reports we used to do manually. We aren’t using this data to tell us what to do on a daily basis, it just tells us about what happened in the past.

So, how could we improve this? Well, we’re already capturing the details about every transaction, we know when someone buys 3 pairs of socks. Can’t we use this data to help us do something right now? For example, can we work out how much stock we have left and order more if needed? No more manual stock counts, no more missed sales due to empty shelves.

The periodic reporting is already handled very well, but using our data on a “day-to-day” basic like this is still harder than it should be. It requires rethinking the way data is stored and delivered, and this is exactly what we try to do with Tinybird.

Difference between periodic and real-time reporting
Real-time streaming forces us to think of creating ad-hoc insights and actions rather than past analysis

Tinybird has used Tremor for one of its analytics starter kits. Tell us a little bit more about why you used it and how you liked the developer experience?

Although we cover the back-end part of apps, we still want to offer templates to give users a head start for front-end building. When we first came across Tremor earlier in the year, it struck us how similar your out-of-the-box styling is to what we had just put all this effort into building over ECharts. It’s simple, elegant, and clean. From a design point of view, it was perfect. We did some more digging and loved how thoughtfully Tremor has been built under the hood.

I’m sick of fighting with libraries with ugly defaults that just end up looking awful regardless. Instead, it feels like Tremor has redirected that effort into making an API that actually works with data.

A lot of visualization libraries put more effort towards customizability. Instead, Tremor is opinionated, and provides a beautiful out-of-the-box design with fewer options to change the look and feel. In my opinion, that’s a good thing. I’m sick of fighting with libraries with ugly defaults that just end up looking awful regardless. Instead, it feels like Tremor has redirected that effort into making an API that actually works with data. There’s no fighting to adhere to some weird data model, it feels totally natural.

Template dashboard for monitoring log data
Tinybird’s Log Analytics Starter Kit built with Tremor

The Tinybird’s Log Analytics Starter Kit is one of our latest templates based on Tremor. This one is aimed at developers, to show how they can capture data from their code and use it to analyze usage patterns with telemetry, or debug errors with additional context. It was just stupid how quickly it came together using Tremor. I probably spent more time writing the README than putting the dashboard together.

Full-stack architecture for building real-time apps
Tinybirds’s common architectural approach: Several data streams are joined together and published as a secure, always-up-to-date API every application can connect to.

Building real-time data apps can be really overwhelming in the beginning. Do you have some advice for developing such apps?

When you have a system that can deliver data fast, it unlocks a lot of possibilities, but a crucial one is iteration. You don’t need to get it right the first time. You can try things out, tweak them, scrap them and start again. This is absolutely crucial if you want to build something that provides value and not noise. Ask any BI Analyst how many dashboards they’ve created, and then ask how many actually get used. Seriously, the amount of wasted effort in this space is crazy, and it’s because legacy stacks don’t allow iteration.

master the data complexity and be able to deliver fast
Challenges for the back-end and front-end side in application building. Solution: Low-latency APIs to process and fetch data faster while having reusable and highly customizable components on the front-end side to iterate as quickly as possible.

Ask any BI analyst how many dashboards they’ve created, and then ask how many actually get used. Seriously, the amount of wasted effort in this space is crazy, and it’s because legacy stacks don’t allow iteration.

And so, you can see how principles in web/app development align quite nicely with real-time data. If you’re building a data-driven application, you’ve got two moving pieces that will need iteration; your data, and your app. You don’t want either side to slow your development down, so you need to pick the right stack, and the right tooling, on both sides.

Before you start building, you’ll have some ideas or assumptions but we all know that once you start putting the code down, they pretty much go out the window. So, we put something as a starting point, and then we try out other ideas until we find the one that works. If you realize one of your Pie charts is useless (which we did when building our Starter Kits), how do you experiment to find something better? If you’re using Tremor on your front-end, it’s trivial to swap out a pie chart with a line chart, but you also need the data to back it up.

Waterfall vs. agile software development
The development of data applications has been characterized by slow, legacy stack. Those who can prototype fast with lightweight tools both on the data- and visualization side will master the ever-increasing complexity of data apps.

Can you update your backend to return the appropriate time series data for a line chart, as fast as you can swap out the Tremor component in your front-end? In my experience, the classic backend data stack is pretty slow to change. You’ve got at least an API + ORM to update, and probably data processing jobs to change, deploy and run. It’s slow, and you don’t even know if the result is what you want.

The real-time space has evolved rapidly in the past 5 years and there is a new generation of tooling that has stripped out all of the complexity. The tools now enable developers to be as fast and agile with data as modern UI libraries, such as Tremor, do it for the front-end.

What tech stack would you recommend for such apps?

Jan 9, 2023

About Alasdair

About Alasdair

Alasdair leads the Developer Relations team at Tinybird. He has been a data engineer and consultant in the data analytics space for more than 10 years. Previously, he worked for the British Telecom’s Cyber Security Platform as well as Cloudera.

About Tinybird

About Tinybird

50-100

Employees

$37m

Funding

2019

Founded