Billy Bosworth had a front seat to the database explosion

This story is part of "The New Database," a Protocol special report. Read more here .

For a long time, database technology barely changed. So when it did, it was a major shift — and Billy Bosworth was there to witness it all.

A college graduate of the University of Louisville in 1994, Bosworth knew that if he was hearing about the Oracle database in Kentucky, a state far from the bleeding edge of tech, it was going mainstream. So he dove into the industry and now, as CEO of Dremio , he's about as far as you could get from Oracle — at least, in the database world.

The startup, which has raised $250 million and is valued at $1 billion, specializes in data lakehouses, technology that Dremio proclaims will obliterate the need for data warehouses like Oracle's Autonomous Data Warehouse and up-and-comers like Snowflake. There are plenty of hurdles ahead of that goal. But given that he's spent three decades working with or around the tech, Bosworth understands the challenge.

And while this approach might not be for everyone, it represents, at least, a strong countervail to the marketing efforts of Dremio's much larger rivals.

Read on for Bosworth's insight. His remarks have been edited for brevity and clarity.

I always loved programming . I was a developer at heart back before the internet, and I learned on my Atari 800. And I just always loved to code. It never felt like work for me. When I was fortunate enough to get an athletic scholarship, I went to Louisville, got a degree in computer science, came out and wanted to get right into the world of programming.

This was in 1992, and there was a lot going on at the time in the mainstream. There was a big shift happening from mainframe architectures to client/server architectures. And it was my first taste of what a paradigm shift feels like in the industry. I had very seasoned people telling me, "If you want a good career, you better stay committed to the mainframe."

One of the hot technologies that was growing in the mainstream was the Oracle database. And I say mainstream because I was in Louisville, Kentucky, not Silicon Valley. And Silicon Valley tends to be a bit ahead, obviously. But the Oracle database was really gaining a lot of steam and traction and being considered for some very mission-critical things.

I decided I wanted to be where the hot stuff was going, not where the hot stuff had been . The first 10 years or so in my career were fabulous because another interesting shift happened: Y2K . That was a big forcing function for the paradigm shift, and a massive accelerator for moving things off the mainframe and onto client/server platforms.

But I really started to see what lay ahead for the industry when I saw the demand placed on the database when applications became web applications. The velocity at which the data was coming into the system, the sheer number of transactions that were happening, the speed with which those transactions were happening: It required a fundamental re-look at what kind of architectures are needed to be able to handle this kind of load. Then the applications started to become interactive, then mobile kickstarted it into another gear; all of a sudden, you weren't just reading from a database, you were writing to databases. And you were doing that at internet scale and internet speeds.

That's when you started to run into problems . At the time, when all you had was relational databases built on that old architecture, you just figured it out. Developers came up with these very creative architectures that were based on sharding. Instead of scaling it vertically, we are going to force it to scale horizontally, because we have to. We can't get the throughput we need in a single box no matter how big we make it. It was a bit ugly, it was a bit convoluted, it was nasty, it was complicated, it was fragile, but it's what they had.

Then we watched the rise of the digital natives like Google and Netflix, and you started to see what they were doing with their technology at the scale they were doing it. They started to open source a lot of the projects that they were working on. And so the world got to see: That's how you can handle that kind of scale. People could now see the paradigm shift in the architecture.

Then Snowflake came along. One of the most important things about its success story is that it showed at a scale that we had not seen before that an ISV could build their system on top of cloud architectures that they did not own and have a very successful, independent, high-growth, standalone company that can achieve global scale. That was really important.

Now, at Dremio, we believe that where and how you store your data at the core data level is actually one of the most important decisions you can make because of the insatiable need in enterprises to be able to access that data. There's hardly any part of the organization that doesn't need access to that data now. And where you put the information has a lot of downstream consequences to how you are going to be able to access it and at what price.

That's the whole rationale behind a lakehouse . It's the notion of making the data itself a first-class tier in our architectures. Now, you can bring specialized engines to be able to access that data without having to move it. And it's the architecture of the future.

Snowflake is classic data warehouse architecture done very well. If you want a closed architecture, they are a really good choice. Databricks is a brilliant solution for data scientists. But we are going to own SQL. You have to be good at something. The thing we are going to be very good at is SQL on the lakehouse.