DevOps Decrypted: Ep. 19 - The Open Source Drama Broadcast

Rasmus Praestholm:

Hey, hey? Is this thing on?

We're doing a podcast here. But, like normally, Laura's here.

But I kicked her out and took over because I wanted to re-licence this broadcast with an anti-competitive bent – so I could play a bigger role in it, you guys!

Okay?! Are we good for this?!

Jobin Kuruvilla:

Yes we are!

Rasmus Praestholm:

All right. Who do we else have today? Jon – you're still here. Right? Did you get anti-competed out?

Jon Mort:

I didn't want to compete with you, Rasmus!

Rasmus Praestholm:

Ah, yeah, you better think about that…

Jobin Kuruvilla:

Yeah. I'm here, Rasmus, just listening in…

Rasmus Praestholm:

All right. It's like there's a chilling effect in the room just because somebody takes over and re-licences all the things. What is that about?

Jon Mort:

I guess you must be referring to HashiCorp, Rasmus.

Rasmus Praestholm:

I do! That's right! It’s the elephant in the room. What is going on with HashiCorp, Jon? I have opinions, but you have neutrality.

Jon Mort:

Yeah. Well, let's I guess, start with what's actually happened. So HashiCorp has announced that in the next release of all of their product suite, they’ll be relicensing from the Mozilla public licence version 2 – so the kind of free, open source licence – to the business source licence, which is not an open source licence by pretty much any definition, including HashiCorp's.

Which includes clauses that prevent anybody from using or embedding, well, providing a service or embedding the products that HashiCorp make in a competing offering.

And yeah, there's all sorts of things to unpack there about, you know, what competing means. And all those things. That's what they've done. They've re-licenced everything from open source to a non-open source licence.

Rasmus Praestholm:

What a shocker.

You can take a thing to like multiple things. Multiple people are contributing and call it your own, and like, no, you can't do that thing any more. What is up with that? That isn't open source; the mindset in there!

Argh! I have so many things to say.

Jobin, what do you have?

Jobin Kuruvilla:

Well, you just said, “We can't do it”, but apparently, you can. Right?!

I mean, that's what HashiCorp has proven. But again, I mean, as Jon said, you have to see their point of view. Obviously, they have a product that people are using, and they are not able to make any money out of it.

You could argue that that's the definition of open source. Right? I mean, that's why they're open source – but again, can we take an open source product and utilise it for our own benefits when it is HashiCorp who is doing most of the hard work, right?

So you have to think about it from their perspective.

But again, the problem from my perspective is.

It's so much vague. And there is no real clarity on what can be done and what cannot be done.

And if you ask different people, you might actually get different opinions like, for example, as our CTO of the company, Jon might have a completely different perspective.

And I'm not sure how much of it Jon says that HashiCorp is in the right in doing this; I mean, will he actually approve – I mean, will you Jon approve us using, you know, Terraform in our products?

Jon Mort:

This is a really good question, alright, because we have… We have as part of our kind of standards, we have a list of approved licences, open source licences that you can use as long as you meet the requirements in them, but without any conversation with anyone.

These are the ones, let's call them blessed. These are the blessed licences. You're good to go with these; anything under those.

The BSL is under that: "You must talk to someone before you consider using this”, because it's not a no – it's not a hard no – but it's also a… Oh, you've got to be careful here because we need to work this out and understand it and get to where it, like, you know, look at the legal review and do that and all, all, all of that sort of friction and things.

And we're gonna have to do that with BSL as it is and have a look at our organisation and what we're doing.

And do we compete now? All of those things…

Jobin Kuruvilla:

Yeah, I mean, I totally understand that. But from where I'm sitting, if I'm starting to work on something completely new, or my team is starting to work on something completely new. I don't want to wait for corps approval or legal approval before I start working on it. I need to start working on it right now.

So if I have an alternative, should I go for that, or should I wait for this? Well, that could take weeks. I mean, we want to move fast.

Jon Mort:

Yeah, it's a. It's a huge friction point. And I think this will be repeated in organisations worldwide; this is gonna start to be, start something which is there – or you continue to use it under the MPL, and you keep, and you keep going with that.

So it's… and you're stuck with the versions as it is. And you don't get security updates or any feature updates and those kinds of things.

Jobin Kuruvilla:

And it gets even worse. I mean, it's okay. At least when I'm starting to work on something new, I have a choice that I can make. But what if I had been working on this thing for the last two years, and then suddenly, this happened?

And now I'm like, after the fact. I mean, what should I have done?

What? What have I done? You know?

What will I do? Nuts, I mean, should I move to another product? Or should I continue using Terraform, knowing that HashiCorp can call me a competitor? And maybe you know, sue me, who knows?

Rasmus Praestholm:

It's crazy.

And there's not enough drama in here; you guys make too much sense. This is open source.

It's drama land, it's time!

So… I want to throw in a little bit of background information for those who might not be as experienced in the nuance of all the backstory and background of this kind of topic and open source in general. Just as our resident open-source purist and nerd on the topic.

So, pulling it back a little bit.

You know, the open source ecosystem was a happy-go-lucky place for a long time, where you had some, you know [...] nerds, and some really like, yeah, do whatever, including commercial. And there was some friction, a little bit back and forth, and all that, but it was generally a happy place.

But then the thing happened here a few years ago, when Elastic and Mongo, you know, big projects out there, so their source get used by somebody like AWS, who hid behind this loophole, where, if they enhanced this software that was open source, didn't release the changes because the licence didn't require them to do so if they were shipping source code.

However, AWS was offering a service online.

Which is like… hang on? You can't come here and benefit from all this open source and use a loophole out of getting to get out of contributing back to the community when you've made valuable contributions.

And Mongo, Elastic and others started down the path of more business-friendly licences that were, at the time, it seemed like it was inspired by, you know, justice – in the sense of, you can't just go here and take the stuff and not contribute back.

But there's a long path from back when it was so like a small little thing to where this BSL thing comes in, and I believe Maria DB – AKA, used to be called MySQL, started that, getting a little bit worse, because the original like a fix to this was to say, hey, AWS, you can't use our things without contributing back.

That's the really brief version of it. That sounds reasonable. Okay, fine, fair. But what happened was that it may have opened the floodgates with some not-so-clear intent, much like HashiCorp has now stepped into this minefield of intent.

The licence they are using, the BSL, was come up with by the MariaDB project because, again, they wanted to be able to say, hey, we made this; we spent a lot of effort on this. We want to be able to have at least a little bit of a moat of, you know, safety around being able to… We want to be the ones that monetise this primarily. But we still want to benefit the community and all those kinds of things.

That made more sense when it was just database software because it's kind of like, yeah, it's via software. It's a database. If you want to use that database, you follow those terms. If you don't, you don't. It's fine. It's a database.

But now HashiCorp got into it with something as broad as Terraform and Vault, which gets used all over the place in all kinds of different versions, and so on.

Suddenly this licence that, in theory, sounded simple, like you can't make a competitive offering.

That's easy to understand with database software.

But when it's like the underpinnings of the modern DevOps organisation in using Terraform to manage infrastructure, that infiltrates everything.

And that's the bit of background on that, like, why, this is different in how we got here.

Jobin Kuruvilla:

Yeah, that's a great background. And again. What makes it more complicated is that HashiCorp is already monetising it with the help of Terraform cloud and Terraform Enterprise. So it's not like they are not monetising Terraform.

So the question then becomes, anybody else who is using Terraform is now directly competing with Terraform cloud or maybe, you know, Terraform enterprise, right? So that is where it becomes so vague and wishy, so ho, How do you actually say you're not competing with them while using Terraform?

Jon Mort:

Yeah, it's like, is anybody with a who does any Terraform state management in the sites is that it? Is that it? Is that the target? If so, that's a lot of people. One of the things that I've been trying to work out is who the target is of the comp competitors. Like as Rasmus was explaining, like clearly, AWS acting, probably, you know, possibly in bad faith with things like Elastic in particularly hosting their own, modifying it, and not, I'm sure, contributing back, so that defence is kind of defensible.

But I can't see who it is that this is different. This piece, the other kind of big distinction, I think, between the other examples and HashiCorp is that they're all like services that you run like you run a database. They're all, let's call them all databases for the sake of this – it's something that you run and operate and keep that.

Terraform isn't – so Terraform is something you modify heavily, you integrate with. You've got providers.

Rasmus Praestholm:

It's a language.

Jon Mort:

Yeah, absolutely. And it doesn't seem to fit that as a way of working. It doesn't seem to fit the intent or the underlying reason why you'd have that as a license. I could actually see how some of the other services that Terraform, sorry – that HashiCorp has could fit under the BSL – maybe you could license Vault, no matter under it. But Terraform doesn't, conceptually, seem like it fit particularly with this Terraform cloud there, as the secondary, the paid-for service.

Rasmus Praestholm:

And that's been one of the main things that are brought up in the crazy discussions happening right now, in that HashiCorp already has a proprietary product related to Terraform, which is Terraform cloud, which is a server-side thing like you could do with something like Vault.

That makes sense, but they already have that potential profit centre, I suppose.

Maybe it's not doing well enough.

But Terraform, the language, the CLI, it's like, huh! There's a lot of, you know, not great understanding going on about this, and I suspect that it's got to have something to do with HashiCorp looking at the past examples of this, Elastic Mongo, and so on.

They switch licences to prevent a given usage.

They probably meant something relatively mild to help try to protect their products and potential for profit. But then they didn't recognise…

Wait a minute. This like opens up Pandora's box of just weird interpretations and potential future bad faith. And it's just… It's crazy.

Jobin Kuruvilla:

Yeah, it goes back to whether it is actually a product or a language in itself, right? And I mean, it's not just side, I mean, talk about consolidation. Talk about container orchestration, and you can see Kubernetes is sort of kind of the standard now in all things container orchestration.

Similarly, Terraform was becoming the standard regarding infrastructure score and how you write in for 6. Right? I have been in so many arguments where people were weighing up cloud formation versus Terraform and finally ended up on Terraform because it was open source, one, but also because, you know, it was cloud agnostic, and you could actually connect it with GCP, Azure, AWS, whatever.

So there were so many positives to Terraform – now that it is BSL, how would those discussions go? And I'll probably throw it even back to Chalky – He, in one of our podcasts, actually mentioned that you know, we should be using Terraform for infrastructure and Ansible for configuration management.

But will we go with the same attitude now that it is BSL, right? That is always my question/

Rasmus Praestholm:

Yep, and it's to pull an almost absurd example out.

This is almost like if Google were somehow able to wrest back control of Kubernetes and licenced it, and Kubectl, with the BSL, was like, no, we want to make money off this now. Wait. What? 'Cause there just been years and years of community engagement and activity?

I don't know how much contributing back there has been of, you know, Terraform bug fixes and so on from the community. I don't know how hard it is to do with it because in projects with a strong central vendor, sometimes it can be difficult to contribute back.

But contributions have happened.

Tons of people have staked their well, maybe staking their careers on Terraforms a bit much, but nonetheless, they went in it with one impression of what the expectations were and just had the rug pulled out from under them.

So I think there are a lot of hurt feelings out in the community right now.

I don't know if HashiCorp intended it or if they just waded into it. But it's something else, that's for sure.

Jobin Kuruvilla:

And to that point, I mean about the intent. I mean, obviously, the intent is probably clear. They want, you know, to stop people from using Terraform for their profits. And whatever you know, and that's fine.

I mean, the intent is clear. But again, how vague it is makes it all the more complicated. Right? That was this thing about BSL that they were saying. It's vague because people wanted, you know, competitors or other vendors who are using Terraform to reach out actually to go on a case-by-case basis. So they could actually vet it.

Yeah, you are okay to use it. You're okay to use it. No, you are not okay to use it because you are using it for your profits, you know.

You're making. You're monetising it, right?

So that was the intent behind it. You know, making it. BSL, that was an argument that I found yesterday.

It makes sense. I mean, that is probably what they meant.

But again, as Jon was saying, we don't know who the competitors are. Who are they actually after? Right?

That's where it can get a bit tricky. Are we the competitor? I don't know. I mean, we use Terraform a lot when we create infrastructure for our customers. Right? I know that a lot of the vendors that we work with like, for example, whether it is GitLab or Atlassian.

All of our partners use Terraform quite a lot in their tool stacks. Right?

So are they actually the competitors?

How will? When does GitLab take this particular development?

Will they continue using Terraform? So there are a lot of questions out there that are unanswered right now.

Rasmus Praestholm:

I think the GitLab thing is a particularly good example because they have a Terraform state option in GitLab software.

That means I'm pretty sure that they would count as a competitor.

So can they no longer use that?

And if we are using GitLab for Terraform state management while we are consulting for a customer, can we not use it?

It's so, so messy.

And another thing to go back to... Jon pulled out some stats here that a lot of developers have contributed to Terraform Core. But also, there are hundreds of Terraform providers.

You know, the ecosystem around Terraform is wide. Now my understanding is that's not affected because the providers are licenced differently. But the open-core model, which I do support. I like it when there is, like, we built an ecosystem here. We've opened the core. It's freely licenced so that you can go, you can go nuts, and then build things around it.

You can build all the plugin extensions and add-ons, and those can be licenced differently. And you know, maybe some people make a bunch of things for free. Maybe some make it paid. That's great.

So essentially, the central vendor is doing this, the cultivating of the overall ecosystem, and then everybody can play with the stuff around it. But in this case, HashiCorp is saying, like, you know.

We want to control all of that, even though technically they don't for the like, the providers.

But if they control the centre and can say who is or isn't a competitor, that must have a paralysing effect on the community.

Jobin Kuruvilla:

Yeah. And speaking of control, I mean, that's, again, another big thing, right? Actually, the GitLab CEO, he actually came out and said, What is more concerning is not the chance right now that they don't view us as a competition today; they might actually see us as a competition two years from now; what happens then? Right?

They can actually move the goalposts. And that is the problem with the BSL licence. And that is not clear terms that say, okay, as of today, what we're working on is a competition. Maybe tomorrow it's not so that that is again the problem.

So how are they going to control the other vendors or others who are using the product?

Rasmus Praestholm: Yep, and it's gonna be really hard to justify investing in Terraform as a vendor or a provider or whatnot like we've been using Terraform in our own products, and we suddenly have to worry. Hey, wait a minute.

Can we do this? And even if we can now, what if we build a whole business around it? And then I said, two years down the road HashiCorp comes. “Sorry, we launched a thing like that ourselves, so no, you can't use your thing any more”.

And yeah, that's rough.

Jobin Kuruvilla:

So speaking of that, I mean, what do folks do? I mean? Will they continue using the current version of Terraform or fork it, and probably start making changes to our own version of Terraform going forward?

Jon Mort:

Yeah, I think this is a great question, Jobin. Where'd you go? So Gruntwork, who built up a lot of community products around Terraform – we use Terragrunt in places, and there are some other tools as well that work with other things. So, that is leading the charge on a community-owned fork called OpenTF.

Which I think has got some legs. And I think that the key thing there is that it is owned by a foundation, which is, this was the really important thing that Google did with Kubernetes, was put it with the ownership of it, under a foundation which at least they have a decent amount of control over but not exclusive control. They couldn't be like that. That's taking it back, and doing something different with it regarding licensing would be quite difficult. And that kind of that coming back, coming, coming together as a foundation for the community fork, I think, is something that's interesting. It's gonna be interesting to watch where that goes.

Jobin Kuruvilla:

Yeah, I was actually looking at the opentf.org, I mean. The first plan is actually to go back to HashiCorp and say; Please continue making it open source, whether it works or not… But at least, you know, the second, the fallback plan of, you know, having an open fork of Terraform, that could actually throw HashiCorp off of the current path. I mean, what if that foundation becomes a prominent one and not the HashiCorp version? Right? It is quite possible, especially if some of the biggest companies out there put some weight behind it.

So it will be interesting to watch, and it will be interesting to see how HashiCorp responds to that.

Rasmus Praestholm:

Yup. This is what gets my open-source nerd side really riled up again because this has happened before, and it'll happen again. And there are two parts to this. One is the example of …

Yeah. It's happened before

My all-time favourite tool of all, Jenkins, once upon a time was called Hudson until somebody decided they didn't want to release the trademark to the community. Something silly like that.

And what happened? A whole bunch of angry nerds got up, forked it, and named it Jenkins.

And guess what? Hudson eventually withered and got thrown off and dumped in a foundation somewhere to go off pretty much and die.

And that's harsh. That was big-time harsh, and the same thing could happen here. And part of me wants to say it must happen here because there is a slippery slope in effect here, where more and more of these unopen source things have been coming out.

Other companies have said, you know what we think open source means something else, or we are the visible source, or we, you can still contribute, but we own it all.

And the more companies that do that, it dilutes the value of open source, or at least it dilutes this angle that was trying to make it more of a productive relationship between nerds like myself, developers that want to do cool things, and businesses that want to solidify a tool for the long-term – they can make money off it, by all means. There are ways to make money off it.

But if you hold it too tight, you severely risk strangling it. And I do hope that there will be enough drama, honestly, to where there will be alarmingly looking forks that could become the new Terraform enough to where HashiCorp goes back and says, woah, okay, maybe we were a little bit too, like, willy nilly about what we actually licenced here. Maybe we need to have everything on it.

Because I might suggest, like – HashiCorp re-licenced Vagrant. Like, you can't use Vagrant in production anymore. And, like, wait. Do people use Vagrant in production?

What in the world? It just smells like... There were some, maybe not ideal, decisions made here.

So hopefully, they will have a rethink, and everybody will come back in the same room, and Terraform will go back to being open source. And they can come up with a better way, as others have, of having, like an open core, but then building businesses around the core – not being weird about the core itself.

Jobin Kuruvilla:

Would you say that? I mean, even if HashiCorp decides to go back and say, hey, we will continue to make it open source – should it be under the foundation? Like Jon mentioned, how did Kubernetes become under the CNCF, right?

Should Terraform in that case, continue to be under a foundation just under the ownership of HashiCorp?

Rasmus Praestholm:

I think that's probably the number one way the HashiCorp can win back its reputation at this point because, at this point, they're looking terrible here, quite honestly.

And the way it worked with Jenkins and Hudson in the past was there was a commercial interest in it. They kept it going for a few years because they hit momentum with a few customers that were signed up for the, you know, proprietary version and all that.

And eventually, they decided, you know what, this isn't working out, and they did donate Hudson back to an open-source foundation. But that was too late because the years had passed, and Jenkins was so much better at that point.

So it needs to happen fast.

I would say, within a month or 2, if they can turn around and say, okay, guys, we get it.

You know. Let's all build a foundation together and put Terraform in it. We'll pick one like the, you know, CNCF or something like that. and then, you know, we will work out something to make it, you know, viable for us to stay in business, to do something here, because that is viable somehow. But this doesn't seem like the way.

Jon Mort:

Although it does seem to be the sort of the prevailing trend of where many organisations are going like, there seem to be some parallels with what Red Hat has been doing with their interpretation of GPL and things. And I think there is a real problem that needs to be addressed.

I think licences are a good way of doing it but switching licences on the business models around open source need. You know, you were talking about moat building and things; I think that the AGPL works pretty well for organisations who provide, who maintain an open source version of their software, of the up of their the product. They allow other people to run it and to, you know, you can use it. The things you and you can even provide a service based on it running and charge people to do so, but you cannot modify it.

And then and then not provide the modifications back upstream. And it's that community element of it which is the importance. You. You need to have that defensibility.

Because you know there is an investment going in. You know, there is. Yeah, there are developers. Time isn't cheap.

And if you've got a business that's supporting something that needs to be that needs to be sustainable.

Rasmus Praestholm:

Yep.

Jon Mort:

So there's a genuine problem here to be solved.

Jobin Kuruvilla:

And I think most recently, I think we have probably seen the same issue with the Spotify Backstage. Right? I mean. They're open source. But at the same time, a lot of the other companies are probably monetizing off of Backstage. So they are also. Still, you know, figuring out how they can actually monetise Backstage, right?

Again, licensing can be an issue. But again, this could be a good eye-opener for them. Right? What happened with HashiCorp and Terraform? So, definitely something to watch out for.

Rasmus Praestholm:

Yeah. And I think Spotify did it well with their premium plugin bundle, for instance, which is like Backstage. Again, the core is free, and we've made these cool plugins over here on the side, and you get to pay for those, like, OK. That's completely reasonable.

Jobin Kuruvilla:

So going back to the number of years, I mean, is that actually a factor? I mean, what if Terraform had decided to change the licence in terms probably eight years back – would that have changed anything?

Rasmus Praestholm:

Huge difference. The bait and switch in the long term is really the danger here in that if you can run a thing like Terraform for nine years as open source, and you can flip on a dime?

That's terrible.

If you're doing things like you have a product, and you put it in pre-release or beta for free online, and like you put a little disclaimer, “hey? We don't know how we're going to monetise this. So just, you know, be aware that at some point when we go live with this, we're going to probably come up with some sort of monetization model”, like oh, okay, yeah, sure, that's fine.

But that's just not how Terraform was presented. And there are so many open source products out there like it that you begin as individual contributors and companies. And to, you know, believe the model of yeah, okay, so we can use this, and of course, we'll try to be good citizens and contribute back.

But it doesn't just disappear nine years later. That's the harsh part. There is a way to make that work, and I've seen it in other projects, depending on how the licence, the copyright, or whatnot is held. It may require every single person who has ever contributed to sign off on a licence change that helps. And you can find a middle ground somewhere – like all active contributors within the last four years, you must sign off that.

Yes, the licence can be changed – but if they don't, you're not changing it.

Jon Mort:

Well, I think well, there is, but I think that a lot of that is now moot with contributor licence agreements like that you have to sign in order to contribute to projects, which effectively hand the copyright of your… so you keep the copyright of your change, but you licence it to the the owner of the project to be able to do what they see fit with it. And I mean along with it as well. There are things like patent grants and indemnity, and all that, that kind of thing.

But one of the things that it does do is it opens the door for that to be so changed. And it fixes the problem that Rasmus was describing as a solution to this. So some of these, some of these things. But it is something like that, you know like a sort of a CTOs or CIOs looking at policies need to be thinking about is, but when we're looking at approving a piece of software for use or if particularly if it's core to an offering of that, we've got – how strong are the protections around that piece of software? And can we rely on it in the future?

Because something like this happened. Now, there's like, as I feel out, there's a bit of trust being shaken as confidence is, and then removed. In this whole kind of open-source area, I think it will be difficult to build back.

Yeah, I think definitely, that is lessons to be learned, even for companies like ours, you know, who might actually be putting you out open source software out there? Right? I mean, what's the licensing that we should be doing? And what is the long-term goal here? A long-term plan here; all of that needs to be taken care of. And to be honest, you know, to summarise – I don't think HashiCorp, was necessarily being "evil” in this change of licence, the whole thing – but they probably didn't think about all the potential after effects for everybody else in the ecosystem, right?

That has probably backfired a bit…

Rasmus Praestholm:

A bit?!

Jobin Kuruvilla:

The intent? Wasn't that bad?!

Rasmus Praestholm:

Yeah, I do get that part. I mean, yeah, I get that. You want to be able to take advantage of your own work.

It's just… differently, please… not this.

I can maybe end it on a quick, semantic thing that I found useful in the past because when the Hudson and Jenkins community kind of split.

One of the arguments was – who's actually the fork?

And I think, I think that's going to come up again here with Terraform because of the community, the will of the community. This ephemeral old fluffy thing floating around out there is probably; it could very well be that HashiCorp. No, that's not the real main line any more. No, you forked – with your weird, new licensing thing. We are still over here.

We're just going to keep going like it always was. And we're not the fork.

You're the fork.

So… open source drama. Gotta love it.

Jobin Kuruvilla:

Brings you back memories of Jenkins, huh?

Rasmus Praestholm:

Oh, yeah…

So, what else do we have in the news?

Not that we should talk about nothing but fun, open-source drama.

Jobin Kuruvilla:

Yeah, I believe we'll be discussing a lot about this at the DevOps Days in London, where Adaptavist is actually hosting a booth.

So if any of you are hearing this is coming to London DevOps Days, please do visit the Adaptavist booth and let us know about it. Maybe HashiCorp, Terraform, and probably talk a lot more about DevOps and everything else.

On another note, I am actually attending the DevSecOps event in DC, organised by Carahsoft, which is the first time they're doing it specifically focusing on the government community here in the DC, Washington DC – so I'm looking forward to that.

And if I have any news from there, I will definitely, I'll be sharing with you all in the next podcast!

Rasmus Praestholm:

Well, that might be it, this one – this episode of the newly rebranded Rasmus's open-source drama broadcast – and hopefully. We'll see you next time.

Insert usual text here about "follow us on all the things" and see what else is going on these days – and maybe the podcast will be back to the same name next time I dunno. Maybe, you know, we'll see.

Maybe I'll change my name by rebranding it…