Douglas Gubert, Rocket.Chat

Thank you for tuning in to today’s episode of the Breaking Changes podcast. I’m your host and chief evangelist for Postman, Kin Lane. With Breaking Changes, we explore topics from the world of APIs, but through the lens of business and engineering leadership. Joining me today we have Douglas Gubert, tech lead at Rocket.Chat. Douglas reminded me of the diverse types of APIs that exist out there, sharing Rocket.Chat’s approach to extending their open source messaging solution, and they left me thinking about what really matters to our API consumers when it comes to the APIs that we deliver. I like to start really simple, the basics. Who are you and what do you do?

All right, so my name is Douglas Gubert. I am currently the tech lead for the Rocket.Chat Apps Engine team, and we are working on the plugin framework for Rocket.Chat. I’ve been in the industry for like the past 10 years already, doing web development, going from websites, e-commerce, this kind of stuff, and three years ago I landed on Rocket.Chat to work primarily with back-end services and working on this kind of, more like a framework really, for other people to build upon. It’s been a very exciting journey so far.

Yeah, it’s an interesting journey and an approach. What stood out to me is, I would say, there’s a little more to Rocket.Chat than I think with other commercial services. So how would you describe how Rocket.Chat’s different in what you do than what we can find with other chatbot automation kind of services?

Sure. So Rocket.Chat, we are an open source chat platform, and so what we differ from others mostly, like these big players like Slack and Microsoft Teams, is really the possibility for our customers, our users, to deploy as a self-service, right, inside their own infrastructure, controlling all the data, whatever comes in the network, whatever goes out of the network. And this data sovereignty is something that is really our core preoccupation, let’s say, recently, something that we are focusing on, in order to give back the control to the communities or companies or whoever is running their chat platform. Given that chatting is something that is so sensitive to a team that is working together, we really don’t want to risk that being accessible outside of this application or outside of the scope of our teams. So this is really our big focus for now.

Yeah, I mean, so data sovereignty is, I think, an increasingly important phrase for a lot of folks, whether it’s, if you’re in Germany, if you’re in Canada, UK, well, there’s a lot more, but you said control. And so in that context, is control over my data, control over my conversations, what we’re talking about? But you’re also, it’s open source, so it’s openly licensed, you can run it on-prem, another level of control, but then you also talked about kind of the scaffolding and framework to customize it and make it your own, so that would be another level of control, would you say?

Exactly. This kind of goes with the idea of making your custom workflows. So for the framework we have, mostly, not mostly, but we have two main ideas that we can leverage or enable with the framework. The first one is empowering companies or whoever is using Rocket.Chat to really develop a plugin, what we call a Rocket.Chat app, that extends the capabilities of the server or of the chat to match their own workflow. You will most likely be working with other platforms, and you’ll most likely be talking about the progress that you see in other platforms in Rocket.Chat, so what we want to do is make it easier for you to add a bot, or maybe not even a bot, more something more automated, like directly into the server, like responding to file uploads, responding to specific message content, content moderation, I forget, I missed the word, but it’s like when you’re kind of controlling what kind of content, this is important for communities, content that is not safe for work to be shared. And this is something that you can do now without having to dive deep into the Rocket.Chat code base, and then having to make a custom server, custom code that will get out of sync with our open source code base. And this is really about making it simple to really customize and see the tool working inside your workflow much more smoothly. So this is the first thing. The second one is really using our Marketplace to distribute these plugins to the wider community. So we have, for instance, some apps that integrate with Giphy or integrate with Tenor or other services like this to enhance your experience with slash commands, being able to make it easier to share GIFs, or making it easier to integrate with Atlassian products is something that we have apps for that too. So it enables our community to use their own creativity to expand the experience that the other people can have, and distribute this either for free or, you know, in a paid subscription model, and to reach the wider community that we have, like, I think, less, last I checked it was like 10 million servers at least deployed currently. So yeah, those are the two main focuses we have on the project.

Yeah, what caught my attention is, we had a brainstorming conversation after we first met and I’m always looking for the API story. And I would say you have all the characteristics of an API, but you chose deliberately for a high-performance in-memory plugin approach to extending the platform. But for me, the actual API story here is, is that how you process, but also the fact that I can now connect with a third-party API to augment and add some capabilities to Rocket.Chat. So for me it was a multi-dimensional API story that I found pretty interesting.

Yeah, it’s definitely one of the most unique projects I’ve ever worked on.

So why, talk to me a little bit more about the architectural decision-making that went into the plugin framework, because I think your experience and your vision, it seemed like something others can learn from when it comes to maybe not a chat app but any type of implementation. Because I’m not prescribing that web APIs are the way to do everything, and I think a lot of people think that I like APIs, or any sort of extensibility and standardized approach for how we allow for the extensibility of our applications, our platforms. So can you share a little bit about the architectural decisions that went into this?

Yeah, so just like you mentioned, we kind of went into, let’s say, a not very common path when we were idealizing the framework, which is that we run plugins inside the server. And so this is mainly because we wanted our on-premise clients and users, and even the ones that run air-gapped environments, so Rocket.Chat not communicating over the internet but only in the private networks, we wanted them to be able to do this approach as well and use the plugins. Because, for instance, on larger environments what we can have is Rocket.Chat running in the network, Jira running on-prem inside the same network, Confluence and other products from Atlassian, or for instance Jenkins, like other tooling, the platforms or products that are restricted to this scope of private network, nothing goes out, and if anything has to go out it’s very tightly controlled. And this enables us to do that, because we are not making the server send requests to an external web service that is running anywhere on the internet, sending potentially sensitive information such as messages, such as files, to the outer internet. All of this can be done, we can integrate Rocket.Chat with Jira, with Confluence, with Jenkins, inside the same network, via the app running inside the memory and the runtime of the server. This does come with a lot of different challenges, but we saw it as a really powerful way for us to expand on the current experience for the product.

Yeah, I mean, I think the dimensions of control we talked about earlier really speak to that, and it’s an interesting view of the API landscape, because operating on the web is very beneficial in a lot of ways, there’s a lot of collaboration in real time, but in today’s world, that’s not always what you need to be doing, and you want that level of control locally, you want to have the flexibility of extending it and getting people doing what they want to do, but that full tight control over what the network means is really important for a lot of companies. So when it comes to performance, what other considerations, like what protocols, what does Rocket.Chat use when it comes to a communication protocol?

Oh, sure. So I think internally, when we’re talking about communication between the Rocket.Chat apps and the core product itself, it’s not really a protocol, it’s more like direct calls, what we call Bridges. So they instantiate the code, your app that is being loaded, and via some sandboxing strategies we interact with this code and we get a return. And so this is really happening in memory on the runtime. So this is what happens most of the time when it comes to the plugin framework. Something that is interesting to mention is, Rocket.Chat is built on Meteor, and so communication that we see between client and server in order to keep the messages, this real-time connectivity via WebSockets, this is then controlled by DDP, which is Meteor’s own protocol to communicate between the server and client, and this is where a lot of its magic goes in, because it makes for a really smooth developer experience in the platform. So yeah, I think this is the most different thing that we can mention. Other than that, it’s mostly HTTP requests sending JSON data all over the world whenever we want to communicate with others. I just recalled one of the examples of an app we have, which is ClamAV, which is an antivirus server, and we actually connect to it in an app via this framework that we developed, we connect to it directly via TCP. So this is something many communities do right now, which is, you have a Rocket.Chat server, a ClamAV server, and then Rocket.Chat passes all file uploads through this server to make sure there is no virus, that server does this verification on the binary contents of the file, and then it says, okay, this file is good or not, like, block this, don’t let it pass. So yeah, this is another cool thing we could achieve.

Yeah, well, I think that’s what kind of caught my attention, protocol-wise, because a lot of people have their blinders on when it comes to HTTP and REST, or just event-driven, TCP and Kafka or WebSockets, and so you all went with a pretty nice spread. I think the in-memory approach for the plugins is important, because that’s some serious performance there, but then I think you intelligently used a variety of approaches that met the need or the problem you were trying to solve, not riding the latest trend or whatever that you might see sometimes out in the space. So with your plugin Marketplace, can I make money there, selling access to my plugins, or is this just an open source plugin Marketplace?

Nope, it is possible to make money, definitely. Right now we’re kind of, it’s not available like on the get-go, if you register for a developer or publisher account we need to enable this functionality for publishers, but we usually have a quick conversation with them, like checking the app, checking, since it’s going to be a paid app we need to make sure there’s going to be support, we need to make sure there’s going to be an issue tracker, we need to make sure privacy policies and the terms are like, you know, a standard level for us to accept this entry on the Marketplace, and then we can allow this to be published and accessible to our customer base. So yeah, monetization is possible.

Nice. So you mentioned several times, you mentioned Jira, you mentioned your local Git, you know, you can do GitHub, GitLab on premise, so you were mentioning pretty developer tool chain kind of solutions there. Do you see Rocket.Chat as a pretty industrial-grade feedback loop mechanism for the software development life cycle within the enterprise? Is that something we see a lot of?

Not sure I follow, but, you mean like it being more focused on developer experience for now?

Well, like, being used by people as the backbone for how they develop software within the enterprise, like, this is how developers in an enterprise organization, because you mentioned the Jira integration, so I’m assuming it’s integrated with Jira ticketing, you mentioned Git integration, so it just felt like it was a favorite of developers to kind of be that communication or feedback loop as part of their work.

Yeah, exactly, this happens, you know, a lot of our customers. I don’t think it is more of the focus for the product itself and the framework, but it’s more like these tools, they fit this use case a lot more, right, than other, let’s say, HR applications that you might be using with your company, where the chances of this being on-prem is, like, I don’t know, close to zero. But these heavy use cases where we kind of have this more advanced setup, then sure, it really fits well with the rest of the tool chain usually employed in software development industry. But it really is, coming from the open source perspective, we have a lot of other kinds of software, like HR, like some kind of analytics and other tools that are not developer-focused that are now being developed in the open source world, so it would be possible as well for those to be integrated in this private network scheme. And also, of course, it is possible to connect via the internet to other applications, but like all the other solutions out there that enable you to do that, since it’s something that is the common state for other solutions as well, I guess it’s less interesting to dive deep into.

Yeah, agreed. So regarding your roadmap, what’s top of mind for y’all as far as building for the community for the next year to five years, what’s important?

So one of the things we’re keeping in mind now is making interaction with the UI more powerful. Right now our Rocket.Chat apps, they can interact with the UI to some extent, they can pop up, they can open up modals, they can interact with the sidebars, they can interact with the messages, like putting specific actions in there, but we really want to expand on that and enable apps to put content on separate pages, and going over to maybe enabling some kind of iframe integration to allow them to show external content with some context provided by our tool. So this is one of our biggest focuses, which is really making this experience more customizable for the app so that it can feel more native to the Rocket.Chat experience itself. And yeah, this is one of the coolest ones I think. And I think the second one that is cool to mention is, we are looking a lot for ways to improve our developer experience, which is something that is really important to us. Like, not only do the community developers work with our framework to make the apps and extend the functionalities, but we also do that with some of our own apps that we publish, and yeah, we’re always looking into ways to make our lives easier on that front. So for instance we have a CLI tool that scaffolds out your app, the first app structure that you need, and scaffolds out other components that you might need. So if you want your app to have a slash command, you can scaffold that out, you can generate the boilerplate via our CLI and then customize the file later on to your needs. But having this, you know, it scaffolds out the structure, of course you’re able to customize that, but we have our recommendation on how to do it. And I felt personally, on other projects, like starting a project with no structure, you don’t know the best practices or the how to approach that, it can sometimes be a blocker. So this is one of the things that we really want to remove the blocking from, like, let’s make it easier to develop things and for the app developers to really focus on features. This is, I think, one of the models of maybe all frameworks that we see out there, like, don’t forget about the details, focus on the functionality, focus on your features, and we really feel it now that we’re working on our framework ourselves.

Yeah, that’s a huge one too, because that’s like the foundation for the vocabulary that they’re going to use in their application and how they’re going to communicate, the automation around that. So getting that right and not getting frustrated out of the gate is going to pay off big time down the road.

Definitely, yeah.

So where do you go to stay up to speed on the latest tech in the space, how do you keep your knowledge fresh?

Oh, this is a good one. I actually subscribed to a newsletter from X-Team, the company, not sure you heard about it?

No.

Yeah, they’re an outsourcing company, they have a lot of developers, but the thing is that in their newsletter they share a lot of articles on things that they have found during, like the whole team, I don’t know, hundreds of developers, they’ve found out and they want to highlight. It’s a weekly newsletter, and this has been a very good source for me to keep up with what’s coming out new in the environment. I think the latest thing I stumbled upon because of this newsletter was a new framework for Node.js, which is Redwood.

Sounds cool, I’m checking it out.

Like, we’re always checking out different frameworks to, you know, get inspired, like, okay, how do they solve this problem, how can we adapt it, bring it to the Apps Engine framework so that we can make cooler stuff, etc. So this has been a very important partner, I gotta say, on that front.

Yeah, no, that’s exactly the type of sources I’m looking for, that’s a regular question I ask folks, because I’m always just intrigued by how we all get our knowledge, and then sharing that among other folks, it makes it even more powerful. One last question for you, what’s your cat’s name that we saw in the back there for a little bit?

Her name is Ellie.

Ellie.

Yeah, I’m inspired by Pocoyo.

Ah, it’s a kid’s cartoon.

Yeah, yeah.

Well, I’m always fascinated by people’s animals, especially in this COVID age that we live in, or I wouldn’t even say COVID anymore, but virtual, Zoom-based age we live in, everyone’s animals coming in and out of the screen is a common one. So, well, I appreciate talking with you today, I really enjoyed learning about what you’ve all built. I think it’s pretty fascinating, a look at not just open source but the extensibility of it all, the thought you all are putting into your developer experience, I think is just a good lesson to think about and learn from. And, like, your newsletter, I think that hopefully this podcast for other folks helps them think through some interesting concepts. I appreciate you coming and sharing.

Yeah, thanks for having me.

Thanks again to Douglas for stopping by. You can find more on him on LinkedIn, and you can visit Rocket.Chat at rocket.chat. You can subscribe to the Breaking Changes podcast at postman.com/events/breaking-changes. I’m your host Kin Lane, and until next time, cheers.