Solving Twitter’s Bot Problem With Code

Devvret Rishi:

Thanks, everyone, for allowing me to present here today. What I wanted to walk through is how we could solve Twitter’s bot problem in less than 10 lines of code. Now, the title is a bit of a way to make sure people get interested, but it’s actually going to also relate to the demo that we’re going to do later today. The core thing that we’re going to cover is how to do multimodal deep learning the easy way, using an open-source framework called Ludwig that makes it very simple for anyone to develop their end-to-end deep learning pipelines. Let me start off by telling you just a little bit about who I am, so we can actually give a bit of shared context for why I’m interested in talking about this.

Devvret Rishi:

I’m one of the Co-founders and Chief Product Officer at Predibase, which is a low-code AI platform built on top of Ludwig, our underlying open-source tech. For the entirety of this presentation, as much as possible, I’m really only going to be talking about Ludwig and the open-source side, so we can make sure everything is reproducible. Prior to this, I was also a PM at Google, spent time on a number of different teams. I started off on Firebase, which is a developer product. Went to Google Research on productionizing ML research. Then, over to Vertex AI, which is GCP’s machine learning platform. But the part that’s most relevant today is that I was also the first PM for Kaggle, a data science and machine learning community with over 10 million users today. That’s actually going to be some part of where we see the bot journey begin.

Devvret Rishi:

In another life, my academic background was also in computer science and statistics. I did my master’s in machine learning, but pretty quickly sold out and went to product and really excited to be able to show you a bit about what we’ve built from there.

Devvret Rishi:

Now, it might feel like it happened a century ago, but just about last year, Twitter bots were all over the news and one of the main reasons was that there was an M&A activity happening at the time where Elon Musk was interested in being able to buy the platform. It was really how hung up on this question of how many of the accounts on Twitter’s platform are actually bots, and can we actually detect them in any sort of easy way? This really highlighted and brought to the forefront an issue that I think has been active on the open internet for quite some time.

Devvret Rishi:

Today, a third of the most popular websites on the internet use captures as a way to be able to stop bots from proliferating on their platforms. But if you’ve spent any time trying to get a ticket for a popular event on something like Ticketmaster, you know that those attempts have only been sort of successful. Bots tend to still dominate online, and especially with some of the rise of large language models and generative AI, I think that they’re something that’s going to be at a risk for really proliferating in a larger way.

Devvret Rishi:

Now, just about four years ago, when I was at Kaggle, I saw this firsthand. For those of you that don’t know, Kaggle is a data science and machine learning community plus an ML platform. Essentially, it has a community of over 10 million users who share discussion topics, compete in ML competitions, share datasets, models and code together. There’s a social progression system whereby someone can start with a novice, graduate to be a master, grand master and so forth. About three-and-a-half years ago, we started to see a lot of community discussion and discontent around the idea of how many bots there just were on the actual platform.

Devvret Rishi:

Kaggle provides its users a lot of different free resources, whether that’s compute, storage or others, and so there’s a lot of incentive essentially for people to be able to create their own bot accounts and use it to either spam or abuse any of the resources that were being offered for free. If we take a look at a standard Kaggle profile, in this case, Anthony’s, we can see what makes detecting bots on Kaggle’s platform or really any platform a challenge.

Devvret Rishi:

You have a really rich set of data for any user profile. When you start off, you might have something like just their name and title. Anthony, for example, was the CEO of Kaggle. But then we can start to get into even more interesting data sources. We might have something like a profile picture image. We might have numerical data, like where they placed on rankings in different competitions, how many medals they have. We would have categorical data, which is like if they were a gold, silver, bronze. We even have unstructured text that they may have written for themselves in their bio description, and this can really be freeform and anything that a user may have written directly.

Devvret Rishi:

Now, in my experience, this kind of diversity of data is a blessing and a curse. It’s a blessing because it’s something that allows us to have a lot of rich signal from which we’re able to go ahead and extrapolate whether or not this user is actually a legitimate account or a bot. But the curse is that working with this many modalities of data in a machine learning setting tends to be relatively challenging. In fact, a lot of the teams that I worked with when I was with Vertex AI would take real world data like this and strip it of all the images and texts and the unstructured data and just start their first model on structured data. Because it’s a lot easier to do predictions with let’s just say numbers to binary classification rather than incorporating in text images and a lot of other types of modalities.

Devvret Rishi:

But one of the, I think, key things you’d realize if you did something like that on Kaggle is that you’d be throwing away a lot of the value on the table. What we actually saw when we published our report on our bot detection program at Kaggle was that one of the most predictive features inside that user’s profile was the bio that they wrote, and it was particularly predictive when you use it in the context of what Kaggle was as a machine learning community. An example, a real example of one bio from a user that we determined to be a bot, was someone who wrote in their bio that they’re a personal injury lawyer. They help families and individuals in serious injuries and wrongful death, and they work on many different types of cases.

Devvret Rishi:

Now, in many different types of forms, this is a very legitimate bio. If you were, for example, on a forum for legal professionals or anything that’s career-oriented, I think that you wouldn’t just willy-nilly essentially mark this as a bot account. But when you take into the account of what the unique user profiles look like on Kaggle and mix that in with the other contextual data about this user, for example, how long they’d been on the platform and what their social interactions would look like, that’s where we really started to combine this type of context to be able to understand whether or not this user was a bot or a human. That meant that we needed a generic model that could take into account many different types of modalities, text, images, structured data, and be able to come up with a very specific series of predictions that matched the business needs of the unique context it was in.

Devvret Rishi:

Now, the way to be able to accomplish that is through deep learning. Deep learning to be able to handle multiple modalities of data essentially and be able to do computer vision, NLP and structured data tasks all at once inside of one platform. That’s ultimately what we did at Kaggle, too. We had a team of expert data scientists that essentially built out these models and started to productionize them. But if you’re like me, you want to see some of the benefits of deep learning without having to write the hundreds or thousands of lines of TensorFlow or PyTorch to actually be able to put this model into “production”.

Devvret Rishi:

I don’t want to spend any time debugging an optimizer or trainer. I don’t want to spend any time debugging a loss function. I have a high level sense of the types of goals that I’d want to be able to achieve and the types of customizations I want to get started with. I essentially want to be able to figure it out from there, without having to write the extensive amounts of code that these projects tend to take. That’s where we’re going to introduce Ludwig.

Devvret Rishi:

Ludwig is an open-source, declarative machine learning framework. It’s become relatively popular in the ML community as a way to be able to develop end-to-end machine learning pipelines that go from data pre-processing to model training, all the way to predictions, without really having to write much code. If you wanted to, you can actually go ahead and see a lot more about this just at our website, ludwig.ai. I’ll show you a bit more about that in just a couple minutes. Now, to tell you a little bit about where Ludwig came from, I can tell you a bit about the story of Pierre’s experience building machine learning applications at Uber. Pierre is my Co-founder and the Author of Ludwig.

Devvret Rishi:

Now, Pierre worked on a handful of different projects while he was at Uber. He first started off in intent classification system. Took him about 1500 lines of TensorFlow code, several months to be able to develop, debug, and perfect, and then several more months to be able to deploy. His reward for finishing that project was to work on a fraud prediction problem where it essentially took him 900 lines of code, several months to be able to build, and then several more months to productionize afterwards, and so on and so forth. Now, Pierre likes to say that he’s a lazy data scientist and he didn’t want to have to keep reinventing the wheel, writing all this code from scratch each time. He thought to himself that there must be a better way.

Devvret Rishi:

What he arrived at was essentially what became Ludwig, a declarative, deep-learning framework that now is built on top of PyTorch. Ludwig at its core is really two things. It’s an interface that makes it really easy to be able to build machine learning models using simple configurations rather than having to write raw, low-level code yourself. And it’s an underlying model architecture that, I think, is really clever and flexible in terms of the types of tasks that it’s able to go at and handle. If we start with the interface, the key idea is that instead of having to write, again, 1000 lines of code, you can go ahead and get started just by writing a very simple configuration file, and this configuration file makes it easy for any user to get started.

Devvret Rishi:

The left configuration, the six lines that you see there, essentially correspond to the 1500 lines of code that initially were written for this intent classification system. This six lines of code will set up your entire end-to-end ML pipeline, all the way from data pre-processing, the model training, gradients boosting, and then finally to the actual predictions. But the really nice thing about it is, anyone in machine learning knows your first model is usually not your last. While you get set up very quickly and are able to start with your first model immediately, you also have the expert level of control to be able to edit and modify any part of your model settings in just a single line of configuration.

Devvret Rishi:

If you want to use, for example, a BERT model to be able to encode the text features, you can do so in just one line by saying encoder BERT. If you want to set a loss or regularization, you can go ahead and do so through a series of just one line statements. You never have to implement anything. Instead, you get to control what you want and then the system will automate the rest. That’s really the key idea behind this declarative abstraction built on top of the underlying deep learning frameworks.

Devvret Rishi:

The second part of Ludwig is the underlying model architecture itself. Now, we call this model architecture Encoder Combiner Decoder, or ECD for short. The key innovation here is that every single data type can be handled in a bespoke way and then combined into a centralized model we call a combiner and come up with predictions on the other end. For example, you can choose to encode the text with a very specific text model, encode images with a very specific image model, like VIT or ResNet as an example, and then combine to jointly train these models such that they come up with a prediction that’s optimized on the very other side.

Devvret Rishi:

This type of model architecture tends to be really flexible, and one of the nice things is you can solve a number of different types of machine learning tasks just using a combination of inputs and outputs. Want to build a text classification system? That just means that you have text as an input and a category as an output. If you want to build image captioning, you take an image as an input and then output text that you might be interested in. Of course, you can start to combine multiple of these modalities as well. For example, you can have text as input, image as input, and decide to say that you want a binary classification, which is going to start to look suspiciously similar to our bot detection example in just a few minutes.

Devvret Rishi:

This type of approach gives us a lot of flexibility to be able to combine classical machine learning techniques like classical regression, along with a lot of other types of unstructured data modalities, and apply best-in-class model for each step of the process. What you really get essentially at the end of it, and this screen looks a little bit blurry unfortunately, but what you would see on that screen is that all of these things, all of these machine learning model pipelines are essentially just controlled by a simple series of configuration files. You never have to write the code to be able to build these state-of-the-art models yourself. You can just do so in a couple of one-line statements.

Devvret Rishi:

What we’re going to do next is see how we can apply Ludwig to Twitter’s bot problem. Now, this is the initial promise of the presentation, and so this is where we’ll go ahead and dive into a very specific dataset and a little bit of a live tutorial as well. Today, what we’re going to be looking at is a dataset that is actually publicly available on Kaggle. It’s 37,000 or just over 37,000 examples of Twitter user profiles, and each row is a profile that has features about Twitter user accounts. We have really rich information, like the date that the account was created in, the text description they may have written for themselves in their bio, how many favorites, followers, friends they have, the profile image, how many tweets they send. Finally, we have a ground truth label we’re interested in building a model for, whether or not that user was a bot or a human.

Devvret Rishi:

This original dataset is from Kaggle. We can see that it’s a little bit unbalanced at a ratio of just under two to one, where we have 25,000 of these accounts being human accounts and about 12,000 of them being bot accounts. What we want to be able to do is see how easy it can be to build a multimodal model for this particular use case and this particular dataset. As I mentioned earlier, this is going to be done in under 10 lines of code, and so I felt I needed a slide to be able to demonstrate that that’s actually really possible. I’m actually going to count some of the import statements against my 10-line budget here. Let me walk through what the 10 lines of code to be able to do this are, and then I’ll go into a live demo where we can see how these models perform and actually the way we can track any of the differences between these as well.

Devvret Rishi:

The first thing you’re going to do is just import the relevant parts of the Ludwig modules. As I mentioned earlier, in Ludwig, a model is just essentially a configuration file. Now, you could write out your own configuration file, which is just a YAML, that will essentially indicate any of the input features you want to use into the model and what the output you’re interested in predicting on your dataset is, as well as any customizations that you might want as well. But to save on how many lines of code we actually have to write, you can also use Ludwig to suggest a configuration that makes the most sense for your dataset, simply by passing in the dataset and then the target that you’re interested in being able to predict.

Devvret Rishi:

Here, what we’re doing is creating an auto config, which I’m going to call Twitterbots.yaml, and just essentially one line of code with the additional wrapping here. The second step is to train this model. This is really just two lines. You’re going to declare and instantiate your model by saying that here’s the config that I just generated that I want to be able to build, and then you can call model.train directly on the dataset that you’re interested in, which you can pass it as a data frame. Once you’ve trained this model, we’ll go through a series of steps, checkpoints and evaluation, and then you can immediately get predictions. You can predict on any new dataset, for example, that you might be interested in, or you can run dot evaluation if you want to be able to understand how to evaluate the model’s performance and we’ll automatically split things into training tests and so forth.

Devvret Rishi:

And then finally, you can serve your model. What this will do, just because this is an open-source framework, it’ll spin up a local endpoint on your machine that you can use to be able to ping for realtime inference behind an API. These tend to be just the handful of lines of code you need to be able to get started building your model. What I want to emphasize is that this is a really easy way to get started, but you have all the control about the model that you might want simply by editing this YAML configuration file. If you want to use a different model architecture, pre-process your data in a different way or change anything else around, you can do so simply in a couple of one-line changes, and that’s actually what I think I will go ahead and start to show you next.

Devvret Rishi:

If I can just stop sharing my presentation now, what we’ll do is go into a brief live demo and actually try and show you a bit about what this looks like in a way that you could also start to check it out externally yourself. Cool. Now, I’ve had a couple of difficulties just sharing my desktop tab earlier. If someone is able to let me know in Slack if they’re able to see this, that would be excellent. The first thing that we’ll see is that you can actually access any of our documentation directly from the ludwig.ai website. The specific demo that we’re going to go through here has already been published under the examples and tutorials section for multimodal classification.

Devvret Rishi:

If you click into multimodal classification, you’ll be able to see this Twitter bots project where you can see the Kaggle API accounts necessary to set it up, a little preview of what the data schema looks like, and you can actually open this entire piece in Colab and be able to see it directly in order to be able to actually reproduce it. Here, you can see this was all run. Hopefully, everyone’s able to see this. Here, you can see that this was all run relatively recently. You can go through the series of steps that I went through in order to be able to train the model, evaluate it, and then ultimately get things like learning curves and others out the other end. If anyone’s interested in following along, please feel free to jump into the Colab notebook directly for it. But what I’m going to do is show you a couple of different variations that I’ve trained, and I’m just going to go through a UI that we have built inside of Predibase, the managed platform, on top of Ludwig.

Devvret Rishi:

Now, in Predibase, there’s really three key things that you do. You connect data, so in this case, a Twitter bots dataset, you can train and build models, and then you can query those models. For now, we’ll spend most of the time on just what it looks like to build those models. Here, I have my model repository, which is a collection of these open-source configuration files essentially that we use in order to be able to build my Twitter bots multimodal model. You can actually see that I have a model lineage here, where I started with a basic defaults model that just started with structure data, and then I added text, then I added images as well, the profile picture. And then I finally added a more complex model that used BERT and some other expansions.

Devvret Rishi:

You can actually see a really nice progression in the key metric that I care about for the bot detection problem, which is AUC. It’s a good, essentially, measure of a harmonic average of precision and recall, so we want to be closer to one as much as possible to have a highly predictive bot detector. Here, we can see when I started my basic defaults model, I had an AUC about 0.79. Added in the text description, that gave me about a one point bump in AUC. Added in the profile picture image, that gave me another point bump. Added in a more complex model architecture and that added a little bit more, too.

Devvret Rishi:

If I were to dive in, for example, to an individual model, you could see all the different metrics that might be relevant for it, and all of these are available using the Ludwig open-source visualizations library as well. I can see things like the loss, accuracy, precision, recall, finally where I was directly on the AUC here. I can even see things like feature importance. Now, this is, I think, pretty important to be able to call out because you can see that the text description that a user had written for themselves in their bio was one of the most predictive features of whether or not a given user was a human or a bot. Throwing away that information would really be leaving a lot of the value on the table. If I wanted to dive a lot deeper, I could even get things like text explanations where we’ll highlight the tokens for a number of different account types. So show you what were the types of texts that a user had picked up as being more or less predictive of being a human or a bot.

Devvret Rishi:

One of the things that I liked the most about Ludwig is that it becomes really easy to iterate on model versions just by making simple, one-line changes to model configurations. Let me show you a little bit about what that actually looks like. If I go back into the system around diffing, I can show you that essentially the only differences between the model version two that I made and the model version five that I made, which had about a one-point increase in AUC, is that I added this encoder, this BERT encoder for the text field. This really comes back down to the underlying model architecture that Ludwig uses, that Encoder Combiner Decoder model, which lets you use a best-in-class model for each type of input modality.

Devvret Rishi:

Here, I used the BERT encoder for the text field and I used this both pre-trained, but then set trainable to true, which meant fine tune this particular encoder towards my task that I have here. That made it much easier for the model to be able to essentially take some of the prior knowledge that it had and apply it directly towards this dataset. I’ve iterated on five different model versions here, but if I wanted to, I could continue to iterate on, let’s say, the sixth, seventh, and eighth. I would simply have this visualization. This is a visualization of what the ML pipeline that the config generates for you, so you can see that everything from input features through pre-processing, Encoder Combiner and Decoder are represented here.

Devvret Rishi:

The really nice thing is that I can start to customize any of the different parts of this that I might want without having to change anything else. If instead of using BERT, I wanted to use a different model, like maybe GPT-2 or T5, I could start to actually do that really easily just in a series of one-line clicks. That makes one easy way for this to be able to populate the configuration so that you can actually start to go ahead and run these experiments directly yourself.

Devvret Rishi:

One thing that I’ll leave as an open exercise to the user is that I mentioned that this is an imbalanced dataset. I’ve talked a lot about how Ludwig makes it easy to be able to handle multimodal data by being able to have easy text encoders, image encoders and others. But it also makes it easy to be able to handle imbalanced data by giving you the parameter choices you need to be able to handle those types of modalities. If I wanted to, for example, change things like if I up-sampled or down-sampled, which are common techniques to be able to essentially resize your dataset in order to be able to support less imbalance, I could do so just by essentially specifying and oversampling or undersampling ratio here. I don’t have to implement this part of dataset pre-processing. I just go ahead and start to type it out and I’m able to actually do it directly.

Devvret Rishi:

There’s other types of modalities that are built into Ludwig as well. For example, I can go ahead and change the class weighting and the loss. I essentially tell the model, weigh the positive classes a little bit little, which might be labeled as bots, a little bit more than the negative classes, because I really care about being able to detect those rather than the others. One thing that I’ve shown, I think, here and what exists in the Ludwig open-source tutorial is how to be able to build your first series of models using text, images and other unstructured and structured data in this platform.

Devvret Rishi:

What I’d love to see is an exercise for everyone else out there who’s interested in being able to replicate this, is if they’re also able to see how changing things like oversampling ratios or undersampling ratios impacts their final model performance, and if anyone’s able to beat that 0.81 AUC score that we just saw from our fifth model version here directly. With that, I think what I will do is just pause my presentation for now and leave it open to any questions about how we can do multimodal bot detection or any multimodal deep learning model from the audience.

Speaker 2:

Awesome. We do have some awesome questions that came through. One is, what was the labeling criteria used for the bot detection? That’s the toughest part of the problem, isn’t it?

Dat Ngo (NOT Daniel Ngo):

Yes, I think labeling criteria can be quite difficult. What we usually see is that people do some amount of manual labeling directly themselves. In the Kaggle example where we had, we actually spent time coming up with 1000 plus manual labels that we generated by just having our community moderation team look at which accounts had been abusive in the past and start to label those as bot or humans. I think in the Twitter case, that’s actually something that’s been done themselves.

Dat Ngo (NOT Daniel Ngo):

I think you want to try and get away from having to manually label every single example as fast as possible. That’s where we’re trying to build a deep learning model, so we can generalize the things that we’ve labeled as humans to something that can be a little bit more predictive and help us triage those workflows more directly. In practice, the labeling process was really manual to start off with, but the model helped us make it less so after a time.

Speaker 2:

Got to love that. Starr slow and then ease off. Just get that manual labeling and then go to the automatic when you can.

Dat Ngo (NOT Daniel Ngo):

Exactly.

Speaker 2:

Next up is, how does serving latency, medium and 99th percentile for Ludwig, let’s see if I can get that out of my mouth, and Predibase look like? At what RPS and concurrency values?

Dat Ngo (NOT Daniel Ngo):

Yeah, it’s a great question. Ludwig is an open-source framework that can essentially run and serve models on whichever piece of infrastructure you choose to deploy it on. With Ludwig in the open source, let’s just say it’s a serve-your-own-models type of story. You can export the model out as a TorchScript of the object, which is a compiled model graph that’s really efficient for low-latency serving. And then you can choose to essentially place it in your serving stack and the QPS and latency will be dependent on the compute that you choose to use.

Dat Ngo (NOT Daniel Ngo):

Inside of Predibase, we essentially have built a managed serving offering built on top of Ludwig. We have this abstraction on top of compute that we call engines, and engines essentially can be configured to scale up or down based on the request of the system that you need. We’ve built that optimized serving stack around Ludwig and TorchScript, as well as NVIDIA Triton, to provide really low-latency serving that you might need. Depending on the kind of applications that you might be looking for, whether it’s sub 50 milliseconds or others, that can really just be configured as a function of the amount of compute that you want to be able to throw at it.

Speaker 2:

Awesome. Next one coming through is, how do you compare Ludwig with Vertex AI?

Dat Ngo (NOT Daniel Ngo):

Yeah, it’s a good question. On Vertex Ai, I think we had a large number of different capabilities. One of those capabilities was called Google AutoML, which I had a chance to spend a little bit of time on as well. The key idea behind Google AutoML was, can we make it easier to be able to build ML models for a larger chunk of users than the ones that might be experts, again, in the deep learning frameworks themselves? What I think Ludwig does that I believe is the next evolution of traditional AutoML is that we saw that AutoML was really useful for people to prototype or get started, but no one actually really used it en masse seriously in production while I was there. One of the main reasons that we heard from customers was it felt too much like a black box, and that they were giving away the control they needed in order to actually feel comfortable with their ML model internally.

Dat Ngo (NOT Daniel Ngo):

The way I think about Ludwig is that AutoML is just the starting point. You get the same simplicity as an entry point with your AutoML models with that immediate config generation, but the core principle we try to adhere to is control what you want and we’ll automate the rest. You can control anything down to the level of code, but you don’t have to re-implement the entire stack. If you’re a beginner user and you just want to get started with all the defaults, you can get up and running with a model, I think, inside of several minutes or an hour. But if you’re a more expert user and you want to really iterate on some of the things that we were showing, for example, let’s say a BERT model with a different learning rate, different number of fully connected layers, you have that level of granularity and you can essentially iterate to your heart’s desire.

Dat Ngo (NOT Daniel Ngo):

That’s, I guess, really where I see some of the key differences, is seeing this as the next evolution of some of the projects we worked on with automated machine learning.

Speaker 2:

What other text-embedding approaches do you support?

Dat Ngo (NOT Daniel Ngo):

We support many different types of text-embedding approaches. If you take a look at the Ludwig AI documentation, under the configuration, you’ll be able to see the different data types we support. If you click into text, you’ll be able to see everything that we support for it, all the way from data pre-processing, which includes tokenizing strategies towards the way that we actually encode and generate embeddings for those texts. The very short answer is you could use extremely shallow neural networks to be able to do this on the light end, or something like a parallel CNN, which we found is a pretty good trade-off between performance and speed to training, or larger transformer-based architectures as well.

Dat Ngo (NOT Daniel Ngo):

You saw BERT, but we actually support large language models, all the way through the size of any of the open-source models like OpenLLaMA, 30 billion and 60 billion and other parameter models as well. The cool thing about those models is, you’ll probably never want to train them from scratch, but you can certainly fine tune them to your dataset. That gets supported right out of the box. We’re actually really excited to be able to announce some of those new functionalities recently, but you can start to play with them in Ludwig today.

Speaker 2:

Last question for you. How can we compare Ludwig and other deep learning frameworks in terms of explainability of the underlying items on the data?

Dat Ngo (NOT Daniel Ngo):

Sorry, I think I just lost the audio at the very end. I think the question was around explainability on the underlying models?

Speaker 2:

We’re having a little bit of a streaming issue.

Dat Ngo (NOT Daniel Ngo):

Yeah.

Speaker 2:

… was that. Yes, it was exactly right.

Dat Ngo (NOT Daniel Ngo):

Okay. Great. Let me answer maybe briefly for now, just in case there are streaming issues in the speaker. But the core idea I would say is that Ludwig is really configurable. Again, you can control what you want. We automate the rest. That actually includes explainability. We provide a number of different explainability techniques out of the box and you can choose the ones that you want to be able to use for your type of problem. For example, we provide explainability through Shapley values, which will provide some things like feature importances, but also through integrated gradients, which is part of PyTorch’s Captum library.

Dat Ngo (NOT Daniel Ngo):

With Ludwig, you can think of it as a very easy interface to be able to pick and choose what you want to be able to do and get a lot of that available out of the box without having to reimplement any of these explainability modules directly yourself. I wouldn’t think about comparing Ludwig’s explainability with other explainability, but rather than being able to plug the explainability you need directly into your Ludwig model.

Speaker 2:

Cool.

Dat Ngo (NOT Daniel Ngo):

Hopefully, you were able to hear that.

Speaker 2:

Awesome.

Dat Ngo (NOT Daniel Ngo):

Yeah.

Speaker 2:

Dude, yeah, I heard it loud and clear. I think it’s more on my end than your end. I think everybody else heard it hopefully, and they can give us the thumbs up or thumbs down in Slack if they did or didn’t. I may have frozen, I’m not sure. But this was great, man, and I look forward to seeing the rest of the Predibase team at the LLMs in Production Conference coming up.