From the course: AI Engineering Use Cases and Projects on AWS: Production-Grade LLM Systems

Open-source strategy walkthrough

It's important to have a strategy when you're using open-source software, but also closed-source software. And if we look at large language models, it's going to fit that same story here. We see that open is starting to become the leader when we talk about the dominance of large language models and what strategy will succeed. If we look at history here, we can see that the infrastructure of really the entire globe is running on Linux. Linux is open-source, it's free. The only thing you have to pay for is to understand how it works and have the knowledge of it. Anybody can contribute to it, they can maintain it. And if we look at container runtime standards here, this is a good place to start. Also, if we look at global infrastructure, what it's actually doing, a lot of times, SaaS services, microservices, cloud providers, all of these are running an open-source solution. We also look at the language evolution as well. If we look at TIOBE, which is a popular language index, we look at the top 25 languages, the majority of these languages are also open-source. So if we look at the leaders here, all of it is really pointing towards a dominance of these open-source solutions. And then if we look at the latest things with DeepSeek, for example, DeepSeek is number one in the app store. It has parity with commercial models, MIT license. So clearly, what's going to happen in the future is there's going to be a mixture of both open models and closed models. And so when you're building things, you need to have a strategy for that; so what percentage of your workloads can be local? What percentage of your workloads can be open-source models? And then what percentage of your workloads are going to use really a combination of all those, and how do you actually leverage open models using things like Bedrock, for example, on AWS is a good way to leverage this because you're not tying into any particular type of model. You have an ability to proxy between the different ones. Likewise, if you look at open-source, for example, some of the things that are important to be aware of is that with open source you have the ability to inspect what's happening, you're able to inspect the data sources, you're able to really look at the security as well. So it's really an outlier to talk about regional problems or regional conflicts with open source, really it's about whether it's open or closed. The future is probably going to look a lot like the past, which is open-source models, open-source languages. And also open-source operating systems are what have historically performed very well, and this is a really good fact to know when you're building solutions for large language models.

Contents