The current massive explosion of generative AI models for text and image has been inevitable. As these models become more and more capable, "base model" is a relatively new term that is being used. So what is a base model?
The term is still somewhat vague. Some define it by the number of parameters and thus how large a neural network is, and others by the number of unique and difficult tasks the model can perform. Is it really that exciting to make bigger and bigger AI models and the ability of the model to tackle multiple tasks? Strip away all the hype and marketing language, what's really challenging about these new generations of AI models is that they fundamentally change the way we interact with computers and data. You can think of companies like Cohere, Covariant, Hebbia and You.com.
We have now entered a critical phase of AI in which who gets to build and serve these powerful models has become a major point of contention, particularly as ethical questions begin to emerge, such as who is entitled to what data, whether models violate reasonable assumptions of privacy, whether consent to data use is a factor, what constitutes "inappropriate behavior" and many additional gray spots that need light. With questions like these on the table, it's reasonable to assume that those who control the AI models will be perhaps the most important decision makers of our time.
Is there a game for the open source base models?
Due to the ethical issues associated with AI, attention to open source basic models is gaining momentum. But building basic models is not cheap. They require tens of thousands of state-of-the-art GPUs (Graphics Processing Units) and many machine learning engineers and scientists. The realm of basic model building to date has only been accessible to cloud giants and extremely well-funded startups that find themselves with potential hundreds of millions of dollars of return potential.
Almost all of the models and services created by these few companies have been closed source. Still, closed source entrusts a great deal of power and decisions to a limited number of companies that will define our future, which can be quite unsettling.
We have entered a critical phase of AI where who gets to build and serve these powerful models has become a major point of contention.
The biggest hurdle to basic open source models remains the money. For open source AI systems to be profitable and sustainable, they still require tens of millions of dollars to function and be properly managed. Although this is a fraction of what large companies are investing in their efforts, it is still quite significant for a startup.
We can see how Stability AI's attempt to open up Neo-GPT and turn it into a real business failed, as it was outpaced by companies like Open AI and Cohere. The company now has to deal with a lawsuit from Getty Images, which threatens to distract the company from its goals and further strain resources, both financial and human. Meta's opposition to closed source systems through LLaMA has fueled the open source movement, but it's still too early to tell if they will continue to honor their commitment.
The good news is that models are getting smaller with very recent techniques like Low-Rank Adaptation (LoRa) and Chain-of-Thought Prompting (CoT). But they still require many iterations to make them commercially viable, which naturally involves millions of dollars and a lot of computing power.
Right now, most open source generative AI companies fail, and we have no data points on how “big” and successful open source AI projects could become. That makes it difficult for venture capitalists to write the kind of checks such projects need, particularly at a time when the banking system is so fragile. While this could improve in the future, it currently translates into strategic investments, which comes back to Big Tech companies.
Regardless of where the funds potentially come from, we must face the reality that after the hype wears off and the marketing messages fade, challenges remain on the sustainability, risk-reward, and profitability fronts, as well as maintaining quality and precision to guarantee value. . In short, open source AI companies need to figure out how to become real businesses, which has long been the biggest impediment.
Implications for startups
As we have seen with the massive proliferation of ChatGPT, basic models are the future, but how they will influence it remains to be seen. A new generation of startups is emerging to do all sorts of amazing things, whether they're built as closed source or open source. In an effort to decide the best path forward, founders can ask themselves tough questions related to all aspects of their business. For example:
Financiación
¿De dónde viene el dinero? ¿Tendrán los bolsillos lo suficientemente profundos como para hacerlo solo como una empresa de código cerrado? Si no, ¿qué tipo de pareja se buscará? ¿Se puede mantener el control del proyecto? Si se opta por la ruta del código abierto, ¿cómo se financiará?Equipo
¿El proyecto se presta al aporte y desarrollo de la comunidad, o requiere estrictos controles de calidad? ¿Cómo se podrá atraer al mejor talento para desarrollar el proyecto?Experimentación
¿Cómo podrán las personas experimentar el proyecto para aprender sobre él y probarlo? ¿Se abrirá pruebas y a un marketing extensivo? ¿Estará abierto un entorno de pruebas (sandbox) para los desarrolladores? ¿Se generar entusiasmo a través de redes comunitarias de código abierto?Fidelización
¿Cómo se crea la propia comunidad de usuarios? ¿Qué sucede si se cambia de enfoque, es decir, cuánto se puede perderen reputación y en términos de usuarios si comienza como un proyecto de código abierto y luego se debe ir en la dirección opuesta si un socio con recursos lo exige?Ética y educación
¿Cuáles son sus responsabilidades cuando se trata de cómo se utiliza el proyecto? ¿Qué hacer si se coopta para fines no deseados y dañinos? ¿Hay intervención si aprecia un "comportamiento inapropiado" y, de ser así, en qué medida? ¿Cómo se monitorea el proyecto y transmitirá las políticas a los clientes/usuarios? En resumen, ¿dónde están los límites de seguridad?This is just a beginning. There are many more questions than answers at this time.
Determining future success
As startups move toward building basic models for a particular niche, it's important for them to recognize that new milestones will be required to assess their relevance and ultimate value. As Ryan Shannon of Radical Ventures recently noted:
Unlike a traditional startup, which can simply write code, ship a product, and iterate on customer feedback, Foundation Model companies need to spend more time building and training their models to get a product to a position where it is viable and ready for use. This can often take several years, millions of dollars (or…hundreds of millions of dollars), and several iterations before the products are good enough for companies to charge customers for using them.
This is a tall order, and may require a leap of faith from investors or members of a larger community. The initial investment in grassroots models is substantially higher than what other startups need, but back-end adoption may be unprecedented. These are transformation technologies unlike anything seen before.
With the right amount of time, money, and talent, fundamental models, whether open or closed, will not only usher in the future, they will control it to some extent. The basic models will guide the way we consume the information that shapes our perspectives and decisions, which will have a profound impact on the way society communicates, learns, understands and creates.
The stakes are incredibly high. Open source models need to find a business model that works in the long term, while closed models need to address ethical concerns initially, with behavioral controls and oversight in place. No perfect solution has emerged in this rapidly changing and messy landscape, but facing the big questions and examining our responsibilities is essential to innovation. When we consider all that is possible, both for better and for worse, safeguards are discovered and real progress occurs.
