San Francisco-based Databricks launched an open-source rival to OpenAI's ChatGPT software on Friday.
Databricks created a so-called large language model that it says is cheaper to use and doesn't require organizations to hand over troves of data to a third-party.
Large language models are used to develop generative AI tools, like those from OpenAI, but they require being fed large amounts of data in order to train the system to respond to prompts and produce accurate responses.
So, instead of uploading data to a centralized database that’s owned and controlled by another organization like OpenAI, companies could use an open-source solution, like Databricks' software, to train their own models while maintaining control of their data.
"For many companies, the problems and datasets most likely to benefit from AI represent their most sensitive and proprietary intellectual property, and handing it over to a third party may be unpalatable," Databricks wrote in a blog post.
The company is calling its open-source software "Dolly" — named after the sheep that was cloned in the 1990s.
Dolly is a clone of a Stanford project called Alpaca which itself was developed by using a large language model developed by Meta called LLaMA, or a Large Language Model Meta AI.
Databricks says that its model was able to produce ChatGPT-like outcomes with a much smaller dataset of only 6 billion parameters, compared to the 175 billion parameters used in OpenAI's slightly older GPT-3 model.
OpenAI launched a newer version called GPT-4 on March 14.
"We believe models like Dolly will help democratize LLMs, transforming them from something very few companies can afford into a commodity every company can own and customize to improve their products," Databricks wrote.
The company also said it was providing additional open-sourced tools to enable organizations to use Dolly on the Databricks platform.
Databricks, co-founded by CEO Ali Ghodsi in 2013, provides cloud-based data management and analytics. The company has raised $3.5 billion and was valued at $38 billion in 2021 after closing a $1.6 billion Series H round.
