Former Facebook Research Scientists Raise $20 Million Seed Round for Enterprise Foundation Model Startup
Bain Capital Ventures is leading the $20 million seed round in Contextual AI
The foundation model technology war is just getting started.
Douwe Kiela — a former research scientist at Facebook and the former head of research at open-source AI company Hugging Face — raised a $20 million seed round for a foundation model for enterprises company called Contextual AI.
Amanpreet Singh, a fellow former researcher at Facebook and at Hugging Face, is also a co-founder.
Kiela — and Bain Capital Ventures partner Aaref Hilaly — chatted with me about their hopes for the company.
Contextual AI is building a foundation model for enterprises that tackles problems with hallucination, attribution, compliance, customization, and data privacy in existing foundation models.
The company’s work is building off a research paper that the Facebook research team published in 2020, called, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Kiela was the senior author, as he has been on numerous papers over the years.
Kiela described the problem with existing foundation models as “the two separate halves of the brain don’t really talk to each other.” Foundation models get trained on a set of information and then the model doesn’t properly hold onto those facts when it comes time to produce an answer.
Contextual AI will face a lot of competition. Foundation model company Cohere was founded in 2019 and has already raised hundreds of millions of dollars to bring foundation models to large companies. Vectara, founded by a former Cloudera co-founder, raised $28.5 million to build a foundation model powered search engine for companies. Meanwhile, other startups, namely the startup Perplexity, have released tools meant to improve attribution for the results of a large language model. And some large companies, like my former employer Bloomberg, are releasing their own models trained on their data.
“I don’t think there's going to be one solution that rules them all. Different technical approaches will be better suited to different sets of enterprise use cases,” Hilaly said. “There’s room for lots of multi-billion dollar revenue companies.”
Contextual AI has not released a product, and doesn’t seem to have finished one yet. “Now that we have our GPUs, we’re training models,” Kiela said.