Artificial intelligence has revolutionized the world in many ways, but not everyone has access to these advances. Large companies such as Open AI or Facebook AI have had great achievements due to the resources available to them, but science should be open to everyone: with more or less resources. This is the spirit Hugging Face promulgates with its “Big Science” initiative, and one that our very own NLP engineer Manuel Romero partakes in.

What is the Big Science initiative?

Hugging Face has set out to become the “CERN” of artificial intelligence research. (The world renowned CERN, or the European Organization for Nuclear Research, is a European research organization that operates the largest particle physics laboratory in the world.) Over a period of one year, from May 2021 to May 2022, 600 researchers from more than 50 countries and 250 institutions and companies will come together to create something very special in the NLG field: a large multilingual neural network language model and large multilingual text dataset on the 28 petaflops (that’s a mind-boggling one quadrillion operations per second), Jean Zay supercomputer (IDRIS) located near Paris, France.

Every 2-3 months, a day of live talks, posters and discussions for the group is organized. The next planned event, Big Science Episode # 3, will be co-located with NeurIPS as a Social Event.

Manuel Romero, member of Big Science

Our NLP engineer Manuel Romero has been appointed a member of this initiative. Manuel Romero is one of the biggest contributors in the history of Hugging Face, with more than 200 models that you can view here. Manuel will investigate, together with the rest of the members of the initiative, efforts to continue making science more accessible to all.

A model capable of beating GPT-3, the main great milestone

This work is already paying off. Using the basis of a T5 model, the so-called T0 model has been created. T0 shows zero-shot task generalization on English natural language prompts, outperforming GPT-3 on many tasks, while being 16x smaller.

To obtain T0, Big Science fine-tuned a pre-trained language model covering many different NLP tasks. You can use the models to perform inference on tasks by specifying your query in natural language, and the models will generate a prediction. For instance, you can ask:

“Is this review positive or negative? Review: This is the best cast iron skillet you will ever buy.”

And the model will hopefully generate “Positive”.

The model will give us an answer in the form of natural language and not just a digit, a symbol or a category. The model “speaks” to you, regardless of the problem or question you ask. Cool, right?

Science for all

At Narrativa we are committed to making science accessible to everyone, so in our Hugging Face profile you can find models to detect fake news or hate language. We stand behind initiatives such as Big Science, which is also supported by heavyweights such as Salesforce Research, Microsoft, IBM Research, Stanford University and Maryland University.

Artificial intelligence has to be an accessible tool for everyone, and like Hugging Face, at Narrativa we are working to make it possible!