Facebook’s New AI Tool Mimics an Unexpected Human Behavior: Forgetfulness

Researchers at Facebook have come up with Expire-Span, a new tool that teaches artificial intelligence models to identify and ‘forget’ irrelevant information to enable computer systems to execute tasks faster and more efficiently while reducing costs.

Forgetfulness may be considered a trait, rather an imperfection among humans, and if you’re a fan of science fiction, an all-knowing, ever-remembering machine may be the ultimate solution. Not in Facebook, not in Silicon Valley, where researchers are going the human way and striving to make machines driven by artificial intelligence more humanlike.

ResearchOpens a new window by Facebook AI may have just made a breakthrough in enabling artificial intelligence models to forget data, a step toward “one day achieving a wide-range of difficult, humanlike AI capabilities that otherwise would not be possible.” Called Expire-Span, the tool is still in its infancy and can have real-world applications.

Future applications may be a distant reality. But the tool can have multiple present-day implications on the AI model performance and significantly improve costs. Expire-Span allows an AI-model to selectively forget useless data or ‘memories’, much like the human brain, that discards memories as they become less relevant.

For instance, ask yourself; what did you eat for dinner last night and on the night of 10th February 10? Chances are you’ll be able to recall what you had the other night since it is still fresh, although your dinner on 10th February? Not so much.

The only way a human will remember what they ate, what they wore, or did anything at any arbitrary moment in the past, is if they have another memorable association with that time.

AI-driven systems aren’t like that. They’re all-remembering, all-knowing entities that indiscriminately accept and process all types of information fed to them. The data feed in a neural network is distinctly labeled and stored or memorized by the AI system, which helps it access it later on based on the current conditions.

What the neural network does is scour the entire data storage, which may be years old with hundreds of millions of data points, depending on the context of the task at hand. So, unless the stored data isn’t small, scanning it usually would take up a lot of computational resources such as processing power as well as storage hardware.

Compression of data helps to a certain extent, but then again, it leads to ‘blurry’ outcomes that can potentially affect the accuracy of the model.

So how can Facebook’s Expire-Span help? The answer is in the name of the tool.

See Also: EU Wants Organizations to Use AI Responsibly, Here’s Possibly How

How Does Expire-Span Function?

After identifying (or predicting) the most relevant information derived from contextual data, Expire-Span sets an expiration date for each data value. Facebook explainedOpens a new window , “Picture a neural network presented with a time series of, for example, words, images, or video frames. Expire-Span calculates the information’s expiration value for each hidden state each time a new piece of information is presented, and determines how long that information is preserved as a memory.”

Upon the arrival of the expiration date, the said information is then deleted from the AI memory storage. “This gradual decay of some information is key to keeping important information without blurring it. And the learnable mechanism allows the model to adjust the span size as needed,” Facebook added.

What this means is that like humans intuitively discard trivial information, AI-driven systems with Expire-Span may soon be able to do exactly that. And that’s not all.

Let us look at a test conducted by Facebook, where researchers charged an Expire-Span AI model to identify a yellow door in a hallway with multiple other doors, each with different colors. An AI model with a standard transformer notes and stores all information such as the number of doors, color of doors, etc., until it achieves its task, which is to find the yellow door.

An AI model with Expire-Span will also identify and store this information. The difference is that it will assess the context of the task it was assigned and then proceed to discard all irrelevant information such as the color of other doors, etc.

Through selective forgetting, Expire-Span allows memories to be permanently deleted if the model learns they are not useful for the final task. Thus, like the human mind, the AI model may note everything but selectively (intuitively in the case of humans) delete unimportant details with respect to the task.

Facebook found that Expire-Span can be scalable to “tens of thousands of pieces of information.” Moreover, Expire-Span outperforms existing alternatives such as Adaptive-Span and Compressive Transformer and is more efficient despite the fact that it retains less information.

Finally, they analyze the efficiency of Expire-Span compared to existing approaches and demonstrate that it trains faster and uses less memory.

— Ilir Aliu (@_IlirAliu) May 16, 2021Opens a new window

The AI division of Facebook is also evaluating the incorporation of multiple other types of memories such as semantic memory, episodic memory, procedural memory, explicit and implicit memory within an AI neural network.

Closing Thoughts

Expire-Span may not necessarily have material use cases. However, one of the biggest areas that Facebook, which is dependent on advertising for the bulk of its revenue, can use is to optimize its AdTech engine.

However, the capabilities it presents may find uses across multiple AI implementations, primarily to enhance efficiency and arrest costs in existing models.

Let us know if you enjoyed reading this news on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We would love to hear from you!