Facebook ended a project called Learning from Videos. He uses artificial intelligence to understand and learn audio, textual and visual representations in videos of public users on the social network.
Learning from videos has several goals, such as improving Facebook’s AI systems related to content recommendations and policy enforcement. The project is in its initial stages, but it is already bearing fruit. Facebook says it has already taken advantage of technology to improve Instagram Reels recommendations, like videos of people doing the same dance to the same song. The system is also showing improved results in speech recognition errors, which can reinforce the automatic captioning capabilities and facilitate the detection of hate speech in videos.
Facebook says the project will help AI researchers avoid reliance on labeled data and is part of efforts to build systems that learn similarly to humans. As such, Learning with Videos “will enable entirely new experiences”. The company did not give many details about them, except for a possible feature that would allow AI to find digital memories, including those captured by augmented reality glasses. You could, for example, ask this system to show “every time we sing for grandma”, and it could bring up those clips. Facebook, notably, is.
The company says the project is watching videos in hundreds of languages and from almost every country. This aspect of the project will make AI systems more accurate and allow them to “adapt to our rapidly changing world and recognize the nuances and visual clues in different cultures and regions”.
Facebook says it is keeping privacy in mind when it comes to learning from videos. “We are building and maintaining a solid privacy foundation that uses automated solutions to ensure privacy on a large scale,” wrote the article in a blog. “By incorporating this work at the infrastructure level, we can consistently apply privacy requirements to our systems and support efforts like AI. This includes implementing technical protections throughout the data life cycle.”
Understanding what’s going on in the videos can be an extremely difficult task for AI systems. They can include obstacles such as background noise that makes speech and language switching difficult to understand. However, less than a year after the Learning with Videos project started, Facebook is taking what the system has learned and putting it into practice in other areas.