Facebook develops new AI techniques to one day make “anything shoppable”
Facebook on Tuesday shared details of new artificial intelligence techniques it has developed to ultimately make “anything shoppable” — advances in item segmentation, detection and classification to improve the way people buy, sell and discover items across Facebook platforms. The long-term goal, Facebook said in a pair of blog posts, is to create a holistic AI-powered system to enable frictionless consumerism.
executive guide
What is AI? Everything you need to know about Artificial Intelligence
A guide to artificial intelligence, from machine learning and general AI to neural networks.
“Our long-term vision is to build an all-in-one Al lifestyle assistant that can accurately search and rank billions of products, while personalizing to individual tastes,” one blog post says. “That same system would make online shopping just as social as shopping with friends in real life. Going one step further, it would advance visual search to make your real-world environment shoppable. If you see something you like (clothing, furniture, electronics, etc.), you could snap a photo of it and the system would find that exact item, as well as several similar ones to purchase right then and there.”
Universal product recognition
In pursuit of that vision, Facebook is deploying a new, universal computer vision system called GrokNet. This new product recognition model was designed with the intent to make “virtually any photo shoppable.”
GrokNet currently powering features for buyers and sellers in Marketplace, Facebook’s peer-to-peer shopping platform. When a seller uploads a photo to Marketplace, the system automatically suggests attributes to list, such as the item’s color or material. For buyers, the system allows you to conduct specific searches — such as “black leather sectional sofa” — and find what you’re looking for, even if your search terms didn’t match the seller’s product description.
Facebook explains that it needs a universal system for its platforms, since millions of users are posting listings across dozens of categories. Most product recognition models are designed for specific product verticals, such as furniture or fashion. Adding to the challenge, Facebook wanted to build a system that serves all countries and users, regardless of language, cultural differences, age, socioeconomic class or other factors.
The new system can identify fine-grained product attributes across billions of photos, Facebook says, across several categories. It’s 2x more accurate than Facebook’s previous product recognition systems.
In addition to deploying it in Marketplace, Facebook is using it to test automatic product tagging on Facebook Pages. This should help small businesses market their products more easily while making it easier for consumers to find products they like. Facebook also plans to use it to power Shops, so customers can get personalized suggestions from businesses on the products most relevant to them.
Advances in segmentation
To build a unified model, Facebook had to advance segmentation — the computer vision process of identifying which pixels belong to which objects. Clothing can be especially hard to identify, since it’s often layered or hidden behind hair.
Facebook developed a new clothing parsing technique called Instance Mask Projection, which it says is the first system that can predict obstructed or layered objects in photos, like a shirt beneath a jacket. The company says it’s achieved more than 80 percent accuracy.
To identify an object, the system first predicts a box around each item of clothing. Once it has that rough segmentation, it precisely labels each pixel.
3D views on Marketplace
Facebook is also improving the quality of images on Marketplace with a new feature that allows any seller with a camera phone to turn a 2D video into a 3D-like interactive view. The company says Marketplace is the first person-to-person commerce platform to offer this automatic video stabilization technique. Currently, Facebook is testing it on Marketplace for iOS.
The feature allows viewers of the image to spin it and move it up to 360 degrees to get a complete view of the object for sale. It’s designed to work without requiring the seller to edit or reformat their video, and it should work in low lighting or even if the object is partially obscured.
To create the camera angles in a 3D space, Facebook uses the visual-inertial simultaneous localization and mapping (SLAM) framework, which is commonly used in virtual reality settings.
Article source: https://www.zdnet.com/article/facebook-develops-new-ai-techniques-to-one-day-make-anything-shoppable/#ftag=RSSbaffb68