Shopping for a new couch online? That could mean looking through hundreds of shapes, styles, fabrics, and colors. Redoing a whole living room or bedroom involves not only new pieces of furniture but considering how they all fit together. Mid-priced furnishings retailer West Elm thinks AI can narrow the search by learning your personal aesthetic. The new West Elm Pinterest Style Finder uses neural networks to get a sense of style from your own or anyone else’s Pinterest board of images and returns a shortlist of furniture, rugs, curtains, mirrors, and other items in about 10 seconds.
The online tool is an extension of the in-person service West Elm provides at its 100 stores U.S., Australia, Canada, and the U.K. “They ask people to provide pictures for inspiration so they can get the process started,” says Luke Chatelain, West Elm’s VP of innovation. “We realized this is something that we can automate,” he says, using a neural network-powered visual recognition service provided by New York-based AI startup Clarifai.
Typically, image recognition tools are trained by data scientists to recognize classes of objects: cars, food, clothes, whatever. But neural networks are now able to figure out on their own what you show them. They won’t necessarily know the name “sneakers,” for instance, but they will learn that things with rubber bottoms, a leather or canvas upper, laces and other details all belong in the same category.Last year, Clarifai introduced a drag-and-drop online tool called Custom Training that allows anyone to upload a few photos of specific items, say a brand of shoes, to train a classifier to recognize that type of items in other photos. Chatelain’s team created a site that connects Clarifai’s tool to Pinterest (when users paste in a board’s URL), develops a classifier of the style in the images, and filters out matching products from West Elm’s online catalog of about 5,000 items. (West Elm pays Clarifai based on the number of times it accesses the neural network.)
“The way that it will work best is if you put in [images] from furniture or room shots,” says Chatelain. “If you added components with black-and- white text on there or pictures of your dog…you are going to drive some different reactions.”
I tried my own and other people’s Pinterest boards of Art Deco design—including furniture, sculpture, buildings, even tattoos—and got recommendations that were often on-point. Results were pretty solid for things like rugs or wall hangings where it was easy to correlate designs from Pinterest images to things like fabric patterns. Other recommended items, such as couches, didn’t scream Art Deco, but then, West Elm may simply not have any couches that fit the style.
There were also what appeared to be some unintended results. For instance, Style Finder recommended picture frames that didn’t look Art Deco at all, but they contained sample photos and artwork, which the neural network may have analyzed in order to make a match.
The Style Finder didn’t have to match items with the same or even closely similar patterns; rather, it recommended products that were roughly in the same stylistic vein. “We let the neural networks decide,” says Clarifai’s founder and CEO Matt Zeiler. “It’s not hand-tuned algorithms, for [example]: You might like this color, or you might like this pattern, or you might like this type of chair. It’s really high-level understanding based on the collection of pictures.” The classifier doesn’t even know concepts like “chair” or “art deco.” It’s just matching the un-named impressions it gets from images.
Unlike typical recommendation engines, Style Finder doesn’t rely on someone’s buying or browsing history—recognizing that tastes can change. “All [it cares] about recognizing is some classifier of your preference, say what Matt likes on Friday, June 30th,” says Zeiler. He claims that the technology can start learning a style from a single photograph and get a good sense of it from less than 10.
Chatelain says that West Elm may start augmenting the results with things like shopping history or most-sold items to fine-tune results, however. Style Finder was developed in just a few weeks and will undergo a lot more refinement, he says.
Style Finder can be useful even when it fails to find a product for someone, says Chatelain. “A lot of people on our merchant team and our product service team realize that they can use this to find holes in our catalog, to understand the styles that people are really interested in,” he says. “It’s taking direct input from our consumers.”