Samsung Bixby: One year on and there is still much to be done
It has been a whole year since Samsung announced Bixby as a rival to Apple Siri, Amazon Alexa and Google Assistant. The newest incarnation of Bixby arrived on the new Galaxy S9 and S9+ smartphones which went on sale this month, so we thought this presented the perfect opportunity to see how far (or not) Bixby has come.
Generally speaking, there are three parts to Bixby. The first is an ability to answer simple questions in the same way Alexa and Google Assistant do; this means understanding what you have said, then looking up the information online and feeding it back to you. The second is how Bixby works as an assistant to help you use your smartphone; you can ask her to open apps, adjust settings, save the image you're looking at as your screensaver and much more.
Read More:
Finally, there is Bixby Vision which uses the camera to identify objects, help you buy bottles of wine, apply augmented reality makeup before buying it, translating text, and providing nutritional information on food.
Because using the camera is unique among digital assistants, let's start there.
Bixby Vision
On paper, Bixby Vision's skills sound most impressive. Here is a system, Samsung claims, which can identify anything you show it, help you buy bottles of wine and even identify the food on your plate, before giving nutritional information. Unfortunately, the reality feels a little half-baked.
Bixby Vision is its own app, rather than being part of the camera app. The first function is for translating text, just like Google Translate. The system works okay, and might come in handy when you're struggling with signs at a train station, or a restaurant menu that's written in a language you don't speak. But the experience is clunky at best and, because it uses Google's translation services, provides the same results. I'm not sure why you would opt for Bixby Vision, especially when Google gives you the option to download language packs onto the phone to make translation more quickly.
Sometimes the app works, while other times it just doesn't, without any obvious reason why. Generally, I found Google Translate to be more reliable.
Image searching is much better. Point the Bixby Vision camera at an object and it will use the huge image databases of Pinterest to identify it. In my testing it was successful at not just identifying a camera lens as a camera lens, but it even spotted writing on the lens to identify it as a Nikon 50mm. Bixby also did a good job at identifying a leather wallet (when open and closed), my desk plants, and some headphones.
Bixby could tell that my headphones were by Bose, it correctly identified my phone as an iPhone, and even got the names of two desk plants correct, which pleasantly surprised me when I expected it to fail. It isn't perfect though, as the system thought an AA battery charger was a Samsung smartphone, and a Google Home Mini was a lamp.
But even when it gets things right, I'm not entirely sure what the point is. I know what these items are because I own them, and if I don't then I can probably ask the person who does. It seems this feature is for when you need an object identied but have no one to ask. Useful for plants, sure, but less so the rest of the time.
At first glance, it looks like shopping services are included here to help you buy the identified product. But tapping on product images from shops like Target.com, or images with prices written on them opens the Pinterest website and a 404 error page. I know Bixby can't do shopping in the UK yet, but there should be a neater way of dealing with this instead of a 404 page.
The Place function pastes information about local businesses and places of interest over what the camera can see. Useful if you are walking down an unknown street, or want to know more about an unusual building, but this is something Google Maps does more successfully, albeit without the augmented reality aspect.
Identifying food with Bixby is a mixed bag. Raw ingredients work quite well, so it was able to tell me the nutritional information of a carrot and a bowl of raspberries, but it can't identify several different foods on the same plate. Like the item identifying feature, this is fairly interesting at first — impressive, even. But I was quickly wondering about the point of the feature. It's easier to input your meals manually into one of the many meal tracking apps out there, or the included Samsung Health app.
Bixby's wine identification tool seems quite niche, but makes for a nice tie-in with Vivino, the wine app. Point Bixby at the label and the system should identify it, then provide information and reviews, and offer a place from which to buy a bottle. I showed Bixby three bottles of wine and it identified them all correctly, although two took a second attempt. I could then read reviews and buy the wine via the Vivino website.
However — and this is becoming a theme — Bixby does nothing that Viviano can't do itself. If anything the dedicated (but free) app has more features, like a means of automatically saving each search.
Finally, Bixby Vision can scan QR codes. This is fine, but I honestly can't remember the last time I wanted or needed to scan a QR code.
Bixby Voice
Despite arriving on smartphones in April, Bixby's voice control system didn't arrive in July 2017. Much like Siri and Google Assistant, this part of Bixby works as a voice assistant. You can speak to it by saying "Hi, Bixby" or by pressing the dedicated Bixby button located below the volume keys. Do this, and Bixby starts listening — but you'd better be quick, because it only takes around a second for the assistant to say it "Didn't quite catch that." I quickly learned that the trick is to keep the button pressed until you have finished talking, like on a walkie-talkie.
Get this right, and Bixby is quick to understand, and either reply with voice and written messages, or present you with relevant information, like the weather forecast, or the age of that celebrity you just asked about.
Here Bixby performs roughly on par with Siri, Google Assistant and Alexa. She can look things up online and provide answers.
Where Bixby gets more interesting is with her ability to help you use your phone such as setting an image as your wallpaper, to switching on the S9's mobile hotspot function, or quickly changing a very specific setting that is hiding several sub-menus away.
I can get onboard with the simpler controls, but some are so complex I doubt anyone is going to remember or use them. For example, you can say or type to Bixby: "Open Samsung Notes and search notes containing the word 'meeting' and move them to the Work category."
Another obscure one is: "Open Gallery and delete the URL information from the most recent screenshot."
Then there's: "Open Settings and turn off Speak while screen off for Voice Assistant."
I don't really know what that means, but I guess if I did then Bixby would have me covered.
This is all fine if you find a few really useful commands, then memorize them and delegate them to Bixby. But I feel for most users very few of these will ever be used. An obvious exception is "Hi Bixby, call me an Uber to take me home," which is genuinely useful and simpler than opening the Uber app.
Conclusion
In short, Bixby Voice works more reliably than Bixby Vision and is clearly very powerful when you learn how to use it, but with both systems I wonder who is going to bother. I mostly use Siri to set timers for when I'm cooking, or alarms for the next morning. I'd use Bixby for this too — plus the odd Google search. Perhaps I would also switch the S9's mobile hotspot on and off when I needed it as well, which is useful and convenient, but certainly not groundbreaking in the same way Siri was when it launched, and how Alexa is today.
Bixby Vision isn't going to replace Google Translate when I need to check a menu on vacation. And while the wine search feature works well, I have just installed the proper (and free) Viviano app because it has more features.
Smart home control is still completely missing from Bixby. Samsung is working to improve its SmartThings home control app, so hopefully this will change soon. But for now there is no way to dim the lights and lock the doors by talking to Bixby. Siri, Alexa and Google Assistant can all do this, and much more besides.
Only 12 months old, Bixby was several years later to the assistant game. Samsung has tried to give Bixby some different features to help overcome being late, but most of them just aren't particularly useful. Samsung could create a neat ecosystem for controlling TVs, fridges, washing machines and much more with Bixby. But that still feels a long way away.
