• Aug 21, 2017
• 1 Comments

In our previous article “War of Artificially Intelligent Personal Assistants”, we discussed several Intelligent Personal Assistants (IPAs) that are available in the market. These IPAs can be activated by voice commands and used to control smartphones. In this article, we will try to highlight its potential in another upcoming technology, Smart Speakers. Simply put, Smart Speakers are portable speakers that can not only produce sound but are also intelligent enough to simplify our day to day tasks at home. The integration of IPAs with home speakers enables them to interact with humans. A user can ask questions, get suggestions or generally just talk to these speakers and they will respond with genuine answers. A smart speaker is much more than what the world has experienced ever before. The idea might seem abstract or insane but artificial intelligence is making it a reality. This artificially intelligent voice activated personal assistant speaker is programmed to perform activities like change TV channels, turn on/off lights or other home appliances, change the fan speed, read a book, take notes, order food online, and even shop for groceries. Imagine doing all these activities without touching our smartphones. These home speakers are like small robots that reside in our home and perform multiple activities simultaneously with nothing more than a few voice commands. The market has started to flood with smart speakers. Now, almost every technology company is developing its own IPA equipped smart speaker. The trend was started by Amazon three years ago; when it integrated its artificial assistant, Alexa with a home speaker it named Echo. Last year, Google also came up with “Google Home”. And this year, Apple also joined the fray by introducing its “HomePod” at WWDC conference held in the month of June. Seemingly, other potential key players are Microsoft and Samsung who are also building their IPA integrated speaker. We can say that the market area is pretty nascent and has much more potential to grow in the future. Here is a quick timeline that shows the journey of smart speaker’s market development. We, at Citius Minds, don’t just look at products from a consumer or technological point of view; we tend to break them down into their patents to see the actual invention. Thus, we perform an extensive analysis to study how the technology behind Smart Speaker was slowly transforming into a Home Automation Hub. Our analysis yielded some important technical features of “Smart Speakers”, which are as follows:

• Compact and Portable Design
• Internet Connectivity
• Audio Playback
• Multi-User Support
• Microphone Performance
• Home Assistant
• Online music streaming
• Voice activated & Voice controlled

This article deals with technological insights in this field from product as well as patent perspective. Currently, the top three products in the market are – Amazon’s Echo, Google’s Home, and Apple’s HomePod. 

Amazon Echo

Currently, Amazon sells three products in this line; Amazon Echo, Echo dot, and Echo show. All these products incorporate Amazon’s intelligent personal assistant Alexa. Alexa is programmed to listen to a user’s voice command and act accordingly to prepare a to-do-list, stream web content, control smart home devices, book online movie tickets, shop on Amazon, place an online order for food and much more.  We have discussed the technology behind Amazon Echo in detail, which is as follows: Compact and Portable Design Amazon Echo is cylindrical in shape with 9.25 inches tall and weighs about 2.33 pounds. The compact design has made it portable and easy to place anywhere around the work area. The user control interface includes a mute button, an action button, a light ring and a volume control ring. The light ring is a visual communication to the user about device's status and for notification. Internet Connectivity Echo supports Wi-Fi connectivity and allows the user to do online web streaming. It supports 802.11a/b/g/n Wi-Fi networks. Additionally, Echo also supports Bluetooth connectivity which allows the user to connect with their smartphone or other devices. Audio Playback Echo provides hands-free voice control for playing song available on various music services like Amazon Music, Spotify, Pandora, iHeartRadio, TuneIn, and more. Echo has dual downward-firing speakers which produce 360° omnidirectional audio. Multi-user support Echo supports multi-user accounts by creating separate profile for each user. It holds accounts of multiple family members and easily shares their to-do-list, music, audio book and much more. A user can switch his/her account by simply saying "Alexa, switch account". Microphone Performance Echo has Far-field voice recognition capability and enhanced noise cancellation capability, which allows Echo to listen to the user’s voice while playing music. Echo has seven microphone sensors embedded at top of the device. These sensors have “beam forming technology” which enables it to recognize user’s voice from any direction. Home Assistant Alexa has over 3,000 Alexa skills for assisting its users. Alexa skill Kit(ASK) is created by a third party. One important aspect of these skills is smart home skill API. This provides a Smart home automation hub by controlling appliances such as lights, switches, thermostats, and more. The key players in the smart home industry that are compatible with Alexa are Philips Hue, Ecobee, and Samsung SmartThings. Online Music streaming Amazon basically offers two music options: Amazon Prime Music and Music Unlimited. However,  it can also connect to other third-party music services like Spotify, Pandora, iHeartRadio and TuneIn. For setting up these third-party music services users need to link their Amazon accounts. Voice activated & Voice controlled As earlier discussed Echo uses voice controlled intelligent personal assistant Alexa to play music, make calls, send and receive messages, do web streaming, and more. In the default mode, it continuously listens to all speeches and monitors for the wake-up word to be spoken which is Alexa by default. The wake-up word can be changed to "Amazon", "Echo" or "Computer" by the user.

Related Patents - Amazon

To complete our analysis of Echo, we also analyzed Amazon’s plethora of patents in this domain, some of which are explained below: Amazon’s patent US 9,424,840 B1 discloses a speech recognition platform in a home environment. This speech recognition platform can be implemented with voice control devices. Moreover, this platform is capable of recognizing user’s voice and the purpose of the voice command to perform activities like streaming audio content, setting a reminder, online shopping, making online reservations, etc. In another patent US 9,641,954 B1, Amazon discloses a technique for providing phone communication through voice-controlled devices. The technique involves associating a user’s cell phone number with the voice-controlled device. When a call is made to the user’s cell phone, the device notifies the user by indicating an incoming call. Further, the user can utter a command to answer the call. This enables the device to establish an audio connection with a cellular carrier network. A similar procedure is followed for outgoing calls where a connection is made with the carrier network after the user tells the voice-controlled device to do so. Amazon’s patent US 9,275,637 B1 discloses a technique and system to evaluate a natural language input which is termed as a “wake word”. This wake word triggers the voice control device that the user wishes to provide a voice command to the said device. With the help of these patents, Amazon is able to achieve, implement and enforce its Smart Speakers technology.

Google Home

With the advent of artificial assistants (IPAs), it is a passé by these days to only interact with smartphones. Seeing the market demand, Google also released its smart speaker - Google Home on November 4, 2016, with a vision to give more personalized experience with its new IPA, Google Assistant. According to Business Insider, Google Assistant is light-years ahead of Amazon's Alexa. In addition to this, Google home claims to offer hands-free assistant at home with more robust features. Compact and Portable Design According to Mashable, Google home is not merely a gadget but it is more like a piece of modern art. Google Home is a 5.62 inches tall and 3.79 inches in diameter. It weighs around 1.05 pounds. It is comparatively lighter and smaller than Amazon’s Echo. Apart from voice command, the user control interface also includes touch control on the top of the device and a mic mute button on the back of the device. It has four LEDs on the top for indicating its working status and a power status light on the back side. The base has a modular cover with different color options offered through Google Store intended for the device to blend into the environment. Internet Connectivity Google home connects to wi-fi networks for high-speed streaming of services. It supports 802.11b/g/n/ac (2.4GHz/5Ghz) Wi-Fi  networks. A user can also connect another device with Google home via Bluetooth. Audio Playback Google Home offers better sound quality and also provides multi-room speaker support. Users can connect and group together multiple home speakers for synchronizing playback of music in every room. Google Home has an integrated high-excursion speaker which delivers crystal-clear highs and rich bass. Multi-user support Google home also supports multi-user system which allows up to six people to engage with one device. The most amazing part is that there is no need to switch between different accounts, instead, Google Assistant learns to recognize users’ voices and serve them their personalized information accordingly. Microphone Performance Google home includes two far-field microphone sensors attached on top of it. The far-field microphones and natural language support system provide better interaction with the device. Home Assistant Google Home has a Home automation feature which allows a user to control smart devices with simple voice commands. Thus it can act as a single control hub for controlling the other smart home appliances like chromecast digital media player, and products from Nest, SmartThings, Philips Hue, LIFX, LightwaveRF, and Logitech Harmony. Online Music streaming Google Home has Google Play music as default music streaming service. Like other streaming services, the user needs to sign up for a premium subscription to unlock the full potential of Google Play Music. The subscription to Google Play music also unlocks ad-free listening in YouTube Music, which works with Google Home. Google Home supports other music services like Spotify, and Pandora, TuneIn and more. Voice activated & Voice controlled Google Home includes Google Assistant for assisting its users. This device can also access google account for information about the daily routine of its users in order to provide better services. A large number of services are integrated, allowing users to listen to music and accessing information entirely by voice commands. The wake-up word for the Google Home is “OK Google”.

Related Patents - Google

Again, to understand whether Google has safeguarded itself with patents, we analyzed its patents in this domain, one of which is explained below: 2017 published Google patent US 20170025124 A1 describes a method for providing coordination among a plurality of voice controlled electronic devices in order to mitigate noise. In multiple voice-activated electronic devices environment, a suitable voice-activated electronic device is selected to respond to the input voice command. The selection is based upon a device that detects the input voice best or a device that is closest to the user. In another embodiment, it also discloses that if the input voice command is relevant to a particular device, then that particular voice-controlled electronic device will be selected to respond to speaker’s command. For instances – A voice command like “stop music” will refer to a device playing music. Also, the server and/or device is trained to identify a speaker of the input voice based on voice models comparison and provides a personalized response to its identified speaker. In one of its embodiments, this patent also discloses a noisy environment where the device and/or server is capable to respond to a ‘hotword’ (voice input), that triggers the device and/or server to wake up to receive a voice command.

Apple HomePod

At 2017 WWDC, Apple unveiled its voice-assisted Smart speaker - HomePod, which is based on a voice controlled intelligent personal assistant - Siri. For now, Apple is more focused on sound quality. HomePod can play music with High-Fidelity for rich and deep sound experience throughout the room. It has a unique feature to sense the room environment and automatically adjust the tune. Additionally, Siri will give users a more personalized music experience. Further, Apple claims homepod to be the new sound of home. The device is yet to debut in the market. The HomePod is scheduled to release in December 2017. The following analysis tries to show where Apple is really focusing so as to differentiate its HomePod from the others in competition: Compact and Portable Design HomePod supports compact design features with 6.8 inches × 5.8 inches dimensions and weighs around 5.5 pounds. The top of the device has a touchscreen that displays a dynamic LED waveform. This dynamic LED waveform indicates that Siri is listening and it animates with every word of the user. The device is made up of seamless mesh fabric which is designed for both aesthetic and acoustic performance of the device. Internet Connectivity HomePod supports 802.11a/b/g/n/ac Wi‑Fi with MIMO networks for connecting the device to cloud services. Audio Playback HomePod allows its users to pair multiple speakers and play music in multiple rooms by using Airplay. It features 7 tweeters in its base and a 4-inch woofer on the top. The speaker is equipped with an A8 chip. The chip does all the audio engineering which includes real-time acoustic modeling, audio beam forming and echo cancellation. It provides Hi-Fi sound throughout the room so that everyone can get an immersive music experience irrespective of the location of the device in the room. Multi-user support Nothing disclosed yet. Microphone Performance HomePod can also hear voice commands over the music. HomePod has an internal low-frequency calibration microphone for automatic bass correction and a six-microphone array for far-field Siri and room sense. Home Assistant The primary function of a HomePod is focused on the speaker rather than as a home assistant. According to BI Intelligence, only 10% of US homes are using smart home devices. However, in the coming year, more users are expected to join. Therefore, Apple has not completely ignored this vertical. Beyond playing good quality music, HomePod is also helpful as a home assistant for everyday household activities. HomePod also provides smart home control with voice commands. This is achieved by integrating HomePod with Apple’s Homekit platform. This protocol allows software developers to make applications that work with home automation products. The user needs to just add the appliance to the Home app and then he can control it with his voice. Online Music streaming HomePod is integrated with Apple’s music streaming service iTunes. Voice activated & Voice controlled HomePod is controlled through voice interaction with Siri. The trigger word for HomePod is  "Hey, Siri!". Users can ask HomePod to play a song or even can ask who is singing or playing guitar on a track. Users can ask the speaker to read them the news, play podcasts, set reminders and everything that's possible with Siri on other Apple devices.

Related Patents - Apple

Apple is known to protect all its ideas proactively. A brief study of its patents in this domain yielded the same conclusion. Some of Apple's patents related to this domain are discussed below: In Apple’s published patent US 20150348554 A1 titled “Intelligent assistant for home automation”, a system and process for controlling electronic home devices through virtual assistant on a user device is described. Further, the user device can be a mobile phone, computer, portable media player, television or a wearable electronic device. The user can give a voice command to these devices. The input voice command is received by these devices and transmitted to the server. At the server, voice input command is converted into textual information. Based upon textual information, the servers will identify those electronic devices. After identification, the command is transmitted back to the user device. Further, the user device sends the assigned command to the electronic devices for execution. The electronic device performs the command and sends its current state to the user device. Wake-up words are used as speech that triggers the device for receiving user’s command. These devices are always listening for the wake-up word. However, these triggers sometimes can be missed due to background noises and other factors. Also, sometimes these speech triggers can be falsely recognized by the devices. In another published application US 20160077794 A1 titled “Dynamic thresholds for always listening speech trigger”, Apple discloses a way for mitigating such flaws. This patent discloses a system and process for dynamically adjusting a speech trigger threshold in response to a variety of perceived events.  The process is executed by determining a confidence level of a sampled audio input which includes a trigger word. Further, the confidence level is determined by a variety of methods such as – a sampled audio can be compared to audio flies or other digital representations of the accepted speech trigger.  Under decision section, a confidence level is compared with a threshold value and triggers the virtual assistant to receive the command.

 Technology Summary

Let us take a quick look back at the differences and similarities of the discussed products:

Conclusion

The one thing that we can definitely say is that the evolution of intelligent personal assistants (IPAs) has come up as a perfect precursor to automate our homes. The smart speakers have added another dimension to home entertainment and automation. The technology and feature comparison analysis give us a fair idea about the scope of this technology. Each of the three has its advantages. For instance, with the third party intervention to develop Alexa skills, Amazon Echo has become more robust. Google Home has a huge ecosystem in its favor and thus has the advantage of being integrated with a search engine, email accounts, and other products. Apple is known for perfection and their philosophy is that Smart home automation is yet to take the world by storm. So even by not ignoring its eventual future benefits, they launched HomePod and focusing primarily on audio quality of speakers. HomePod offers a better sound quality than others in the same trend and promises a level of privacy that the others don't have. Click here to download the full article… Citius Minds is a technical consulting firm based out of Chicago. We provide specialized patent analytics and litigation services to patent professionals across the globe. We are highly appreciated and loved by our clients for the following service offerings: | Patent Invalidity Search | Patentability Search | Freedom to Operate Search | Infringement search | Patent Landscape Analysis | Patent Portfolio Analysis | Infringement Contentions | Invalidity Contentions | Prior art search | Rule 11 charts | Source Code review | Pre-litigation analysis | Clearance search | Patent Validity Search |