September 7, 2019 at 4:51 pm #575
According to the Smart Audio Report prepared for NPR by Edison Research, here are the top 10 ways in which users used their devices in the week prior to participating in the survey:
- Play Music (77% of users)
- Get Weather (75%)
- Answer a general question (74%)
- Set a timer/alarm (53%)
- Check the time (53%)
- Get the news (42%)
- Listen to online radio (37%)
- Have the device tell a joke (36%)
- Get a sports score or update (35%)
- Receive reminders (34%)
Currently, you can play your own media collection using the “mpdcontrol” plugin, but I think people would also like to be able to use Spotify, Amazon Prime Music, and other cloud based services.
We have the “WWIS Weather” plugin, which is international but currently serves a limited number of metropolitan locations. The goal was to have one “general” weather plugin, but have most users replace that with something more specific to their location. In the US, the National Weather Service has an API that could be used, and I think a lot of other countries have their own services.
Our main impediment to answering a general question is improving the local speech to text processing. However, assuming you are either using a cloud STT service or the offline STT has been improved, Wolfram Alpha seems like it can handle most of these requests, as demonstrated in Oscar Liang’s speech recognition project at https://oscarliang.com/raspberry-pi-voice-recognition-works-like-siri/
A kitchen timer (“Naomi, tell me when three minutes is up”) or alarm clock (“Naomi, wake me up in eight hours”) should be fairly easy to do.
Check the time – Yep, we got that one covered in the “clock” plugin. However, I have extended the functionality of my own copy to also be able to report the time in several locations where I have family outside my current timezone, so I’m not calling them in the middle of the night. If someone could extend this with the ability to request a location, and possibly remember locations (so I would have to specify Richmond, British Columbia the first time I ask, but then it would assume BC when I just ask for Richmond in the future) or people (so I can ask “What time is it where my brother lives?”) those would be nice additions.
Get the news – This is covered in “news” which pulls from the news.google.com feed and “hn” which pulls from the hacker-news.firebaseio.com feed. I’d like to see a full blown news aggregator written for Naomi and the ability to listen to full articles, not just have articles emailed to myself.
Listen to online radio – this should be fairly easy to write, since most online radio stations use a .m3u or .pls playlist that can be used to stream audio. Most radio stations should just be a matter of plugging in the name you want to use for the station and the URI of the playlist, although finding those URI’s is a bit of a scavenger hunt. I’m sure there are searchable databases of streaming feeds, so maybe this can be made easy for people.
Having Naomi tell a joke is covered in Joke, although it would be nice to easily add and share jokes and address humor outside the knock-knock joke domain.
Get a sports score or update – This would be a nice feature to add, but I have no idea how fractured real time sports score reporting is.
Receive reminders – this can be done using the “googlecalendar” plugin, but it would be nice to allow Naomi to consume .ics files directly, possibly by having Naomi watch your email or its own email account for .ics attachments.
Well that should cover the most common use cases. If you have ideas for other speech handler plugins, please share!
September 8, 2019 at 8:22 pm #587
- This topic was modified 1 year, 1 month ago by aaronchantrill.
A couple of additional thoughts:
When I first saw the email plugin, I thought it would be like the email reader in the movie “HER”, which would actually read the text of emails to you. So I’d still like to see something like that, where you ask for a list of new emails and ask to have them read to you by author or subject line.
I’d also like to see a system where you can communicate with your Naomi via text or email, sending requests for information like “what meetings do I have today?” or “what’s on my grocery shopping list?”
I’m working on pulling the email access stuff into app_utils.py, so that there is no reason for a plugin to need access to the email address, password, server, etc information. That way we could work on restricting access to that sort of sensitive information to the app_utils.py file only.
And yes, I’d love to have a way of generating shopping lists. This seems like it would be pretty easy to implement. Even if you have to create the lists and standard things you need in profile.yml, it would be great to just yell out “Hey Naomi, add eggs to my shopping list” or “Naomi, add masking tape to my hardware list” and be able to pull up the full list later. And this would be a really good thing to combine with a text/email interface.
- This reply was modified 1 year, 1 month ago by aaronchantrill.
You must be logged in to reply to this topic.