If you want an artificial intelligence to power some parts of your home, you can always head online and purchase a Google Home or Amazon Echo, two devices that can control everything from your music to your lights.
Or if you’re a certain kind of tech pro, you can also build your own home assistant.
Facebook CEO Mark Zuckerberg is apparently that kind of tech pro. As one of his annual “personal challenges,” he decided to carve off a few hours every week to code his very own digital assistant, which he named “Jarvis” after the highly advanced (and very wry) A.I. in the “Iron Man” and “Avengers” movies:
According to a note he posted on Facebook, Zuckerberg built Jarvis to not only monitor and control everything from lights to the thermostat and security cameras, but also respond to both natural-language inputs (“Turn on the lights!”) and text messages. As with many tech projects, this task became exponentially more complex as he wrestled with the nuances of the English language.
“It started simple by looking for keywords, like ‘bedroom,’ ‘lights,’ and ‘on’ to determine I was telling it to turn the lights on in the bedroom,” he wrote. “It quickly became clear that it needed to learn synonyms, like that ‘family room’ and ‘living room’ mean the same thing in our home. This meant building a way to teach it new words and concepts.”
Ever the overachiever, Zuckerberg also attempted to wire up various appliances around his home to respond to Web-based outputs, including a toaster. As it turned out, however, browning bread via an app required some old-school technology. “It’s hard to find a toaster that will let you push the bread down while it’s powered off so you can automatically start toasting when the power goes on,” he wrote. “I ended up finding an old toaster from the 1950s and rigging it up with a connected switch.”
In similar fashion, he would have needed to make some hardware modifications in order to create a Web-enabled automatic dog-food dispenser, or a t-shirt cannon. Because what’s the point of owning an absurdly software-powered home if you can’t use it to fire t-shirts directly into your waiting arms?
Having multiple people in a house can also cause issues with an A.I., which is a key reason why Google and Amazon tether their devices to one account. “When I tell it to turn the AC up in ‘my office,’” Zuckerberg wrote, “that means something completely different from when Priscilla tells it the exact same thing. That one caused some issues!” The inability of Jarvis to automatically determine human locations was also a problem—the system would start playing music in the wrong room, for example.
For those interested in experimenting with A.I., Zuckerberg used the following platforms (all maintained by Facebook, of course):
- Messenger Bot Framework
- Open-source Nuclide packages
- The Buck build system
- Open-source FastText AI text classification tool
- Facebook Research GitHub repo
Zuckerberg claims that the effort “was easier than I expected.” If you’re interested in the possibility of controlling your home via a homegrown A.I., check out his blog, which offers much more detail on how he carried out the project. With the right tools and a bit of ingenuity, you may even invent your own twist on the concept.