Cubic.ai app

Cubic Robotics was founded by the talented Yuri Burov, whom RBC once named among Russia’s best IT entrepreneurs.

I had already worked with Yuri on a robotics project before: an interface for a telepresence robot. That project never went into mass production, but its technology is reportedly used by large companies in voice control systems.

The team then decided that hardware was too hard, moved to California and set out to build an app for controlling a smart home. The project was named Cubic.ai because the first prototypes of the hardware assistant were cube-shaped.

By the time I was invited to design the app’s interface, the team already had a basic design and a visual concept. Everything revolved around the cube. The shape gave rise to the name, and the name gave rise to the logo, a cube in isometric projection:

When a designer has found such a neat, coherent concept, the job is half done. Even if the concept itself is debatable, applying it consistently across the product gives a much better result than designing everything piecemeal. A systematic approach to design works a bit like statistics on the stock market: a strategy may lose money for a long time, but the total of all trades ends up positive.

Cubic’s designer had the right idea with the logo and took it further, turning it into the virtual assistant button, like Siri’s.

Then the designer made a set of icons in the same style. The app’s purpose was to control a smart home: smart devices from different companies, such as color light bulbs, thermostats and gas leak sensors, were brought together in a single control panel. Each device needed a clear icon that could also show its status.

Here are the icons the designer came up with for the thermostat. The icon’s shape is again borrowed from the logo, with the current temperature and, for example, humidity written inside.

The app got a system of rooms. Devices could have been grouped by type: light bulbs, thermostats and so on. But it is much better to make users associate the app with their own home. So the sections are living room, hallway, kitchen and bedroom. Then the control panel that Cubic aimed to be feels like a real house.

For the rooms, the team prepared preset names and drew several icons in the same cube style as the devices:

It was around this stage that I joined the project.

Seeing sound

Here is a paradox: the idea of a voice evokes very definite visual associations.

Voice is sound. Sound is the vibration of particles in a medium. When we speak, air leaves the lungs, and muscles narrow and widen the vocal cords. This makes air molecules vibrate at different frequencies, and the vibrations travel to the ears. There, the air waves hit the eardrums, our auditory receptors pick up the vibrations, and the nervous system converts them into something like an electrical signal.

Sound works completely differently from light. Light exists on its own: it is a stream of particles, photons. Sound is the vibration of something else: air, water, a piece of iron. That is why light can travel through outer space and sound cannot: there is no medium to carry it.

People don’t intuitively grasp this difference, so we picture both light and sound as waves. Our brains can even mix the two up. It is a well-known fact that on LSD people hear light and see sound. That is why it is so easy to depict sound waves as light waves. An animated wave representing a sound recording has the same effect as LSD: it feels like we can see the sound.

Every single voice assistant uses this effect. There is no assistant without some kind of sound-and-light animation.

It is not just about looks. It is about feedback, which every interface must have. Without feedback, people assume the interface has frozen.

Feedback differs for different kinds of interaction. I think the rule is this: the longer the interaction, the longer the feedback should last. With a button, for example, it is very simple. We tap the screen for a split second, and the button turns slightly darker for a split second. That is enough to show it has been pressed. Speaking takes much longer. Commands to a virtual assistant can last 10–20 seconds, and we need continuous feedback the whole time. Giving the assistant button an inner shadow to show it is pressed is not enough: the app may freeze after the tap without the user noticing. So speech recognition needs continuous feedback.

That is what wave animations are for. Ideally, the animation should react to the voice: the louder we speak, the bigger the wave.

So even a voice assistant prototype must have animation, and I proposed one for Cubic. Google Assistant’s animation made a good starting point. The app was released only for Android, so this kind of animation would look natural and could be built quickly.

I put the same waves, only white, inside the cube icon. It was the simplest solution. The animation didn’t react to the voice, but the motion alone was enough to provide feedback.

Then the developer stepped in and said that while tracking changes in the voice was hard, he could easily detect silence. That gave us the idea of adding a second kind of feedback for when the user has pressed the button but hasn’t started speaking yet:

We got together with the team and asked ourselves: is there a third state? It turned out there was. Once a command has been given, the app needs up to 10 seconds to reach the server and recognize it. So we needed one more indicator to show the user that the command has been received and is being processed:

These three indicators are the bare minimum for any voice assistant prototype, even the simplest one. Without this three-step feedback, an app shouldn’t be released at all.

Thermostats

Cubic’s selling point was support for many devices from different vendors in a single app. Usually each company releases its own app for its devices, which makes controlling different devices inconvenient.

So Cubic needed lots of different universal icons. The first version supported Nest thermostats, smoke and carbon monoxide detectors, humidity and water leak sensors, motion sensors, Philips Hue color bulbs, door locks and smart plugs.

Let’s start with thermostats. Every American home has a thermostat, but it is usually controlled by hand. Nest’s new thermostats had only recently started appearing in homes, and they are controlled through an app.

A thermostat regulates temperature, so the main thing the icon shows is degrees Fahrenheit. A thermostat also has a target temperature it must maintain. Let’s say it is 68 °F (20 °C) in the apartment. We set a target of 72 degrees, and the thermostat starts working. The main question is: which temperature should the display show?

Most cheap heaters show only the current temperature, so there is no feedback. How do you know the thermostat has received the command and is heating? The target temperature has to be shown. Not instead of the current one, but alongside it, otherwise we won’t know how warm the room is right now.

The display should show both temperatures, current and target. If they match (or there is no target), only the current temperature is shown, and the thermostat is highlighted, for example, in green.

By highlighting the thermostat in green, we intuitively introduced the idea of a norm. Let’s develop it. If there is a normal temperature, there should also be “cold” and “hot.” Since we decided that the temperature is normal when the current and target temperatures match, the logical conclusion is: hot means the current temperature is above the target, and cold means it is below.

That gives us two new colors: orange and blue.

There is another approach: highlight a thermostat in orange when it is heating and in blue when it is cooling. This works until we introduce the idea of overheating and overcooling.

Normal, hot and cold are relative, but there are limits that apply to everyone. If the apartment is above 95 °F (35 °C), it is overheated. Likewise, 42 °F (5 °C) is hardly acceptable in a home: it is overcooled.

Overheating and overcooling should be reported to the user as critical errors that require action.

Alerts of this level must grab attention, so the color coding needs to be extended. Let’s use red and dark blue for serious deviations from a reasonable temperature, and fill the icon with color to draw attention.

This is where highlighting the heating/cooling mode instead of the hot/cold state breaks down. If the apartment is overheated and the thermostat is cooling, the icon turns blue. If the apartment is overcooled and the thermostat is heating, the icon turns orange. In other words, we would be telling the user something false, as if there were no problem.

That is a gross mistake! The kind that leads to nuclear accidents and plane crashes. Caught you? At first glance, it seemed like a minor detail, a matter of taste.

Besides a temperature sensor, Nest thermostats have a built-in humidity sensor. This is a secondary feature, so humidity can be shown when the user taps the icon, with one exception. Air that is too dry or too humid is also quite dangerous, since it can damage sensitive electronics. So excessively dry or humid air should be flagged the same way as overheating and overcooling.

The colors are already taken, so let’s add a water drop icon and show it large in place of the temperature, moving the temperature below it.

If the apartment has both temperature and humidity problems at once, it is better to give priority to temperature and show the degrees, moving the drop icon down or into the settings window. I didn’t work out this case in the first version of Cubic.

Here are all the thermostat states:

Smoke detectors

Nest also makes smoke and carbon monoxide detectors. A single sensor detects both smoke and carbon monoxide. Each has three hazard levels: low, medium and high.

We will extend the color coding: red for high danger, yellow and orange for low and medium.

One sensor covers two kinds of hazards, so we need extra coding with icons. The icons aren’t particularly complex, but we can’t simply use a smoke icon: smoke is usually drawn as wavy lines that don’t convey any sense of danger.

So it is better to use a fire icon. It is not a literal depiction, but fire is far more recognizable than three odd lines. And it looks more dangerous.

There were plenty of fire icons. I liked three of them. You could pick one, or use all three according to the hazard level:

Carbon monoxide has no visual association at all. Carbon monoxide (chemical formula CO) is a colorless, odorless and tasteless gas that can kill you before you notice anything. It is probably best to pick a scarier icon.

Trying out the icons:

Looks quite dangerous.

There may be a small fire and a high level of carbon monoxide at the same time. In that case, the icon should be red, matching the highest threat level, and both icons should be shown.

Other icons

Besides thermostats and smoke detectors, for the first version of Cubic I made icons for light bulbs, a motion sensor, a water sensor, a door lock and a smart plug.

The complete icon set:

Devices are grouped into rooms on the app’s home screen:

Dashboard

Cubic is a free app with a paid option. For $10 a month, the user gets a dashboard, a system of alerts and tips. The dashboard sends notifications to the phone if something is wrong at home and advises what to do in an emergency. For example, in case of a carbon monoxide leak, open the windows.

The dashboard is accessed from the app’s home screen via the Dashboard button in the menu bar.

When there is a threat, the dashboard button should be highlighted.

The dashboard itself explains the status of every device. Future versions of Cubic were to support security cameras, which would also be available in the dashboard.

Other sections

Controlling light bulbs and the thermostat:

A built-in marketplace is one of the popular ways to monetize an app:

Screens for adding a new device (Philips Hue):

Concept of a voice assistant built into the system:

What happened next

The app was released in 2017. Several major American media outlets wrote about it. After a year, the project still hadn’t become profitable and was shut down.

Introduction
Sound animation
Thermostats
Smoke detectors
Other icons
Dashboard
Other sections