denialtek

XToys - Designing a Multi Platform App

Added 2020-11-06 01:09:54 +0000 UTC

XToys uses a lot of modern web technologies and it simply wouldn't have been possible to develop the app a few years ago.

XToys is built on top of Quasar which just hit it's 1.0 milestone back in February 2019. Quasar is a framework that sites on top of Vue.js and provides a lot of helpful additional features and pre-built UI controls that made it much easier to develop the site.

The iOS and Android apps are built using Capacitor which allows the website to run within a web view on the phone but access native features of the phone like running in the background, access to the Bluetooth stack and to intercept the phone's volume keys.

Everything else leverages modern features within the Chrome web browser:

Web Audio API - The Web Audio API is an extremely flexible API that allow for dynamic sound creation. XToys uses it to on the fly convert the Pattern that's running to an audio waveform in realtime.
Gamepad API - Connect to and use gamepads on the web. The Gamepad API has some limitations on mobile that make using it less than ideal (the site tab needs to be open and visible). So the Android app leverages the Android Gamepad API instead of the web API.
Bluetooth API - Connect to your toys. The Bluetooth API unfortunately isn't supported by the browsers that Capacitor uses so the Android and iOS app leverage the Android/iOS Bluetooth APIs instead.
Speech Recognition API - React to voice commands. Unfortunately in Chrome the Speech Recognition API has a built in word filter so many words that you might want to say while using the app gets bleeped out to "f*****".
WebRTC - Peer to peer connection of clients. XToys uses this to directly connect you to other people in your Session and it allows for realtime reacting to actions done on the other device. No servers needed.

The backend runs on Google's cloud infrastructure which allows for automatic quick scaling up as needed.