In contrast with other types of Android apps, a Home APIs app has its own unique considerations and challenges, including dependencies on physical smart home devices, network reliability, security and privacy, and device and protocol interoperability. These differences have an impact on testing.
Set up your test environment
A robust testing environment is vital for testing a Home APIs app.
It's a good strategy to isolate your Home API test devices on a dedicated Wi-Fi network. This prevents interference from other devices and lets you simulate different network conditions such as throttled bandwidth.
You'll want to test using a variety of smart home device types, from a diverse array of manufacturers. Be sure to test with real, physical devices. Virtual devices and emulators have their place but shouldn't be relied on exclusively.
Likewise, testing your app with a variety of Android devices from multiple manufacturers, with varying screen dimensions, and running different Android versions helps to verify broader compatibility.
Assemble your tools
The following tools are helpful in testing a Home APIs app:
| Tool | Description |
|---|---|
| Google Home Plugin for Android Studio | Provides access to Google Assistant Simulator, Home Graph Viewer, Google Cloud Logging, and the Android Debug Bridge (adb) Log viewer. |
| Google Home Playground | Not a substitute for testing with actual physical devices, but invaluable for many testing scenarios. |
| Matter Virtual Device (MVD) | Another emulation app that can be helpful when testing with Matter devices in a Home APIs app. |
| Espresso | Lets you automate UI testing. You need to mock the underlying API to simulate hardware interactions. |
| A Network emulator | Lets you simulate varying network conditions and API responses, which is especially helpful if your Home API uses a cloud backend. |
| Android Studio Logcat | Handles and analyzing log data. |
Apply different testing approaches and strategies
You should apply a variety of testing approaches to your Home APIs app. For general information about different methods, see Test apps on Android. In addition to the user experience and security testing that all Android apps are expected to undergo, some areas of testing are especially pertinent for Home APIs apps and are highlighted in more detail in the following sections.
Functional testing
Depending on the capabilities of your app, you may want to pay particular attention to some of the following areas:
- Device discovery and pairing, including initial setup and pairing with new devices, re-discovery of existing devices after app restarts, device reboots, or network changes. Also check scenarios such as unresponsive devices, or a user entering incorrect credentials.
- State synchronization, making sure that the app updates to accurately reflect current state in a timely manner.
- How permissions are managed and enforced, especially if your app supports multiple users or shared home access. This includes verifying that your app gracefully handles revoked permissions.
- If your app supports scenes, such as a "Goodnight" scene that turns off all the lights, test scene creation, modification, and execution. Test automations, including starters and actions.
Performance testing
Verify that your app is responsive and doesn't introduce any unnecessary latency. If possible, test with a large number of devices to assess scalability and how your app performs under load. Also look at the CPU, memory, and battery consumption of your app, especially during active device communication, to see whether its use of resources is reasonable.
Reliability and stability testing
Due to their dependency on network connectivity, Home APIs apps should be tested against the following scenarios:
- Network fluctuation
- Simulate Wi-Fi dropouts, weak signals, and network switching.
- Verify that your app handles disconnections and reconnections gracefully.
- Verify that commands queue and execute when connectivity is restored, or provide appropriate error messages.
- Test the app from remote networks, not just from the home network.
- Device disconnection
- Physically unplug devices.
- Power cycle devices.
- Concurrent operations
- Send a rapid sequence of commands.
- Have multiple users try to control the same device simultaneously.
- Test overlapping scenes and automations.
- Error handling
- Force API errors, for example, by mocking cloud API server responses.
- Test invalid input from the user.
- Verify graceful degradation and informative error messages.
- Reliability - Run the app and connected devices for extended periods to uncover possible memory leaks or stability issues.