Testing iOS Push Notifications in CI/CD Pipelines
iOS applications are a great way for an user to have a richer customer experience as part of a SaaS platform. One of the many ways in which we enrich such experiences is through Push Notifications or Remote Notifications.
The first push notifications we add to an application are usually simple both in structure and behavior; simple in the sense that they’re just text to be displayed to the user or simply modifying the app badge counter, but as our application grows in features and complexity so does the behavior and structure of the notifications.
Since we have more complex behavior, the likeliness of introducing a regression or bug when we change our notification code increases, ideally we’d manually test each notification scenario to make sure that we’ve not “broken” anything however this becomes cumbersome and error prone.
Since Xcode 11.4 it is possible to have a simulator instance receive a push notification using the xcrun simctl push command.
While it’s awesome that we no longer need certificates, a network connection and more importantly an actual device to test (some) of the push notification functionality; we still have to test manually by using the CLI and inspecting the behavior of our app visually.
In this article we will explore how to test push notifications in CI/CD pipelines using XCTest and Bash.
Sample Projects
I’ve prepared a sample project that has a few tests setup and that makes use of a bash script to execute the tests. Below you can find links to the source of each project.
Strategy Overview
Since our push has to be sent by xcrun simctl push command we need a way to coordinate the app being tested (PushNotifications), the UITest runner (PushNotificationsUITest) app, the iOS Simulator and the execution of the xcrun simctl push command.
To coordinate all of this we’re using a Bash script, which performs the following actions:
- Starts monitoring device logs
- Starts UITests using
xcodebuild - Detects the
PushNotificationsUITests“ready” marker - Invokes
xcrun simctl pushwith the appropriate payload for the UITest xcodebuildtests completes and script performs cleanup- Test finishes reporting success or failure
Here’s a diagram that can hopefully make things clearer:
1. Bash script starts UITests
As is usual in a CI/CD env, our tests will be started by the bash script, our script assumes that theres a target bundle called PushNotificationsUITests that will be the tested target:
# Placeholder for whatever command is needed
device_udid=$(find_device_udid)uitest_output=$(mktemp)
xcodebuild -project ./PushNotifications.xcodeproj \
-scheme PushNotifications \
-only-testing:PushNotificationsUITests \
-destination "platform=iOS Simulator,id=$device_udid" \
test > $uitest_output &xcodebuild_tests_pid=$!
It’s important to note that we will be running the tests as a background process so as to not block our main bash script. We will also store the output of the test so that we can inspect it later on.
(You can also use test-without-building or Test Plans for executing your tests, as long as its in the background and using Xcodebuild).
2. Wait until the app is ready to receive the push notification
After the tests have started and, our push notification test test_display_push_message is running, it should use taps, clicks, selections, etc, provided by the XCTest framework to get to the screen or point at which it can receive the push notification, for example in our test we simply tap a button to navigate to the screen that is able to handle the push notification:
func test_display_push_message() throws {
app.launch()
app.descendants(matching: .button)["Receive Message"].tap() // ... rest of the code ...
}
3. Push Notification UITest sends “ready” marker
Once we’re certain that our test and app are ready to receive the push we emit a text marker from the test_display_push_message test that will indicate to the bash script that it should send the push notification, adding to our previous test function code it should now look like this:
import os // provides os_log// ... class body ...func test_display_push_message() throws {
app.launch()
// This will trigger a `xcrun simctl push` to be executed.
os_log("XCUI-SEND-MESSAGE-XCUI", type: .default) // ... rest of the code ...
}
Our marker in this case is the following string:
"XCUI-SEND-MESSAGE-XCUI"4. Bash script detects “ready” marker
Since our tests run in the background, the main bash script will also run a process that monitors the simulator’s output and more specifically the output of our UITest runner app. To achieve this we will have 2 processes running:
- A background process that writes the test runner logs to a file. (
xcrun simctl spawn). - A process in the main bash script that reads the file and looks for the marker.
# 1. Start the background log writer
uitest_logs=$(mktemp)
xcrun simctl spawn $device_udid log stream --predicate 'process CONTAINS "PushNotificationsUITests"' > $uitest_logs &# ... xcodebuild test commands ...echo "=> Waiting for 'XCUI-SEND-MESSAGE-XCUI' marker"
sh -c "tail -n +0 -f \"$uitest_logs\" | { sed '/XCUI-SEND-MESSAGE-XCUI/ q' && kill \$\$ ;}"
The monitor command is a bit complicated but essentially what it does is the following:
- In a new shell start tailing the file (
sh -c). - Pipe the output to
sedand when it encounters the marker pattern, exitsed. - When
sedexits it kills the parent processsh -cby expanding the\$\$variable. - Since the subshell exits the main bash scripts continues.
Notice how the “monitor” command waits for the same marker that we have in our swift/test function: XCUI-SEND-MESSAGE-XCUI
5. Bash script invokes simctl
When our “monitor” command detects the marker execution on the main bash script will continue, we can use this to construct a JSON object that the xcrun simctl push command understands and to deliver the push notification:
push_payload=$(cat <<EOF
{
"Simulator Target Bundle": "com.aztristian.mobile.PushNotifications",
"aps": {
"alert": {
"body": "Test"
}
},
"custom_id": 999,
}EOF)echo $push_payload | xcrun simctl push $device_udid -
Here we have a hardcoded push payload object, but we could easily load the contents from a file or another process depending on the marker or some other parameters. Be sure to replace the Simulator Target Bundlewith the value of your app’s bundle id.
6. Verify the push contents
At this point our app should receive the push notification and since the application should be in the Active state the received push notification handler should forward the notification to the controller causing for the controller to present a pop up with the information that we intend to verify.
Our remote notification handler in swift looks like this:
extension AppDelegate: UNUserNotificationCenterDelegate {
func userNotificationCenter(
_ center: UNUserNotificationCenter,
willPresent notification: UNNotification,
withCompletionHandler completionHandler: @escaping (UNNotificationPresentationOptions) -> Void
) {
let userInfo = notification.request.content.userInfo
let title = "Notification Received"
let message = "CustomID: \(userInfo["custom_id"] ?? "")"
window?.rootViewController?
.showAlert(title: title, message: message) completionHandler(.list)
}
}
Don’t forget to setup the AppDelegate as the UNUserNotificationCenter delegate:
// .. inside AppDelegate class ...
func application(_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions:[UIApplication.LaunchOptionsKey: Any]?
) -> Bool { let current = UNUserNotificationCenter.current()
current.delegate = self
return true
}
We are also using the following extension to present an alert from a UIViewController :
extension UIViewController {
func showAlert(title: String, message: String) {
let alertController = UIAlertController(
title: title,
message: message,
preferredStyle: .alert
) alertController.addAction(
UIAlertAction(title: "OK", style: .default))
DispatchQueue.main.async {
self.present(alertController, animated: true)
}
}
}
Once our controller presents the alert our UITest will have code that matches the title of the alert, in this case: Notification Received ending up with our full test looks like this:
import os
import XCTestclass PushNotificationsUITests: XCTestCase {
func test_display_push_message() throws {
XCUIApplication().launch()
// This will trigger a `xcrun simctl push` to be executed.
os_log("XCUI-SEND-MESSAGE-XCUI", type: .default)
let alertElem = XCUIApplication()
.descendants(matching: .staticText)["Notification Received"]
waitFor(forElement: alertElem, timeout: 10)
}
}private func waitFor(forElement element: XCUIElement,
timeout: TimeInterval) {
let predicate = NSPredicate(format: "exists == true")
let elemExists = XCTNSPredicateExpectation(
predicate: predicate, object: element)
XCTWaiter().wait(for: [elemExists], timeout: timeout)
if !element.exists { XCTFail("Does not exist") }
}
7. Check UITest output
As our last step, the bash script should check the output of our xcodebuild command. We cannot rely on the bash exit code because it does not indicate whether the tests failed or not, to check it we simply grep the contents of the temporary file that we used:
test_succeeded=$(grep '\*\* TEST SUCCEEDED \*\*' $uitest_output)if [ -z "$test_succeeded" ]; then
echo "** TEST FAILED **" && exit 1
else
echo "** TEST SUCCEEDED **" && exit 0
fi
If the string ** TEST SUCCEEDED ** is present in the output file then we exit the full bash script with 0 indicating to the success CI/CD process.
NOTE: When the tests fail xcodebuild tends to hang for a bit (more than a minute) before producing the ** TEST FAILED ** output so have a little patience when it happens.
7. Finally full result!
I’ve included a file called test_push_notifications.sh which runs the tests from the terminal; it has the device_udid hardcoded but can be invoked from the root of the project with your particular device UDID as the first argument:
test_push_notifications.sh AD7BED71-5CAB-4F60-92AA-3106465D36BCYou should see something like the following:
Pretty cool right? though it wasn’t simple and its prone to many “bashy” errors it at least provides us with a way to start automating the testing of push notifications; a process which requires manual interactions with UIs. Next steps could converting the bash script into a python or go tool to have something a bit more resilient and flexible, you could also add a more dynamic way of supplying push payloads.
