Skip to content

Audio: Increase pre-merge test coverage to 10 test cases #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tmoida wants to merge 1 commit into
base: main
Choose a base branch
from
+1,669 −301
Open

Audio: Increase pre-merge test coverage to 10 test cases #243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 47 additions & 8 deletions Runner/plans/meta-ar-ci-premerge.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,61 @@
metadata:
format: Lava-Test Test Definition 1.0
name: SmokeSanity
description: "Pre-merge LAVA plan to run AudioRecord and AudioPlayback on every PR"
description: "Pre-merge LAVA plan to run AudioPlayback and AudioRecord test cases on every PR"
maintainer:
- abbajaj@qti.qualcomm.com
- tmoida@qti.qualcomm.com
os:
- openembedded
scope:
- functional
devices:
- rb3gen2

run:
steps:
- cd Runner
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true
- $PWD/suites/Multimedia/Audio/AudioRecord/run.sh || true

# ========== AudioPlayback Test Cases (7 configs) ==========

# Playback Test 1: Config1 (16KHz, 16-bit, 2ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config1" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# Playback Test 2: Config7 (24KHz, 24-bit, 6ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config7" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# Playback Test 3: Config13 (44.1KHz, 16-bit, 1ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config13" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# Playback Test 4: Config15 (48KHz, 8-bit, 2ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config15" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# Playback Test 5: Config18 (88.2KHz, 24-bit, 2ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config18" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# Playback Test 6: Config20 (96KHz, 24-bit, 6ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config20" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# Playback Test 7: Config5 (192KHz, 32-bit, 8ch)
- $PWD/suites/Multimedia/Audio/AudioPlayback/run.sh --clip-name "Config5" --audio-clips-path /home/AudioClips/ --no-extract-assets || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.res || true

# ========== AudioRecord Test Cases (3 configs) ==========

# Record Test 1: record_config1 (8KHz, 1ch)
- $PWD/suites/Multimedia/Audio/AudioRecord/run.sh --config-name "record_config1" --record-seconds 10s || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioRecord/AudioRecord.res || true

# Record Test 2: record_config7 (48KHz, 2ch)
- $PWD/suites/Multimedia/Audio/AudioRecord/run.sh --config-name "record_config7" --record-seconds 10s || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioRecord/AudioRecord.res || true

# Record Test 3: record_config10 (96KHz, 6ch)
- $PWD/suites/Multimedia/Audio/AudioRecord/run.sh --config-name "record_config10" --record-seconds 10s || true
- $PWD/utils/send-to-lava.sh $PWD/suites/Multimedia/Audio/AudioRecord/AudioRecord.res || true

# Parse and report results
- $PWD/utils/result_parse.sh

8 changes: 5 additions & 3 deletions Runner/suites/Multimedia/Audio/AudioPlayback/AudioPlayback.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@ metadata:
- functional

params:
AUDIO_BACKEND: "" # Selects backend: pipewire or pulseaudio, default: auto-detect
AUDIO_BACKEND: "" # Selects backend: pipewire or pulseaudio, default: auto-detect
SINK_CHOICE: "speakers" # Playback sink: speakers or null, default: speakers
CLIP_NAMES: "Config1" # Test specific clips (e.g., "Config1 Config2" or "play_48KHz_8b_2ch"), default: Config1
CLIP_FILTER: "" # Filter clips by pattern (e.g., "48KHz" or "16b" or "2ch"), default: unset
FORMATS: "wav" # Audio formats: e.g. wav, default: wav
DURATIONS: "short" # Playback durations: short, medium, long, default: short
LOOPS: 1 # Number of playback loops, default: 1
Expand All @@ -29,5 +31,5 @@ run:
steps:
- REPO_PATH=$PWD
- cd Runner/suites/Multimedia/Audio/AudioPlayback/
- ./run.sh --backend "${AUDIO_BACKEND}" --sink "${SINK_CHOICE}" --formats "${FORMATS}" --durations "${DURATIONS}" --loops "${LOOPS}" --timeout "${TIMEOUT}" --strict "${STRICT}" --audio-clips-path "${AUDIO_CLIPS_BASE_DIR}" --ssid "${SSID}" --password "${PASSWORD}" || true
- $REPO_PATH/Runner/utils/send-to-lava.sh AudioPlayback.res || true
- ./run.sh --backend "${AUDIO_BACKEND}" --sink "${SINK_CHOICE}" --clip-name "${CLIP_NAMES}" --clip-filter "${CLIP_FILTER}" --formats "${FORMATS}" --durations "${DURATIONS}" --loops "${LOOPS}" --timeout "${TIMEOUT}" --strict "${STRICT}" --audio-clips-path "${AUDIO_CLIPS_BASE_DIR}" --ssid "${SSID}" --password "${PASSWORD}" || true
- $REPO_PATH/Runner/utils/send-to-lava.sh AudioPlayback.res || true
174 changes: 138 additions & 36 deletions Runner/suites/Multimedia/Audio/AudioPlayback/Read_me.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ This suite automates the validation of audio playback capabilities on Qualcomm L
## Features

- Supports **PipeWire** and **PulseAudio** backends
- **20-clip test coverage**: Comprehensive validation across diverse audio formats (sample rates: 8KHz-352.8KHz, bit depths: 8b-32b, channels: 1ch-8ch)
- **Flexible clip selection**:
- Use generic config names (Config1-Config20) for easy selection
- Use descriptive names (e.g., play_48KHz_16b_2ch) for specific formats
- Auto-discovery mode tests all available clips
- **Clip filtering**: Filter tests by sample rate, bit rate, or channel configuration
- Plays audio clips with configurable format, duration, and loop count
- **Network operations are optional**: By default, no network connection is attempted. Use `--enable-network-download` to enable downloading missing audio files
- Automatically downloads and extracts audio assets if missing
Expand All @@ -20,6 +26,38 @@ This suite automates the validation of audio playback capabilities on Qualcomm L
- Generates `.res` result file and optional JUnit XML output


## Audio Clip Configurations

The test suite includes 20 diverse audio clip configurations covering various sample rates, bit depths, and channel configurations:

Config Descriptive Name Sample Rate Bit Rate Channels
Config1 play_16KHz_16b_2ch 16 KHz 16-bit 2ch
Config2 play_176.4KHz_24b_1ch 176.4 KHz 24-bit 1ch
Config3 play_176.4KHz_32b_6ch 176.4 KHz 32-bit 6ch
Config4 play_192KHz_16b_6ch 192 KHz 16-bit 6ch
Config5 play_192KHz_32b_8ch 192 KHz 32-bit 8ch
Config6 play_22.050KHz_8b_1ch 22.05 KHz 8-bit 1ch
Config7 play_24KHz_24b_6ch 24 KHz 24-bit 6ch
Config8 play_24KHz_32b_8ch 24 KHz 32-bit 8ch
Config9 play_32KHz_16b_2ch 32 KHz 16-bit 2ch
Config10 play_32KHz_8b_8ch 32 KHz 8-bit 8ch
Config11 play_352.8KHz_32b_1ch 352.8 KHz 32-bit 1ch
Config12 play_384KHz_32b_2ch 384 KHz 32-bit 2ch
Config13 play_44.1KHz_16b_1ch 44.1 KHz 16-bit 1ch
Config14 play_44.1KHz_8b_6ch 44.1 KHz 8-bit 6ch
Config15 play_48KHz_8b_2ch 48 KHz 8-bit 2ch
Config16 play_48KHz_8b_8ch 48 KHz 8-bit 8ch
Config17 play_88.2KHz_16b_8ch 88.2 KHz 16-bit 8ch
Config18 play_88.2KHz_24b_2ch 88.2 KHz 24-bit 2ch
Config19 play_8KHz_8b_1ch 8 KHz 8-bit 1ch
Config20 play_96KHz_24b_6ch 96 KHz 24-bit 6ch

Coverage Summary:
- Sample Rates: 8 KHz, 16 KHz, 22.05 KHz, 24 KHz, 32 KHz, 44.1 KHz, 48 KHz, 88.2 KHz, 96 KHz, 176.4 KHz, 192 KHz, 352.8 KHz, 384 KHz
- Bit Depths: 8-bit, 16-bit, 24-bit, 32-bit
- Channel Configurations: 1ch (Mono), 2ch (Stereo), 6ch (5.1 Surround), 8ch (7.1 Surround)
- Total Configurations: 20 unique audio format combinations

## Prerequisites

Ensure the following components are present in the target Yocto build:
Expand Down Expand Up @@ -112,6 +150,33 @@ AUDIO_CLIPS_BASE_DIR="/tmp/ci-audio-staging/AudioClips" ./run-test.sh AudioPlayb

**Directly from Test Directory**
cd Runner/suites/Multimedia/Audio/AudioPlayback

# Test all 20 clips (auto-discovery mode)
./run.sh --no-extract-assets

# Test specific clips using Config naming (Config1 to Config20)
./run.sh --no-extract-assets --clip-name "Config1"
./run.sh --no-extract-assets --clip-name "Config1 Config5 Config10"

# Test specific clips using descriptive names
./run.sh --no-extract-assets --clip-name "play_48KHz_8b_2ch"
./run.sh --no-extract-assets --clip-name "play_8KHz_8b_1ch"
./run.sh --no-extract-assets --clip-name "play_192KHz_32b_8ch"

# Filter clips by sample rate
./run.sh --no-extract-assets --clip-filter "48KHz"
./run.sh --no-extract-assets --clip-filter "192KHz"

# Filter clips by bit depth
./run.sh --no-extract-assets --clip-filter "16b"
./run.sh --no-extract-assets --clip-filter "24b"

# Filter clips by channel configuration
./run.sh --no-extract-assets --clip-filter "2ch"
./run.sh --no-extract-assets --clip-filter "8ch"

# Combine filters (tests clips matching any pattern)
./run.sh --no-extract-assets --clip-filter "48KHz 16b"
# Show usage/help
./run.sh --help

Expand All @@ -130,31 +195,33 @@ cd Runner/suites/Multimedia/Audio/AudioPlayback


Environment Variables:
Variable Description Default
AUDIO_BACKEND Selects backend: pipewire or pulseaudio auto-detect
SINK_CHOICE Playback sink: speakers or null speakers
FORMATS Audio formats: e.g. wav wav
DURATIONS Playback durations: short, medium, long short
LOOPS Number of playback loops 1
TIMEOUT Playback timeout per loop (e.g., 15s, 0=none) 0
STRICT Enable strict mode (fail on any error) 0
DMESG_SCAN Scan dmesg for errors after playback 1
VERBOSE Enable verbose logging 0
EXTRACT_AUDIO_ASSETS Download/extract audio assets if missing true
ENABLE_NETWORK_DOWNLOAD Enable network download of missing audio files false
AUDIO_CLIPS_BASE_DIR Custom path to pre-staged audio clips (CI use) unset
JUNIT_OUT Path to write JUnit XML output unset
SSID Wi-Fi SSID for network connection unset
PASSWORD Wi-Fi password for network connection unset
NET_PROBE_ROUTE_IP IP used for route probing (default: 1.1.1.1) 1.1.1.1
NET_PING_HOST Host used for ping reachability check 8.8.8.8
Variable Description Default
AUDIO_BACKEND Selects backend: pipewire or pulseaudio auto-detect
SINK_CHOICE Playback sink: speakers or null speakers
FORMATS Audio formats: e.g. wav wav
DURATIONS Playback durations: short, medium, long short
LOOPS Number of playback loops 1
TIMEOUT Playback timeout per loop (e.g., 15s, 0=none) 0
STRICT Enable strict mode (fail on any error) 0
DMESG_SCAN Scan dmesg for errors after playback 1
VERBOSE Enable verbose logging 0
EXTRACT_AUDIO_ASSETS Download/extract audio assets if missing true
ENABLE_NETWORK_DOWNLOAD Enable network download of missing audio files false
AUDIO_CLIPS_BASE_DIR Custom path to pre-staged audio clips (CI use) unset
JUNIT_OUT Path to write JUnit XML output unset
SSID Wi-Fi SSID for network connection unset
PASSWORD Wi-Fi password for network connection unset
NET_PROBE_ROUTE_IP IP used for route probing (default: 1.1.1.1) 1.1.1.1
NET_PING_HOST Host used for ping reachability check 8.8.8.8


CLI Options
Option Description
--backend Select backend: pipewire or pulseaudio
--sink Playback sink: speakers or null
--formats Audio formats (space/comma separated): e.g. wav
--clip-name <names> Test specific clips using Config1-Config20 or descriptive names (space-separated)
--clip-filter <patterns> Filter clips by sample rate, bit rate, or channels (space-separated patterns)
--formats Audio formats (space/comma separated): e.g. wav
--durations Playback durations: short, medium, long
--loops Number of playback loops
--timeout Playback timeout per loop (e.g., 15s)
Expand All @@ -170,22 +237,59 @@ Option Description
```

Sample Output:

**Example 1: Testing specific clip using Config naming**
```
sh-5.3# ./run.sh --backend pipewire
[INFO] 2025-09-12 05:24:47 - ---------------- Starting AudioPlayback ----------------
[INFO] 2025-09-12 05:24:47 - SoC: 498
[INFO] 2025-09-12 05:24:47 - Args: backend=pipewire sink=speakers loops=1 timeout=0 formats='wav' durations='short' strict=0 dmesg=1 extract=true
[INFO] 2025-09-12 05:24:47 - Using backend: pipewire
[INFO] 2025-09-12 05:24:47 - Routing to sink: id=72 name='Built-in Audio Speaker playback' choice=speakers
[INFO] 2025-09-12 05:24:47 - Watchdog/timeout: 0
[INFO] 2025-09-12 05:24:47 - [play_wav_short] loop 1/1 start=2025-09-12T05:24:47Z clip=AudioClips/yesterday_48KHz.wav backend=pipewire sink=speakers(72)
[INFO] 2025-09-12 05:24:47 - [play_wav_short] exec: pw-play -v "AudioClips/yesterday_48KHz.wav"
[INFO] 2025-09-12 05:26:52 - [play_wav_short] evidence: pw_streaming=1 pa_streaming=0 alsa_running=1 asoc_path_on=1 pw_log=1
[PASS] 2025-09-12 05:26:52 - [play_wav_short] loop 1 OK (rc=0, 125s)
[INFO] 2025-09-12 05:26:52 - Scanning dmesg for snd|audio|pipewire|pulseaudio: errors & success patterns
[INFO] 2025-09-12 05:26:52 - No snd|audio|pipewire|pulseaudio-related errors found (no OK pattern requested)
[INFO] 2025-09-12 05:26:52 - Summary: total=1 pass=1 fail=0 skip=0
[PASS] 2025-09-12 05:26:52 - AudioPlayback PASS
sh-5.3# ./run.sh --no-extract-assets --clip-name "Config1"
[INFO] 2025-12-30 11:47:32 - ---------------- Starting AudioPlayback ----------------
[INFO] 2025-12-30 11:47:32 - Platform Details: machine='Qualcomm Technologies, Inc. Robotics RB3gen2' target='Kodiak' kernel='6.18.0-00393-g27507852413b' arch='aarch64'
[INFO] 2025-12-30 11:47:32 - Args: backend=auto sink=speakers loops=1 timeout=0 formats='wav' durations='short' strict=0 dmesg=1 extract=false network_download=false clips_path=default
[INFO] 2025-12-30 11:47:32 - Using backend: pipewire
[INFO] 2025-12-30 11:47:32 - Routing to sink: id=52 name='Built-in Audio Speaker playback' choice=speakers
[INFO] 2025-12-30 11:47:32 - Using clip discovery mode
[INFO] 2025-12-30 11:47:32 - Discovered 1 clips to test
[INFO] 2025-12-30 11:47:32 - [play_16KHz_16b_2ch] Using clip: yesterday_16KHz_30s_16b_2ch.wav (1922036 bytes)
[INFO] 2025-12-30 11:47:32 - [play_16KHz_16b_2ch] loop 1/1 start=2025-12-30T11:47:32Z clip=yesterday_16KHz_30s_16b_2ch.wav backend=pipewire sink=speakers(52)
[INFO] 2025-12-30 11:47:32 - [play_16KHz_16b_2ch] exec: pw-play -v "AudioClips/yesterday_16KHz_30s_16b_2ch.wav"
[INFO] 2025-12-30 11:48:02 - [play_16KHz_16b_2ch] evidence: pw_streaming=1 pa_streaming=0 alsa_running=1 asoc_path_on=1 pw_log=1
[PASS] 2025-12-30 11:48:02 - [play_16KHz_16b_2ch] loop 1 OK (rc=0, 30s)
[INFO] 2025-12-30 11:48:02 - Summary: total=1 pass=1 fail=0 skip=0
[PASS] 2025-12-30 11:48:02 - AudioPlayback PASS
```

**Example 2: Testing multiple clips**
```
sh-5.3# ./run.sh --no-extract-assets --clip-name "Config1 Config2 Config3"
[INFO] 2025-12-30 11:48:13 - Using clip discovery mode
[INFO] 2025-12-30 11:48:13 - Discovered 3 clips to test
[INFO] 2025-12-30 11:48:13 - [play_16KHz_16b_2ch] Using clip: yesterday_16KHz_30s_16b_2ch.wav (1922036 bytes)
[PASS] 2025-12-30 11:48:43 - [play_16KHz_16b_2ch] loop 1 OK (rc=0, 30s)
[INFO] 2025-12-30 11:48:43 - [play_176.4KHz_24b_1ch] Using clip: yesterday_176.4KHz_30s_24b_1ch.wav (15892062 bytes)
[PASS] 2025-12-30 11:49:14 - [play_176.4KHz_24b_1ch] loop 1 OK (rc=0, 31s)
[INFO] 2025-12-30 11:49:14 - [play_176.4KHz_32b_6ch] Using clip: yesterday_176.4KHz_30s_32b_6ch.wav (127135484 bytes)
[PASS] 2025-12-30 11:49:44 - [play_176.4KHz_32b_6ch] loop 1 OK (rc=0, 30s)
[INFO] 2025-12-30 11:49:44 - Summary: total=3 pass=3 fail=0 skip=0
[PASS] 2025-12-30 11:49:44 - AudioPlayback PASS
```

**Example 3: Filtering clips by sample rate**
```
sh-5.3# ./run.sh --no-extract-assets --clip-filter "48KHz"
[INFO] 2025-12-30 12:00:08 - Using clip discovery mode
[INFO] 2025-12-30 12:00:08 - Discovered 2 clips to test
[INFO] 2025-12-30 12:00:08 - [play_48KHz_8b_2ch] Using clip: yesterday_48KHz_30s_8b_2ch.wav (2883002 bytes)
[PASS] 2025-12-30 12:00:38 - [play_48KHz_8b_2ch] loop 1 OK (rc=0, 30s)
[INFO] 2025-12-30 12:00:38 - [play_48KHz_8b_8ch] Using clip: yesterday_48KHz_30s_8b_8ch.wav (11531688 bytes)
[PASS] 2025-12-30 12:01:08 - [play_48KHz_8b_8ch] loop 1 OK (rc=0, 30s)
[INFO] 2025-12-30 12:01:08 - Summary: total=2 pass=2 fail=0 skip=0
[PASS] 2025-12-30 12:01:08 - AudioPlayback PASS
```

**Example 4: Invalid config name (shows helpful error)**
```
sh-5.3# ./run.sh --no-extract-assets --clip-name "Config0"
[INFO] 2025-12-30 11:59:52 - Using clip discovery mode
[SKIP] 2025-12-30 11:59:52 - AudioPlayback SKIP - Invalid clip/config name(s) provided. Available range: Config1 to Config20
```

Results:
Expand All @@ -211,5 +315,3 @@ Diagnostic logs: dmesg snapshots, mixer dumps, playback logs per test case

SPDX-License-Identifier: BSD-3-Clause-Clear
(C) Qualcomm Technologies, Inc. and/or its subsidiaries.


Loading
Loading