ficik/-rpi-sendspin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1054d4b9d0215eef2ecb6d1eb404d831a5affe9a
-rpi-sendspin/third_party/sendspin-go/docs/hires-audio-verification.md

11 KiB
Raw Blame History

Hi-Res Audio Verification Report

Status: 🟡 Needs Testing Version: v0.9.0 Date: 2025-10-26

Executive Summary

resonate-go implements full hi-res audio support with 24-bit depth and sample rates up to 192kHz. However, compatibility with Music Assistant and other Resonate servers needs verification due to potential codec negotiation differences.

Supported Hi-Res Formats:

  • ✅ 192kHz/24-bit PCM (lossless)
  • ✅ 176.4kHz/24-bit PCM (lossless)
  • ✅ 96kHz/24-bit PCM (lossless)
  • ✅ 88.2kHz/24-bit PCM (lossless)
  • ⚠️ Opus encoding requires 48kHz (no automatic resampling)

Current Implementation Analysis

1. Audio Pipeline Capabilities

Sample Rate Support (pkg/resonate/player.go:170-185):

SupportFormats: []protocol.AudioFormat{
    {Codec: "pcm", Channels: 2, SampleRate: 192000, BitDepth: 24},  // Hi-res
    {Codec: "pcm", Channels: 2, SampleRate: 176400, BitDepth: 24},  // Hi-res
    {Codec: "pcm", Channels: 2, SampleRate: 96000, BitDepth: 24},   // Hi-res
    {Codec: "pcm", Channels: 2, SampleRate: 88200, BitDepth: 24},   // Hi-res
    {Codec: "pcm", Channels: 2, SampleRate: 48000, BitDepth: 16},   // CD quality
    {Codec: "pcm", Channels: 2, SampleRate: 44100, BitDepth: 16},   // CD quality
    {Codec: "opus", Channels: 2, SampleRate: 48000, BitDepth: 16},  // Compressed
},

Bit Depth Pipeline:

  • Server default: 24-bit (pkg/resonate/server.go:32)
  • Player supports: 16-bit and 24-bit (pkg/audio/decode/pcm.go:23-24)
  • Internal representation: int32 for full 24-bit range (pkg/audio/types.go:11-12)
  • Audio output: Full 24-bit support via malgo (pkg/audio/output/malgo.go)

Sample Rate Pipeline:

  • Source: Any rate (192kHz default for TestTone)
  • Encoder: Opus requires 48kHz, PCM supports any rate
  • Decoder: PCM supports any rate, Opus fixed at 48kHz
  • Output: malgo backend accepts any sample rate and handles format reinitialization

2. Codec Negotiation

Server-Side Logic (internal/server/audio_engine.go:200-230)

func (e *AudioEngine) negotiateCodec(client *Client) string {
    sourceRate := e.source.SampleRate()

    // Check if client advertised support for exact source format
    for _, format := range caps.SupportFormats {
        // If source is 48kHz PCM and client wants Opus, prefer Opus
        if format.Codec == "opus" && sourceRate == 48000 {
            return "opus"
        }

        // If client supports exact source format, use PCM
        if format.Codec == "pcm" && format.SampleRate == sourceRate && format.BitDepth == DefaultBitDepth {
            return "pcm"
        }
    }

    // FALLBACK: If no exact match, try legacy fields
    // This is for backward compatibility with Music Assistant
    if codec == "opus" && sourceRate == 48000 {
        return "opus"
    }

    // Final fallback: PCM at source rate
    return "pcm"
}

Issues Identified

🔴 Critical: No Automatic Resampling for Opus

When source is 192kHz and client supports Opus:

  • Current behavior: Falls back to PCM at 192kHz
  • Expected behavior (Music Assistant?): Resample 192kHz → 48kHz, encode to Opus
  • Impact: Client receives uncompressed PCM instead of Opus (4x bandwidth increase)

Test Case:

Source: 192kHz/24-bit test tone
Client: Advertises Opus support
Expected: Server resamples to 48kHz and sends Opus
Actual: Server sends 192kHz PCM (no resampling)

✅ Audio Output: Malgo with Full 24-bit Support

The audio output uses malgo library (via miniaudio) which:

  • Supports 16-bit, 24-bit, and 32-bit output natively
  • Handles format reinitialization for format changes
  • Preserves full 24-bit pipeline all the way to device playback

This means hi-res samples maintain full resolution through the entire pipeline.

3. Bandwidth Analysis

PCM Bandwidth (192kHz/24-bit stereo):

Sample rate:   192,000 Hz
Channels:      2
Bit depth:     24 bits = 3 bytes
Bytes/second:  192000 × 2 × 3 = 1,152,000 bytes/s = 9.216 Mbps

Opus Bandwidth (48kHz/16-bit stereo @ 256kbps):

Bitrate:       256 kbps (configurable, set in opus_encoder.go:31)
Bytes/second:  32,000 bytes/s = 0.256 Mbps
Compression:   36x smaller than 192kHz PCM!

Impact of Missing Resampling

If Music Assistant sends 192kHz source and expects Opus:

  • Without resampling: 9.216 Mbps per client (current)
  • With resampling: 0.256 Mbps per client (ideal)
  • Difference: 36x higher bandwidth usage!

For 5 simultaneous clients:

  • PCM: 46 Mbps
  • Opus: 1.3 Mbps

4. Compatibility Testing Plan

Test Matrix

Source Format Client Codec Expected Behavior Status
192kHz/24-bit PCM PCM 192kHz Direct PCM stream ✅ Should work
192kHz/24-bit PCM Opus 48kHz Resample + Opus ⚠️ Falls back to PCM
96kHz/24-bit PCM PCM 96kHz Direct PCM stream ✅ Should work
96kHz/24-bit PCM Opus 48kHz Resample + Opus ⚠️ Falls back to PCM
48kHz/16-bit PCM Opus 48kHz Direct Opus encode ✅ Works
48kHz/16-bit PCM PCM 48kHz Direct PCM stream ✅ Works

Required Tests

Test 1: Music Assistant Compatibility

# Start resonate-go server with 192kHz source
./resonate-server --audio test_192khz_24bit.flac

# Connect Music Assistant player
# Expected: MA requests Opus, server resamples and encodes
# Actual: ???

Test 2: Multi-Room Sync at Hi-Res

# Start server with 192kHz PCM
./resonate-server --audio hires_test.flac

# Start 5 players
for i in {1..5}; do
  ./resonate-player --name "Player-$i" &
done

# Verify:
# - All players receive 192kHz PCM
# - Sync stays within 10ms
# - No dropped frames
# - Network bandwidth is acceptable

Test 3: Sample Rate Switching

# Start server with 48kHz source
./resonate-server --audio 48khz.flac

# Connect player (should get Opus)
./resonate-player --name "Test"

# Switch to 192kHz source on server
# (Would require server restart currently)

# Expected: Player handles format change gracefully
# Actual: Malgo reinitializes the device cleanly for the new format

Test 4: Bit Depth Verification

# Generate 24-bit test tone with known frequency spectrum
./resonate-server --audio 24bit_sweep.wav

# Record output from player
# Analyze frequency spectrum
# Verify: Full 24-bit dynamic range preserved until output stage

5. Known Issues & Limitations

Issue 1: No Automatic Resampling for Opus 🔴

Location: internal/server/audio_engine.go:209, 219

Current Code:

if format.Codec == "opus" && sourceRate == 48000 {
    return "opus"
}

Problem: Only uses Opus if source is already 48kHz. Doesn't resample hi-res sources.

Fix Required:

if format.Codec == "opus" {
    // Resample to 48kHz if needed
    if sourceRate != 48000 {
        // Create resampler from sourceRate to 48kHz
        encoder.resampler = NewResampler(sourceRate, 48000, channels)
    }
    return "opus"
}

Impact: High bandwidth usage for Opus clients with hi-res sources.

Issue 2: Output Format Support ✅ RESOLVED

Previous Issue: oto library only supported 16-bit output format.

Current Status: Malgo backend now supports 24-bit and 32-bit output natively. Full hi-res resolution is preserved through device playback.

Issue 3: Format Reinitialization ✅ RESOLVED

Previous Issue: oto context could only be initialized once per process; format changes required restart.

Current Status: Malgo backend supports format reinitialization, allowing graceful format changes during streaming.

6. Recommendations

Immediate Actions (v0.9.x)

  1. 🔴 Priority 1: Add Resampling to Opus Path

    • Implement automatic resampling in audio_engine.go
    • Use existing pkg/audio/resample.Resampler
    • Test with 192kHz → 48kHz Opus encoding
    • Verify bandwidth reduction

  2. 🟡 Priority 2: Test with Music Assistant

    • Deploy resonate-go server with MA
    • Verify codec negotiation compatibility
    • Test hi-res audio file playback
    • Document any protocol differences

  3. ✅ Complete: 24-bit Output Support

    • Malgo backend supports 24-bit output natively
    • Full hi-res pipeline preserved through device playback

  4. 🟢 Priority 3: Add Integration Tests

    • Create test suite for hi-res formats
    • Verify sample rate handling
    • Check codec negotiation logic
    • Measure bandwidth usage

Future Enhancements (v1.0+)

  1. Configurable Quality Profiles

    • "Low bandwidth" → Force Opus with resampling
    • "Balanced" → Opus for >48kHz, PCM for ≤48kHz
    • "Hi-Res" → Always use PCM at source rate

  2. Device Selection Control

    • Allow users to select audio output device
    • Support ASIO on Windows for low-latency recording
    • Show available devices and their capabilities

7. Verification Checklist

Before marking v1.0.0 as ready:

  • Verify 192kHz/24-bit PCM playback end-to-end with malgo
  • Verify 96kHz/24-bit PCM playback with malgo
  • Verify 24-bit audio reaches device without downsampling
  • Test automatic resampling for Opus (after implementing)
  • Test with Music Assistant server
  • Test with other Resonate protocol implementations
  • Measure multi-room sync accuracy at hi-res
  • Document bandwidth requirements
  • Create hi-res test audio files repository
  • Profile CPU usage with hi-res streams
  • Test with 5+ simultaneous hi-res clients
  • Verify no audio artifacts at high sample rates
  • Check for buffer underruns at 192kHz
  • Test format negotiation with all supported rates

8. Test Resources Needed

Audio Test Files:

  • test_192khz_24bit.flac - Full hi-res test
  • test_96khz_24bit.flac - Mid-tier hi-res
  • test_48khz_24bit.flac - Baseline quality
  • sweep_24bit.wav - Frequency sweep for bit depth verification
  • dynamic_range_test.wav - Test 24-bit dynamic range

Test Equipment:

  • Music Assistant server instance
  • Multiple player instances (5+)
  • Network bandwidth monitor
  • Audio spectrum analyzer
  • Sync measurement tools

9. Conclusion

resonate-go has excellent foundational support for hi-res audio with a clean 24-bit pipeline and support for sample rates up to 192kHz. However, critical compatibility testing is required to ensure:

  1. Codec negotiation works with Music Assistant (especially Opus fallback)
  2. Bandwidth optimization through automatic resampling to Opus
  3. Multi-room sync accuracy maintained at hi-res rates
  4. No audio quality degradation in the pipeline

The biggest concern is the lack of automatic resampling for Opus encoding which could cause 36x higher bandwidth usage when Music Assistant expects Opus but receives PCM.

Status: Ready for Testing 🧪

Reference in New Issue View Git Blame Copy Permalink