🎉 live server seems to be working now
This commit is contained in:
460
third_party/sendspin-go/docs/plans/2025-10-23-resonate-player-design.md
vendored
Normal file
460
third_party/sendspin-go/docs/plans/2025-10-23-resonate-player-design.md
vendored
Normal file
@@ -0,0 +1,460 @@
|
||||
# Resonate Go Player Design
|
||||
|
||||
**Date:** 2025-10-23
|
||||
**Status:** Design Approved
|
||||
**Target:** Implement a Resonate Protocol player in Go
|
||||
|
||||
## Overview
|
||||
|
||||
A multi-room synchronized audio player implementing the Resonate Protocol specification. The player can discover and connect to Music Assistant servers, receive and decode multiple audio formats, maintain precise clock synchronization for multi-room playback, and provide a rich terminal UI for monitoring and control.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
- Register as a player with Resonate servers using mDNS discovery (both server-initiated and client-initiated modes)
|
||||
- Accept and decode audio streams in three formats: Opus, FLAC, PCM
|
||||
- Maintain sub-millisecond clock synchronization with server for multi-room audio
|
||||
- Control playback volume via software mixing
|
||||
- Display stream metadata (title, artist, album)
|
||||
- Provide interactive TUI for monitoring and control
|
||||
- Handle stream format changes dynamically
|
||||
- Report player state back to server
|
||||
|
||||
### Non-Functional Requirements
|
||||
- Cross-platform (macOS, Linux, Windows)
|
||||
- Low latency playback (<200ms buffer)
|
||||
- Robust reconnection on network failures
|
||||
- Minimal CPU usage during playback
|
||||
|
||||
## Architecture
|
||||
|
||||
### Component Overview
|
||||
|
||||
Event-driven architecture using Go channels and goroutines for clean separation of concerns.
|
||||
|
||||
**Core Components:**
|
||||
|
||||
1. **Discovery Manager** - mDNS service advertisement and discovery
|
||||
2. **WebSocket Client** - Protocol communication with server
|
||||
3. **Clock Sync** - NTP-style time synchronization
|
||||
4. **Audio Decoder** - Multi-format audio decoding
|
||||
5. **Playback Scheduler** - Timestamp-based playback scheduling
|
||||
6. **Audio Output** - PCM playback with volume control
|
||||
7. **TUI Controller** - User interface and monitoring
|
||||
|
||||
**Data Flow:**
|
||||
```
|
||||
Network → WebSocket → Decoder → Scheduler → Audio Output → Speaker
|
||||
↓
|
||||
Control/Time Sync → Clock/State Management
|
||||
↓
|
||||
TUI Display
|
||||
```
|
||||
|
||||
### Key Data Structures
|
||||
|
||||
```go
|
||||
// Channels for inter-goroutine communication
|
||||
audioChunks chan AudioChunk // Raw encoded chunks from network
|
||||
decodedAudio chan PCMBuffer // Decoded PCM ready for playback
|
||||
controlMsgs chan ControlMessage // Volume, mute, state changes
|
||||
timeSyncReq chan TimeMessage // Clock sync requests
|
||||
timeSyncResp chan TimeMessage // Clock sync responses
|
||||
uiEvents chan UIEvent // User input from TUI
|
||||
|
||||
// Core types
|
||||
type AudioChunk struct {
|
||||
Timestamp int64 // Microseconds, server clock
|
||||
Data []byte // Encoded audio frame
|
||||
}
|
||||
|
||||
type PCMBuffer struct {
|
||||
Timestamp int64 // When to play (server clock)
|
||||
Samples []int16 // PCM samples
|
||||
Format AudioFormat
|
||||
}
|
||||
|
||||
type AudioFormat struct {
|
||||
Codec string // "opus", "flac", "pcm"
|
||||
SampleRate int // 44100, 48000, etc.
|
||||
Channels int // 1=mono, 2=stereo
|
||||
BitDepth int // 16, 24
|
||||
}
|
||||
```
|
||||
|
||||
## Component Designs
|
||||
|
||||
### 1. Discovery Manager
|
||||
|
||||
**mDNS Service Configuration:**
|
||||
- **Advertise (Server-initiated):**
|
||||
- Service: `_resonate._tcp.local.`
|
||||
- Port: 8927 (default, configurable)
|
||||
- TXT record: `path=/resonate`
|
||||
- **Discover (Client-initiated):**
|
||||
- Browse for: `_resonate-server._tcp.local.`
|
||||
- Port: 8927
|
||||
- TXT record: `path=/resonate`
|
||||
|
||||
**Connection Strategy:**
|
||||
- Run both modes simultaneously on startup
|
||||
- First successful connection wins
|
||||
- Exponential backoff on connection failures (1s, 2s, 4s, 8s, max 30s)
|
||||
- Support manual server override via CLI flag
|
||||
|
||||
**Library:** `github.com/hashicorp/mdns`
|
||||
|
||||
### 2. WebSocket Client
|
||||
|
||||
**Connection Lifecycle:**
|
||||
|
||||
1. Connect to `ws://[host]:[port]/resonate`
|
||||
2. Send `client/hello`:
|
||||
```json
|
||||
{
|
||||
"type": "client/hello",
|
||||
"payload": {
|
||||
"client_id": "[UUID]",
|
||||
"name": "[hostname]-resonate-player",
|
||||
"version": 1,
|
||||
"supported_roles": ["player"],
|
||||
"device_info": {
|
||||
"product_name": "Resonate Go Player",
|
||||
"manufacturer": "resonate-go",
|
||||
"software_version": "0.1.0"
|
||||
},
|
||||
"player_support": {
|
||||
"codecs": ["opus", "flac", "pcm"],
|
||||
"sample_rates": [44100, 48000],
|
||||
"channels": [1, 2],
|
||||
"bit_depths": [16, 24]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
3. Wait for `server/hello` (block other messages)
|
||||
4. Send initial `client/state`:
|
||||
```json
|
||||
{
|
||||
"type": "client/state",
|
||||
"payload": {
|
||||
"state": "synchronized",
|
||||
"volume": 100,
|
||||
"muted": false
|
||||
}
|
||||
}
|
||||
```
|
||||
5. Begin clock sync loop
|
||||
6. Ready for streaming
|
||||
|
||||
**Message Routing:**
|
||||
- JSON messages → parse type, route to appropriate channel
|
||||
- Binary messages → byte 0 = type (0 for audio), bytes 1-8 = timestamp (big-endian int64), rest = audio data
|
||||
|
||||
**Library:** `github.com/gorilla/websocket`
|
||||
|
||||
### 3. Clock Synchronization
|
||||
|
||||
**Protocol:** Three-timestamp NTP-style synchronization
|
||||
|
||||
**Flow:**
|
||||
1. Client records `t1` (local time μs)
|
||||
2. Send `client/time` with `t1`
|
||||
3. Server receives at `t2`, responds immediately
|
||||
4. Server sends `server/time` with `{t1, t2, t3}`
|
||||
5. Client receives at `t4`
|
||||
|
||||
**Offset Calculation:**
|
||||
```go
|
||||
rtt := (t4 - t1) - (t3 - t2)
|
||||
offset := ((t2 - t1) + (t3 - t4)) / 2
|
||||
|
||||
// Exponential moving average for stability
|
||||
smoothedOffset = (oldOffset * 0.9) + (offset * 0.1)
|
||||
```
|
||||
|
||||
**Timing:**
|
||||
- Send `client/time` every 1 second
|
||||
- Discard samples if RTT > 100ms (network congestion)
|
||||
- Track jitter via rolling window of recent offsets
|
||||
- Flag sync as degraded if no response in 5 seconds
|
||||
|
||||
**Usage:**
|
||||
```go
|
||||
localPlayTime := serverTimestamp - clockOffset
|
||||
```
|
||||
|
||||
### 4. Audio Decoder
|
||||
|
||||
**Decoder per Format:**
|
||||
|
||||
- **Opus:** `github.com/hraban/opus` (CGO-based, stable)
|
||||
- **FLAC:** `github.com/mewkiz/flac` (pure Go)
|
||||
- **PCM:** Direct copy (no decoding)
|
||||
|
||||
**Process Flow:**
|
||||
1. Wait for `stream/start` message with format specification
|
||||
2. Initialize appropriate decoder with codec header if provided
|
||||
3. Read from `audioChunks` channel
|
||||
4. Decode each chunk to PCM
|
||||
5. Write to `decodedAudio` channel with original timestamp preserved
|
||||
|
||||
**Format Changes:**
|
||||
- On new `stream/start`, tear down old decoder
|
||||
- Clear buffers
|
||||
- Initialize new decoder
|
||||
- Resume playback
|
||||
|
||||
**Error Handling:**
|
||||
- Log decode errors but continue (skip bad frames)
|
||||
- Report error stats to TUI
|
||||
|
||||
### 5. Playback Scheduler
|
||||
|
||||
**Buffering Strategy:**
|
||||
- Maintain priority queue ordered by timestamp
|
||||
- Target buffer depth: 100-150ms (configurable)
|
||||
- Jitter buffer to absorb network variance
|
||||
|
||||
**Scheduling Algorithm:**
|
||||
```go
|
||||
for chunk := range decodedAudio {
|
||||
// Calculate local play time
|
||||
localPlayTime := chunk.Timestamp - clockOffset
|
||||
|
||||
// Wait until precise moment
|
||||
sleepDuration := localPlayTime - time.Now()
|
||||
|
||||
if sleepDuration > 0 {
|
||||
time.Sleep(sleepDuration)
|
||||
sendToOutput(chunk)
|
||||
} else if sleepDuration > -50*time.Millisecond {
|
||||
// Slightly late but playable
|
||||
sendToOutput(chunk)
|
||||
} else {
|
||||
// Too late, drop
|
||||
logDroppedFrame(chunk)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Buffer Management:**
|
||||
- Monitor queue depth
|
||||
- Report underruns/overruns to TUI
|
||||
- Adjust jitter buffer dynamically if frequent issues
|
||||
|
||||
### 6. Audio Output
|
||||
|
||||
**Library:** `github.com/ebitengine/oto/v3` (pure Go, cross-platform)
|
||||
|
||||
**Initialization:**
|
||||
```go
|
||||
ctx, ready, err := oto.NewContext(
|
||||
sampleRate,
|
||||
channels,
|
||||
bitDepth/8, // bytes per sample
|
||||
)
|
||||
player := ctx.NewPlayer(pcmReader)
|
||||
```
|
||||
|
||||
**Volume Control:**
|
||||
Software mixing applied before output:
|
||||
```go
|
||||
volumeMultiplier := float64(volume) / 100.0
|
||||
|
||||
for i := range samples {
|
||||
if muted {
|
||||
samples[i] = 0
|
||||
} else {
|
||||
samples[i] = int16(float64(samples[i]) * volumeMultiplier)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Error Handling:**
|
||||
- Detect audio output failures
|
||||
- Attempt re-initialization
|
||||
- Report to TUI and server state
|
||||
|
||||
### 7. TUI Controller
|
||||
|
||||
**Library:** `github.com/charmbracelet/bubbletea`
|
||||
|
||||
**Display Layout:**
|
||||
```
|
||||
┌─ Resonate Player ────────────────────────────────────┐
|
||||
│ Status: Connected to music-assistant.local │
|
||||
│ Sync: ✓ Synced (offset: +2.3ms, jitter: 0.8ms) │
|
||||
├──────────────────────────────────────────────────────┤
|
||||
│ Now Playing: │
|
||||
│ Track: The Less I Know the Better │
|
||||
│ Artist: Tame Impala │
|
||||
│ Album: Currents │
|
||||
│ │
|
||||
│ Format: Opus 48kHz Stereo 16-bit │
|
||||
│ │
|
||||
│ Volume: [████████░░] 80% │
|
||||
│ Buffer: 150ms (50 chunks) │
|
||||
├──────────────────────────────────────────────────────┤
|
||||
│ Stats: RX: 12,450 Played: 12,380 Dropped: 2 │
|
||||
│ │
|
||||
│ ↑/↓:Volume m:Mute r:Reconnect d:Debug q:Quit │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Keyboard Controls:**
|
||||
- `↑/↓`: Volume ±5%
|
||||
- `m`: Toggle mute
|
||||
- `r`: Force reconnect
|
||||
- `d`: Toggle debug panel (goroutine stats, channel depths)
|
||||
- `q`: Quit
|
||||
|
||||
**State Updates:**
|
||||
- Subscribe to state changes via channels
|
||||
- Debounce rapid updates (max 30 FPS)
|
||||
- Send `client/state` to server when volume/mute changes
|
||||
|
||||
**Debug Panel (toggled):**
|
||||
```
|
||||
Goroutines: 8
|
||||
Channels:
|
||||
audioChunks: 15/100
|
||||
decodedAudio: 8/50
|
||||
controlMsgs: 0/10
|
||||
Clock Offset: +2345μs ±800μs
|
||||
```
|
||||
|
||||
## Control Flow
|
||||
|
||||
### Server Command Handling
|
||||
|
||||
**Incoming `server/command`:**
|
||||
```json
|
||||
{
|
||||
"type": "server/command",
|
||||
"payload": {
|
||||
"command": "volume",
|
||||
"volume": 75
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Processing:**
|
||||
1. Parse command type
|
||||
2. Apply change to internal state
|
||||
3. Update audio output (volume multiplier)
|
||||
4. Send `client/state` with changed fields:
|
||||
```json
|
||||
{
|
||||
"type": "client/state",
|
||||
"payload": {
|
||||
"volume": 75
|
||||
}
|
||||
}
|
||||
```
|
||||
5. Update TUI display
|
||||
|
||||
### Metadata Updates
|
||||
|
||||
**Incoming `stream/metadata`:**
|
||||
```json
|
||||
{
|
||||
"type": "stream/metadata",
|
||||
"payload": {
|
||||
"title": "Song Title",
|
||||
"artist": "Artist Name",
|
||||
"album": "Album Name",
|
||||
"artwork_url": "https://..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Processing:**
|
||||
1. Parse metadata
|
||||
2. Update TUI display immediately
|
||||
3. Optionally cache artwork for display
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Network Errors
|
||||
- WebSocket disconnect → Show in TUI, start reconnect backoff
|
||||
- Discovery failure → Continue trying both modes indefinitely
|
||||
- Timeout on handshake → Reconnect with fresh connection
|
||||
|
||||
### Audio Errors
|
||||
- Decode failure → Log, skip frame, continue
|
||||
- Output device failure → Report error state to server, attempt re-init
|
||||
- Buffer underrun → Report stats, continue (audio glitch acceptable)
|
||||
- Late frames → Drop if >50ms late
|
||||
|
||||
### Clock Sync Errors
|
||||
- No sync response in 5s → Mark degraded, continue using last offset
|
||||
- Excessive jitter → Increase buffer size dynamically
|
||||
- Complete sync loss → Continue playing but warn in TUI
|
||||
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
github.com/hashicorp/mdns # mDNS discovery
|
||||
github.com/gorilla/websocket # WebSocket client
|
||||
github.com/ebitengine/oto/v3 # Audio output
|
||||
github.com/hraban/opus # Opus decoder (CGO)
|
||||
github.com/mewkiz/flac # FLAC decoder
|
||||
github.com/charmbracelet/bubbletea # TUI framework
|
||||
github.com/google/uuid # Client ID generation
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
**CLI Flags:**
|
||||
```
|
||||
--server string Manual server address (skip mDNS)
|
||||
--port int Port for mDNS advertisement (default: 8927)
|
||||
--name string Player friendly name (default: hostname-resonate-player)
|
||||
--buffer-ms int Jitter buffer size (default: 150ms)
|
||||
--log-file string Log file path (default: ./resonate-player.log)
|
||||
--debug Enable debug logging
|
||||
```
|
||||
|
||||
**Config File (optional):**
|
||||
```yaml
|
||||
# ~/.config/resonate-player/config.yaml
|
||||
server: "music-assistant.local:8927"
|
||||
name: "Living Room Speaker"
|
||||
buffer_ms: 200
|
||||
log_level: "info"
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
- Clock offset calculation
|
||||
- Audio format conversions
|
||||
- Volume mixing algorithm
|
||||
- Message parsing/serialization
|
||||
|
||||
### Integration Tests
|
||||
- Mock WebSocket server for protocol testing
|
||||
- Synthetic audio stream playback
|
||||
- Reconnection scenarios
|
||||
- Format switching
|
||||
|
||||
### Manual Testing
|
||||
- Multi-room sync with multiple instances
|
||||
- Network disruption (wifi drops)
|
||||
- Long-running stability (24h+)
|
||||
- Various audio formats from Music Assistant
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Visualizer support (spectrum analyzer in TUI)
|
||||
- Metadata with artwork display (kitty/sixel protocols)
|
||||
- Hardware volume control integration (ALSA/PulseAudio/CoreAudio)
|
||||
- Web-based control UI
|
||||
- Playlist/queue display
|
||||
- Audio effects (EQ, crossfade)
|
||||
|
||||
## References
|
||||
|
||||
- Resonate Protocol Spec: https://github.com/Resonate-Protocol/spec
|
||||
- Music Assistant: https://music-assistant.io/
|
||||
2939
third_party/sendspin-go/docs/plans/2025-10-23-resonate-player-implementation.md
vendored
Normal file
2939
third_party/sendspin-go/docs/plans/2025-10-23-resonate-player-implementation.md
vendored
Normal file
File diff suppressed because it is too large
Load Diff
530
third_party/sendspin-go/docs/plans/2025-10-25-library-refactor-design.md
vendored
Normal file
530
third_party/sendspin-go/docs/plans/2025-10-25-library-refactor-design.md
vendored
Normal file
@@ -0,0 +1,530 @@
|
||||
# Library Refactor Design
|
||||
|
||||
**Date:** 2025-10-25
|
||||
**Status:** Approved
|
||||
**Approach:** Ground-Up Redesign (Approach C)
|
||||
|
||||
## Overview
|
||||
|
||||
Convert resonate-go from a CLI-focused project to a library-first architecture with layered APIs. The existing CLI tools will become thin wrappers that use the public library APIs, serving as reference implementations.
|
||||
|
||||
## Goals
|
||||
|
||||
1. **Library-first architecture** - Primary use case is embedding in Go applications
|
||||
2. **Layered APIs** - High-level convenience for most users, low-level building blocks for power users
|
||||
3. **Aggressive migration** - Complete restructure, not gradual wrapper approach
|
||||
4. **Backward compatibility** - CLI tools use library but maintain same functionality
|
||||
5. **Clean package design** - Intuitive organization, clear boundaries, good documentation
|
||||
|
||||
## Use Cases
|
||||
|
||||
### Primary Use Cases
|
||||
- Embed player in Go applications (desktop music apps, smart home controllers)
|
||||
- Embed server in Go applications (music streaming services, home media servers)
|
||||
- Build custom audio pipelines (decoders, resamplers, encoders for custom processing)
|
||||
- Enable Music Assistant and similar systems to integrate Resonate directly
|
||||
|
||||
### Example Use Cases
|
||||
- Desktop music player with custom UI
|
||||
- Multi-room audio controller
|
||||
- Home media server with Resonate output
|
||||
- Audio processing pipeline with hi-res support
|
||||
- Music Assistant native Resonate provider
|
||||
|
||||
## Architecture
|
||||
|
||||
### Three-Layer Design
|
||||
|
||||
**Layer 1: High-Level Convenience (`pkg/resonate/`)**
|
||||
- Simple constructors: `NewPlayer()`, `NewServer()`
|
||||
- Sensible defaults for common use cases
|
||||
- Hides complexity: connection management, format negotiation, error recovery
|
||||
- Target: Users who want "just play audio" or "just serve audio"
|
||||
|
||||
**Layer 2: Component APIs (`pkg/audio/`, `pkg/protocol/`, `pkg/sync/`, `pkg/discovery/`)**
|
||||
- Building blocks for custom implementations
|
||||
- Each package focused on one concern
|
||||
- Composable: mix and match components
|
||||
- Target: Users building custom audio pipelines or integrations
|
||||
|
||||
**Layer 3: Internal Implementation (`internal/`)**
|
||||
- CLI app logic (`internal/app/`)
|
||||
- TUI implementation (`internal/ui/`)
|
||||
- Implementation details not exposed as public API
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
resonate-go/
|
||||
├── pkg/
|
||||
│ ├── resonate/ # High-level convenience API
|
||||
│ │ ├── player.go # Player API
|
||||
│ │ ├── server.go # Server API
|
||||
│ │ └── source.go # AudioSource interface + built-ins
|
||||
│ ├── audio/ # Audio fundamentals
|
||||
│ │ ├── format.go # Format, Buffer types
|
||||
│ │ ├── types.go # Constants, conversion helpers
|
||||
│ │ ├── decode/ # Decoders
|
||||
│ │ │ ├── decoder.go # Interface
|
||||
│ │ │ ├── pcm.go # PCM decoder
|
||||
│ │ │ ├── opus.go # Opus decoder
|
||||
│ │ │ ├── flac.go # FLAC decoder
|
||||
│ │ │ └── mp3.go # MP3 decoder
|
||||
│ │ ├── encode/ # Encoders
|
||||
│ │ │ ├── encoder.go # Interface
|
||||
│ │ │ ├── pcm.go # PCM encoder
|
||||
│ │ │ └── opus.go # Opus encoder
|
||||
│ │ ├── resample/ # Resampling
|
||||
│ │ │ └── resampler.go
|
||||
│ │ └── output/ # Audio output
|
||||
│ │ ├── output.go # Interface
|
||||
│ │ └── portaudio.go
|
||||
│ ├── protocol/ # Resonate wire protocol
|
||||
│ │ ├── messages.go # Protocol message types
|
||||
│ │ └── client.go # WebSocket client
|
||||
│ ├── sync/ # Clock synchronization
|
||||
│ │ └── clock.go
|
||||
│ └── discovery/ # mDNS discovery
|
||||
│ └── mdns.go
|
||||
├── cmd/
|
||||
│ ├── resonate-player/ # Thin CLI wrapper
|
||||
│ └── resonate-server/ # Thin CLI wrapper
|
||||
├── internal/
|
||||
│ ├── app/ # CLI app logic
|
||||
│ └── ui/ # TUI implementation
|
||||
├── examples/ # Example code
|
||||
│ ├── basic-player/
|
||||
│ ├── basic-server/
|
||||
│ ├── custom-source/
|
||||
│ ├── multi-room/
|
||||
│ └── audio-pipeline/
|
||||
└── docs/
|
||||
└── plans/
|
||||
```
|
||||
|
||||
## API Design
|
||||
|
||||
### High-Level API (`pkg/resonate/`)
|
||||
|
||||
#### Player API
|
||||
|
||||
```go
|
||||
package resonate
|
||||
|
||||
// PlayerConfig for creating a player
|
||||
type PlayerConfig struct {
|
||||
ServerAddr string // "localhost:8927" or discovered via mDNS
|
||||
PlayerName string // Display name
|
||||
Volume int // 0-100
|
||||
DebugMode bool // Enable debug logging
|
||||
}
|
||||
|
||||
// Player represents a Resonate audio player
|
||||
type Player struct {
|
||||
// unexported fields
|
||||
}
|
||||
|
||||
// NewPlayer creates a new player instance
|
||||
func NewPlayer(cfg PlayerConfig) (*Player, error)
|
||||
|
||||
// Connect to the configured server
|
||||
func (p *Player) Connect() error
|
||||
|
||||
// Play starts playback
|
||||
func (p *Player) Play() error
|
||||
|
||||
// Pause pauses playback
|
||||
func (p *Player) Pause() error
|
||||
|
||||
// Stop stops playback and disconnects
|
||||
func (p *Player) Stop() error
|
||||
|
||||
// SetVolume adjusts volume (0-100)
|
||||
func (p *Player) SetVolume(vol int) error
|
||||
|
||||
// Mute toggles mute
|
||||
func (p *Player) Mute(muted bool) error
|
||||
|
||||
// Status returns current playback status
|
||||
func (p *Player) Status() PlayerStatus
|
||||
|
||||
// Close releases resources
|
||||
func (p *Player) Close() error
|
||||
```
|
||||
|
||||
#### Server API
|
||||
|
||||
```go
|
||||
package resonate
|
||||
|
||||
// ServerConfig for creating a server
|
||||
type ServerConfig struct {
|
||||
Address string // "0.0.0.0"
|
||||
Port int // 8927
|
||||
Source AudioSource // Where audio comes from
|
||||
EnablemDNS bool // Advertise via mDNS
|
||||
DebugMode bool
|
||||
}
|
||||
|
||||
// Server serves audio to Resonate clients
|
||||
type Server struct {
|
||||
// unexported fields
|
||||
}
|
||||
|
||||
// NewServer creates a new server instance
|
||||
func NewServer(cfg ServerConfig) (*Server, error)
|
||||
|
||||
// Start begins serving (blocks)
|
||||
func (s *Server) Start() error
|
||||
|
||||
// Stop gracefully shuts down
|
||||
func (s *Server) Stop() error
|
||||
|
||||
// Clients returns connected client info
|
||||
func (s *Server) Clients() []ClientInfo
|
||||
```
|
||||
|
||||
#### AudioSource Interface
|
||||
|
||||
```go
|
||||
package resonate
|
||||
|
||||
// AudioSource provides audio samples for the server
|
||||
type AudioSource interface {
|
||||
Read(samples []int32) (int, error)
|
||||
SampleRate() int
|
||||
Channels() int
|
||||
BitDepth() int
|
||||
Metadata() (title, artist, album string)
|
||||
Close() error
|
||||
}
|
||||
|
||||
// Built-in source constructors
|
||||
func FileSource(path string) (AudioSource, error)
|
||||
func TestToneSource(frequency float64) AudioSource
|
||||
```
|
||||
|
||||
### Component-Level APIs
|
||||
|
||||
#### `pkg/audio/` - Audio Fundamentals
|
||||
|
||||
```go
|
||||
package audio
|
||||
|
||||
// Core types
|
||||
type Format struct {
|
||||
Codec string
|
||||
SampleRate int
|
||||
Channels int
|
||||
BitDepth int
|
||||
CodecHeader []byte
|
||||
}
|
||||
|
||||
type Buffer struct {
|
||||
Timestamp int64
|
||||
PlayAt time.Time
|
||||
Samples []int32
|
||||
Format Format
|
||||
}
|
||||
|
||||
// Constants
|
||||
const (
|
||||
Max24Bit = 8388607 // 2^23 - 1
|
||||
Min24Bit = -8388608 // -2^23
|
||||
)
|
||||
|
||||
// Conversion helpers
|
||||
func SampleToInt16(sample int32) int16
|
||||
func SampleFromInt16(sample int16) int32
|
||||
func SampleTo24Bit(sample int32) [3]byte
|
||||
func SampleFrom24Bit(b [3]byte) int32
|
||||
```
|
||||
|
||||
#### `pkg/audio/decode/` - Decoders
|
||||
|
||||
```go
|
||||
package decode
|
||||
|
||||
// Decoder interface
|
||||
type Decoder interface {
|
||||
Decode(data []byte) ([]int32, error)
|
||||
Close() error
|
||||
}
|
||||
|
||||
// Constructors for each codec
|
||||
func NewPCM(format audio.Format) (Decoder, error)
|
||||
func NewOpus(format audio.Format) (Decoder, error)
|
||||
func NewFLAC(format audio.Format) (Decoder, error)
|
||||
func NewMP3(format audio.Format) (Decoder, error)
|
||||
```
|
||||
|
||||
#### `pkg/audio/encode/` - Encoders
|
||||
|
||||
```go
|
||||
package encode
|
||||
|
||||
type Encoder interface {
|
||||
Encode(samples []int32) ([]byte, error)
|
||||
Close() error
|
||||
}
|
||||
|
||||
func NewPCM(format audio.Format) (Encoder, error)
|
||||
func NewOpus(format audio.Format) (Encoder, error)
|
||||
```
|
||||
|
||||
#### `pkg/audio/resample/` - Resampling
|
||||
|
||||
```go
|
||||
package resample
|
||||
|
||||
type Resampler struct {
|
||||
// unexported
|
||||
}
|
||||
|
||||
func New(inputRate, outputRate, channels int) *Resampler
|
||||
func (r *Resampler) Resample(input, output []int32) int
|
||||
```
|
||||
|
||||
#### `pkg/audio/output/` - Audio Output
|
||||
|
||||
```go
|
||||
package output
|
||||
|
||||
type Output interface {
|
||||
Open(sampleRate, channels int) error
|
||||
Write(samples []int32) error
|
||||
Close() error
|
||||
}
|
||||
|
||||
func NewPortAudio() Output
|
||||
```
|
||||
|
||||
#### `pkg/protocol/` - Resonate Wire Protocol
|
||||
|
||||
```go
|
||||
package protocol
|
||||
|
||||
// Message types
|
||||
type HelloMessage struct {
|
||||
PlayerName string
|
||||
SupportFormats []Format
|
||||
SupportCodecs []string
|
||||
SupportSampleRates []int
|
||||
SupportBitDepth []int
|
||||
}
|
||||
|
||||
type StartMessage struct {
|
||||
Format Format
|
||||
ServerTime int64
|
||||
StreamOffset int64
|
||||
}
|
||||
|
||||
type ChunkMessage struct {
|
||||
Timestamp int64
|
||||
Data []byte
|
||||
}
|
||||
|
||||
type ControlMessage struct {
|
||||
Command string // "play", "pause", "stop"
|
||||
}
|
||||
|
||||
// Client for low-level protocol control
|
||||
type Client struct {
|
||||
// unexported
|
||||
}
|
||||
|
||||
func NewClient(serverAddr string) (*Client, error)
|
||||
func (c *Client) SendHello(hello HelloMessage) error
|
||||
func (c *Client) ReadMessage() (interface{}, error)
|
||||
func (c *Client) Close() error
|
||||
```
|
||||
|
||||
#### `pkg/sync/` - Clock Synchronization
|
||||
|
||||
```go
|
||||
package sync
|
||||
|
||||
type Clock struct {
|
||||
// unexported
|
||||
}
|
||||
|
||||
func NewClock() *Clock
|
||||
func (c *Clock) Sync(serverAddr string) error
|
||||
func (c *Clock) ServerTime() int64
|
||||
func (c *Clock) LocalTime() time.Time
|
||||
func (c *Clock) Offset() int64
|
||||
```
|
||||
|
||||
#### `pkg/discovery/` - mDNS Server Discovery
|
||||
|
||||
```go
|
||||
package discovery
|
||||
|
||||
type Service struct {
|
||||
Name string
|
||||
Address string
|
||||
Port int
|
||||
}
|
||||
|
||||
// Discover servers on the network
|
||||
func Discover(timeout time.Duration) ([]Service, error)
|
||||
|
||||
// Advertise this server
|
||||
func Advertise(name string, port int) error
|
||||
func StopAdvertising() error
|
||||
```
|
||||
|
||||
## CLI Migration
|
||||
|
||||
The existing CLI tools (`cmd/resonate-player/main.go`, `cmd/resonate-server/main.go`) will be rewritten to use the high-level `pkg/resonate/` API. They will handle only:
|
||||
- Flag parsing
|
||||
- TUI setup (using `internal/ui/`)
|
||||
- Signal handling
|
||||
- Calling library functions
|
||||
|
||||
### Example - New Player CLI
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/harperreed/resonate-go/pkg/resonate"
|
||||
"github.com/harperreed/resonate-go/internal/ui"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// Parse flags
|
||||
cfg := resonate.PlayerConfig{
|
||||
ServerAddr: *serverFlag,
|
||||
PlayerName: *nameFlag,
|
||||
Volume: *volumeFlag,
|
||||
DebugMode: *debugFlag,
|
||||
}
|
||||
|
||||
// Create player using library
|
||||
player, err := resonate.NewPlayer(cfg)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer player.Close()
|
||||
|
||||
// Connect and play
|
||||
if err := player.Connect(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Run TUI (internal implementation)
|
||||
ui.RunPlayerUI(player)
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
Create `examples/` directory with real-world usage:
|
||||
- `examples/basic-player/` - Simple player implementation
|
||||
- `examples/basic-server/` - Simple server implementation
|
||||
- `examples/custom-source/` - Custom AudioSource implementation
|
||||
- `examples/multi-room/` - Multiple synchronized players
|
||||
- `examples/audio-pipeline/` - Using low-level audio components
|
||||
|
||||
These examples serve dual purpose:
|
||||
1. Documentation for library users
|
||||
2. Integration tests for the library
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Step-by-Step Plan
|
||||
|
||||
1. **Create new package structure** - Set up `pkg/` directories with package stubs
|
||||
2. **Move audio primitives first** - `pkg/audio/`, `pkg/audio/decode/`, etc. (foundation layer)
|
||||
3. **Move protocol layer** - `pkg/protocol/`, `pkg/sync/`, `pkg/discovery/`
|
||||
4. **Build high-level APIs** - `pkg/resonate/` wrapping the components
|
||||
5. **Migrate CLI tools** - Rewrite to use `pkg/resonate/`
|
||||
6. **Add examples** - Create `examples/` directory with working code
|
||||
7. **Documentation** - README updates, godoc comments on all exported types/functions
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
- Move existing tests alongside the code as packages are migrated
|
||||
- Add integration tests in `examples/` (they serve dual purpose as docs)
|
||||
- Ensure CLI tools work identically to current behavior
|
||||
- Test both high-level and low-level APIs work independently
|
||||
|
||||
### Version Strategy
|
||||
|
||||
- This is a major refactor → tag as `v1.0.0` when complete
|
||||
- Signals stable library API and commitment to compatibility
|
||||
- Previous v0.0.x were "pre-library" releases for CLI tools only
|
||||
|
||||
### Rollout Plan
|
||||
|
||||
1. Work in a feature branch/worktree (`library-refactor`)
|
||||
2. Validate each layer works before moving to next
|
||||
3. Merge to main when:
|
||||
- All packages implemented
|
||||
- CLI tools work with library
|
||||
- Examples run successfully
|
||||
- Tests pass
|
||||
4. Tag `v1.0.0` release
|
||||
|
||||
## Documentation Requirements
|
||||
|
||||
### Package Documentation
|
||||
Every exported package, type, function must have godoc comments:
|
||||
- Package-level doc explaining purpose and usage
|
||||
- Type-level doc with example
|
||||
- Function-level doc with parameters and return values
|
||||
|
||||
### README Updates
|
||||
- Add "Using as a Library" section
|
||||
- Show both high-level and low-level API examples
|
||||
- Link to examples directory
|
||||
- Keep CLI usage docs
|
||||
|
||||
### Examples
|
||||
Each example should be a complete, runnable program showing real-world usage.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. ✅ All code moved from `internal/` to `pkg/` where appropriate
|
||||
2. ✅ High-level API works for simple player/server use cases
|
||||
3. ✅ Low-level APIs allow building custom pipelines
|
||||
4. ✅ CLI tools work using public library APIs
|
||||
5. ✅ All examples run successfully
|
||||
6. ✅ All tests pass
|
||||
7. ✅ Documentation complete (godoc + README + examples)
|
||||
8. ✅ Tagged as v1.0.0
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
- **5-7 days** total for complete migration
|
||||
- Foundation (audio/protocol): 2 days
|
||||
- High-level API: 1-2 days
|
||||
- CLI migration: 1 day
|
||||
- Examples + documentation: 1-2 days
|
||||
- Testing + polish: 1 day
|
||||
|
||||
## Trade-offs
|
||||
|
||||
### Advantages
|
||||
- ✅ Best API design - clean separation, intuitive naming
|
||||
- ✅ Maximum flexibility - every component usable independently
|
||||
- ✅ Good documentation story - clear package boundaries
|
||||
- ✅ Future-proof - stable v1.0.0 API
|
||||
|
||||
### Disadvantages
|
||||
- ❌ Most work - complete restructure vs. simple wrapper
|
||||
- ❌ Higher risk - more things to potentially break
|
||||
- ❌ Longer timeline - 5-7 days vs. 1-2 days for simple approach
|
||||
|
||||
### Mitigation
|
||||
- Work in isolated worktree/branch
|
||||
- Validate each layer before proceeding
|
||||
- Keep existing code as reference
|
||||
- Comprehensive testing at each stage
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Set up git worktree for isolated development
|
||||
2. Create detailed implementation plan with tasks
|
||||
3. Begin migration starting with foundation layer
|
||||
1565
third_party/sendspin-go/docs/plans/2025-10-25-library-refactor-plan.md
vendored
Normal file
1565
third_party/sendspin-go/docs/plans/2025-10-25-library-refactor-plan.md
vendored
Normal file
File diff suppressed because it is too large
Load Diff
440
third_party/sendspin-go/docs/plans/phase1-hires-fixes.md
vendored
Normal file
440
third_party/sendspin-go/docs/plans/phase1-hires-fixes.md
vendored
Normal file
@@ -0,0 +1,440 @@
|
||||
# Phase 1: Hi-Res Audio Fixes
|
||||
|
||||
**Goal:** Enable true 24-bit output and optimize Opus bandwidth usage
|
||||
|
||||
**Date:** 2025-10-26
|
||||
**Status:** Planning
|
||||
|
||||
---
|
||||
|
||||
## Problem Summary
|
||||
|
||||
1. **16-bit Output Choke Point** 🔴
|
||||
- Current: oto only supports 16-bit (FormatSignedInt16LE)
|
||||
- Impact: 24-bit pipeline is downsampled to 16-bit at playback
|
||||
- Loss: Lower 8 bits of precision thrown away
|
||||
|
||||
2. **Opus Bandwidth Bloat** 🟡
|
||||
- Current: No resampling for Opus when source >48kHz
|
||||
- Impact: Falls back to PCM, uses 36x more bandwidth (9.2 Mbps vs 0.26 Mbps)
|
||||
- Issue: Client wants compression, gets uncompressed hi-res instead
|
||||
|
||||
---
|
||||
|
||||
## Solution 1: Swap oto → malgo (24-bit support)
|
||||
|
||||
### Why malgo?
|
||||
- ✅ Native 24-bit support (`FormatS24`)
|
||||
- ✅ No external dependencies on Windows/macOS
|
||||
- ✅ Can re-initialize for format changes
|
||||
- ✅ Modern, actively maintained
|
||||
|
||||
### Files to Create
|
||||
|
||||
#### 1. `pkg/audio/output/malgo.go`
|
||||
|
||||
**Purpose:** New output backend using malgo library
|
||||
|
||||
**Key Features:**
|
||||
- Support both 16-bit and 24-bit output
|
||||
- Handle format re-initialization (solve oto limitation)
|
||||
- Callback-based architecture (malgo uses push model)
|
||||
- Buffer management for smooth playback
|
||||
|
||||
**API Design:**
|
||||
```go
|
||||
type Malgo struct {
|
||||
ctx *malgo.AllocatedContext
|
||||
device *malgo.Device
|
||||
sampleRate int
|
||||
channels int
|
||||
bitDepth int // NEW: track bit depth (16 or 24)
|
||||
volume int
|
||||
muted bool
|
||||
|
||||
// Buffering for callback-based playback
|
||||
ringBuffer *RingBuffer
|
||||
mu sync.Mutex
|
||||
}
|
||||
|
||||
// Open initializes with bit depth support
|
||||
func (m *Malgo) Open(sampleRate, channels, bitDepth int) error
|
||||
|
||||
// Write queues samples to ring buffer
|
||||
func (m *Malgo) Write(samples []int32) error
|
||||
|
||||
// dataCallback is called by malgo to fill audio buffer
|
||||
func (m *Malgo) dataCallback(pOutput, pInput [][]byte, frameCount uint32)
|
||||
|
||||
// Reinitialize allows format changes (solves oto issue)
|
||||
func (m *Malgo) Reinitialize(sampleRate, channels, bitDepth int) error
|
||||
```
|
||||
|
||||
**Sample Conversion:**
|
||||
```go
|
||||
// For 24-bit output
|
||||
func int32To24Bit(sample int32) []byte {
|
||||
return []byte{
|
||||
byte(sample),
|
||||
byte(sample >> 8),
|
||||
byte(sample >> 16),
|
||||
}
|
||||
}
|
||||
|
||||
// For 16-bit output (backward compat)
|
||||
func int32To16Bit(sample int32) []byte {
|
||||
sample16 := int16(sample >> 8) // Shift down to 16-bit
|
||||
return []byte{
|
||||
byte(sample16),
|
||||
byte(sample16 >> 8),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Ring Buffer:**
|
||||
```go
|
||||
// Simple ring buffer for callback model
|
||||
type RingBuffer struct {
|
||||
buffer []int32
|
||||
read int
|
||||
write int
|
||||
size int
|
||||
mu sync.Mutex
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Files to Modify
|
||||
|
||||
#### 2. `pkg/audio/output/output.go`
|
||||
|
||||
**Change:** Add bitDepth parameter to Open()
|
||||
|
||||
```diff
|
||||
type Output interface {
|
||||
- Open(sampleRate, channels int) error
|
||||
+ Open(sampleRate, channels, bitDepth int) error
|
||||
Write(samples []int32) error
|
||||
Close() error
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. `pkg/audio/output/oto.go`
|
||||
|
||||
**Change:** Update to match new interface (backward compat)
|
||||
|
||||
```diff
|
||||
-func (o *Oto) Open(sampleRate, channels int) error {
|
||||
+func (o *Oto) Open(sampleRate, channels, bitDepth int) error {
|
||||
+ // oto only supports 16-bit, log warning if 24-bit requested
|
||||
+ if bitDepth != 16 {
|
||||
+ log.Printf("Warning: oto only supports 16-bit, ignoring bitDepth=%d", bitDepth)
|
||||
+ }
|
||||
// ... existing code
|
||||
}
|
||||
```
|
||||
|
||||
#### 4. `pkg/resonate/player.go`
|
||||
|
||||
**Change:** Use malgo instead of oto, pass bitDepth
|
||||
|
||||
```diff
|
||||
import (
|
||||
"github.com/Resonate-Protocol/resonate-go/pkg/audio/output"
|
||||
+ _ "github.com/Resonate-Protocol/resonate-go/pkg/audio/output/malgo"
|
||||
)
|
||||
|
||||
func (p *Player) setupOutput() error {
|
||||
- p.output = output.NewOto()
|
||||
+ p.output = output.NewMalgo()
|
||||
- err := p.output.Open(p.format.SampleRate, p.format.Channels)
|
||||
+ err := p.output.Open(p.format.SampleRate, p.format.Channels, p.format.BitDepth)
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
#### 5. `go.mod`
|
||||
|
||||
**Change:** Add malgo dependency
|
||||
|
||||
```diff
|
||||
require (
|
||||
github.com/ebitengine/oto/v3 v3.4.0
|
||||
+ github.com/gen2brain/malgo v0.11.21
|
||||
// ... other deps
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Solution 2: Add Opus Resampling
|
||||
|
||||
### Why resample for Opus?
|
||||
- Opus is fixed at 48kHz
|
||||
- Hi-res sources (192kHz) can't be encoded directly
|
||||
- Without resampling → falls back to PCM → 36x bandwidth increase
|
||||
- With resampling → downsample to 48kHz → compress with Opus → saves bandwidth
|
||||
|
||||
### Files to Modify
|
||||
|
||||
#### 6. `internal/server/audio_engine.go`
|
||||
|
||||
**Change 1:** Add resampler to OpusEncoder struct
|
||||
|
||||
```diff
|
||||
type Client struct {
|
||||
// ... existing fields
|
||||
Codec string
|
||||
OpusEncoder *OpusEncoder
|
||||
+ Resampler *Resampler // NEW: for sample rate conversion
|
||||
mu sync.RWMutex
|
||||
}
|
||||
```
|
||||
|
||||
**Change 2:** Update AddClient to create resampler for Opus
|
||||
|
||||
```diff
|
||||
func (e *AudioEngine) AddClient(client *Client) {
|
||||
codec := e.negotiateCodec(client)
|
||||
|
||||
switch codec {
|
||||
case "opus":
|
||||
encoder, err := NewOpusEncoder(e.source.SampleRate(), e.source.Channels(), chunkSamples)
|
||||
if err != nil {
|
||||
log.Printf("Failed to create Opus encoder for %s, falling back to PCM: %v", client.Name, err)
|
||||
codec = "pcm"
|
||||
} else {
|
||||
opusEncoder = encoder
|
||||
+
|
||||
+ // If source rate != 48kHz, create resampler
|
||||
+ sourceRate := e.source.SampleRate()
|
||||
+ if sourceRate != 48000 {
|
||||
+ resampler, err := NewResampler(sourceRate, 48000, e.source.Channels())
|
||||
+ if err != nil {
|
||||
+ log.Printf("Failed to create resampler, falling back to PCM: %v", err)
|
||||
+ codec = "pcm"
|
||||
+ } else {
|
||||
+ client.Resampler = resampler
|
||||
+ log.Printf("Created resampler: %dHz → 48kHz for Opus", sourceRate)
|
||||
+ }
|
||||
+ }
|
||||
}
|
||||
}
|
||||
|
||||
client.mu.Lock()
|
||||
client.Codec = codec
|
||||
client.OpusEncoder = opusEncoder
|
||||
client.mu.Unlock()
|
||||
}
|
||||
```
|
||||
|
||||
**Change 3:** Update generateAndSendChunk to use resampler
|
||||
|
||||
```diff
|
||||
func (e *AudioEngine) generateAndSendChunk() {
|
||||
// ... read samples from source ...
|
||||
|
||||
for _, client := range e.clients {
|
||||
var audioData []byte
|
||||
|
||||
client.mu.RLock()
|
||||
codec := client.Codec
|
||||
opusEncoder := client.OpusEncoder
|
||||
+ resampler := client.Resampler
|
||||
client.mu.RUnlock()
|
||||
|
||||
switch codec {
|
||||
case "opus":
|
||||
if opusEncoder != nil {
|
||||
+ // Resample if needed
|
||||
+ samplesToEncode := samples[:n]
|
||||
+ if resampler != nil {
|
||||
+ resampled, err := resampler.Resample(samplesToEncode)
|
||||
+ if err != nil {
|
||||
+ log.Printf("Resample error for %s: %v", client.Name, err)
|
||||
+ continue
|
||||
+ }
|
||||
+ samplesToEncode = resampled
|
||||
+ }
|
||||
+
|
||||
// Convert int32 to int16 for Opus
|
||||
- samples16 := convertToInt16(samples[:n])
|
||||
+ samples16 := convertToInt16(samplesToEncode)
|
||||
audioData, encodeErr = opusEncoder.Encode(samples16)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Change 4:** Update negotiateCodec to prefer Opus for hi-res sources
|
||||
|
||||
```diff
|
||||
func (e *AudioEngine) negotiateCodec(client *Client) string {
|
||||
sourceRate := e.source.SampleRate()
|
||||
|
||||
// Check support_formats
|
||||
for _, format := range client.Capabilities.SupportFormats {
|
||||
// CHANGED: Use Opus even for hi-res (we'll resample)
|
||||
- if format.Codec == "opus" && sourceRate == 48000 {
|
||||
+ if format.Codec == "opus" {
|
||||
return "opus"
|
||||
}
|
||||
|
||||
// If client supports exact source format, use PCM (lossless)
|
||||
if format.Codec == "pcm" && format.SampleRate == sourceRate {
|
||||
return "pcm"
|
||||
}
|
||||
}
|
||||
|
||||
// Legacy support
|
||||
for _, codec := range client.Capabilities.SupportCodecs {
|
||||
- if codec == "opus" && sourceRate == 48000 {
|
||||
+ if codec == "opus" {
|
||||
return "opus"
|
||||
}
|
||||
}
|
||||
|
||||
return "pcm"
|
||||
}
|
||||
```
|
||||
|
||||
#### 7. `internal/server/resampler.go`
|
||||
|
||||
**Verify:** Ensure it handles int32 samples properly
|
||||
|
||||
Current resampler already uses `[]int32`, so should work as-is. Just verify the API:
|
||||
|
||||
```go
|
||||
func NewResampler(fromRate, toRate, channels int) (*Resampler, error)
|
||||
func (r *Resampler) Resample(samples []int32) ([]int32, error)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Plan
|
||||
|
||||
### Test 1: 24-bit Output Verification
|
||||
```bash
|
||||
# Start player with 192kHz/24-bit source
|
||||
./resonate-player -server localhost:8927 -name "24bit-test"
|
||||
|
||||
# Verify in logs:
|
||||
# "Audio output initialized: 192000Hz, 2ch, 24bit"
|
||||
# "Using malgo backend with FormatS24"
|
||||
|
||||
# Measure: Use audio analyzer to verify full 24-bit dynamic range
|
||||
```
|
||||
|
||||
### Test 2: Opus Resampling Verification
|
||||
```bash
|
||||
# Start server with 192kHz source
|
||||
./resonate-server -audio test_192k.flac
|
||||
|
||||
# Connect client that prefers Opus
|
||||
./resonate-player -server localhost:8927 -prefer-opus
|
||||
|
||||
# Verify in logs:
|
||||
# "Created resampler: 192000Hz → 48kHz for Opus"
|
||||
# "Client codec negotiated: opus"
|
||||
# NOT "falling back to PCM"
|
||||
|
||||
# Measure bandwidth: Should see ~0.26 Mbps instead of 9.2 Mbps
|
||||
```
|
||||
|
||||
### Test 3: Format Switching
|
||||
```bash
|
||||
# Start with 48kHz source
|
||||
./resonate-server -audio 48k.flac
|
||||
|
||||
# Connect player (should get Opus at 48kHz)
|
||||
./resonate-player
|
||||
|
||||
# Restart server with 192kHz source
|
||||
# Player should detect format change and reinitialize
|
||||
|
||||
# Verify: malgo reinitializes successfully (oto couldn't do this)
|
||||
```
|
||||
|
||||
### Test 4: Backward Compatibility
|
||||
```bash
|
||||
# Test that oto still works for users who want it
|
||||
./resonate-player -backend oto
|
||||
|
||||
# Should work but log warning about 16-bit limitation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bandwidth Comparison
|
||||
|
||||
### Before (No Resampling)
|
||||
```
|
||||
Source: 192kHz/24-bit stereo
|
||||
Client: Wants Opus
|
||||
Result: Falls back to PCM
|
||||
Bandwidth: 192000 × 2 × 3 = 1,152,000 bytes/s = 9.2 Mbps
|
||||
```
|
||||
|
||||
### After (With Resampling)
|
||||
```
|
||||
Source: 192kHz/24-bit stereo
|
||||
Client: Wants Opus
|
||||
Result: Resample to 48kHz → Opus encode
|
||||
Bandwidth: 256 kbps = 0.256 Mbps
|
||||
Savings: 36x reduction!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. ✅ Create `pkg/audio/output/malgo.go` (new file)
|
||||
2. ✅ Update `pkg/audio/output/output.go` (add bitDepth param)
|
||||
3. ✅ Update `pkg/audio/output/oto.go` (match interface)
|
||||
4. ✅ Update `pkg/resonate/player.go` (use malgo)
|
||||
5. ✅ Add malgo to `go.mod`
|
||||
6. ✅ Test 24-bit output with malgo
|
||||
7. ✅ Update `internal/server/audio_engine.go` (add resampler)
|
||||
8. ✅ Test Opus resampling with 192kHz source
|
||||
9. ✅ Update documentation
|
||||
|
||||
---
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
### Low Risk
|
||||
- malgo is well-tested, actively maintained
|
||||
- Resampler already exists and works
|
||||
- Changes are isolated to output layer
|
||||
|
||||
### Medium Risk
|
||||
- Callback model (malgo) vs blocking Write() (oto) requires ring buffer
|
||||
- Need to tune buffer size to avoid underruns
|
||||
|
||||
### Mitigation
|
||||
- Start with conservative buffer size (500ms)
|
||||
- Add comprehensive logging for debugging
|
||||
- Keep oto as fallback option (flag: `-backend oto`)
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] Player outputs true 24-bit audio (not downsampled to 16-bit)
|
||||
- [x] Opus clients with 192kHz sources get resampled audio (not PCM fallback)
|
||||
- [x] Bandwidth for Opus clients drops from 9.2 Mbps to ~0.26 Mbps
|
||||
- [x] Format switching works without restart
|
||||
- [x] No audio artifacts or underruns
|
||||
- [x] Tests pass for all formats (16/24-bit, 48/96/192 kHz)
|
||||
|
||||
---
|
||||
|
||||
## Next Steps (Phase 2)
|
||||
|
||||
After Phase 1 is complete:
|
||||
- Add configurable quality profiles (hi-res vs balanced vs low-bandwidth)
|
||||
- Optimize jitter buffer for hi-res rates
|
||||
- Add CPU/bandwidth monitoring
|
||||
- Create comprehensive test suite with real audio files
|
||||
Reference in New Issue
Block a user