# SnoopX - HDR Screen Magnifier

A macOS screen magnification tool with true HDR (High Dynamic Range) support using ScreenCaptureKit and Metal rendering.

## Features

- **HDR Capture & Display**: Preserves pixel values > 1.0 from HDR content
- **Metal-based Rendering**: GPU-accelerated display with MTKView
- **Real-time Magnification**: Configurable zoom levels (1x, 2x, 4x, 8x)
- **Color Analysis**: Display RGB values and statistics for selected regions
- **Graph Modes**: Horizontal/vertical pixel value plots (step or linear interpolation)
- **Display Mode Control**: Auto-detect, force SDR, or force HDR
- **Independent Statistics**: Toggle statistics display separately from graph plotting

## Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                     HDR CAPTURE & DISPLAY                       │
└─────────────────────────────────────────────────────────────────┘

1. CAPTURE (Snooper.m)
   ScreenCaptureKit → CVPixelBuffer (Float16 RGBA)
   ↓
   kCVPixelFormatType_64RGBAHalf (8 bytes/pixel)
   kCGColorSpaceExtendedLinearDisplayP3
   Values: 0.0 - 5.0+ (Extended Dynamic Range)

2. PROCESSING (Snooper.m)
   Float16 buffer → Magnified CGImage
   ↓
   Statistics computation (min/max/avg/median)
   Graph rendering (optional)
   Cursor color sampling

3. DISPLAY (HDRImageView.m + Controller.m)
   NSImage → Metal Texture (RGBA16Float)
   ↓
   MTKView with EDR-enabled CAMetalLayer
   No-clamping fragment shader (preserves > 1.0)
   Direct GPU rendering to screen
```

## Key Files

### Core Implementation
- **HDRImageView.h/m** (410 lines): MTKView-based HDR display view
  - NSImageView-compatible API
  - Float32 → Float16 texture conversion
  - Metal rendering pipeline
  - EDR-enabled display layer

- **Snooper.h/m** (3,700+ lines): Screen capture and image processing
  - ScreenCaptureKit integration (macOS 12.3+)
  - HDR pixel buffer handling
  - Magnification and color sampling
  - Graph and statistics rendering

- **Controller.h/m** (555 lines): Application controller
  - Window management
  - User preference handling
  - View updates and timer control
  - Display mode switching

### Interface
- **English.lproj/MainMenu.xib**: Application UI and menu structure

## Build Requirements

### Xcode Settings
- **Minimum Deployment Target**: macOS 12.3 (for ScreenCaptureKit)
- **Frameworks Required**:
  - Cocoa.framework
  - ScreenCaptureKit.framework
  - VideoToolbox.framework
  - Metal.framework
  - MetalKit.framework

### Build Settings
1. Open project in Xcode
2. Select "SnoopX" target
3. Build Settings → "macOS Deployment Target" → `12.3` or higher
4. Build (Cmd+B)

## Usage

### Basic Operation
1. Launch SnoopX
2. The magnified view follows your cursor
3. Status bar shows coordinates and RGB values
4. HDR mode activates automatically when viewing HDR content

### Keyboard Shortcuts
- **Arrow Keys**: Move cursor by 1 pixel (Shift = 10 pixels)
- **i**: Toggle statistics display
- **o**: Cycle through graph modes (Off → H-Step → H-Linear → V-Step → V-Linear)
- **Shift-H**: Horizontal step graph (no line connectors)
- **Shift-V**: Vertical step graph (no line connectors)

### Menu Options
- **Zoom**: 1x, 2x, 4x, 8x magnification
- **Update Timer**: None, 50ms, 100ms, 200ms, 500ms, 1000ms refresh
- **Display Mode**: Auto, Force SDR, Force HDR
- **Always on Top**: Keep window above other applications
- **Lock Position**: Freeze cursor position
- **Highlight**: Show cursor indicator in graph plots

### Display Modes
- **Auto** (default): HDR when available, SDR otherwise
- **Force SDR**: Always use 8-bit color (0-255 range)
- **Force HDR**: Always use Float16 capture (0.0-5.0+ range)

### Reading RGB Values

**SDR Mode:**
```
(01234, 05678) - (255, 128, 064)
```
Values shown as 0-255

**HDR Mode:**
```
(01234, 05678) - HDR (1.523, 0.856, 0.342)
```
- All channels in [0.0, 1.0]: Shows as 0-255
- Any channel outside [0.0, 1.0]: Shows as floating point (f-stops)

## HDR Limitations

### YCbCr 8-bit Format
Apple's default HDR preset uses **8-bit YCbCr** video format which has inherent range limitations:

- **Maximum value**: ~1.09 (not the 5.0+ theoretical limit)
- **Typical HDR peaks**: 0.94 - 1.09
- **Why**: Video range (16-235) + BT.709 color conversion math

This is **expected and correct behavior**, not a bug. The format can only represent ~1 f-stop above SDR white.

### Getting Higher HDR Values
To capture values > 1.1, you would need:
- 10-bit or 16-bit pixel formats (not in Apple's preset)
- Manual ScreenCaptureKit configuration (no presets)
- Different color space handling

The current implementation correctly handles whatever values the system provides.

## Statistics Display

**SDR Statistics:**
```
min=(  0,  0,  0), max=(255,255,255)
avg=(128,128,128), med=(127,127,127)
```

**HDR Statistics:**
```
min=(0.000,0.000,0.000), max=(1.234,1.156,1.089)
avg=(0.567,0.543,0.512), med=(0.523,0.501,0.478)
```

Statistics are computed from the magnified region:
- **min/max**: Minimum and maximum values per channel
- **avg**: Mean value per channel
- **med**: Median value per channel (sorted middle value)

Statistics and graph plotting are independent:
- Press **i** to toggle statistics
- Press **o** to cycle graph modes
- Both can be enabled simultaneously

## Color Spaces

The application uses different color spaces depending on content:

**HDR Path:**
- Capture: `kCGColorSpaceExtendedLinearDisplayP3`
- Processing: Float16 (IEEE 754 half-float)
- Display: `kCGColorSpaceExtendedLinearDisplayP3`
- Metal Layer: RGBA16Float with EDR enabled

**SDR Path:**
- Capture: sRGB or device color space
- Processing: 8-bit RGBA
- Display: sRGB
- No extended dynamic range

## Troubleshooting

### No HDR values appearing
- Check System Settings → Displays → HDR brightness is enabled
- Verify you're viewing actual HDR content (Photos app, Apple TV+)
- Remember: 8-bit YCbCr format limits values to ~1.09

### Black screen
- Verify Metal is supported on your Mac
- Check Console.app for error messages
- Try toggling Display Mode menu

### Values capped at 1.0
- This is normal for most HDR content in 8-bit YCbCr
- Values of 0.94-1.09 indicate HDR is working correctly
- See "HDR Limitations" section above

### Build errors
- Verify all frameworks are linked (Metal, MetalKit, ScreenCaptureKit)
- Check macOS deployment target is set to 12.3 or higher
- Ensure Xcode includes macOS 12.3+ SDK

### Performance issues
- Reduce update timer frequency (100ms or 200ms recommended)
- Lower zoom level (1x or 2x)
- Disable statistics when not needed

## Technical Details

### HDR Capture Pipeline
1. **ScreenCaptureKit** captures display as Float16 pixel buffer
2. **CVPixelBuffer** provides direct access to HDR pixels
3. **Float16 → Float32** conversion for processing
4. **Magnification** by pixel duplication (nearest neighbor)
5. **Graph/Statistics** rendered onto magnified image
6. **Float32 → Float16** conversion for Metal texture
7. **MTKView** renders with EDR-enabled CAMetalLayer

### Metal Shaders
```metal
// Fragment shader - preserves HDR values
fragment half4 fragmentShader(VertexOut in [[stage_in]],
                              texture2d<half> texture [[texture(0)]]) {
    constexpr sampler textureSampler(mag_filter::nearest,
                                     min_filter::nearest);
    return texture.sample(textureSampler, in.texCoord);
    // NO clamp() or saturate() - preserves values > 1.0
}
```

### Float16 Conversion
Uses IEEE 754 binary16 format:
- Range: 0.0 to 65504.0
- Precision: ~3 decimal digits
- HDR range (0.0-2.0): No precision loss
- Values > 1.0 preserved through entire pipeline

## Development Notes

### Code Cleanup
The codebase has been cleaned of development debug logging. Remaining logs:
- **Errors** (❌): Critical failures
- **Warnings** (⚠️): Important issues
- **Success** (✅): Initialization confirmations

### Recent Changes
- Independent statistics mode (separate from graph plotting)
- Removed problematic zoom levels (kept only 1x, 2x, 4x, 8x)
- Fixed HDR statistics computation from Float16 buffers
- Color space corrections (ExtendedLinearDisplayP3 throughout)
- Improved cursor pixel alignment on Retina displays

## Known Issues

### Red Square Sizing (#11)
The red cursor indicator square sometimes corresponds to a single pixel, but other times covers multiple pixels depending on display scale factors and magnification level. This is a known issue to be investigated.

### Color Profile EDR Limits
Some color profiles limit EDR headroom to 2.3-2.4 f-stops even though the hardware supports more. This limitation is not exposed by macOS APIs and cannot be detected programmatically.

## License

Copyright 2008 NTU CSIE CMLAB. All rights reserved.

## Credits

Original SnoopX implementation by tzhuan (2008)
HDR enhancements and modernization (2024-2025)
