How We Test Wireless Headphones

The Falcon System, our testing methodology, combines objective testing with our expert reviewers’ subjective analyses to evaluate wireless headphones.

We use 27 scoring criteria with different weights to ensure a fair and insightful wireless headphones review. This enables us to calculate two distinct scores — a General Score and an Audiophile Score.

Two Overall Score

Our review process culminates in two definitive scores:

• General Score: Calculated from categories that are important for general usage.
• Audiophile Score: Calculated from categories that are important for an audiophile

Score Category

There are 2 main categories that we are evaluating for wireless headphones. The headphones category is further subdivided into 7 sub-categories.

Score Weightage

Since the need for general usage differs from audiophile usage, there are different weights across the score category.

For General Usage:

For Audiophile Usage:

Grading Scale

We are using a 5-point grading scale for each score criterion, where a score of 5 indicates outstanding performance, and 1 signifies very poor performance.

• 5 – Outstanding
• 4 – Good
• 3 – Average
• 2 – Substandard
• 1 – Very Poor

Battery

Our evaluation of the ‘Battery’ score focuses on two primary aspects:

1. Battery Life
2. Playback Time (from a 15-minute charge)

Battery life is the most crucial factor, directly affecting user convenience. Additionally, the efficiency of rapid charging forms an integral part of our assessment of battery performance.

Battery Life

• Weightage: 95%
• Grading Criteria:
  • >40 hours: Outstanding
  • >30-40 hours: Good
  • >20-30 hours: Average
  • 10-20 hours: Substandard
  • <10 hours: Very Poor

Playback Time (from a 15 minute charge)

• Weightage: 5%
• Grading Criteria:
  • >2 hours: Outstanding
  • 1.6-2 hours: Good
  • 1.1-1.5 hours: Average
  • 0.6-1.0 hours: Substandard
  • <=0.5 hours: Very Poor

Mic

Our ‘Mic’ evaluation focuses on:

1. Noise Pickup
2. Sound Quality

We assess the mic’s ability to clearly capture voice amidst background noise and its overall sound quality. Both factors are crucial for clear and reliable voice communication.

Noise Pickup

• Weightage: 50%
• Grading Criteria:
  • No noise: Outstanding
  • Soft noise: Good
  • Audible: Average
  • Loud noise: Substandard
  • Much too noisy: Very Poor

Sound Quality

• Weightage: 50%
• Grading Criteria:
  • Sounds like you are speaking directly: Outstanding
  • Clear and no breakup: Good
  • Muffled: Average
  • Some breakup: Substandard
  • Can barely hear: Very Poor

Sound

The ‘Sound’ category assessment encompasses several key aspects of audio performance:

1. Noise Leakage
2. Noise Isolation
3. Transparency Mode
4. Bass, Mids, and Treble Quality
5. Soundstage
6. Imaging
7. Dynamics

This evaluation focuses on the headphones’ ability to deliver clear, balanced, and immersive audio experiences. From minimizing external ambient sound to providing rich and detailed audio across various frequencies, each aspect contributes to the overall sound quality perceived by the user.

Noise Leakage

• Weightage: 5%
• Grading Criteria:
  • Very Poor: Can hear everything clearly
  • Substandard: Can make out most of the sound
  • Average: Can make out some of the sound
  • Good: Can hear something but cannot make out the lyrics
  • Outstanding: Cannot hear a thing

Noise Isolation

• Weightage: 5%
• Grading Criteria:
  • Very Poor: Doesn’t exist
  • Substandard: Moderate filtration of both low and high frequency noise
  • Average: Excellent filtering of low freq. noise, moderate filter of high freq. noise
  • Good: Excellent filtering of both low and high freq. noise
  • Outstanding: Granular controls

Transparency Mode

• Weightage: 5%
• Grading Criteria:
  • Very Poor: Doesn’t exist
  • Substandard: Can only enable/disable
  • Average: Average
  • Good: Good
  • Outstanding: Outstanding

Bass [Sub-bass, Bass Impact, Bass Tightness]

• Weightage: 25%
• Grading Criteria:
  • Very Poor
  • Substandard
  • Average
  • Good
  • Outstanding
• Note: Take an average of the three sub-components for overall grading.

Mids, Treble

• Weightage: 25%
• Grading Criteria:
  • Very Poor
  • Substandard
  • Average
  • Good
  • Outstanding

Soundstage, Imaging, Dynamics

• Weightage: 5%
• Grading Criteria:
  • Very Poor
  • Substandard
  • Average
  • Good
  • Outstanding

Connectivity

The ‘Connectivity’ category assesses the wireless performance of headphones, focusing on:

1. Codec Support
2. Ease of Pairing and Multi-Point Connections
3. Stability and Range

This category evaluates the range of audio codecs supported, the ease and reliability of connecting to devices, and the stability of the connection at various distances, ensuring a seamless and uninterrupted listening experience.

Codec Support

• Weightage: 30%
• Grading Criteria:
  • Very Poor: SBC
  • Substandard: SBC AAC
  • Average: SBC AAC aptX
  • Good: LDAC etc
  • Outstanding: No configuration needed

Ease of Pairing + Multi-Point

• Weightage: 20%
• Grading Criteria:
  • Very Poor: Frequently unstable
  • Substandard: Slightly unstable
  • Average: Stable
  • Good: No configuration needed
  • Outstanding: Pair to new device; Pair to paired device; Change pairing to another paired device from connected state
• Note: Get average score.

Stability + Range (Connection Dropouts)

• Weightage: 50%
• Grading Criteria:
  • Very Poor: >10 dropouts/hour
  • Substandard: >8-10 dropouts/hour
  • Average: 6-8 dropouts/hour
  • Good: 3-5 dropouts/hour
  • Outstanding: 0-2 dropouts/hour
• Note: Assessment per 1 hour.

Waterproof

The ‘Waterproof’ category evaluates the headphones’ resistance to water and moisture, based solely on the:

1. IPX Rating

This rating determines the headphones’ suitability for different environments and activities, particularly those involving exposure to water, thereby affecting their versatility and durability.

IPX Rating

• Weightage: 100%
• Grading Criteria:
  • Very Poor: IPX1 or Not Rated
  • Substandard: IPX2
  • Average: IPX3
  • Good: IPX4/5
  • Outstanding: >=IPX6

Design

In the ‘Design’ category, we consider the physical aspects of the headphones, focusing on:

1. Fit and Comfort
2. Build Quality
3. Aesthetic Appeal

This evaluation encompasses the comfort level for extended use, the durability and quality of materials used, and the overall visual and tactile appeal of the headphones.

Fit and Comfort (Hours of Comfortable Wearing)

• Weightage: 50%
• Grading Criteria:
  • Very Poor: Comfortably wear less than 1 hour
  • Substandard: Comfortably wear more than 1 hour
  • Average: Comfortably wear more than 2 hours
  • Good: Comfortably wear more than 3 hours
  • Outstanding: Comfortably wear more than 4 hours

Build Quality

• Weightage: 25%
• Grading Criteria:
  • Very Poor: Fragile and cheap feeling
  • Substandard: Plastic, average feel
  • Average: Less than average
  • Good: Above average
  • Outstanding: Premium feeling and robust

Aesthetic

• Weightage: 25%
• Grading Criteria:
  • Very Poor: Ugly, unattractive
  • Substandard: Normal, average
  • Average: Less than average
  • Good: Above average
  • Outstanding: Fresh, attractive, innovative, premium

UX

The ‘UX’ category is centered around user interaction with the headphones, examining:

1. Control Mechanism
2. Control Reliability

This category measures the intuitiveness and reliability of the controls, impacting how users interact with and manage their listening experience.

Control Mechanism

• Weightage: 50%
• Grading Criteria:
  • Very Poor: None
  • Substandard: Touch or Mechanical only
  • Average: Often Inconsistent
  • Good: Mostly Reliable
  • Outstanding: Both touch and Mechanical

Control Reliability

• Weightage: 50%
• Grading Criteria:
  • Very Poor: Unreliable
  • Substandard: Often Inconsistent
  • Average: Mostly Reliable
  • Good: Very Reliable
  • Outstanding: No reported issues or failures

Software

Our evaluation in the ‘Software’ category involves assessing the headphones’ software features, including:

1. EQ (Equalizer)
2. Battery Level Monitoring
3. ‘Find My Headphones’ Feature
4. Integration with Voice Assistants
5. Auto-Pause Functionality

These software elements enhance user convenience and personalization, contributing significantly to the overall user experience. If there’s no software support, the score will be 0.

EQ (Equalizer)

• Weightage: 20%
• Grading Criteria:
  • Very Poor: None
  • Substandard: Presets
  • Average: <=6-band equalizer
  • Good: 6-10-band (or more) equalizer with DSP controls
  • Outstanding: Parametric EQ support

See Battery Level

• Weightage: 20%
• Grading Criteria:
  • Very Poor: None
  • Substandard: Yes (general)
  • Average: Yes (detailed battery life estimation)
  • Good: Yes (detailed with percentage)
  • Outstanding: Real-time monitoring with notifications

Find My Headphones

• Weightage: 20%
• Grading Criteria:
  • Very Poor: None
  • Substandard: Basic (sound alert within app)
  • Average: Advanced (sound alert + last known location)
  • Good: GPS tracking
  • Outstanding: GPS tracking with real-time updates

Voice Assistant

• Weightage: 20%
• Grading Criteria:
  • Very Poor: None
  • Substandard: Basic commands
  • Average: Full integration with mobile voice assistants
  • Good: Custom voice assistant
  • Outstanding: Advanced custom voice assistant with unique features

Auto-Pause

• Weightage: 20%
• Grading Criteria:
  • Very Poor: None
  • Substandard: Basic (pause when removed)
  • Average: Advanced (pause when removed, play when worn)
  • Good: Context aware (auto-pause based on environmental factors)
  • Outstanding: Adaptive (learns and adjusts to user behavior)

Suggestions and Feedbacks

If you spot any errors with our testing methodology or have any good feedback on how we can further improve the test, please feel free to contact us.

Other Testing Methodologies

• Sparrow System: How We Test True Wireless Headphones (TWS)
• Medusa System: How We Test Wired Headphones and IEMs
• Oracle Scoring: How We Test General Audio Gear