Insights on Data preprocessing and Model design

Improving the real-time smartphone positioning accuracy by leveraging deep learning techniques, particularly a Transformer-based GNSS Positioning Correction Network. Here, I want to talk about the data preprocessing and Model design aspect of the project with some insights.

1. Weighted Least Squares (WLS) Recap

The Weighted Least Squares method (WLS) is a common baseline for GNSS positioning. It solves an overdetermined system formed by the pseudorange equations from multiple satellites.

Figure 1: Four Satellites Intersecting Spheres at a Single Point. Source: geoterrace.lpnu.ua

Basic GNSS Equation (display math):

where:

• \(P_i\) is the pseudorange from satellite \(i\),
• \((x_{\text{sat},i}, y_{\text{sat},i}, z_{\text{sat},i})\) are the satellite’s ECEF coordinates,
• \((x_{\text{rec}}, y_{\text{rec}}, z_{\text{rec}})\) is the unknown receiver position,
• \(c \cdot \Delta t_{\text{rec}}\) is the clock bias term,
• \(\varepsilon_i\) is measurement noise.

You often have more satellites than the four unknowns \(\{x_{\text{rec}}, y_{\text{rec}}, z_{\text{rec}}, \Delta t_{\text{rec}}\}\), so WLS linearizes and iteratively solves:

Linearized WLS Update (display math):

where

• \(\mathbf{z}\) stacks the measured pseudoranges \((P_i)\),
• \(\mathbf{r}(\mathbf{x}_0)\) is the modeled pseudorange from the current guess,
• \(\mathbf{H}\) is the Jacobian (partial derivatives),
• \(\mathbf{W}\) is a diagonal weight matrix,
• \(\Delta \mathbf{x}\) updates \(\{x_{\text{rec}}, y_{\text{rec}}, z_{\text{rec}}, \Delta t_{\text{rec}}\}\).

After a few iterations, WLS converges to \(\hat{\mathbf{x}}\). This WLS solution is our baseline, refined further by deep learning.

2. Motivation for Deep Learning in GNSS Positioning

Urban environments wreak havoc on GNSS signals due to multipath and Non-Line-of-Sight (NLOS) effects. Traditional WLS-based methods aren’t always robust enough in these conditions. Deep learning, however, can capture non-linear relationships and subtle patterns in the data, potentially delivering more accurate positioning.

Figure 2: Multipath and NLOS. Source: MDPI Sensors Journal

Why Not Direct Regression on Coordinates?

1. Huge numeric ranges: Raw pseudorange values can be on the order of \(10^5\) meters, risking unstable gradients.
2. Permutation Invariance: The order of satellite observations in an epoch shouldn’t affect final positioning—but typical neural networks treat sequential inputs as ordered.

We needed a strategy to tackle these challenges before feeding data into a Transformer-based model.

3. Data Preprocessing

Figure 3: Padding of Satellite Constellation Vectors to Uniform Length

3.1 Rethinking “Data Filtering”

Rather than discarding “noisy” measurements (like low-elevation signals), we kept them all. Our philosophy: noise can encode environmental clues (e.g., multipath). The attention mechanism in our Transformer can learn to downweight poor signals automatically instead of relying on manual filtering heuristics.

3.2 Order Invariance: Sorting & Padding

GNSS data for each epoch arrives as a set of satellite measurements. But neural networks often see them as sequences. We overcame this by:

• Sorting satellites first by constellation ID, then by elevation angle.
• Padding each sorted list to a fixed length (the max satellite count).

This ensures consistent input shapes and removes the need for specialized permutation-invariant architectures.

3.3 Converting Regression to Classification

Directly predicting continuous coordinates from raw data can be tricky. Instead, we transform the coordinate error (compared to WLS) into discrete classes (e.g., intervals from -25m to +25m).

Figure 4: Error Distribution Statistics of All Data

The transformation function ( f(e) ) is defined as:

Here, \(\text{sign}(e)\) preserves the direction of the error \(e\), ensuring that positive and negative errors are distinguished.

Fourier-Like Note: We discretize the positioning error into multiple “levels,” somewhat analogous to decomposing signals in a Fourier transform.

After classification, we use a small MSE loss to refine the final offsets.

4. Model Design: A “BERT-Like” Transformer

4.1 Algebraic Inspiration

WLS solves an overdetermined system by blending multiple equations. Transformers attend to multiple “inputs” (words, in NLP). In GNSS, each satellite measurement is analogous to an “equation.” Our Transformer learns relationships among satellites, similar to finding the best geometric fit in WLS.

4.2 BERT-Like Encoder

We adapt a BERT-style Transformer, commonly used for text classification (like IMDB review analysis), to GNSS error classification:

• Input Encoding: Sort & pad GNSS features, then embed them in a higher-dimensional space.
• Multi-Head Self-Attention: Identifies which satellites or features are most reliable.
• Classification Heads: Output discrete error classes for \(\Delta x, \Delta y, \Delta z\), from which we compute a final correction offset.

4.3 Hybrid Loss

Hybrid Loss (display math):

where \(\mathcal{L}_{\text{CE}}\) is cross-entropy for coarse error intervals, and \(\mathcal{L}_{\text{MSE}}\) refines the final positioning. Balancing these losses helps stabilize training while still achieving fine-grained accuracy.

5. Some Results

5.1 Cumulative Distribution Functions

Blue : WLS (baseline)
Green : Neural Network

Figure 5: Cumulative Distribution Functions of Horizontal and Vertical Positioning Errors on Test Sets

5.2 Comparison of Positioning Error Metrics (MSE, Mean, 50th Percentile, and 95th Percentile) between WLS and NN Models on the Test Set

5.3 Comparison of Trajectories Estimated by WLS and NN Models with Ground Truth (GT) for Horizontal and Vertical Positioning

Blue : WLS (baseline)
Green : Neural Network
Red : Ground Truth

Figure 6: Comparison of WLS, Neural Network, and True Coordinates in the Vertical Direction

Figure 7: Comparison of WLS, Neural Network, and True Coordinates in the Horizontal Direction

Conclusion

Due to the challenges posed by multipath effects and Non-Line-of-Sight (NLOS) errors, which traditional mathematical methods cannot accurately model, deep learning provides a promising alternative. The system effectively transforms the difficult-to-converge regression task into a classification problem that categorizes positioning errors into discrete levels by leveraging data preprocessing techniques and a classification-based model design.

Compared to the Weighted Least Squares (WLS) baseline, the proposed deep learning-based approach significantly improves positioning accuracy, particularly in challenging environments. This highlights the potential of deep learning to overcome the limitations of traditional GNSS positioning methods, offering a more robust and accurate solution for real-time smartphone positioning.

After Thoughts

While the final results demonstrate overall improvement, some insights were not individually validated due to time constraints. Additionally, because the loss was aggregated and averaged across the dataset, there are still traces where the accuracy is worse than the baseline. This highlights the importance of considering data distribution differences.

In future work, it will be necessary to analyze and categorize different scenarios (e.g., urban canyons, open fields, mixed environments) to account for their unique characteristics. By separating and tailoring the model’s training or evaluation for distinct data segments, we can further refine positioning accuracy and improve the model’s robustness across all scenarios.