Ego-contingent trajectory prediction

"What will that car do if I brake?" The model takes a nearby road user's recent motion, the ego vehicle's recent motion, and a plan for what the ego vehicle does next, and predicts how the other road user moves in response. The same scene can be run under three plans: what the driver really did, braking, and accelerating. It was a course research project at RV University, trained on the nuScenes driving dataset and written up as an IEEE-format paper with six authors. I am a co-author.

A forecasting model usually learns what tends to happen next. This one is asked what would happen if the ego vehicle did something it did not do. There is no recording of how the other road user reacted to a brake that never happened, so a what-if forecast cannot be checked directly against data. What can be checked is the forecast under the real plan, and whether the forecast moves at all when the plan changes.

A conditional sequence-to-sequence model: three LSTM encoders, one each for the target's past, the ego vehicle's past and the ego plan, feeding one LSTM decoder. It has 796,418 parameters and is written in TensorFlow. A data pipeline extracts those three inputs from nuScenes: 18,942 rows from 10 scenes at first, then 1,200,336 rows from all 850 scenes, split by scene with the dataset's official 700 and 150. Around the model sit SHAP feature attributions for each plan, two simulators (about 12,500 lines of Python, one of which uses the trained model as a collision-risk predictor), a latency benchmark and a 12-test suite.

There are two sets of numbers, and they should not be mixed. The paper reports the first model, trained on 10 scenes: a test error of 3.27 × 10⁻⁴ in scaled units and an average displacement error of 0.515 m/s. That model came from a notebook that split its data at random by sample, which can leak between training and test. In May 2026 the model was retrained on all 850 scenes with the official scene split: validation error 8.86 × 10⁻⁵ and an average displacement error of 0.449 m/s. Those results are in the code, not yet in the paper. Both displacement errors are in velocity space, because the model predicts velocities, so they cannot be compared with the position errors in meters that the field usually quotes. One prediction takes 26.5 ms on average.

The paper received three peer reviews in May 2026, rated borderline, borderline and accept. The reviewers asked for displacement-error metrics, more datasets, stronger baselines and better figures. My own record gives the venue as IEEE CONECCT 2026; no acceptance notice or DOI is on file, so this site marks the venue as pending. The paper's methods section also describes things the code does not do, so this page describes the code and does not link the paper until the two agree. The plots here show how the forecast changes with the plan. In the sample shown, all three forecasts miss what the road user actually did.