Investigating Feature Substitution in Algorithmic Worker-Scoring Models

Abstract

Algorithmic management systems increasingly use operational data to predict, rank, allocate, and evaluate work. This paper investigates whether worker-relevant predictive capacity persists after explicit worker-identifying and monitoring features are removed from algorithmic scoring models. Using a public warehouse task-execution dataset, we train classification models to predict whether tasks fall above the median observed duration, a worker-relevant operational outcome that may inform allocation, performance management, or staffing decisions. We compare model performance across full-feature, direct-monitoring-excluded, operational/proxy-only, and stricter proxy-ablation settings. The results show that removing explicit worker-identifying fields only modestly reduces predictive performance and the direct-monitoring-excluded model retains most of the full model's ROC-AUC, while operational proxy models remain meaningfully predictive even after removing workzone and priority variables. These findings suggest that feature removal may reduce formal observability without eliminating worker-relevant scoring capacity, because operational context variables can preserve predictive signal. We contribute a sociotechnical framework for analysing feature substitution in algorithmic management and argue that audits should examine retained predictive capacity, not merely the presence of direct monitoring features.