Downfeather: A Machine-Learning-Driven Adaptive Multi-Connection Download Manager

Abstract: This paper presents Downfeather, a download manager that keeps learning while a download runs. A file is split into HTTP Range segments and fetched over several connections, and the manager tunes how much each connection reads per call using only the speeds it measures. Development took four iterations. Six recorded traces were clustered with K-Means and mined with Apriori to give a first speed governor; because that governor rested on little data and could only slow a connection down, an online gradient “brain” replaced it. Later versions added exploration noise and finally a saved memory of how speed changed after each of the brain’s own moves. A real download of PixelMaster.exe (251.23 MB) over two connections finished in 43.1 s at 46.64 Mbps, with nearly equal shares (51.5% and 48.5%). A small online controller written with the Python standard library and NumPy can therefore cope with a noisy link without a pre-trained model.

Keywords: Adaptive Download Manager, Multi-Connection Download, Online Learning, Gradient Descent, K-Means, Association Rule Mining, Work Stealing, HTTP Range Requests