Enhancing MIT Computing Infrastructure » Task force on Mass Storage System upgrade

The Tier-2 computing center will need to move to a new and modern mass data storage model. The existing Tier-2 mass storage system, holding over 15 PB of data, will not be sustainable for the longer term future. A major upgrade is required. After careful considerations and testing it has become clear that the best solution is the CephFS file system. It is resource intensive but offers full POSIX compliance and is directly integrated into the linux kernel.

Mandate of the Task Force¶

The task force will deliver a fully CephFS based mass storage for subMIT, by September 1, 2024, and for the MIT Tier-2 center by April 1, 2025 without major service interruptions. Apart from building the new file system(s) it is essential that existing data on the legacy systems are transferred to allow for this seamless transition.

To accomplish this goal the task force leaders will first create a full schedule with relevant critical milestones and approximate cost estimates for new hardware required. The substantially smaller subMIT computing system will serve as a test case. This small CephFS setup (0.5 PB) will replace the existing gluster file system. New hardware will be needed and the transition should be accomplished with minimal service interruptions of subMIT.

The general infrastructure installation like rack preparation, networking and such can be going on in parallel.