Project

General

Profile

Data rushing in for the HTo4L analysis (Chad)

Copying order 1000 files to /data/submit/cms/store with ~100 processes in parallel.
Added by Christoph Paus over 1 year ago

Chad asked for the Higgs to 4 lepton analysis for the Junior Lab Open data project to have samples copied to submit. I use this as a test for seeing the bandwidth we can copy at. FiBS* is processing 10 downloads in parallel on each of the submit50-59 servers (they carry the gluster drives). This means 100 parallel processes.

Performance is pretty good, little spikes. I think we are hitting the bottleneck.

$ du -sh /data/submit/cms/store/

897G /data/submit/cms/store/
Download all files
gluster-copy-hto4l.png (18.4 KB) gluster-copy-hto4l.png Christoph Paus, 08/05/2022 10:00 PM
gluster-copy-hto4l-done.png (21.6 KB) gluster-copy-hto4l-done.png Christoph Paus, 08/05/2022 10:15 PM

Comments