Large File Transfers
The cluster focuses on challenges and solutions for uploading and downloading large files over networks, emphasizing slow upload speeds, bandwidth bottlenecks, and asymmetric internet connections.
Activity Over Time
Top Contributors
Keywords
Sample Comments
Downloading and uploading large files more quickly.
Is this actually transferring over LAN? So you could share a very large file on a slow connection?
my mistake, I swapped the 38tb and 112tb from parent commentwhatever the download size is, you're bottlenecked by the remote server's up speed
Their network file transfer is super slow
Steam downloads easily saturate my 1 Gbs. Same for S3 transfers.
Just try transfering it with sftp and see what kind of speed you get. Maybe use tar first to make a few big files, which will reduce overhead
Did you try enabling multi-threaded support? I was able to upload ~12 TB using 12-16 threads in a few months (and I'm based in EU as well).
(I need to quickly ship a 50mb file to 50,000 clients worldwide.)
Bandwidth [to transfer big binaries around] is not free however.
It took weeks to upload? Never underestimate the bandwidth of a station wagon full of tapes...