Hello [email protected]

I intend to upload an archive of a website I scraped for the last few weeks. The torrent is about 4 TB large and contains 400000 folders which in turn contain 40 .jpg’s on average (plus some metadata).

Should I just create the torrent with the files as they are right now, or should I put the individual folders in archives (or maybe even the entire torrent?)?

Thanks in advance

  • cesium@sh.itjust.worksOPEnglish
    6·
    1 year ago

    Makes sense. What about individual folders? I’m concerned that the massive amount of small files could slow down the download quite a bit. Unfortunately, I’m not too familiar with the BitTorrent protocol, so I don’t know if this would have an impact.

    • zodo123@lemmy.worldEnglish
      14·
      1 year ago

      Small files don’t impact BitTorrent transfers. They’re sent in pieces of a fixed size, and those pieces can contain multiple files.

    • fkn@lemmy.worldEnglish
      2·
      1 year ago

      Depends on final size of the folder zips. 20-30mb? Sure. 200-300? Nah.