Support for concatenating multiple files

[ikreymer]

It would be great if there was documentation / examples, or even a utility function for concatenating existing file links, to make a new file. After debugging, I was able to come up with something that works, but not sure if this is the 'right' way to do it, and doesn't using the task queue...

The writer is the top-level writer from UnixFS.createWriter() and links is an array of links returned after multiple fileWriter.close() calls.
The below produces a concatenated file, using createAdvancedFile.

async function concat(writer, links) {
  const {fileEncoder, hasher, linker} = writer.settings;
  const advanced = fileEncoder.createAdvancedFile(links);
  const bytes = fileEncoder.encode(advanced);
  const hash = await hasher.digest(bytes);
  const cid = linker.createLink(fileEncoder.code, hash);
  const block = { bytes, cid };
  
  // correct to access underlying writer like this?
  writer.writer.write(block);

  const link = {
    cid,
    contentByteLength: fileEncoder.cumulativeContentByteLength(links),
    dagByteLength: fileEncoder.cumulativeDagByteLength(bytes, links),
  };

  return link;
}

[alanshaw]

Wow this is an interesting idea. I think you won't end up with a balanced tree this way and will probably add more intermediate blocks than are necessary.

I like the idea but I'm not sure if this library would be able to support this, since it would require knowledge and access to existing blocks.