ASP.NET tip: perform a large number of write operations on multiple files at the same time to optimize performance

One of my own projects required multiple writes to 65,536 files at the same time.

If you open all the files, then repeat, and finally close all the files. It will take about 16 minutes for the first write to complete, and 40 minutes for the second. No more testing.

                      for (int i = 0; i < 65536; i++)
                      {
                              fileStream[i] = new FileStream(buffDir+"\\"+ i.ToString() + ".dat", FileMode.Create,FileAccess.Write, FileShare.Write,14000);
                      }
                        write;
                              write;
                              write;
                              ........
                            for (int i = 0; i < 65536; i++)
                      {
                              fileStream[i] .close();
                      }

If you open only one file while writing, close after writing. So it only takes about 2 minutes and 30 seconds for all the writes to complete.

Loop

{
    for (int i=0;i < 65536;i++)
{
  open;
  write;
  close;
}
}

Thus, the second method is much more powerful than the first. Opening all the files at once takes up a large amount of memory, and the most important thing is that when net processes filestream, it may need to allocate and recycle a lot of memory, which consumes a lot of memory and resources.

I also tested that if the number of files is small, the performance of the first type is much better than that of the second type.