ASP.NET How to upload files and handle large file upload in core MVC

Binding interface with file model: IFormFile (small file upload)

When you use the IFormFile interface to upload files, be sure to note that IFormFile will read all files in an Http request to the server memory before triggering ASP.NET The Action method in the Controller of the core MVC. In this case, it's OK to upload some small files, but if you upload large files, it will cause a large amount of server memory to be occupied or even overflowed, so IFormFile interface is only suitable for small file upload.

 

The Html code of a file upload page is generally as follows:

<form method="post" enctype="multipart/form-data" action="/Upload">
    <div>
        <p>Upload one or more files using this form:</p>
        <input type="file" name="files" />
    </div>
    <div>
         <input type="submit" value="Upload" />
    </div>
</form>

In order to support file upload, make sure to declare the attribute enctype = "multipart / form data" on the form tag, otherwise you will find ASP.NET No files can be read in the Controller of core MVC. The Input type="file" tag supports uploading multiple files in html 5, and the attribute multiple can be added.

 

Using IFormFile interface to upload a file is very simple. You can declare it as a set parameter of Action in contour:

[HttpPost]
public async Task<IActionResult> Post(List<IFormFile> files)
{
    long size = files.Sum(f => f.Length);

    foreach (var formFile in files)
    {
        var filePath = @"F:\UploadingFiles\" + formFile.FileName.Substring(formFile.FileName.LastIndexOf("\\") + 1);//be careful formFile.FileName Contains the file path of the uploaded file, so the Substring Remove only the last filename

        if (formFile.Length > 0)
        {
            using (var stream = new FileStream(filePath, FileMode.Create))
            {
                await formFile.CopyToAsync(stream);
            }
        }
    }

    return Ok(new { count = files.Count, size });
}

Note that the parameter name files of the Action method Post above must be the same as the name attribute value of the Input type="file" tag in the upload page.

 

Don't use it directly Request.Form.Files

The above example is when we know the name attribute value of the Input type="file" tag. If you don't know the name attribute value of the Input type="file" tag (for example, the Input type="file" tag dynamically generated by javascript in the front end), is there any way to get all the uploaded files?

Maybe some students will think it can be used Request.Form.Files To get all the uploaded files in the current Http request, as follows:

[HttpPost]
public async Task<IActionResult> Post()
{
    IFormFileCollection files = Request.Form.Files;
    long size = files.Sum(f => f.Length);

    foreach (var formFile in files)
    {
        var filePath = @"F:\UploadingFiles\" + formFile.FileName.Substring(formFile.FileName.LastIndexOf("\\") + 1);

        if (formFile.Length > 0)
        {
            using (var stream = new FileStream(filePath, FileMode.Create))
            {
                await formFile.CopyToAsync(stream);
            }
        }
    }

    return Ok(new { count = files.Count, size });
}

Then execute the above code and you will find that the code is executed to Request.Form.Files It stuck all the time, as follows:

Now we can add a parameter string parameter to the Post method as follows:

[HttpPost]
public async Task<IActionResult> Post(string parameter)
{
    IFormFileCollection files = Request.Form.Files;
    long size = files.Sum(f => f.Length);

    foreach (var formFile in files)
    {
        var filePath = @"F:\UploadingFiles\" + formFile.FileName.Substring(formFile.FileName.LastIndexOf("\\") + 1);

        if (formFile.Length > 0)
        {
            using (var stream = new FileStream(filePath, FileMode.Create))
            {
                await formFile.CopyToAsync(stream);
            }
        }
    }

    return Ok(new { count = files.Count, size });
}

Running the code, you will find that although the string parameter of the Post method does not get any value of null, this time the Post method is not stuck in the Request.Form.Files , file upload succeeded, Post method successfully completed:

This is because when ASP.NET When the Action method of the Controller in the core MVC does not define parameters, Request.Form It doesn't do data binding, that is, when we don't define parameters in the Post method above, Request.Form Not at all ASP.NET Core initialization, so only one access Request.Form The code will get stuck, so when we casually define a string parameter parameter to the Post method, Request.Form It is initialized, and then it can be accessed Request.Form Data in.

Since we have to define parameters for the Post method, we should define meaningful parameters instead of defining a useless one. We change the code of the Post method to the following:

[HttpPost]
public async Task<IActionResult> Post([FromForm]IFormCollection formData)
{
    IFormFileCollection files = formData.Files;//Equivalent to Request.Form.Files

    long size = files.Sum(f => f.Length);

    foreach (var formFile in files)
    {
        var inputName = formFile.Name;//Through IFormFile.Name Property to get the ownership of each uploaded file on the page Input type="file"Tagged name Property value
        var filePath = @"F:\UploadingFiles\" + formFile.FileName.Substring(formFile.FileName.LastIndexOf("\\") + 1);

        if (formFile.Length > 0)
        {
            using (var stream = new FileStream(filePath, FileMode.Create))
            {
                await formFile.CopyToAsync(stream);
            }
        }
    }

    return Ok(new { count = files.Count, size });
}

We define an IFormCollection type parameter formData for the Post method, and mark the [FromForm] attribute label, indicating that the IFormCollection formData parameter is initialized with the form data in the Http request, so the formData is equivalent to Request.Form Yes.

We can access any data submitted by the Form from the Form data and get all the uploaded files. In fact, it doesn't matter what the parameter name of the Post method is (in this case, we call it formData), but its parameter must be IFormCollection type to bind the Form data in the Http request, which is the key.

Execute the above code, the file is uploaded successfully, and the code is executed successfully:

 

 

File stream (large file upload)

Before introducing this method, let's take a look at what an Http request containing an uploaded file looks like:

Content-Type=multipart/form-data; boundary=---------------------------99614912995
-----------------------------99614912995
Content-Disposition: form-data; name="SOMENAME"

Formulaire de Quota
-----------------------------99614912995
Content-Disposition: form-data; name="OTHERNAME"

SOMEDATA
-----------------------------99614912995
Content-Disposition: form-data; name="files"; filename="Misc 001.jpg"

SDFESDSDSDJXCK+DSDSDSSDSFDFDF423232DASDSDSDFDSFJHSIHFSDUIASUI+/==
-----------------------------99614912995
Content-Disposition: form-data; name="files"; filename="Misc 002.jpg"

ASAADSDSDJXCKDSDSDSHAUSAUASAASSDSDFDSFJHSIHFSDUIASUI+/==
-----------------------------99614912995
Content-Disposition: form-data; name="files"; filename="Misc 003.jpg"

TGUHGSDSDJXCK+DSDSDSSDSFDFDSAOJDIOASSAADDASDASDASSADASDSDSDSDFDSFJHSIHFSDUIASUI+/==
-----------------------------99614912995--

This is a Http request in multipart / form data format. We can see that the first line of information is Http header. Here we only list the content type line Http Header information, which is consistent with the value of the enctype attribute on the form tag in our html page. In the first line, there is a boundary = ----------------------- 99614912995, and the value after boundary = is randomly generated. This is actually to declare what the separator of form data in Http request is, which represents each line read in Http request------------------------- --99614912995, indicating a section data. A section may be the key value data of a form or the file data of an uploaded file. The first line of each section is the section header, in which the content disposition attribute is form data, which means that this section is from the form data submitted by the form label. If the section header has the filename or filenamestar attribute, then this section is the file data of the previous document. Otherwise, this section is the key value data of a form The row after the header is the real data row of this section. For example, in the above example, the first two sections are form key value pairs, and the last three sections are three uploaded picture files.

 

Next, let's see how to use the file stream to upload large files to avoid loading all uploaded files into the server memory at one time. The trouble with streaming is that you can't use it ASP.NET The model binder of core MVC is used to deserialize the uploaded file into C ා (as described in the IFormFile interface earlier). First, we need to define the MultipartRequestHelper class, which is used to identify each section type in the Http request (whether the form key value pair section or upload file section)

using System;
using System.IO;
using Microsoft.Net.Http.Headers;

namespace AspNetCore.MultipartRequest
{
    public static class MultipartRequestHelper
    {
        // Content-Type: multipart/form-data; boundary="----WebKitFormBoundarymx2fSWqWSd0OxQqq"
        // The spec says 70 characters is a reasonable limit.
        public static string GetBoundary(MediaTypeHeaderValue contentType, int lengthLimit)
        {
            //var boundary = Microsoft.Net.Http.Headers.HeaderUtilities.RemoveQuotes(contentType.Boundary);// .NET Core <2.0
            var boundary = Microsoft.Net.Http.Headers.HeaderUtilities.RemoveQuotes(contentType.Boundary).Value; //.NET Core 2.0
            if (string.IsNullOrWhiteSpace(boundary))
            {
                throw new InvalidDataException("Missing content-type boundary.");
            }

            //Pay attention here boundary.Length refer to boundary=---------------------------99614912995 After medium---------------------------99614912995 The length of the string, that is section The length of the separator, which is also mentioned above, is generally not more than 70 characters
            if (boundary.Length > lengthLimit)
            {
                throw new InvalidDataException(
                    $"Multipart boundary length limit {lengthLimit} exceeded.");
            }

            return boundary;
        }

        public static bool IsMultipartContentType(string contentType)
        {
            return !string.IsNullOrEmpty(contentType)
                    && contentType.IndexOf("multipart/", StringComparison.OrdinalIgnoreCase) >= 0;
        }

        //If section Is a form key value pair section,Then this method returns true
        public static bool HasFormDataContentDisposition(ContentDispositionHeaderValue contentDisposition)
        {
            // Content-Disposition: form-data; name="key";
            return contentDisposition != null
                    && contentDisposition.DispositionType.Equals("form-data")
                    && string.IsNullOrEmpty(contentDisposition.FileName.Value) // For .NET Core <2.0 remove ".Value"
                    && string.IsNullOrEmpty(contentDisposition.FileNameStar.Value); // For .NET Core <2.0 remove ".Value"
        }

        //If section Yes upload file section,Then this method returns true
        public static bool HasFileContentDisposition(ContentDispositionHeaderValue contentDisposition)
        {
            // Content-Disposition: form-data; name="myfile1"; filename="Misc 002.jpg"
            return contentDisposition != null
                    && contentDisposition.DispositionType.Equals("form-data")
                    && (!string.IsNullOrEmpty(contentDisposition.FileName.Value) // For .NET Core <2.0 remove ".Value"
                        || !string.IsNullOrEmpty(contentDisposition.FileNameStar.Value)); // For .NET Core <2.0 remove ".Value"
        }

        // If one section Of Header Yes: Content-Disposition: form-data; name="files"; filename="Misc 002.jpg"
        // Then this method returns: files
        public static string GetFileContentInputName(ContentDispositionHeaderValue contentDisposition)
        {
            return contentDisposition.Name.Value;
        }

        // If one section Of Header Yes: Content-Disposition: form-data; name="myfile1"; filename="F:\Misc 002.jpg"
        // Then this method returns: Misc 002.jpg
        public static string GetFileName(ContentDispositionHeaderValue contentDisposition)
        {
            return Path.GetFileName(contentDisposition.FileName.Value);
        }
    }
}

 

Then we need to define an extension class called FileStreamingHelper, in which the StreamFiles extension method is used to read the file stream data of the uploaded file, and write the data to the hard disk of the server, which accepts a parameter targetDirectory to declare which folder the uploaded file is stored in.

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Features;
using Microsoft.AspNetCore.Mvc.ModelBinding;
using Microsoft.AspNetCore.WebUtilities;
using Microsoft.Net.Http.Headers;
using System;
using System.Globalization;
using System.IO;
using System.Text;
using System.Threading.Tasks;

namespace AspNetCore.MultipartRequest
{
    public static class FileStreamingHelper
    {
        private static readonly FormOptions _defaultFormOptions = new FormOptions();

        public static async Task<FormValueProvider> StreamFiles(this HttpRequest request, string targetDirectory)
        {
            if (!MultipartRequestHelper.IsMultipartContentType(request.ContentType))
            {
                throw new Exception($"Expected a multipart request, but got {request.ContentType}");
            }

            // Used to accumulate all the form url encoded key value pairs in the 
            // request.
            var formAccumulator = new KeyValueAccumulator();

            var boundary = MultipartRequestHelper.GetBoundary(
                MediaTypeHeaderValue.Parse(request.ContentType),
                _defaultFormOptions.MultipartBoundaryLengthLimit);
            var reader = new MultipartReader(boundary, request.Body);

            var section = await reader.ReadNextSectionAsync();//For reading Http First of the requests section data
            while (section != null)
            {
                ContentDispositionHeaderValue contentDisposition;
                var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out contentDisposition);

                if (hasContentDispositionHeader)
                {
                    /*
                    section used to process the uploaded file type
                    -----------------------------99614912995
                    Content - Disposition: form - data; name = "files"; filename = "Misc 002.jpg"

                    ASAADSDSDJXCKDSDSDSHAUSAUASAASSDSDFDSFJHSIHFSDUIASUI+/==
                    -----------------------------99614912995
                    */
                    if (MultipartRequestHelper.HasFileContentDisposition(contentDisposition))
                    {
                        if (!Directory.Exists(targetDirectory))
                        {
                            Directory.CreateDirectory(targetDirectory);
                        }

                        var fileName = MultipartRequestHelper.GetFileName(contentDisposition);

                        var loadBufferBytes = 1024;//This is every time Http Requested section The size of the read file data in Byte That is, byte, which is set to 1024, which means that every time the Http Requested section Read and fetch 1024 bytes of data from the data stream to the server memory, and then write to the following targetFileStream This value can be adjusted according to the memory size of the server. This avoids loading the data of all uploaded files into the memory of the server at one time, leading to the server crash.

                        using (var targetFileStream = System.IO.File.Create(targetDirectory + "\\" + fileName))
                        {
                            //section.Body yes System.IO.Stream Type, which means Http One of the requests section Data stream from which each can be read section So we can not use the following data section.Body.CopyToAsync Method, but in a loop section.Body.Read Method (if section.Body.Read Method returns 0, indicating that the data flow has reached the end and all data has been read targetFileStream
                            await section.Body.CopyToAsync(targetFileStream, loadBufferBytes);
                        }

                    }
                    /*
                    section used to process form key data
                    -----------------------------99614912995
                    Content - Disposition: form - data; name = "SOMENAME"

                    Formulaire de Quota
                    -----------------------------99614912995
                    */
                    else if (MultipartRequestHelper.HasFormDataContentDisposition(contentDisposition))
                    {
                        // Content-Disposition: form-data; name="key"
                        //
                        // value

                        // Do not limit the key name length here because the 
                        // multipart headers length limit is already in effect.
                        var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name);
                        var encoding = GetEncoding(section);
                        using (var streamReader = new StreamReader(
                            section.Body,
                            encoding,
                            detectEncodingFromByteOrderMarks: true,
                            bufferSize: 1024,
                            leaveOpen: true))
                        {
                            // The value length limit is enforced by MultipartBodyLengthLimit
                            var value = await streamReader.ReadToEndAsync();
                            if (String.Equals(value, "undefined", StringComparison.OrdinalIgnoreCase))
                            {
                                value = String.Empty;
                            }
                            formAccumulator.Append(key.Value, value); // For .NET Core <2.0 remove ".Value" from key

                            if (formAccumulator.ValueCount > _defaultFormOptions.ValueCountLimit)
                            {
                                throw new InvalidDataException($"Form key count limit {_defaultFormOptions.ValueCountLimit} exceeded.");
                            }
                        }
                    }
                }

                // Drains any remaining section body that has not been consumed and
                // reads the headers for the next section.
                section = await reader.ReadNextSectionAsync();//For reading Http Next in request section data
            }

            // Bind form data to a model
            var formValueProvider = new FormValueProvider(
                BindingSource.Form,
                new FormCollection(formAccumulator.GetResults()),
                CultureInfo.CurrentCulture);

            return formValueProvider;
        }

        private static Encoding GetEncoding(MultipartSection section)
        {
            MediaTypeHeaderValue mediaType;
            var hasMediaTypeHeader = MediaTypeHeaderValue.TryParse(section.ContentType, out mediaType);
            // UTF-7 is insecure and should not be honored. UTF-8 will succeed in 
            // most cases.
            if (!hasMediaTypeHeader || Encoding.UTF7.Equals(mediaType.Encoding))
            {
                return Encoding.UTF8;
            }
            return mediaType.Encoding;
        }
    }
}

 

Now we need to create another ASP.NET The core MVC's custom interceptor, DisableFormValueModelBindingAttribute, implements the interface IResourceFilter, which is used to disable ASP.NET The model binder of core MVC, so that when an Http request arrives at the server, ASP.NET Core MVC will not execute the contour Action method after all the requested uploaded file data is loaded into the server memory, but when the Http request arrives at the server, it will execute the contour Action method immediately.

using Microsoft.AspNetCore.Mvc.Filters;
using Microsoft.AspNetCore.Mvc.ModelBinding;
using System;
using System.Linq;

namespace AspNetCore.MultipartRequest
{
    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
    public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter
    {
        public void OnResourceExecuting(ResourceExecutingContext context)
        {
            var formValueProviderFactory = context.ValueProviderFactories
                .OfType<FormValueProviderFactory>()
                .FirstOrDefault();
            if (formValueProviderFactory != null)
            {
                context.ValueProviderFactories.Remove(formValueProviderFactory);
            }

            var jqueryFormValueProviderFactory = context.ValueProviderFactories
                .OfType<JQueryFormValueProviderFactory>()
                .FirstOrDefault();
            if (jqueryFormValueProviderFactory != null)
            {
                context.ValueProviderFactories.Remove(jqueryFormValueProviderFactory);
            }
        }

        public void OnResourceExecuted(ResourceExecutedContext context)
        {
        }
    }
}

 

Finally, we define an Action method called Index in the contour, and register the DisableFormValueModelBindingAttribute interceptor we defined to disable the model binding of Action. The Index method calls the StreamFiles method in the FileStreamingHelper class we defined earlier. Its parameter is the folder path used to store the uploaded file. The StreamFiles method will return a FormValueProvider, which is used to store the form key value data in the Http request. After that, we will bind it to the view model viewModel of MVC, and then pass the viewModel back to the client browser to report that the client browser file has been uploaded successfully.

[HttpPost]
[DisableFormValueModelBinding]
public async Task<IActionResult> Index()
{
    FormValueProvider formModel;
    formModel = await Request.StreamFiles(@"F:\UploadingFiles");

    var viewModel = new MyViewModel();

    var bindingSuccessful = await TryUpdateModelAsync(viewModel, prefix: "",
        valueProvider: formModel);

    if (!bindingSuccessful)
    {
        if (!ModelState.IsValid)
        {
            return BadRequest(ModelState);
        }
    }

    return Ok(viewModel);
}

 

The view model viewModel is defined as follows:

public class MyViewModel
{
    public string Username { get; set; }
}

 

Finally, the html page we use to upload files is almost the same as before:

<form method="post" enctype="multipart/form-data" action="/Home/Index">
    <div>
        <p>Upload one or more files using this form:</p>
        <input type="file" name="files" multiple />
    </div>
    <div>
        <p>Your Username</p>
        <input type="text" name="username" />
    </div>
    <div>
         <input type="submit" value="Upload" />
    </div>
</form>

 

This is all the code, hope to help you!

 

reference:

File uploads in ASP.NET Core

Uploading Files In ASP.net Core

What is the boundary parameter in an HTTP multi-part (POST) Request?

Tags: Attribute encoding Javascript

Posted on Sun, 31 May 2020 10:10:23 -0400 by BeastRider