This content originally appeared on DEV Community and was authored by Sergio Mijatovic
One of the most common tasks a developer faces when building web applications is uploading files from a client (typically a web browser). Whether it’s a small or a large application, chances are you will need to upload images, PDF documents and other kinds of files. These files are then typically stored on the server and indexed in the database, so they can be served up on demand.
Note that the kind of upload I am talking about here is through HTML forms, or using the same method as HTML forms, and is the most common kind. When people talk about uploading files, that’s what they are usually referring to.
The mechanics of uploading a file
File upload is usually sent to a server as an HTTP POST request. The body of request will then contain metadata about the files being uploaded from the client, as well as the contents of files themselves. The content sent may look something like this — the first part before the empty line is an HTTP header, followed by the actual upload data:
POST /file_up HTTP/1.1
Host: 127.0.0.1
Content-Type: multipart/form-data; boundary=---------------------------42828225182062843700116470094
Content-Length: 244463
-----------------------------42828225182062843700116470094
Content-Disposition: form-data; name="req"
upload_file
-----------------------------42828225182062843700116470094
Content-Disposition: form-data; name="myfile"; filename="pic7.jpg"
Content-Type: image/jpeg
\377\330...
What’s in a message like this?
First of all, there is a uniquely-generated delimiter (in this case “ — — — — — — — — — — — — — -37487108932486351905827665017”, but in general it would be some other string) that is used to separate the parts of the message. It is generated in such a manner that it does not exist in any part of the message, hence it can be used as a delimiter between the data pieces.
Those pieces tell the server all it needs to know. One kind of information conveyed this way is telling the server that the data comes from a web form (“Content-Disposition: form-data;”).
Next, there are name/value pairs that come from the HTML form. For instance when you fill out a form with your name and address, they will be sent as values in such pairs. In this case, there is a “req=upload_file” pair.
Finally, there are the file(s), one or more of them. Here, a file is given with a URL field “myfile”, with the actual file name of “pic7.jpg”. The file content type is JPG (specified as “image/jpeg”), then followed by the binary content of the file (in this case beginning with “\377\330…”).
In the way described, any number of name/value pairs can be specified, as well as any number of files uploaded.
The server must parse this correctly and extract the data in order to get all the name/value pairs and all the file names and their contents. Then these files can be saved, passed on somewhere else, or processed in some way.
This is how web upload works in most cases, and web clients/servers follow this process — that’s because it’s an Internet standard described in-RFC7578.
Virtually all languages and frameworks support file uploads, with varying degrees of complexity. The following is an example of uploading files to the server using Vely framework for C programming language.
Prerequisites
Nginx is used as a web server, so install it before proceeding. If not already setup, you may need to configure a firewall for it to allow HTTP traffic (see ufw, firewall-cmd etc.).
This example uses-Vely application development framework. Install it first.
You will need sudo privilege to install software and to create a web page used for testing.
Upload example
To run the example here, create an application “file_up” in a directory of its own (see vf for more on Vely’s program manager):
mkdir upload_example
cd upload_example
sudo vf -i -u $(whoami) file_up
Create file “upload_file.vely” and copy this to it:
#include "vely.h"
void upload_file () {
input-param myfile_filename
input-param myfile_location
input-param myfile_ext
input-param myfile_size
out-header default
@File <<p-out myfile_filename>> uploaded to <<p-out myfile_location>> (extension <<p-out myfile_ext>>, size <<p-out myfile_size>>).
}
To run this code as an application server, make it first:
vv -q
To start your application server that can upload files:
vf file_up
How does this work
Vely has a built-in functionality for file uploads, and the details described previously are automatically done. When you upload a file, it is treated as any other input parameter. So the code you have on server-side:
input-param myfile_filename
input-param myfile_location
input-param myfile_ext
input-param myfile_size
assumes that your uploaded file will be a URL field named “myfile”, and then you can obtain it’s actual file name (which is “myfile_filename”), its location where it’s uploaded on the server (“myfile_location”), its extension (“myfile_ext”) and size in bytes (“myfile_size”). Basically the URL field name is appended with an underscore and the kind of information you want to get.
The last part:
@File <<p-out myfile_filename>> uploaded to <<p-out myfile_location>> (extension <<p-out myfile_ext>>, size <<p-out myfile_size>>).
is an output_statement that begins with “@” and it outputs text to the client. The p-out statement will print a string, and in this case it is inlined by using “<<” and “>>”. It’s pretty simple (and readable) to output data this way.
Setup web server
The next step is to set up a web server. You have just started an application server, and by definition, it sits behind the web server. This is good for many reasons: performance, safety, separation of layers. You can use any web server that supports FastCGI protocol (the same protocol used by PHP-FPM); in this example I will use Nginx which inherently supports FastCGI. That’s nice as there’s nothing to configure.
As I mentioned, to start you will need to have Nginx installed. Once done, you will need to edit the Nginx configuration file. For Ubuntu and similar:
sudo vi /etc/nginx/sites-enabled/default
while on Fedora and other systems it might be:
sudo vi /etc/nginx/nginx.conf
Add the following in the “server {}” section — note the “fastcgi_pass” parameter that specifies path to your application’s Unix socket:
location /file_up { include /etc/nginx/fastcgi_params; fastcgi_pass unix:///var/lib/vv/file_up/sock/sock; }
client_max_body_size 20M;
Finally, restart Nginx:
sudo systemctl restart nginx
This will connect the Nginx web server to your application server via a Unix socket, which is a very fast connection indeed, so that requests made via web server are answered by your code above. The “client_max_body_size” directive helps with avoiding “Request Entity Too Large” error, i.e. Nginx by default allows file uploads only up to 2MB in size, which isn’t much these days — this way we up the limit to 20MB.
You are now done! Your application is ready to use.
Test by uploading a file
To test, create an HTML file (i.e. a web page) that you can open through your web server. This file should be under the Nginx’s root document directory. To see where is this root directory on your system check out the configuration file above and look for “root” directive under the “server{}” section, for instance it might look like:
root /usr/share/nginx/html/;
in which case the directory is “/usr/share/nginx/html/” — note however it may vary depending on the distribution and release. Create a file “test_upload.html” there, and copy the following into it:
<form action="/file_up" method="POST" enctype="multipart/form-data">
<input type="hidden" name="req" value="upload_file">
<label for="myfile">File:</label>
<input type="file" name="myfile" value=""><br><br>
<input type="submit" value="Submit">
</form>
The above is a simple HTML form that will call your Vely code. How does it work?
Notice the “action” attribute in a “
” element: it is “/file_up”, which is the application path of your application (by default it is the name of application preceded by a forward slash), and it is the “location” in the Nginx directive you added.The request method used is “POST”, and the name of the HTML field to upload the file is “myfile” — this is in “” element of type “file” that lets you choose a local file to upload.
Finally, there’s a Submit button given as an “” element of type “submit”.
This is a very simple setup that you can build upon for your web applications.
To test, go to the browser and (assuming you are doing this on your local computer, otherwise change to your web address):
http://127.0.0.1/test_upload.html
You can now select a file (in this example it’s “pic7.jpg”), upload it and you will see the output:
File pic7.jpg uploaded to /var/lib/vv/file_up/app/file/18648/2107645QBsCL (extension .jpg, size 244124).
This means your file has been uploaded to the server and is saved at location “/var/lib/vv/file_up/app/file/18648/2107645QBsCL”. This file is in Vely file_storage, and its name is generated to be unique among all uploaded files, even if they are uploaded at the same time. The way files are organized is to make them available for quick retrieval and so you don’t have to worry about naming them — after all the name of a file is just a reference you’re likely to store somewhere in the database. You can leave the file there (as Vely file storage is meant to keep files for quick access), or you can move it somewhere else by using rename-file statement.
Conclusion
And there you have it. You can now upload files to the server. Of course, the application here is a rudimentary one, but it shows the basics. You can upload multiple files, add other text fields, save the file location to the database and process the files according to your application’s needs.
Cover Image Copyright Sergio Mijatovic (c) 2023
This content originally appeared on DEV Community and was authored by Sergio Mijatovic
Sergio Mijatovic | Sciencx (2023-03-15T22:28:22+00:00) Mastering File Upload: A Guide for Web Developers. Retrieved from https://www.scien.cx/2023/03/15/mastering-file-upload-a-guide-for-web-developers/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.