This content originally appeared on Level Up Coding - Medium and was authored by Itsuki
File Uploads, downloads! Binaries are unavoidable!
In this article, let’s check out how we can
- Handle Binary data posted as Body
- Handle Binary data posted as Multipart/form-data
- Send Binary data as Response
We will be using axum here, a web application framework great for routing requests to handlers and declaratively parse requests using extractors.
Let’s go!
Set Up
As always, starting with the dependencies in Cargo.toml.
[dependencies]
tokio = { version = "1", features = ["macros"] }
lambda_http = "0.13.0"
axum = { version = "0.7.5", features = ["multipart"] }
tower-http = { version = "0.5.2", features = ["limit"] }
regex = "1.10.6"
urlencoding = "2.1.3"
I will be running it using API Gateway and Lambda so I have added lambda_http, but you can serve it on whatever you like!
All the handling will work out the same way!
Receive Binary Data
Bytes Directly in Body
When receiving bytes directly as Request Body, there are couple things we will do in our handler function.
- Extracting the file_name from the Content-Disposition Header.
- Convert Bytes to Vec<u8> which is probably what you need when saving the file.
async fn post_bytes(headers: HeaderMap, bytes: Bytes) -> Json<Value> {
let Some(content_deposition) = headers.get(CONTENT_DISPOSITION) else {
return Json(json!({
"message": "content diposition not available"
}));
};
let Ok(content_deposition_string) = content_deposition.to_str() else {
return Json(json!({
"message": "content diposition not available"
}));
};
let Ok(re) = Regex::new(r"((.|\s\S|\r|\n)*)filename\*=utf-8''(?<name>((.|\s\S|\r|\n)*))") else {
return Json(json!({
"message": "Regex failed to create."
}));
};
let Some(captures) = re.captures(content_deposition_string) else {
return Json(json!({
"message": "File name not found."
}));
};
let file_name = decode(&captures["name"]).expect("UTF-8").to_string();
let data: Vec<u8> = match bytes.bytes().collect() {
Ok(data) => data,
Err(err) => {
return Json(json!({
"message": format!("Bytes data not available. Error: {}", err.to_string())
}));
}
};
return Json(json!({
"file_name": file_name,
"data_length": data.len()
}));
}
As you can see, quiet amount of work for extracting the file_name…
Here, I am assuming the CONTENT_DISPOSITION is in the following format.
Content-Disposition: attachment; filename*=utf-8''pikachu.jpg where the file name is url-encoded. It could be different, but this one is pretty standard in most cases (maybe?).
To try the handler out, let’s add create a Router in our main and add a route to it directing to the handler we have above!
#[tokio::main]
async fn main() -> Result<(), Error> {
set_var("AWS_LAMBDA_HTTP_IGNORE_STAGE_IN_PATH", "true");
tracing::init_default_subscriber();
let app = Router::new()
.route("/", post(post_bytes));
run(app).await
}
As I have mentioned, I am using API Gateway and Lambda to serve the App router, but you can choose any way you want!
cargo lambda watch if you are like me and let’s post!
I am using Insomnia posting the following pikachu! Make sure to add Content-Disposition header.
And here is the response I got!
{
"data_length": 39739,
"file_name": "pikachu.jpg"
}
Bytes as Multipart/form-data
First of all, to use the axum::extract::Multipart , we need to have the crate feature multipart (like what we have specified above).
Here is how we can use the Multipart extractor to handle file upload assuming the field name for the file is file.
async fn post_bytes_multiparts(mut multipart: Multipart) -> Json<Value> {
while let Some(field) = multipart.next_field().await.unwrap() {
let name = field.name().unwrap().to_string();
let file_name = field.file_name().unwrap().to_string();
let content_type = field.content_type().unwrap().to_string();
let data = field.bytes().await.unwrap();
if name == "file" {
return Json(json!({
"file_name": file_name,
"content_type": content_type,
"data_length": data.len()
}));
}
}
Json(json!({
"message": "file field is missing in the form."
}))
}
I am too lazy to do error handlings but you can do better than this!
As you can see, a lot less work to do while trying to get file name in comparison to what we had above, having to use regular expressions, decoding the name, and blahhh, thanks to file_name() extracting the file name found in the Content-Disposition header for us!
However, as you might know, not all front-end, by saying that, I am thinking of Swift, supporting posting Multipart/form-data out of box. You will have to manually creating boundaries, switching all types of form values (string, file, etc.) and build your own form body Data.
Less work on one end, more work on the other!
You get to choose!
To test it out, simply add another route to the router like following!
let app = Router::new()
.route("/", post(post_bytes))
.route("/multiparts", post(post_bytes_multiparts))
Set Body Size Limit
Files can be large, especially if you are expecting videos.
For security reasons, by default, Bytes, or any extractors that uses Bytes]internally such as String, Json, Form, and Multipart, will not accept bodies larger than 2MB.
Obviously not enough!
To set our own limit, two steps.
- Disable the default request body limit.
- Optionally add our own limit using tower_http::limit.
let app = Router::new()
// ...routes
// Set a different limit: 10GB
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(10 * 1000 * 1000 * 1000));
Send Binary Data
Different from above, when sending binary data, we need the make sure the Content-Type and Content-Disposition in our Response Header is set correctly.
That means our return value, instead of Json<Value> , will be Response with our custom headers included!
Let’s simply echoing back the bytes what we received.
fn build_error_response(message: &str) -> Response {
let mut json_header = HeaderMap::new();
json_header.insert(CONTENT_TYPE, "application/json".parse().unwrap());
let mut response = Response::new(json!({
"message": message
}).to_string());
*response.status_mut() = StatusCode::BAD_REQUEST;
return (json_header, response).into_response();
}
async fn echo_bytes(headers: HeaderMap, bytes: Bytes) -> Response {
let Some(content_type) = headers.get(CONTENT_TYPE) else {
return build_error_response("content type not available");
};
let Some(content_disposition) = headers.get(CONTENT_DISPOSITION) else {
return build_error_response("content diposition not available");
};
let Ok(content_disposition_string) = content_disposition.to_str() else {
return build_error_response("content diposition not available");
};
let Ok(re) = Regex::new(r"((.|\s\S|\r|\n)*)filename\*=utf-8''(?<name>((.|\s\S|\r|\n)*))") else {
return build_error_response("Regex failed to create");
};
let Some(captures) = re.captures(content_disposition_string) else {
return build_error_response("File name not found in Content-Disposition");
};
let file_name = decode(&captures["name"]).expect("UTF-8").to_string();
let data: Vec<u8> = match bytes.bytes().collect() {
Ok(data) => data,
Err(err) => {
return build_error_response(&format!("Bytes data not available. Error: {}", err.to_string()));
}
};
let file_name_encode = encode(&file_name).to_string();
let mut headers = HeaderMap::new();
headers.insert(CONTENT_TYPE, content_type.to_owned());
headers.insert(CONTENT_LENGTH, bytes.len().into());
headers.insert(CONTENT_DISPOSITION, format!("attachment; filename*=utf-8''{}", file_name_encode).parse().unwrap());
(headers, data).into_response()
}
Since we are simply echoing, I guess you can simply use the CONTENT_DISPOSITION received, but just so that we get to see how we can build our own in Rust, let’s do it this way!
Again, add another route
.route("/echo", post(echo_bytes))
and you should see the image, or whatever file you sent, coming back!
Thank you for reading!
That’s all I have for today!
Happy bytes handling!
Rust API Server: Send and Receive Bytes Data was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Itsuki
Itsuki | Sciencx (2024-08-31T12:23:27+00:00) Rust API Server: Send and Receive Bytes Data. Retrieved from https://www.scien.cx/2024/08/31/rust-api-server-send-and-receive-bytes-data/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.