- in memory (photo) file upload validation(file size and dimensions)
- I’ve never written Clojure before
- unhandled exceptions
Important note: this is my first take on functional programming and Clojure so you probably shouldn’t read anything written here as a guide for handling file uploads or validation.
Through the post, I’ll try stick to the least necessary amount of code, skip the parts that are explained in documentation of given dependencies and go with the top-down approach explaining the code. Full source code is available in the respository.
There are few objectives that I’d like to achieve and I’m using midje to make sure that everything works as expected. I want to create an API endpoint that will:
- return response if photo size is above the limit without downloading whole file
1 2 3 4 5 6 7 8 9 10
- validate photo dimensions without creating temporary object
1 2 3 4 5 6 7 8 9 10 11 12
- save photo file to the disk
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
- save information about upload in database in a paperclip accessible format.
Firstly, We’re going to create an API endpoint in liberator to handle POST request with a file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
and here we stumble into the first issue: does Ring, on which liberator depends, help me with handling file uploads without downloading the whole file? Unfortunately that wasn’t the case because byte-array-store middleware provided with Ring (which can be used to handle multipart uploads) transforms files immediately to a byte array.
and here we stumble into the first issue, does Ring, on which liberator depends, help me with handling file uploads without downloading the whole file? Unfortunately that wasn’t the case because
byte-array-store middleware provided with Ring(which can be used to handle multipart uploads) transforms files immediately to a byte array. We’d like to avoid that, so we need to create custom stream handler that’s going to return buffered input stream.
1 2 3 4 5
Middlewares in Ring have to be passed into the site handler:
1 2 3
For the sake of simplicity, we’re going to implement the rest of the objectives in the model. Still going with the top-down approach, we know that we’ll want to: – perform validations – add record in database – save file on disk – return errors from validations
Let’s start with a main method method that’s going to be public and as you can recall, is used in the router to save a photo.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Right now, we’re ignoring the fact that the
image_content_type and file location path are hardcoded, it can be easily fixed later by getting the configuration file.
let form firstly we perform validations, so let’s add a code that’s supposed to do that:
1 2 3 4 5 6 7 8 9 10
Here, we’re doing a few of things
- provide array where we’ll combine errors from validators
- check if file is provided
- rename temporarily the key where the image is so that the
size-validatorcould be reused for other files uploads, not only images
- validate dimensions of the photo
Before we start writing validators, it would be good to have a helper function to combine errors:
1 2 3 4 5 6 7
Equipped with such function, we’re ready to start writing validators. At the very beginning we’ve to check for file presence. If we want to know if a file is available, we can just check if there’s any filename:
1 2 3 4 5
File validator on the other side, looks quite complex in respect to other methods because we’ve to perform byte by byte read from the stream and check if we’ve not exceeded the limit. We’d like to also save the stream while consuming it because if the file is below the limit, we’d like to use it in future validations.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Dimensions validation can be performed by just casting the stream as image thanks to
1 2 3 4 5 6 7 8 9
That was the last validation needed to be performed before we could save a file. I’m using Korma to generete SQL queries
1 2 3 4
and now the last part, saving the photo to the disk:
1 2 3
exceptions are handled in
#create! because we need to perform rollback if anything goes wrong with the file save.
There’re a lot of missing parts for the uploader to be production ready. As a beginner I’ve mostly struggled with the IO and not being a Java programmer wrapping my head around different kinds of bytes and streams wasn’t the pleasantest time. I’m still not sure if tests setup with
midje is similar to something people do in applications at production but it worked well for such small and primitive example. I wish there were more books, blog posts about that from more experienced Clojure devs. And lastly, HUGE thanks to @annapawlicka for patience, guidance and helping me out with all kinds of beginner’s problems. 💙