It has been one hell of a journey learning go, it been some-worth difficult as I was very used to the loose nature of JavaScript and PHP (languages I have been writing for about 3 years). GO's strict nature is something I have just been coming into terms with in the last few months and by all means, loving it. It is a great language.
The reason for this post is to try to explain file upload in GO through the eyes of an amateur, so pardon me if my writing is very verbose as i will be trying to explain this as i would to myself. so let's dig in.
We would be handling file uploading as part of the form body as well as handling multiple files.
First let's create the boilerplate, create a folder and create a new main.go file.
package main
func uploadHandler(w http.ResponseWriter, r *http.Request){
}
func main(){
http.HandleFunc("/upload", uploadHandler)
log.Fatal(http.ListenAndServe(":9000", nil))
}
next, let us create a simple index.html file where we would be making our submissions from
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Go file upload</title>
</head>
<body>
<form action="/upload" method="post">
<input type="file" name="image" accept="image/*">
<button type="submit">Submit</button>
</form>
</body>
</html>
lets also serve the index.html file in main.go
package main
func uploadHandler(w http.ResponseWriter, r *http.Request){
}
func indexHandler(res http.ResponseWriter, r *http.Request) {
res.Header().Add("Content-Type", "text/html")
http.ServeFile(res, r, "index.html")
}
func main(){
http.HandleFunc("/", indexHandler)
http.HandleFunc("/upload", uploadHandler)
log.Fatal(http.ListenAndServe(":9000", nil))
}
I do recommend that when testing with the index.html file you add enctype="multipart/form-data" since browsers will not submit the file without this in the form tag and that would cause an error, or you could use postman until we get to parsing multipart forms.
next, we move forward to handling the form in the request, first, we must parse the form
func uploadHandler(w http.ResponseWriter, r *http.Request){
// this neccessary else you will not be able to access other properties needed to get the file
// this method returns an err
err:=r.ParseForm()
// always handle error
if err != nil{
// of course we should use a better error message
http.Error(w, "Unable to parse form", http.StatusInternalServerError)
return
}
}
the reason why parsing the form is that we need to call the r.FormFile() method and this method as well as the r.FormValue() methods are unavailable until the form is parsed.
in this case, we will be using the r.FormFile() method, this method returns three arguments:
First argument is the multipart.File type. the second argument is the multipart.FIleHeader type these arguments contain the meta details of the file such as its name, size and the file itself." the third and final argument is an error.
with that being said let jump in and take a look at the added lines of code
func uploadHandler(w http.ResponseWriter, r *http.Request) {
// this neccessary else you will not be able to access other properties needed to get the file
// this method returns an err
err:=r.ParseForm()
// always handle error
if err != nil{
http.Error(w, "Unable to parse form", http.StatusInternalServerError)
return
}
file, fileHeader, err := r.FormFile("image")
// get the file by entering its form name.
// handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
//postpone closing the file until the function is complete
defer file.Close()
after the previous step, it is now time to create the file on the local server, we are going to be using the current time as the name of the file and extracting the file extension with the filepath.Ext() method.
func uploadHandler(w http.ResponseWriter, r *http.Request) {
/*
reduced for brevity
*/
// create file on the local server, filepath.Ext() will get the extension out of the filename
localfile, err := os.Create(time.Now().UTC().String()+filepath.Ext(fileHeader.Filename))
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
Now we can copy the uploaded file into the newly created file.
/*
reduced for brevity
*/
// create file on the local server, filepath.Ext() will get the extension out of the filename
localfile, err := os.Create(time.Now().UTC().String()+filepath.Ext(fileHeader.Filename))
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// use the io.Copy() method to copy bytes to the localbyte from the uploaded file,
// this method returns the number of bytes copied and an error, we would be ignoring the bytes
_, err = io.Copy(localfile, file)
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
//after which we can simply respond back to the client
fmt.Fprint(w, "File upload successful")
}
and that's it, that is all that is needed to upload a file.
Parsing a multipart form
if the form submitted is a multipart form, parsing the form is slightly different from a normal form but only by one line.
//change this line
err:=r.ParseForm()
//to, r.ParseMultipartForm() takes an argument (maxMemory), maxMemory indicates how much of the file size should be kept in memory, while any extra will be kept in a temp folder on the disc
err:=r.ParseMultipartForm(300)
Uploading multiple files
Uploading multiple files is just as simple as uploading a single file with the major difference being how you handle getting the files and looping through them, lets take a look.
// create a new function for uploading multiple files
func uploadMultipleFiles(w http.ResponseWriter, r *http.Request){
r.ParseMultipartForm(1024 * 1024 *10) //maxMemory if 10mb
}
let us also handle that in the main.go
func main() {
http.HandleFunc("/", indexHandler)
http.HandleFunc("/upload", uploadHandler)
http.HandleFunc("/uploadmultiple", uploadMultipleFiles)
http.ListenAndServe(":9000", nil)
}
after parsing the form, we would now be able to get the files with the r.MultipartForm.File["images"] slice.
func uploadMultipleFiles(w http.ResponseWriter, r *http.Request){
r.ParseMultipartForm(1024 * 1024 *10) //maxMemory if 10mb
//we get the slice of files
files:= r.MultipartForm.File["images"]
}
we need to loop through the files to be able to upload each of the files just like we did before, with the only difference being that we would not be working with an actual multipart.File type but with a multipart.FileHeader type instead. but not to worry the multipart.FileHeader also contains the file itself.
func uploadMultipleFiles(w http.ResponseWriter, r *http.Request){
r.ParseMultipartForm(1024 * 1024 *10) //maxMemory if 10mb
//we get the slice of files
files:= r.MultipartForm.File["images"]
// looping through the files to upload each file individually
for _, file := range files {
uploadFile, err := file.Open()
// handle error as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// postpone file closure until the function is complete
defer uploadFile.Close()
//create a new localfile
localfile, err := os.Create(time.Now().UTC().String() + filepath.Ext(file.Filename))
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
_, err = io.Copy(localfile, uploadFile)
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
//after which we can simply respond back to the client
fmt.Fprint(w, "File upload successful")
}
Extras
Setting a maximum file upload size
in a case where you want to restrict the file upload size, we would simply equate the file size in bytes:
// add where neccessary in your code
const MAX_UPLOAD_SIZE int64 = 1024 * 1024 * 10 //maximum of 10mb per file
// if this was for a single file upload it would be fileHeader.Size, because it is the
// multipart.FileHeader type that has Size a property
if file.Size > MAX_UPLOAD_SIZE{
http.Error(w, "file sizes cannot be bigger than 10mb", http.StatusBadRequest)
return
}
Tracking file upload progress
Following up with file upload so as to check if the file upload progress is a bit tricky but not complicated. First, we need to create a new struct type whose property would be the total size of the file (TotalBytesToRead) and how many bytes have been read (TotalBytesRead). the struct also needs to satisfy the io.Writer interface, OK let's see some code.
type Progress struct{
TotalBytesToRead int64
TotalBytesRead int64
}
// create the write method so satisfy the i0.Writer interface
func (p *Progress) Write(pb []byte) (int, error) {
n := len(pb) //length of the file read so far
// set error to nil since no error will be handled
p.TotalBytesRead = int64(n)
if p.TotalBytesRead == p.TotalBytesToRead {
fmt.Println("Done")
return n, nil
}
//this will be printed on your terminals
fmt.Printf("File upload still in progress -- %d\n", p.TotalBytesRead)
return n, nil
}
// now in use with the file upload
func uploadHandler(w http.ResponseWriter, r *http.Request) {
// this neccessary else you will not be able to access other properties needed to get the file
// this method returns an err
err:=r.ParseForm()
// always handle error
if err != nil{
http.Error(w, "Unable to parse form", http.StatusInternalServerError)
return
}
file, fileHeader, err := r.FormFile("image")
// get the file by entering its form name.
// handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
//postpone closing the file until the function is complete
defer file.Close()
//create a new localfile
localfile, err := os.Create(time.Now().UTC().String()+filepath.Ext(file.Filename))
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// Create a new instance of progress
progress := &Progress{
TotalBytesToRead : fileHeader.Size,
}
//instead of coppying directly we would use the io.TeeReader() method
// this method will allow us to write the amount of bytes read into the progress.Write method
_, err := io.Copy(localfile, io.TeeReader(file, progress))
//handle errors as neccessary
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
and that is it, that is all that is needed to track file upload progress.
at this point, I hope you are now a master in the arts of file uploading and all of its caveats. thank you for reading and i hope this was helpful. please follow me.
Complete code can be found here