feat: Initial commit

.gitignore

@@ -0,0 +1,10 @@
+.cache
+.cover
+coverage.txt
+coverage.out
+coverage.html
+test
+/dist
+build
+rufs
+examples/tutorial/run/

LICENSE

@@ -0,0 +1,7 @@
+Copyright 2017 Matthias Ladkau
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

NOTICE

@@ -0,0 +1,19 @@
+RUFS - Remote Union File System
+Copyright (c) 2017 Matthias Ladkau
+
+The following components are included in this product:
+
+go-fuse
+https://github.com/hanwen/go-fuse
+Copyright (c) 2010 the Go-FUSE Authors
+Licensed under the MIT License
+
+sys - supplemental Go packages
+https://golang.org/x/sys
+Copyright (c) 2009 The Go Authors
+Licensed under the MIT License
+
+dokan-go
+https://github.com/keybase/dokan-go
+Copyright (c) 2016-2017, Keybase
+Licensed under the BSD3 License

README.md

@@ -0,0 +1,161 @@
+Rufs
+====
+Rufs is a remote union filesystem which aims to provide a lightweight and secure solution for distributed file storage. Rufs uses a client-server system where servers expose branches and clients mount one or several branches into a tree structure. The client can overlay branches providing a union view.
+
+Features
+--------
+- Client-Server model using RPC call over ssl.
+- Single executable for client and server.
+- Communication is secured via a secret token which is never transferred over the network and certificate pinning once a client has connected successfully.
+- Clients can provide a unified view with files from different locations.
+- Default client provides CLI, REST API and a web interface.
+- Branches can be read-only.
+- A read-only version of the file system can be exported via FUSE and mounted.
+
+Getting Started
+---------------
+You can download a precompiled package for Windows (win64) or Linux (amd64) [here](https://devt.de/build_status.html).
+
+The archive contains a single executable which contains the server and client code for Rufs.
+
+### Tutorial:
+
+To get an idea of what EliasDB is about have a look at the [tutorial](/examples/local_fileshare/doc/tutorial.md).
+
+### REST API:
+
+The terminal uses a REST API to communicate with the backend. The REST API can be browsed using a dynamically generated swagger.json definition (https://localhost:9090/fs/swagger.json). You can browse the API of Rufs's latest version [here](http://petstore.swagger.io/?url=https://raw.githubusercontent.com/krotik/rufs/master/doc/swagger.json#/default).
+
+### Command line options
+The main Rufs executable has two main tools:
+```
+Rufs 1.0.0
+
+Usage of ./rufs [tool]
+
+The tools are:
+
+    server    Run as a server
+    client    Run as a client
+
+Use ./rufs [tool] --help for more information about a tool.
+```
+The most important one is server which starts the file server. The server has several options:
+```
+Rufs 1.0.0
+
+Usage of ./rufs server [options]
+
+  -config string
+    	Server configuration file (default "rufs.server.json")
+  -help
+    	Show this help message
+  -secret string
+    	Secret file containing the secret token (default "rufs.secret")
+  -ssl-dir string
+    	Directory containing the ssl key.pem and cert.pem files (default "ssl")
+
+The server will automatically create a default config file and
+default directories if nothing is specified.
+```
+Once the server is started the client tool can be used to interact with the server. The options of the client tool are:
+```
+Rufs 1.0.0
+
+Usage of ./rufs client [mapping file]
+
+  -fuse-mount string
+    	Mount tree as FUSE filesystem at specified path (read-only)
+  -help
+    	Show this help message
+  -secret string
+    	Secret file containing the secret token (default "rufs.secret")
+  -ssl-dir string
+    	Directory containing the ssl key.pem and cert.pem files (default "ssl")
+  -web string
+    	Export the tree through a https interface on the specified host:port
+
+The mapping file assignes remote branches to the local tree.
+The client tries to load rufs.mapping.json if no mapping file is defined.
+It starts empty if no mapping file exists. The mapping file
+should have the following json format:
+
+{
+  "branches" : [
+    {
+      "branch"      : <branch name>,
+      "rpc"         : <rpc interface>,
+      "fingerprint" : <fingerprint>
+    },
+    ...
+  ],
+  "tree" : [
+    {
+      "path"      : <path>,
+      "branch"    : <branch name>,
+      "writeable" : <writable flag>
+    },
+    ...
+  ]
+}
+```
+On the console type 'q' to exit and 'help' to get an overview of available commands:
+```
+Available commands:
+----
+branch [branch name] [rpc] [fingerprint] : List all known branches or add a new branch to the tree
+cat <file>                               : Read and print the contents of a file
+cd [path]                                : Show or change the current directory
+checksum [path] [glob]                   : Show a directory listing and file checksums
+cp <src file/dir> <dst dir>              : Copy a file or directory
+dir [path] [glob]                        : Show a directory listing
+get <src file> [dst local file]          : Retrieve a file and store it locally (in the current directory)
+help [cmd]                               : Show general or command specific help
+mkdir <dir>                              : Create a new direectory
+mount [path] [branch name] [ro]          : List all mount points or add a new mount point to the tree
+ping <branch name> [rpc]                 : Ping a remote branch
+put [src local file] [dst file]          : Read a local file and store it
+refresh                                  : Refreshes all known branches and reconnect if possible.
+ren <file> <newfile>                     : Rename a file or directory
+reset [mounts|brances]                   : Remove all mounts or all mounts and all branches
+rm <file>                                : Delete a file or directory (* all files; ** all files/recursive)
+storeconfig [local file]                 : Store the current tree mapping in a local file
+sync <src dir> <dst dir>                 : Make sure dst has the same files and directories as src
+tree [path] [glob]                       : Show the listing of a directory and its subdirectories
+```
+
+### Configuration
+The Rufs client and server use each their own configuration file and require a shared `rufs.secret` file to be able to talk to each other. The server configuration is called `rufs.server.json`. After starting the server for the first time it should create a default configuration file. Available configurations are:
+
+| Configuration Option | Description |
+| --- | --- |
+| BranchName | Branch name which the server will export. |
+| EnableReadOnly | Export the branch only for read operations. |
+| LocalFolder | Local physical folder which is exported. |
+| RPCHost | RPC host for communication with clients. |
+| RPCPort | RPC port for communication with clients. |
+
+Note: It is not (and will never be) possible to access the REST API via HTTP.
+
+Building Rufs
+----------------
+To build Rufs from source you need to have Go installed (go >= 1.12):
+
+Create a directory, change into it and run:
+```
+git clone https://devt.de/krotik/rufs/ .
+```
+
+You can build Rufs's executable with:
+```
+go build -o rufs ./cli/...
+```
+
+Rufs also has a web interface which should be bundled with the executable. The bundled web interface in `web.zip` can be attached by running (assuming the `rufs` executable is in the same folder as the script):
+```
+./attach_webzip.sh
+```
+
+License
+-------
+Rufs source code is available under the [MIT License](/LICENSE).

api/about.go

@@ -0,0 +1,68 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+/*
+Package api contains the REST API for RUFS.
+
+/about
+
+Endpoint which returns an object with version information.
+
+{
+    api_versions : List of available API versions e.g. [ "v1" ]
+    product      : Name of the API provider (RUFS)
+    version:     : Version of the API provider
+}
+*/
+package api
+
+import (
+	"encoding/json"
+	"net/http"
+
+	"devt.de/krotik/rufs/config"
+)
+
+/*
+EndpointAbout is the about endpoint definition (rooted). Handles about/
+*/
+const EndpointAbout = APIRoot + "/about/"
+
+/*
+AboutEndpointInst creates a new endpoint handler.
+*/
+func AboutEndpointInst() RestEndpointHandler {
+	return &aboutEndpoint{}
+}
+
+/*
+aboutEndpoint is the handler object for about operations.
+*/
+type aboutEndpoint struct {
+	*DefaultEndpointHandler
+}
+
+/*
+HandleGET returns about data for the REST API.
+*/
+func (a *aboutEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+
+	data := map[string]interface{}{
+		"product": "RUFS",
+		"version": config.ProductVersion,
+	}
+
+	// Write data
+
+	w.Header().Set("content-type", "application/json; charset=utf-8")
+
+	ret := json.NewEncoder(w)
+	ret.Encode(data)
+}

api/about_test.go

@@ -0,0 +1,21 @@
+package api
+
+import (
+	"testing"
+
+	"devt.de/krotik/rufs/config"
+)
+
+func TestAboutEndpoint(t *testing.T) {
+
+	st, _, body, _ := sendTestRequest(testQueryURL+EndpointAbout, "GET", nil)
+
+	if st != "200 OK" || body != `
+{
+  "product": "RUFS",
+  "version": "`[1:]+config.ProductVersion+`"
+}` {
+		t.Error("Unexpected response:", st, body)
+		return
+	}
+}

api/rest.go

@@ -0,0 +1,274 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+package api
+
+import (
+	"crypto/tls"
+	"fmt"
+	"net/http"
+	"strings"
+	"sync"
+
+	"devt.de/krotik/rufs"
+)
+
+/*
+APIVersion is the version of the REST API
+*/
+const APIVersion = "1.0.0"
+
+/*
+APIRoot is the API root directory for the REST API
+*/
+const APIRoot = "/fs"
+
+/*
+APISchemes defines the supported schemes by the REST API
+*/
+var APISchemes = []string{"https"}
+
+/*
+APIHost is the host definition for the REST API
+*/
+var APIHost = "localhost:9040"
+
+/*
+RestEndpointInst models a factory function for REST endpoint handlers.
+*/
+type RestEndpointInst func() RestEndpointHandler
+
+/*
+GeneralEndpointMap is a map of urls to general REST endpoints
+*/
+var GeneralEndpointMap = map[string]RestEndpointInst{
+	EndpointAbout:   AboutEndpointInst,
+	EndpointSwagger: SwaggerEndpointInst,
+}
+
+/*
+RestEndpointHandler models a REST endpoint handler.
+*/
+type RestEndpointHandler interface {
+
+	/*
+		HandleGET handles a GET request.
+	*/
+	HandleGET(w http.ResponseWriter, r *http.Request, resources []string)
+
+	/*
+		HandlePOST handles a POST request.
+	*/
+	HandlePOST(w http.ResponseWriter, r *http.Request, resources []string)
+
+	/*
+		HandlePUT handles a PUT request.
+	*/
+	HandlePUT(w http.ResponseWriter, r *http.Request, resources []string)
+
+	/*
+		HandleDELETE handles a DELETE request.
+	*/
+	HandleDELETE(w http.ResponseWriter, r *http.Request, resources []string)
+
+	/*
+		SwaggerDefs is used to describe the endpoint in swagger.
+	*/
+	SwaggerDefs(s map[string]interface{})
+}
+
+/*
+trees is a map of all trees which can be used by the REST API
+*/
+var trees = make(map[string]*rufs.Tree)
+var treesLock = sync.Mutex{}
+
+/*
+ResetTrees removes all registered trees.
+*/
+var ResetTrees = func() {
+	treesLock.Lock()
+	defer treesLock.Unlock()
+
+	trees = make(map[string]*rufs.Tree)
+}
+
+/*
+Trees is a getter function which returns a map of all registered trees.
+This function can be overwritten by client code to implement access
+control.
+*/
+var Trees = func() (map[string]*rufs.Tree, error) {
+	treesLock.Lock()
+	defer treesLock.Unlock()
+
+	ret := make(map[string]*rufs.Tree)
+
+	for k, v := range trees {
+		ret[k] = v
+	}
+
+	return ret, nil
+}
+
+/*
+GetTree returns a specific tree. This function can be overwritten by
+client code to implement access control.
+*/
+var GetTree = func(id string) (*rufs.Tree, bool, error) {
+	treesLock.Lock()
+	defer treesLock.Unlock()
+
+	tree, ok := trees[id]
+
+	return tree, ok, nil
+}
+
+/*
+AddTree adds a new tree. This function can be overwritten by
+client code to implement access control.
+*/
+var AddTree = func(id string, tree *rufs.Tree) error {
+	treesLock.Lock()
+	defer treesLock.Unlock()
+
+	if _, ok := trees[id]; ok {
+		return fmt.Errorf("Tree %v already exists", id)
+	}
+
+	trees[id] = tree
+
+	return nil
+}
+
+/*
+RemoveTree removes a tree. This function can be overwritten by
+client code to implement access control.
+*/
+var RemoveTree = func(id string) error {
+	treesLock.Lock()
+	defer treesLock.Unlock()
+
+	if _, ok := trees[id]; !ok {
+		return fmt.Errorf("Tree %v does not exist", id)
+	}
+
+	delete(trees, id)
+
+	return nil
+}
+
+/*
+TreeConfigTemplate is the configuration which is used for newly created trees.
+*/
+var TreeConfigTemplate map[string]interface{}
+
+/*
+TreeCertTemplate is the certificate which is used for newly created trees.
+*/
+var TreeCertTemplate *tls.Certificate
+
+/*
+Map of all registered endpoint handlers.
+*/
+var registered = map[string]RestEndpointInst{}
+
+/*
+HandleFunc to use for registering handlers
+*/
+var HandleFunc = http.HandleFunc
+
+/*
+RegisterRestEndpoints registers all given REST endpoint handlers.
+*/
+func RegisterRestEndpoints(endpointInsts map[string]RestEndpointInst) {
+
+	for url, endpointInst := range endpointInsts {
+		registered[url] = endpointInst
+
+		HandleFunc(url, func() func(w http.ResponseWriter, r *http.Request) {
+			var handlerURL = url
+			var handlerInst = endpointInst
+
+			return func(w http.ResponseWriter, r *http.Request) {
+
+				// Create a new handler instance
+
+				handler := handlerInst()
+
+				// Handle request in appropriate method
+
+				res := strings.TrimSpace(r.URL.Path[len(handlerURL):])
+
+				if len(res) > 0 && res[len(res)-1] == '/' {
+					res = res[:len(res)-1]
+				}
+
+				var resources []string
+
+				if res != "" {
+					resources = strings.Split(res, "/")
+				}
+
+				switch r.Method {
+				case "GET":
+					handler.HandleGET(w, r, resources)
+
+				case "POST":
+					handler.HandlePOST(w, r, resources)
+
+				case "PUT":
+					handler.HandlePUT(w, r, resources)
+
+				case "DELETE":
+					handler.HandleDELETE(w, r, resources)
+
+				default:
+					http.Error(w, http.StatusText(http.StatusMethodNotAllowed),
+						http.StatusMethodNotAllowed)
+				}
+			}
+		}())
+	}
+}
+
+/*
+DefaultEndpointHandler is the default endpoint handler implementation.
+*/
+type DefaultEndpointHandler struct {
+}
+
+/*
+HandleGET handles a GET request.
+*/
+func (de *DefaultEndpointHandler) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+	http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+}
+
+/*
+HandlePOST handles a POST request.
+*/
+func (de *DefaultEndpointHandler) HandlePOST(w http.ResponseWriter, r *http.Request, resources []string) {
+	http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+}
+
+/*
+HandlePUT handles a PUT request.
+*/
+func (de *DefaultEndpointHandler) HandlePUT(w http.ResponseWriter, r *http.Request, resources []string) {
+	http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+}
+
+/*
+HandleDELETE handles a DELETE request.
+*/
+func (de *DefaultEndpointHandler) HandleDELETE(w http.ResponseWriter, r *http.Request, resources []string) {
+	http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+}

api/rest_test.go

@@ -0,0 +1,335 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+package api
+
+import (
+	"bytes"
+	"encoding/json"
+	"flag"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+	"strings"
+	"sync"
+	"testing"
+
+	"devt.de/krotik/common/httputil"
+	"devt.de/krotik/rufs"
+)
+
+const testPort = ":9040"
+
+var testQueryURL = "http://localhost" + testPort
+
+var lastRes []string
+
+type testEndpoint struct {
+	*DefaultEndpointHandler
+}
+
+/*
+handleSearchQuery handles a search query REST call.
+*/
+func (te *testEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+	lastRes = resources
+	te.DefaultEndpointHandler.HandleGET(w, r, resources)
+}
+
+func (te *testEndpoint) SwaggerDefs(s map[string]interface{}) {
+}
+
+var testEndpointMap = map[string]RestEndpointInst{
+	"/": func() RestEndpointHandler {
+		return &testEndpoint{}
+	},
+}
+
+func TestMain(m *testing.M) {
+	flag.Parse()
+
+	hs, wg := startServer()
+	if hs == nil {
+		return
+	}
+	defer stopServer(hs, wg)
+
+	RegisterRestEndpoints(testEndpointMap)
+	RegisterRestEndpoints(GeneralEndpointMap)
+
+	// Run the tests
+
+	res := m.Run()
+
+	// Teardown
+
+	stopServer(hs, wg)
+
+	os.Exit(res)
+}
+
+func TestTreeManagement(t *testing.T) {
+
+	if err := AddTree("1", &rufs.Tree{}); err != nil {
+		t.Error(err)
+		return
+	}
+
+	if err := AddTree("1", &rufs.Tree{}); err == nil || err.Error() != "Tree 1 already exists" {
+		t.Error(err)
+		return
+	}
+
+	if res, _ := Trees(); fmt.Sprint(res) != "map[1:/: ]" {
+		t.Error("Unexpected result: ", res)
+		return
+	}
+
+	if res, _, _ := GetTree("1"); res == nil {
+		t.Error("Unexpected result: ", res)
+		return
+	}
+
+	if err := RemoveTree("1"); err != nil {
+		t.Error(err)
+		return
+	}
+
+	if err := RemoveTree("1"); err == nil || err.Error() != "Tree 1 does not exist" {
+		t.Error(err)
+		return
+	}
+
+	if err := AddTree("1", &rufs.Tree{}); err != nil {
+		t.Error(err)
+		return
+	}
+
+	ResetTrees()
+
+	if res, _ := Trees(); fmt.Sprint(res) != "map[]" {
+		t.Error("Unexpected result: ", res)
+		return
+	}
+}
+
+func TestEndpointHandling(t *testing.T) {
+
+	lastRes = nil
+
+	if _, _, res, _ := sendTestRequest(testQueryURL, "GET", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	if lastRes != nil {
+		t.Error("Unexpected lastRes:", lastRes)
+	}
+
+	lastRes = nil
+
+	if _, _, res, _ := sendTestRequest(testQueryURL+"/foo/bar", "GET", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	if fmt.Sprint(lastRes) != "[foo bar]" {
+		t.Error("Unexpected lastRes:", lastRes)
+	}
+
+	lastRes = nil
+
+	if _, _, res, _ := sendTestRequest(testQueryURL+"/foo/bar/", "GET", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	if fmt.Sprint(lastRes) != "[foo bar]" {
+		t.Error("Unexpected lastRes:", lastRes)
+	}
+
+	if _, _, res, _ := sendTestRequest(testQueryURL, "POST", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	if _, _, res, _ := sendTestRequest(testQueryURL, "PUT", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	if _, _, res, _ := sendTestRequest(testQueryURL, "DELETE", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	if _, _, res, _ := sendTestRequest(testQueryURL, "UPDATE", nil); res != "Method Not Allowed" {
+		t.Error("Unexpected response:", res)
+		return
+	}
+
+	// Test swagger endpoint
+
+	if _, _, res, _ := sendTestRequest(testQueryURL+"/fs/swagger.json", "GET", nil); res != `
+{
+  "basePath": "/fs",
+  "definitions": {
+    "Error": {
+      "description": "A human readable error mesage.",
+      "type": "string"
+    }
+  },
+  "host": "localhost:9040",
+  "info": {
+    "description": "Query and control the Remote Union File System.",
+    "title": "RUFS API",
+    "version": "1.0.0"
+  },
+  "paths": {
+    "/about": {
+      "get": {
+        "description": "Returns available API versions, product name and product version.",
+        "produces": [
+          "application/json"
+        ],
+        "responses": {
+          "200": {
+            "description": "About info object",
+            "schema": {
+              "properties": {
+                "api_versions": {
+                  "description": "List of available API versions.",
+                  "items": {
+                    "description": "Available API version.",
+                    "type": "string"
+                  },
+                  "type": "array"
+                },
+                "product": {
+                  "description": "Product name of the REST API provider.",
+                  "type": "string"
+                },
+                "version": {
+                  "description": "Version of the REST API provider.",
+                  "type": "string"
+                }
+              },
+              "type": "object"
+            }
+          },
+          "default": {
+            "description": "Error response",
+            "schema": {
+              "$ref": "#/definitions/Error"
+            }
+          }
+        },
+        "summary": "Return information about the REST API provider."
+      }
+    }
+  },
+  "produces": [
+    "application/json"
+  ],
+  "schemes": [
+    "https"
+  ],
+  "swagger": "2.0"
+}`[1:] {
+		t.Error("Unexpected response:", res)
+		return
+	}
+}
+
+/*
+Send a request to a HTTP test server
+*/
+func sendTestRequest(url string, method string, content []byte) (string, http.Header, string, *http.Response) {
+	var req *http.Request
+	var err error
+
+	if content != nil {
+		req, err = http.NewRequest(method, url, bytes.NewBuffer(content))
+	} else {
+		req, err = http.NewRequest(method, url, nil)
+	}
+
+	if err != nil {
+		panic(err)
+	}
+
+	req.Header.Set("Content-Type", "application/json")
+
+	client := &http.Client{}
+	resp, err := client.Do(req)
+	if err != nil {
+		panic(err)
+	}
+	defer resp.Body.Close()
+
+	body, _ := ioutil.ReadAll(resp.Body)
+	bodyStr := strings.Trim(string(body), " \n")
+
+	// Try json decoding first
+
+	out := bytes.Buffer{}
+	err = json.Indent(&out, []byte(bodyStr), "", "  ")
+	if err == nil {
+		return resp.Status, resp.Header, out.String(), resp
+	}
+
+	// Just return the body
+
+	return resp.Status, resp.Header, bodyStr, resp
+}
+
+/*
+Start a HTTP test server.
+*/
+func startServer() (*httputil.HTTPServer, *sync.WaitGroup) {
+	hs := &httputil.HTTPServer{}
+
+	var wg sync.WaitGroup
+	wg.Add(1)
+
+	go hs.RunHTTPServer(testPort, &wg)
+
+	wg.Wait()
+
+	// Server is started
+
+	if hs.LastError != nil {
+		panic(hs.LastError)
+	}
+
+	return hs, &wg
+}
+
+/*
+Stop a started HTTP test server.
+*/
+func stopServer(hs *httputil.HTTPServer, wg *sync.WaitGroup) {
+
+	if hs.Running == true {
+
+		wg.Add(1)
+
+		// Server is shut down
+
+		hs.Shutdown()
+
+		wg.Wait()
+
+	} else {
+
+		panic("Server was not running as expected")
+	}
+}

api/swagger.go

@@ -0,0 +1,139 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+package api
+
+import (
+	"encoding/json"
+	"net/http"
+)
+
+/*
+SwaggerDefs is used to describe the endpoint in swagger.
+*/
+func (a *aboutEndpoint) SwaggerDefs(s map[string]interface{}) {
+
+	// Add query paths
+
+	s["paths"].(map[string]interface{})["/about"] = map[string]interface{}{
+		"get": map[string]interface{}{
+			"summary":     "Return information about the REST API provider.",
+			"description": "Returns available API versions, product name and product version.",
+			"produces": []string{
+				"application/json",
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "About info object",
+					"schema": map[string]interface{}{
+						"type": "object",
+						"properties": map[string]interface{}{
+							"api_versions": map[string]interface{}{
+								"description": "List of available API versions.",
+								"type":        "array",
+								"items": map[string]interface{}{
+									"description": "Available API version.",
+									"type":        "string",
+								},
+							},
+							"product": map[string]interface{}{
+								"description": "Product name of the REST API provider.",
+								"type":        "string",
+							},
+							"version": map[string]interface{}{
+								"description": "Version of the REST API provider.",
+								"type":        "string",
+							},
+						},
+					},
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	// Add generic error object to definition
+
+	s["definitions"].(map[string]interface{})["Error"] = map[string]interface{}{
+		"description": "A human readable error mesage.",
+		"type":        "string",
+	}
+}
+
+/*
+EndpointSwagger is the swagger endpoint URL (rooted). Handles swagger.json/
+*/
+const EndpointSwagger = APIRoot + "/swagger.json/"
+
+/*
+SwaggerEndpointInst creates a new endpoint handler.
+*/
+func SwaggerEndpointInst() RestEndpointHandler {
+	return &swaggerEndpoint{}
+}
+
+/*
+Handler object for swagger operations.
+*/
+type swaggerEndpoint struct {
+	*DefaultEndpointHandler
+}
+
+/*
+HandleGET returns the swagger definition of the REST API.
+*/
+func (a *swaggerEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+
+	// Add general sections
+
+	data := map[string]interface{}{
+		"swagger":     "2.0",
+		"host":        APIHost,
+		"schemes":     APISchemes,
+		"basePath":    APIRoot,
+		"produces":    []string{"application/json"},
+		"paths":       map[string]interface{}{},
+		"definitions": map[string]interface{}{},
+	}
+
+	// Go through all registered components and let them add their definitions
+
+	a.SwaggerDefs(data)
+
+	for _, inst := range registered {
+		inst().SwaggerDefs(data)
+	}
+
+	// Write data
+
+	w.Header().Set("content-type", "application/json; charset=utf-8")
+
+	ret := json.NewEncoder(w)
+	ret.Encode(data)
+}
+
+/*
+SwaggerDefs is used to describe the endpoint in swagger.
+*/
+func (a *swaggerEndpoint) SwaggerDefs(s map[string]interface{}) {
+
+	// Add general application information
+
+	s["info"] = map[string]interface{}{
+		"title":       "RUFS API",
+		"description": "Query and control the Remote Union File System.",
+		"version":     APIVersion,
+	}
+}

api/v1/admin.go

@@ -0,0 +1,543 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+/*
+Package v1 contains Rufs REST API Version 1.
+
+Admin control endpoint
+
+/admin
+
+The admin endpoint can be used for various admin tasks such as registering
+new branches or mounting known branches.
+
+A GET request to the admin endpoint returns the current tree
+configuration; an object of all known branches and the current mapping:
+
+	{
+	    branches : [ <known branches> ],
+	    tree  : [ <current mapping> ]
+	}
+
+A POST request to the admin endpoint creates a new tree. The body of
+the request should have the following form:
+
+	"<name>"
+
+/admin/<tree>
+
+A DELETE request to a particular tree will delete the tree.
+
+/admin/<tree>/branch
+
+A new branch can be created in an existing tree by sending a POST request
+to the branch endpoint. The body of the request should have the following
+form:
+
+	{
+	    branch : <Name of the branch>,
+	    rpc : <RPC definition of the remote branch (e.g. localhost:9020)>,
+	    fingerprint : <Expected SSL fingerprint of the remote branch or an empty string>
+	}
+
+/admin/<tree>/mapping
+
+A new mapping can be created in an existing tree by sending a POST request
+to the mapping endpoint. The body of the request should have the following
+form:
+
+	{
+	    branch : <Name of the branch>,
+	    dir : <Tree directory of the branch root>,
+	    writable : <Flag if the branch should handle write operations>
+	}
+
+
+Dir listing endpoing
+
+/dir/<tree>/<path>
+
+The dir endpoing handles requests for the directory listing of a certain
+path. A request url should be of the following form:
+
+/dir/<tree>/<path>?recursive=<flag>&checksums=<flag>
+
+The request can optionally include the flag parameters (value should
+be 1 or 0) recursive and checksums. The recursive flag will add all
+subdirectories to the listing and the checksums flag will add
+checksums for all listed files.
+
+
+File queries and manipulation
+
+/file/{tree}/{path}
+
+A GET request to a specific file will return its contents. A POST will
+upload a new or overwrite an existing file. A DELETE request will delete
+an existing file.
+
+New files are expected to be uploaded using a multipart/form-data request.
+When uploading a new file the form field for the file should be named
+"uploadfile". The form can optionally contain a redirect field which
+will issue a redirect once the file has been uploaded.
+
+A PUT request is used to perform a file operation. The request body
+should be a JSON object of the form (parameters are operation specific):
+
+	{
+	    action : <Action to perform>,
+		files : <List of (full path) files which should be copied / renamed>
+	    newname : <New name of file (when renaming)>,
+		newnames : <List of new file names when renaming multiple files using
+					the files parameter>,
+	    destination : <Destination file when copying a single file - Destination
+						directory when copying multiple files using the files
+						parameter or syncing directories>
+	}
+
+The action can either be: sync, rename, mkdir or copy. Copy and sync returns a JSON
+structure containing a progress id:
+
+	{
+	    progress_id : <Id for progress of the copy operation>
+	}
+
+
+Progress information
+
+/progress/<progress id>
+
+A GET request to the progress endpoint returns the current progress of
+an ongoing operation. The result should be:
+
+	{
+	    "item": <Currently processing item>,
+	    "operation": <Name of operation>,
+	    "progress": <Current progress>,
+	    "subject": <Name of the subject on which the operation is performed>,
+	    "total_items": <Total number of items>,
+	    "total_progress": <Total progress>
+	}
+
+
+Create zip files
+
+/zip/<tree>
+
+A post to the zip enpoint returns a zip file containing requested files. The
+files to include must be given as a list of file name with full path in the body.
+The body should be application/x-www-form-urlencoded encoded. The list should
+be a JSON encoded string as value of the value files. The body should have the
+following form:
+
+	files=[ "<file1>", "<file2>" ]
+*/
+package v1
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"strconv"
+
+	"devt.de/krotik/rufs"
+	"devt.de/krotik/rufs/api"
+)
+
+/*
+EndpointAdmin is the mount endpoint URL (rooted). Handles everything
+under admin/...
+*/
+const EndpointAdmin = api.APIRoot + APIv1 + "/admin/"
+
+/*
+AdminEndpointInst creates a new endpoint handler.
+*/
+func AdminEndpointInst() api.RestEndpointHandler {
+	return &adminEndpoint{}
+}
+
+/*
+Handler object for admin operations.
+*/
+type adminEndpoint struct {
+	*api.DefaultEndpointHandler
+}
+
+/*
+HandleGET handles an admin query REST call.
+*/
+func (a *adminEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+	data := make(map[string]interface{})
+
+	trees, err := api.Trees()
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	refreshName := r.URL.Query().Get("refresh")
+
+	for k, v := range trees {
+		var tree map[string]interface{}
+
+		if refreshName != "" && k == refreshName {
+			v.Refresh()
+		}
+
+		json.Unmarshal([]byte(v.Config()), &tree)
+		data[k] = tree
+	}
+
+	// Write data
+
+	w.Header().Set("content-type", "application/json; charset=utf-8")
+	json.NewEncoder(w).Encode(data)
+}
+
+/*
+HandlePOST handles REST calls to create a new tree.
+*/
+func (a *adminEndpoint) HandlePOST(w http.ResponseWriter, r *http.Request, resources []string) {
+	var tree *rufs.Tree
+	var ok bool
+	var err error
+	var data map[string]interface{}
+
+	if len(resources) == 0 {
+		var name string
+
+		if err := json.NewDecoder(r.Body).Decode(&name); err != nil {
+			http.Error(w, fmt.Sprintf("Could not decode request body: %v", err.Error()),
+				http.StatusBadRequest)
+			return
+		} else if name == "" {
+			http.Error(w, fmt.Sprintf("Body must contain the tree name as a non-empty JSON string"),
+				http.StatusBadRequest)
+			return
+		}
+
+		// Create a new tree
+
+		tree, err := rufs.NewTree(api.TreeConfigTemplate, api.TreeCertTemplate)
+		if err != nil {
+			http.Error(w, fmt.Sprintf("Could not create new tree: %v", err.Error()),
+				http.StatusBadRequest)
+			return
+		}
+
+		// Store the new tree
+
+		if err := api.AddTree(name, tree); err != nil {
+			http.Error(w, fmt.Sprintf("Could not add new tree: %v", err.Error()),
+				http.StatusBadRequest)
+		}
+
+		return
+	}
+
+	if !checkResources(w, resources, 2, 2, "Need a tree name and a section (either branches or mapping)") {
+		return
+	}
+
+	if tree, ok, err = api.GetTree(resources[0]); err == nil && !ok {
+		err = fmt.Errorf("Unknown tree: %v", resources[0])
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	if err := json.NewDecoder(r.Body).Decode(&data); err != nil {
+		http.Error(w, fmt.Sprintf("Could not decode request body: %v", err.Error()),
+			http.StatusBadRequest)
+		return
+	}
+
+	if resources[1] == "branch" {
+
+		// Add a new branch
+
+		if rpc, ok := getMapValue(w, data, "rpc"); ok {
+			if branch, ok := getMapValue(w, data, "branch"); ok {
+				if fingerprint, ok := getMapValue(w, data, "fingerprint"); ok {
+
+					if err := tree.AddBranch(branch, rpc, fingerprint); err != nil {
+						http.Error(w, fmt.Sprintf("Could not add branch: %v", err.Error()),
+							http.StatusBadRequest)
+					}
+				}
+			}
+		}
+
+	} else if resources[1] == "mapping" {
+
+		// Add a new mapping
+
+		if _, ok := data["dir"]; ok {
+
+			if dir, ok := getMapValue(w, data, "dir"); ok {
+				if branch, ok := getMapValue(w, data, "branch"); ok {
+					if writeableStr, ok := getMapValue(w, data, "writeable"); ok {
+
+						writeable, err := strconv.ParseBool(writeableStr)
+
+						if err != nil {
+							http.Error(w, fmt.Sprintf("Writeable value must be a boolean: %v", err.Error()),
+								http.StatusBadRequest)
+
+						} else if err := tree.AddMapping(dir, branch, writeable); err != nil {
+
+							http.Error(w, fmt.Sprintf("Could not add branch: %v", err.Error()),
+								http.StatusBadRequest)
+						}
+					}
+				}
+			}
+		}
+	}
+}
+
+/*
+HandleDELETE handles REST calls to delete an existing tree.
+*/
+func (a *adminEndpoint) HandleDELETE(w http.ResponseWriter, r *http.Request, resources []string) {
+
+	if !checkResources(w, resources, 1, 1, "Need a tree name") {
+		return
+	}
+
+	// Delete the tree
+
+	if err := api.RemoveTree(resources[0]); err != nil {
+		http.Error(w, fmt.Sprintf("Could not remove tree: %v", err.Error()),
+			http.StatusBadRequest)
+	}
+}
+
+/*
+SwaggerDefs is used to describe the endpoint in swagger.
+*/
+func (a *adminEndpoint) SwaggerDefs(s map[string]interface{}) {
+
+	s["paths"].(map[string]interface{})["/v1/admin"] = map[string]interface{}{
+		"get": map[string]interface{}{
+			"summary":     "Return all current tree configurations.",
+			"description": "All current tree configurations; each object has a list of all known branches and the current mapping.",
+			"produces": []string{
+				"text/plain",
+				"application/json",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "refresh",
+					"in":          "query",
+					"description": "Refresh a particular tree (reload branches and mappings).",
+					"required":    false,
+					"type":        "string",
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "A key-value map of tree name to tree configuration",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+		"post": map[string]interface{}{
+			"summary":     "Create a new tree.",
+			"description": "Create a new named tree.",
+			"consumes": []string{
+				"application/json",
+			},
+			"produces": []string{
+				"text/plain",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "data",
+					"in":          "body",
+					"description": "Name of the new tree.",
+					"required":    true,
+					"schema": map[string]interface{}{
+						"type": "string",
+					},
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns an empty body if successful.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	s["paths"].(map[string]interface{})["/v1/admin/{tree}"] = map[string]interface{}{
+		"delete": map[string]interface{}{
+			"summary":     "Delete a tree.",
+			"description": "Delete a named tree.",
+			"produces": []string{
+				"text/plain",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns an empty body if successful.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	s["paths"].(map[string]interface{})["/v1/admin/{tree}/branch"] = map[string]interface{}{
+		"post": map[string]interface{}{
+			"summary":     "Add a new branch.",
+			"description": "Add a new remote branch to the tree.",
+			"consumes": []string{
+				"application/json",
+			},
+			"produces": []string{
+				"text/plain",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "data",
+					"in":          "body",
+					"description": "Definition of the new branch.",
+					"required":    true,
+					"schema": map[string]interface{}{
+						"type": "object",
+						"properties": map[string]interface{}{
+							"branch": map[string]interface{}{
+								"description": "Name of the remote branch (must match on the remote branch).",
+								"type":        "string",
+							},
+							"rpc": map[string]interface{}{
+								"description": "RPC definition of the remote branch (e.g. localhost:9020).",
+								"type":        "string",
+							},
+							"fingerprint": map[string]interface{}{
+								"description": "Expected SSL fingerprint of the remote branch (shown during startup) or an empty string.",
+								"type":        "string",
+							},
+						},
+					},
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns an empty body if successful.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	s["paths"].(map[string]interface{})["/v1/admin/{tree}/mapping"] = map[string]interface{}{
+		"post": map[string]interface{}{
+			"summary":     "Add a new mapping.",
+			"description": "Add a new mapping to the tree.",
+			"consumes": []string{
+				"application/json",
+			},
+			"produces": []string{
+				"text/plain",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "data",
+					"in":          "body",
+					"description": "Definition of the new branch.",
+					"required":    true,
+					"schema": map[string]interface{}{
+						"type": "object",
+						"properties": map[string]interface{}{
+							"branch": map[string]interface{}{
+								"description": "Name of the known remote branch.",
+								"type":        "string",
+							},
+							"dir": map[string]interface{}{
+								"description": "Tree directory which should hold the branch root.",
+								"type":        "string",
+							},
+							"writable": map[string]interface{}{
+								"description": "Flag if the branch should be mapped as writable.",
+								"type":        "string",
+							},
+						},
+					},
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns an empty body if successful.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	// Add generic error object to definition
+
+	s["definitions"].(map[string]interface{})["Error"] = map[string]interface{}{
+		"description": "A human readable error mesage.",
+		"type":        "string",
+	}
+}

api/v1/admin_test.go

@@ -0,0 +1,283 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+package v1
+
+import (
+	"fmt"
+	"testing"
+
+	"devt.de/krotik/rufs"
+	"devt.de/krotik/rufs/api"
+	"devt.de/krotik/rufs/config"
+)
+
+func TestAdminQuery(t *testing.T) {
+	queryURL := "http://localhost" + TESTPORT + EndpointAdmin
+
+	defer func() {
+
+		// Make sure all trees are removed
+
+		api.ResetTrees()
+	}()
+
+	// In the beginning there should be no trees
+
+	st, _, res := sendTestRequest(queryURL, "GET", nil)
+	if st != "200 OK" || res != "{}" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Create a new tree
+
+	st, _, res = sendTestRequest(queryURL, "POST", []byte("\"Hans1\""))
+	if st != "200 OK" || res != "" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Check a new tree was created
+
+	st, _, res = sendTestRequest(queryURL+"?refresh=Hans1", "GET", nil)
+	if st != "200 OK" || res != `
+{
+  "Hans1": {
+    "branches": [],
+    "tree": []
+  }
+}`[1:] {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Add a new branch
+
+	fooRPC := fmt.Sprintf("%v:%v", branchConfigs["footest"][config.RPCHost], branchConfigs["footest"][config.RPCPort])
+	fooFP := footest.SSLFingerprint()
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/branch", "POST", []byte(fmt.Sprintf(`
+{
+	"branch" : "footest",
+	"rpc"    : %#v,
+	"fingerprint" : %#v
+}`, fooRPC, fooFP)))
+	if st != "200 OK" || res != "" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Check the branch was added
+
+	st, _, res = sendTestRequest(queryURL, "GET", nil)
+	if st != "200 OK" || res != fmt.Sprintf(`
+{
+  "Hans1": {
+    "branches": [
+      {
+        "branch": "footest",
+        "fingerprint": %#v,
+        "rpc": %#v
+      }
+    ],
+    "tree": []
+  }
+}`[1:], fooFP, fooRPC) {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Add a new mapping
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/mapping", "POST", []byte(`
+{
+	"dir" : "/",
+	"branch" : "footest",
+	"writeable" : false
+}`))
+	if st != "200 OK" || res != "" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Check the mapping was added
+
+	st, _, res = sendTestRequest(queryURL, "GET", nil)
+	if st != "200 OK" || res != fmt.Sprintf(`
+{
+  "Hans1": {
+    "branches": [
+      {
+        "branch": "footest",
+        "fingerprint": %#v,
+        "rpc": %#v
+      }
+    ],
+    "tree": [
+      {
+        "branch": "footest",
+        "path": "/",
+        "writeable": false
+      }
+    ]
+  }
+}`[1:], fooFP, fooRPC) {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Test error cases
+
+	st, _, res = sendTestRequest(queryURL, "POST", []byte(`""`))
+	if st != "400 Bad Request" || res != "Body must contain the tree name as a non-empty JSON string" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL, "POST", []byte(`x`))
+	if st != "400 Bad Request" || res != "Could not decode request body: invalid character 'x' looking for beginning of value" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	origConfig := api.TreeConfigTemplate
+	api.TreeConfigTemplate = nil
+
+	st, _, res = sendTestRequest(queryURL, "POST", []byte(`"xx"`))
+	if st != "400 Bad Request" || res != "Could not create new tree: Missing TreeSecret key in tree config" {
+		api.TreeConfigTemplate = origConfig
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	api.TreeConfigTemplate = origConfig
+
+	// Test error cases
+
+	origTrees := api.Trees
+
+	api.Trees = func() (map[string]*rufs.Tree, error) {
+		return nil, fmt.Errorf("Testerror")
+	}
+
+	st, _, res = sendTestRequest(queryURL, "GET", nil)
+	if st != "400 Bad Request" || res != "Testerror" {
+		api.Trees = origTrees
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	api.Trees = origTrees
+
+	st, _, res = sendTestRequest(queryURL, "POST", []byte("\"Hans1\""))
+	if st != "400 Bad Request" || res != "Could not add new tree: Tree Hans1 already exists" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans2/mapping", "POST", nil)
+	if st != "400 Bad Request" || res != "Unknown tree: Hans2" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/mapping", "POST", []byte("aaa"))
+	if st != "400 Bad Request" || res != "Could not decode request body: invalid character 'a' looking for beginning of value" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans2/", "POST", []byte("aaa"))
+	if st != "400 Bad Request" || res != "Need a tree name and a section (either branches or mapping)" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/branch", "POST", []byte(fmt.Sprintf(`
+{
+	"branch" : "footest",
+	"rpc"    : %#v,
+	"fingerprint" : %#v
+}`, fooRPC, fooFP)))
+	if st != "400 Bad Request" || res != "Could not add branch: Peer already registered: footest" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/branch", "POST", []byte(fmt.Sprintf(`
+{
+	"branch" : "footest",
+	"rpc2"    : %#v,
+	"fingerprint" : %#v
+}`, fooRPC, fooFP)))
+	if st != "400 Bad Request" || res != "Value for rpc is missing in posted data" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/mapping", "POST", []byte(`
+{
+	"dir" : "/",
+	"branch" : "footest2",
+	"writeable" : false
+}`))
+	if st != "400 Bad Request" || res != "Could not add branch: Unknown target node" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/mapping", "POST", []byte(`
+{
+	"dir" : "/",
+	"branch" : "footest2",
+	"writeable" : "test"
+}`))
+	if st != "400 Bad Request" || res != "Writeable value must be a boolean: strconv.ParseBool: parsing \"test\": invalid syntax" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Delete twice
+
+	if trees, err := api.Trees(); len(trees) != 1 || err != nil {
+		t.Error("Unexpected result:", trees, err)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1", "DELETE", nil)
+	if st != "200 OK" || res != "" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL, "DELETE", nil)
+	if st != "400 Bad Request" || res != "Need a tree name" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/meyer", "DELETE", nil)
+	if st != "400 Bad Request" || res != "Invalid resource specification: meyer" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	if trees, err := api.Trees(); len(trees) != 0 || err != nil {
+		t.Error("Unexpected result:", trees, err)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1", "DELETE", nil)
+	if st != "400 Bad Request" || res != "Could not remove tree: Tree Hans1 does not exist" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+}

api/v1/dir.go

@@ -0,0 +1,177 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+package v1
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"os"
+	"path"
+	"strconv"
+
+	"devt.de/krotik/common/stringutil"
+	"devt.de/krotik/rufs"
+	"devt.de/krotik/rufs/api"
+)
+
+/*
+EndpointDir is the dir endpoint URL (rooted). Handles everything
+under dir/...
+*/
+const EndpointDir = api.APIRoot + APIv1 + "/dir/"
+
+/*
+DirEndpointInst creates a new endpoint handler.
+*/
+func DirEndpointInst() api.RestEndpointHandler {
+	return &dirEndpoint{}
+}
+
+/*
+Handler object for dir operations.
+*/
+type dirEndpoint struct {
+	*api.DefaultEndpointHandler
+}
+
+/*
+HandleGET handles a dir query REST call.
+*/
+func (d *dirEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+	var tree *rufs.Tree
+	var ok, checksums bool
+	var err error
+	var dirs []string
+	var fis [][]os.FileInfo
+
+	if len(resources) == 0 {
+		http.Error(w, "Need at least a tree name",
+			http.StatusBadRequest)
+		return
+	}
+
+	if tree, ok, err = api.GetTree(resources[0]); err == nil && !ok {
+		err = fmt.Errorf("Unknown tree: %v", resources[0])
+	}
+
+	if err == nil {
+		var rex string
+
+		glob := r.URL.Query().Get("glob")
+		recursive, _ := strconv.ParseBool(r.URL.Query().Get("recursive"))
+		checksums, _ = strconv.ParseBool(r.URL.Query().Get("checksums"))
+
+		if rex, err = stringutil.GlobToRegex(glob); err == nil {
+
+			dirs, fis, err = tree.Dir(path.Join(resources[1:]...), rex, recursive, checksums)
+		}
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	data := make(map[string]interface{})
+
+	for i, d := range dirs {
+		var flist []map[string]interface{}
+
+		fi := fis[i]
+
+		for _, f := range fi {
+			toAdd := map[string]interface{}{
+				"name":  f.Name(),
+				"size":  f.Size(),
+				"isdir": f.IsDir(),
+			}
+
+			if checksums {
+				toAdd["checksum"] = f.(*rufs.FileInfo).Checksum()
+			}
+
+			flist = append(flist, toAdd)
+		}
+
+		data[d] = flist
+	}
+
+	// Write data
+
+	w.Header().Set("content-type", "application/json; charset=utf-8")
+	json.NewEncoder(w).Encode(data)
+}
+
+/*
+SwaggerDefs is used to describe the endpoint in swagger.
+*/
+func (d *dirEndpoint) SwaggerDefs(s map[string]interface{}) {
+
+	s["paths"].(map[string]interface{})["/v1/dir/{tree}/{path}"] = map[string]interface{}{
+		"get": map[string]interface{}{
+			"summary":     "Read a directory.",
+			"description": "List the contents of a directory.",
+			"produces": []string{
+				"text/plain",
+				"application/json",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "path",
+					"in":          "path",
+					"description": "Directory path.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "recursive",
+					"in":          "query",
+					"description": "Add listings of subdirectories.",
+					"required":    false,
+					"type":        "boolean",
+				},
+				{
+					"name":        "checksums",
+					"in":          "query",
+					"description": "Include file checksums.",
+					"required":    false,
+					"type":        "boolean",
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns a map of directories with a list of files as values.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	// Add generic error object to definition
+
+	s["definitions"].(map[string]interface{})["Error"] = map[string]interface{}{
+		"description": "A human readable error mesage.",
+		"type":        "string",
+	}
+}

api/v1/dir_test.go

@@ -0,0 +1,158 @@
+package v1
+
+import (
+	"fmt"
+	"testing"
+
+	"devt.de/krotik/common/errorutil"
+	"devt.de/krotik/rufs"
+	"devt.de/krotik/rufs/api"
+	"devt.de/krotik/rufs/config"
+)
+
+func TestDirQuery(t *testing.T) {
+	adminQueryURL := "http://localhost" + TESTPORT + EndpointAdmin
+
+	queryURL := "http://localhost" + TESTPORT + EndpointDir
+
+	// Setup a tree
+
+	defer func() {
+
+		// Make sure all trees are removed
+
+		api.ResetTrees()
+	}()
+
+	tree, err := rufs.NewTree(api.TreeConfigTemplate, api.TreeCertTemplate)
+	errorutil.AssertOk(err)
+
+	api.AddTree("Hans1", tree)
+
+	fooRPC := fmt.Sprintf("%v:%v", branchConfigs["footest"][config.RPCHost], branchConfigs["footest"][config.RPCPort])
+	fooFP := footest.SSLFingerprint()
+
+	err = tree.AddBranch("footest", fooRPC, fooFP)
+	errorutil.AssertOk(err)
+
+	err = tree.AddMapping("/", "footest", false)
+	errorutil.AssertOk(err)
+
+	st, _, res := sendTestRequest(adminQueryURL, "GET", nil)
+	if st != "200 OK" || res != fmt.Sprintf(`
+{
+  "Hans1": {
+    "branches": [
+      {
+        "branch": "footest",
+        "fingerprint": %#v,
+        "rpc": %#v
+      }
+    ],
+    "tree": [
+      {
+        "branch": "footest",
+        "path": "/",
+        "writeable": false
+      }
+    ]
+  }
+}`[1:], fooFP, fooRPC) {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Get a directory listing
+
+	st, _, res = sendTestRequest(queryURL+"Hans1", "GET", nil)
+	if st != "200 OK" || res != `
+{
+  "/": [
+    {
+      "isdir": true,
+      "name": "sub1",
+      "size": 4096
+    },
+    {
+      "isdir": false,
+      "name": "test1",
+      "size": 10
+    },
+    {
+      "isdir": false,
+      "name": "test2",
+      "size": 10
+    }
+  ]
+}`[1:] {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"Hans1/sub1", "GET", nil)
+	if st != "200 OK" || res != `
+{
+  "/sub1": [
+    {
+      "isdir": false,
+      "name": "test3",
+      "size": 17
+    }
+  ]
+}`[1:] {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Test recursive with checksums
+
+	st, _, res = sendTestRequest(queryURL+"Hans1?recursive=TRUE&checksums=1", "GET", nil)
+	if st != "200 OK" || res != `
+{
+  "/": [
+    {
+      "checksum": "",
+      "isdir": true,
+      "name": "sub1",
+      "size": 4096
+    },
+    {
+      "checksum": "73b8af47",
+      "isdir": false,
+      "name": "test1",
+      "size": 10
+    },
+    {
+      "checksum": "b0c1fadd",
+      "isdir": false,
+      "name": "test2",
+      "size": 10
+    }
+  ],
+  "/sub1": [
+    {
+      "checksum": "f89782b1",
+      "isdir": false,
+      "name": "test3",
+      "size": 17
+    }
+  ]
+}`[1:] {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	// Test error cases
+
+	st, _, res = sendTestRequest(queryURL, "GET", nil)
+	if st != "400 Bad Request" || res != "Need at least a tree name" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+
+	st, _, res = sendTestRequest(queryURL+"dave", "GET", nil)
+	if st != "400 Bad Request" || res != "Unknown tree: dave" {
+		t.Error("Unexpected response:", st, res)
+		return
+	}
+}

api/v1/file.go

@@ -0,0 +1,839 @@
+/*
+ * Rufs - Remote Union File System
+ *
+ * Copyright 2017 Matthias Ladkau. All rights reserved.
+ *
+ * This Source Code Form is subject to the terms of the MIT
+ * License, If a copy of the MIT License was not distributed with this
+ * file, You can obtain one at https://opensource.org/licenses/MIT.
+ */
+
+package v1
+
+import (
+	"encoding/json"
+	"fmt"
+	"mime/multipart"
+	"net/http"
+	"path"
+	"time"
+
+	"devt.de/krotik/common/cryptutil"
+	"devt.de/krotik/common/datautil"
+	"devt.de/krotik/common/errorutil"
+	"devt.de/krotik/common/httputil"
+	"devt.de/krotik/rufs"
+	"devt.de/krotik/rufs/api"
+)
+
+// Progress endpoint
+// =================
+
+/*
+Progress is a persisted data structure which contains the current
+progress of an ongoing operation.
+*/
+type Progress struct {
+	Op            string   // Operation which we show progress of
+	Subject       string   // Subject on which the operation is performed
+	Progress      int64    // Current progress of the ongoing operation (this is reset for each item)
+	TotalProgress int64    // Total progress required until current operation is finished
+	Item          int64    // Current processing item
+	TotalItems    int64    // Total number of items to process
+	Errors        []string // Any error messages
+}
+
+/*
+JSONString returns the progress object as a JSON string.
+*/
+func (p *Progress) JSONString() []byte {
+	ret, err := json.MarshalIndent(map[string]interface{}{
+		"operation":      p.Op,
+		"subject":        p.Subject,
+		"progress":       p.Progress,
+		"total_progress": p.TotalProgress,
+		"item":           p.Item,
+		"total_items":    p.TotalItems,
+		"errors":         p.Errors,
+	}, "", "    ")
+	errorutil.AssertOk(err)
+	return ret
+}
+
+/*
+ProgressMap contains information about copy progress.
+*/
+var ProgressMap = datautil.NewMapCache(100, 0)
+
+/*
+EndpointProgress is the progress endpoint URL (rooted). Handles everything
+under progress/...
+*/
+const EndpointProgress = api.APIRoot + APIv1 + "/progress/"
+
+/*
+ProgressEndpointInst creates a new endpoint handler.
+*/
+func ProgressEndpointInst() api.RestEndpointHandler {
+	return &progressEndpoint{}
+}
+
+/*
+Handler object for progress operations.
+*/
+type progressEndpoint struct {
+	*api.DefaultEndpointHandler
+}
+
+/*
+HandleGET handles a progress query REST call.
+*/
+func (f *progressEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+	var ok bool
+	var err error
+
+	if len(resources) < 2 {
+		http.Error(w, "Need a tree name and a progress ID",
+			http.StatusBadRequest)
+		return
+	}
+
+	if _, ok, err = api.GetTree(resources[0]); err == nil && !ok {
+		err = fmt.Errorf("Unknown tree: %v", resources[0])
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	p, ok := ProgressMap.Get(resources[0] + "#" + resources[1])
+
+	if !ok {
+		http.Error(w, fmt.Sprintf("Unknown progress ID: %v", resources[1]),
+			http.StatusBadRequest)
+		return
+	}
+
+	w.Header().Set("content-type", "application/octet-stream")
+	w.Write(p.(*Progress).JSONString())
+}
+
+/*
+SwaggerDefs is used to describe the endpoint in swagger.
+*/
+func (f *progressEndpoint) SwaggerDefs(s map[string]interface{}) {
+
+	s["paths"].(map[string]interface{})["/v1/progress/{tree}/{progress_id}"] = map[string]interface{}{
+		"get": map[string]interface{}{
+			"summary":     "Request progress update.",
+			"description": "Return a progress object showing the progress of an ongoing operation.",
+			"produces": []string{
+				"text/plain",
+				"application/json",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "progress_id",
+					"in":          "path",
+					"description": "Id of progress object.",
+					"required":    true,
+					"type":        "string",
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns the requested progress object.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+	}
+
+	// Add generic error object to definition
+
+	s["definitions"].(map[string]interface{})["Error"] = map[string]interface{}{
+		"description": "A human readable error mesage.",
+		"type":        "string",
+	}
+}
+
+// File endpoint
+// =============
+
+/*
+EndpointFile is the file endpoint URL (rooted). Handles everything
+under file/...
+*/
+const EndpointFile = api.APIRoot + APIv1 + "/file/"
+
+/*
+FileEndpointInst creates a new endpoint handler.
+*/
+func FileEndpointInst() api.RestEndpointHandler {
+	return &fileEndpoint{}
+}
+
+/*
+Handler object for file operations.
+*/
+type fileEndpoint struct {
+	*api.DefaultEndpointHandler
+}
+
+/*
+HandleGET handles a file query REST call.
+*/
+func (f *fileEndpoint) HandleGET(w http.ResponseWriter, r *http.Request, resources []string) {
+	var tree *rufs.Tree
+	var ok bool
+	var err error
+
+	if len(resources) < 2 {
+		http.Error(w, "Need a tree name and a file path",
+			http.StatusBadRequest)
+		return
+	}
+
+	if tree, ok, err = api.GetTree(resources[0]); err == nil && !ok {
+		err = fmt.Errorf("Unknown tree: %v", resources[0])
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	w.Header().Set("content-type", "application/octet-stream")
+
+	if err := tree.ReadFileToBuffer(path.Join(resources[1:]...), w); err != nil {
+		http.Error(w, fmt.Sprintf("Could not read file %v: %v", path.Join(resources[1:]...), err.Error()),
+			http.StatusBadRequest)
+		return
+	}
+}
+
+/*
+HandlePUT handles REST calls to modify / copy existing files.
+*/
+func (f *fileEndpoint) HandlePUT(w http.ResponseWriter, r *http.Request, resources []string) {
+	f.handleFileOp("PUT", w, r, resources)
+}
+
+/*
+HandleDELETE handles REST calls to delete existing files.
+*/
+func (f *fileEndpoint) HandleDELETE(w http.ResponseWriter, r *http.Request, resources []string) {
+	f.handleFileOp("DELETE", w, r, resources)
+}
+
+func (f *fileEndpoint) handleFileOp(requestType string, w http.ResponseWriter, r *http.Request, resources []string) {
+	var action string
+	var data, ret map[string]interface{}
+	var tree *rufs.Tree
+	var ok bool
+	var err error
+	var files []string
+
+	if len(resources) < 1 {
+		http.Error(w, "Need a tree name and a file path",
+			http.StatusBadRequest)
+		return
+	} else if len(resources) == 1 {
+		resources = append(resources, "/")
+	}
+
+	if tree, ok, err = api.GetTree(resources[0]); err == nil && !ok {
+		err = fmt.Errorf("Unknown tree: %v", resources[0])
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	ret = make(map[string]interface{})
+
+	if requestType == "DELETE" {
+
+		// See if the request contains a body with a list of files
+
+		err = json.NewDecoder(r.Body).Decode(&files)
+
+	} else {
+
+		// Unless it is a delete request we need an action command
+
+		err = json.NewDecoder(r.Body).Decode(&data)
+
+		if err != nil {
+			http.Error(w, fmt.Sprintf("Could not decode request body: %v", err.Error()),
+				http.StatusBadRequest)
+			return
+		}
+
+		actionObj, ok := data["action"]
+
+		if !ok {
+			http.Error(w, fmt.Sprintf("Action command is missing from request body"),
+				http.StatusBadRequest)
+			return
+		}
+
+		action = fmt.Sprint(actionObj)
+	}
+
+	fullPath := path.Join(resources[1:]...)
+	if fullPath != "/" {
+		fullPath = "/" + fullPath
+	}
+
+	dir, file := path.Split(fullPath)
+
+	if requestType == "DELETE" {
+
+		if len(files) == 0 {
+			_, err = tree.ItemOp(dir, map[string]string{
+				rufs.ItemOpAction: rufs.ItemOpActDelete,
+				rufs.ItemOpName:   file,
+			})
+
+		} else {
+
+			// Delete the files given in the body
+
+			for _, f := range files {
+				dir, file := path.Split(f)
+
+				if err == nil {
+					_, err = tree.ItemOp(dir, map[string]string{
+						rufs.ItemOpAction: rufs.ItemOpActDelete,
+						rufs.ItemOpName:   file,
+					})
+				}
+			}
+		}
+
+	} else if action == "rename" {
+
+		if newNamesParam, ok := data["newnames"]; ok {
+
+			if newNames, ok := newNamesParam.([]interface{}); !ok {
+				err = fmt.Errorf("Parameter newnames must be a list of filenames")
+			} else {
+
+				if filesParam, ok := data["files"]; !ok {
+					err = fmt.Errorf("Parameter files is missing from request body")
+				} else {
+
+					if filesList, ok := filesParam.([]interface{}); !ok {
+						err = fmt.Errorf("Parameter files must be a list of files")
+					} else {
+
+						for i, f := range filesList {
+							dir, file := path.Split(fmt.Sprint(f))
+
+							if err == nil {
+								_, err = tree.ItemOp(dir, map[string]string{
+									rufs.ItemOpAction:  rufs.ItemOpActRename,
+									rufs.ItemOpName:    file,
+									rufs.ItemOpNewName: fmt.Sprint(newNames[i]),
+								})
+							}
+						}
+					}
+				}
+			}
+
+		} else {
+
+			newName, ok := data["newname"]
+
+			if !ok {
+				err = fmt.Errorf("Parameter newname is missing from request body")
+
+			} else {
+
+				_, err = tree.ItemOp(dir, map[string]string{
+					rufs.ItemOpAction:  rufs.ItemOpActRename,
+					rufs.ItemOpName:    file,
+					rufs.ItemOpNewName: fmt.Sprint(newName),
+				})
+			}
+		}
+
+	} else if action == "mkdir" {
+
+		_, err = tree.ItemOp(dir, map[string]string{
+			rufs.ItemOpAction: rufs.ItemOpActMkDir,
+			rufs.ItemOpName:   file,
+		})
+
+	} else if action == "copy" {
+
+		dest, ok := data["destination"]
+
+		if !ok {
+			err = fmt.Errorf("Parameter destination is missing from request body")
+
+		} else {
+
+			// Create file list
+
+			filesParam, hasFilesParam := data["files"]
+
+			if hasFilesParam {
+
+				if lf, ok := filesParam.([]interface{}); !ok {
+					err = fmt.Errorf("Parameter files must be a list of files")
+
+				} else {
+					files = make([]string, len(lf))
+
+					for i, f := range lf {
+						files[i] = fmt.Sprint(f)
+					}
+				}
+
+			} else {
+
+				files = []string{fullPath}
+			}
+
+			if err == nil {
+
+				// Create progress object
+
+				uuid := fmt.Sprintf("%x", cryptutil.GenerateUUID())
+				ret["progress_id"] = uuid
+
+				mapLookup := resources[0] + "#" + uuid
+
+				ProgressMap.Put(mapLookup, &Progress{
+					Op:            "Copy",
+					Subject:       "",
+					Progress:      0,
+					TotalProgress: 0,
+					Item:          0,
+					TotalItems:    int64(len(files)),
+					Errors:        []string{},
+				})
+
+				go func() {
+
+					err = tree.Copy(files, fmt.Sprint(dest),
+						func(file string, writtenBytes, totalBytes, currentFile, totalFiles int64) {
+
+							if p, ok := ProgressMap.Get(mapLookup); ok && writtenBytes > 0 {
+								p.(*Progress).Subject = file
+								p.(*Progress).Progress = writtenBytes
+								p.(*Progress).TotalProgress = totalBytes
+								p.(*Progress).Item = currentFile
+								p.(*Progress).TotalItems = totalFiles
+							}
+						})
+
+					if err != nil {
+						if p, ok := ProgressMap.Get(mapLookup); ok {
+							p.(*Progress).Errors = append(p.(*Progress).Errors, err.Error())
+						}
+					}
+				}()
+
+				// Wait a little bit so immediate errors are directly reported
+
+				time.Sleep(10 * time.Millisecond)
+			}
+		}
+
+	} else if action == "sync" {
+
+		dest, ok := data["destination"]
+
+		if !ok {
+			err = fmt.Errorf("Parameter destination is missing from request body")
+
+		} else {
+
+			uuid := fmt.Sprintf("%x", cryptutil.GenerateUUID())
+			ret["progress_id"] = uuid
+
+			mapLookup := resources[0] + "#" + uuid
+
+			ProgressMap.Put(mapLookup, &Progress{
+				Op:            "Sync",
+				Subject:       "",
+				Progress:      0,
+				TotalProgress: -1,
+				Item:          0,
+				TotalItems:    -1,
+				Errors:        []string{},
+			})
+
+			go func() {
+
+				err = tree.Sync(fullPath, fmt.Sprint(dest), true,
+					func(op, srcFile, dstFile string, writtenBytes, totalBytes, currentFile, totalFiles int64) {
+
+						if p, ok := ProgressMap.Get(mapLookup); ok && writtenBytes > 0 {
+
+							p.(*Progress).Op = op
+							p.(*Progress).Subject = srcFile
+							p.(*Progress).Progress = writtenBytes
+							p.(*Progress).TotalProgress = totalBytes
+							p.(*Progress).Item = currentFile
+							p.(*Progress).TotalItems = totalFiles
+						}
+					})
+
+				if err != nil {
+					if p, ok := ProgressMap.Get(mapLookup); ok {
+						p.(*Progress).Errors = append(p.(*Progress).Errors, err.Error())
+					}
+				}
+
+			}()
+
+			// Wait a little bit so immediate errors are directly reported
+
+			time.Sleep(10 * time.Millisecond)
+		}
+
+	} else {
+
+		err = fmt.Errorf("Unknown action: %v", action)
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	// Write data
+
+	w.Header().Set("content-type", "application/json; charset=utf-8")
+	json.NewEncoder(w).Encode(ret)
+}
+
+/*
+HandlePOST handles REST calls to create or overwrite a new file.
+*/
+func (f *fileEndpoint) HandlePOST(w http.ResponseWriter, r *http.Request, resources []string) {
+	var err error
+	var tree *rufs.Tree
+	var ok bool
+
+	if len(resources) < 1 {
+		http.Error(w, "Need a tree name and a file path",
+			http.StatusBadRequest)
+		return
+	}
+
+	if tree, ok, err = api.GetTree(resources[0]); err == nil && !ok {
+		err = fmt.Errorf("Unknown tree: %v", resources[0])
+	}
+
+	if err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	// Check we have the right request type
+
+	if r.MultipartForm == nil {
+		if err = r.ParseMultipartForm(32 << 20); err != nil {
+			http.Error(w, fmt.Sprintf("Could not read request body: %v", err.Error()),
+				http.StatusBadRequest)
+			return
+		}
+	}
+
+	if r.MultipartForm != nil && r.MultipartForm.File != nil {
+
+		// Check the files are in the form field uploadfile
+
+		files, ok := r.MultipartForm.File["uploadfile"]
+		if !ok {
+			http.Error(w, "Could not find 'uploadfile' form field",
+				http.StatusBadRequest)
+			return
+		}
+
+		for _, file := range files {
+			var f multipart.File
+
+			// Write out all send files
+
+			if f, err = file.Open(); err == nil {
+				err = tree.WriteFileFromBuffer(path.Join(path.Join(resources[1:]...), file.Filename), f)
+			}
+
+			if err != nil {
+				http.Error(w, fmt.Sprintf("Could not write file %v: %v", path.Join(resources[1:]...)+file.Filename, err.Error()),
+					http.StatusBadRequest)
+				return
+			}
+		}
+	}
+
+	if redirect := r.PostFormValue("redirect"); redirect != "" {
+
+		// Do the redirect - make sure it is a local redirect
+
+		if err = httputil.CheckLocalRedirect(redirect); err != nil {
+			http.Error(w, fmt.Sprintf("Could not redirect: %v", err.Error()),
+				http.StatusBadRequest)
+			return
+		}
+
+		http.Redirect(w, r, redirect, http.StatusFound)
+	}
+}
+
+/*
+SwaggerDefs is used to describe the endpoint in swagger.
+*/
+func (f *fileEndpoint) SwaggerDefs(s map[string]interface{}) {
+
+	s["paths"].(map[string]interface{})["/v1/file/{tree}/{path}"] = map[string]interface{}{
+		"get": map[string]interface{}{
+			"summary":     "Read a file.",
+			"description": "Return the contents of a file.",
+			"produces": []string{
+				"text/plain",
+				"application/octet-stream",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "path",
+					"in":          "path",
+					"description": "File path.",
+					"required":    true,
+					"type":        "string",
+				},
+			},
+			"responses": map[string]interface{}{
+				"200": map[string]interface{}{
+					"description": "Returns the content of the requested file.",
+				},
+				"default": map[string]interface{}{
+					"description": "Error response",
+					"schema": map[string]interface{}{
+						"$ref": "#/definitions/Error",
+					},
+				},
+			},
+		},
+		"put": map[string]interface{}{
+			"summary":     "Perform a file operation.",
+			"description": "Perform a file operation like rename or copy.",
+			"consumes": []string{
+				"application/json",
+			},
+			"produces": []string{
+				"text/plain",
+				"application/json",
+			},
+			"parameters": []map[string]interface{}{
+				{
+					"name":        "tree",
+					"in":          "path",
+					"description": "Name of the tree.",
+					"required":    true,
+					"type":        "string",
+				},
+				{
+					"name":        "path",
+					"in":          "path",
+					"description": "File path.",
+					"required":    true,
+					"type":        "string",
+				},
+
+				{
+					"name":        "operation",
+					"in":          "body",
+					"description": "Operation which should be executes",
+					"required":    true,
+					"schema": map[string]interface{}{
+						"type": "object",
+						"properties": map[string]interface{}{
+							"action": map[string]interface{}{
+								"description": "Action to perform.",
+								"type":        "string",
+								"enum": []string{
+									"rename",
+									"mkdir",
+									"copy",
+									"sync",
+								},
+							},
+							"newname": map[string]interface{}{
+								"description": "New filename when renaming a single file.",
+								"type":        "string",
+							},
+							"newnames": map[string]interface{}{
+								"description": "List of new file names when renaming multiple files using the files parameter.",
+								"type":        "array",
+								"items": map[string]interface{}{
+									"description": "New filename.",
+									"type":        "string",
+								},
+							},
+							"destination": map[string]interface{}{
+								"description": "Destination directory when copying files.",
+								"type":        "string",
+							},
+							"files": map[string]interface{}{
+								"description": "List of (full path) files which should be copied / renamed.",
+								"type":        "array",
+								"items": map[string]interface{}{