Canopy Server
Overview
Installation
Configuration
Operations
Overview
About

The Canopy Server is an open source server-side program written in GoLang. It serves the Canopy REST API and Canopy Websocket API. Additionally, it can be configured to serve the Canopy Device Manager which provides a convenient web-based frontend for developers and system administrators.

The Canopy Server has been architected to scale linearly to millions of devices with class leading performance, security and ease-of-use.

Hosting Options
Hosted by Us

The Canopy team operates a large-scale deployment of the Canopy Server that you can use to develop and launch your cloud-enabled products and services. Our servers are hosted in a world-class datacenter with high availability.

Learn more about our hosted offerings on our Services page.

Hosted by You

The Canopy Server is completely open source and licensed under the highly-permisive Apache v2.0 license. It is easy to deploy your own instance of the server by following the installation instructions in this document.

Installation

This section contains instructions for installing the Canopy Server from a release package or from source on a fresh Ubuntu 14.04 LTS machine. The Canopy Server may be installed in a public cloud, private cloud, local network, or even on your laptop!

Installing Dependencies

Canopy requires Cassandra and Oracle Java 1.7 in order to run.

Installing Cassandra

Using the text editor of your choice, edit /etc/apt/sources.list.d/cassandra.sources.list:

sudo vim /etc/apt/sources.list.d/cassandra.sources.list

Add the following line:

deb http://debian.datastax.com/community stable main

Save and exit. Then run:

curl -L http://debian.datastax.com/debian/repo_key | sudo apt-key add -
sudo apt-get update
sudo apt-get install -y cassandra=2.0.7
Installing Oracle Java 1.7

Run:

sudo apt-get install -y python-software-properties
sudo add-apt-repository ppa:webupd8team/java

Press ENTER when prompted. The run:

sudo apt-get update
sudo apt-get install -y oracle-java7-installer

When prompted, press ENTER. Then select "Yes" and press ENTER.

Installing Canopy Server

The Canopy Server can be built from source or installed from a Debian package.

Installing Canopy Server From .deb Package

Download the latest release on our Release Package Downloads page. Look for the ".deb x64 64-bit Binary Package".

In the directory where you downloaded the release package, install the Canopy Server with this command (replacing XX.YY.ZZ-W with the version number you downloaded):

sudo dpkg -i canopy-server_XX.YY.ZZ-W_amd64.deb

You can now skip ahead to Configuring and Running.

Building Canopy Server From Source

Alternatively you can build the Canopy Server from source by cloning the github repository. This section walks you through the steps.

Install golang. Canopy does not work with the old version of golang that Ubuntu's "apt-get" installs. We recommend manually installing version go1.4.2:

sudo apt-get install -y git make mercurial
wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.4.2.linux-amd64.tar.gz
echo PATH=/usr/local/go/bin:\$PATH >> ~/.profile
source ~/.profile

Clone the Canopy Server repository:

git clone https://github.com/canopy-project/canopy-server

Build the project:

cd canopy-server
make

You should see this output:

make -C src/canopy go_get_deps
make[1]: Entering directory `/home/ubuntu/canopy-server/src/canopy'
mkdir -p ~/.canopy/golang
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get code.google.com/p/go.net/websocket
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get github.com/gocql/gocql
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get github.com/gorilla/sessions
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get github.com/gorilla/context
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get github.com/gorilla/mux
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get github.com/sendgrid/sendgrid-go
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go get code.google.com/p/go.crypto/bcrypt
make[1]: Leaving directory `/home/ubuntu/canopy-server/src/canopy'
make -C src/canopy all
make[1]: Entering directory `/home/ubuntu/canopy-server/src/canopy'
mkdir -p ~/.canopy/golang
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go build canopy-cloud-service.go canopy_cloud.go canopy_json.go
mkdir -p build/
mv canopy-cloud-service build/
mkdir -p ~/.canopy/golang
GOPATH=$(cd ~/.canopy/golang; pwd):$(cd ../../; pwd) go build canopy_cmd.go
mkdir -p build/
mv canopy_cmd build/canodevtool
make[1]: Leaving directory `/home/ubuntu/canopy-cloud/src/canopy'

Install it:

sudo make install

You should see this output:

src/canopy install
make[1]: Entering directory `/home/ubuntu/canopy-cloud/src/canopy'
cp build/canopy-cloud-service build/canodevtool /usr/local/bin
cp ../../scripts/canopy-cloud-service /etc/init.d
mkdir -p /etc/canopy
cp ../../scripts/ccs-env.sh /etc/canopy
../../scripts/create-canopy-group-user.sh
id: canopy: no such user
Adding group `canopy' (GID 114) ...
Done.
id: canopy: no such user
Adding system user `canopy' (UID 108) ...
Adding new user `canopy' (UID 108) with group `canopy' ...
Not creating home directory `/home/canopy'.
mkdir -p /var/log/canopy
touch /var/log/canopy/ccs.log
touch /var/log/canopy/ccs-errors.log
chown canopy /var/log/canopy
chgrp canopy /var/log/canopy
chown canopy /var/log/canopy/ccs.log
chgrp canopy /var/log/canopy/ccs.log
chown canopy /var/log/canopy/ccs-errors.log
chgrp canopy /var/log/canopy/ccs-errors.log
make[1]: Leaving directory `/home/ubuntu/canopy-cloud/src/canopy'
Configuring and Running
Configuring the Cloud Server
One last step before running the Canopy Server. We must configure it. Edit the file:
/etc/canopy/server.conf

Change the "hostname" to match the IP address or hostname of your server.

There a many other configuration options you can modify. Learn more about configuring the server in the Configuration chapter.

Starting Cassandra and Initialize Database

Start Cassandra running:

sudo cassandra

You should see a bunch of text scroll, ending with:

INFO 17:24:59,576 Completed flushing /var/lib/cassandra/data/system/local/system-local-jb-3-Data.db (114 bytes) for commitlog position ReplayPosition(segmentId=1410197098396, position=91604)
INFO 17:24:59,644 CFS(Keyspace='system', ColumnFamily='local') liveRatio is 27.44988344988345 (just-counted was 3.515151515151515).  calculation took 5ms for 257 cells
INFO 17:24:59,644 Enqueuing flush of Memtable-local@414544706(10098/277188 serialized/live bytes, 259 ops)
INFO 17:24:59,645 Writing Memtable-local@414544706(10098/277188 serialized/live bytes, 259 ops)
INFO 17:24:59,677 Completed flushing /var/lib/cassandra/data/system/local/system-local-jb-4-Data.db (5275 bytes) for commitlog position ReplayPosition(segmentId=1410197098396, position=103603)
INFO 17:24:59,693 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-1-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-3-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-4-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-2-Data.db')]
INFO 17:24:59,780 Node localhost/127.0.0.1 state jump to normal

Press ENTER to return to the command-line.

Create the database:

canopy-ops create-db
Starting the Canopy Server

Now, you can start the Canopy Server

sudo /etc/init.d/canopy-server start

To check that it is working, go to https://<ip_address>/api/info in your web browser. You should see something like the following response:

{
    "clock_us": 1427847182641191,
    "clock_utc": "2015-04-01T00:13:02.641191Z",
    "config": {
        "allow-anon-devices": true,
        "allow-origin": "",
        "email-service": "sendgrid",
        "enable-http": false,
        "enable-https": true,
        "forward-other-hosts": "",
        "hostname": "dev02.canopy.link",
        "http-port": 80,
        "https-cert-file": "/etc/canopy/cert.pem",
        "https-port": 443,
        "https-priv-key-file": "/etc/canopy/key.pem",
        "js-client-path": "/home/gregp/development/canopy/canopy-js-client",
        "log-file": "/var/log/canopy/server.log",
        "sendgrid-username": "canopy-mailer",
        "web-manager-path": "/home/gregp/development/canopy/canopy-app"
    },
    "result": "ok",
    "service-name": "Canopy Cloud Service",
    "version": "15.04.03"
}

If you see anything other than that, there was most likely a problem. You can view the log file by typing:

cat /var/log/canopy/server.log
Configuration
Config Priority

The Canopy Server's configuration is read from one or more config files. Config settings can be overridden with environment variables and command-line parameters.

This table shows the ordering in which configuration is read:

Config Source Order
Compiled Defaults 1 - Lowest priority (read first)
/etc/canopy/server.conf 2
$HOME/.canopy/server.conf 3
$CANOPY_SERVER_CONFIG_FILE 4
Environment Variables 5
Command-line Parameters 6 - Highest priority (read last)
Configuration Settings
JSON Key Environment Variable Command-line Option Description
"allow-anon-devices" ALLOW_ANON_DEVICES --allow-anon-devices= If true, allow anonymous devices to post data to the server. If false, require that every device has an associated User Account.

Default: true

"allow-origin" CCS_ALLOW_ORIGIN --allow-origin= Domains to include in the "allow-origin" header of HTTP responses.

Default: ""

"email-service" CCS_EMAIL_SERVICE --email-service= Selects external service to use for sending email. Valid options are:
  • "none" - Do not send email.
  • "sendgrid" - Use SendGrid to send email.
If "sendgrid", then you must also set "sendgrid-username" and "sendgrid-secret-key".

Default: "none"

"enable-http" CCS_ENABLE_HTTP --enable-http= Serve the Canopy REST API over HTTP.

Warning: Enabling this option makes Canopy vulnerable to a number of attacks, including password snooping, man-in-the-middle attacks, and more. This option should only be used if Canopy is deployed on a secured LAN and you know what you're doing!

Default: false

"enable-https" CCS_ENABLE_HTTPS --enable-https= Serve the Canopy REST API over HTTPS.

Default: true

"forward-other-hosts" CCS_FORWARD_OTHER_HOSTS --forward-other-hosts= The Canopy Server includes a proxy server that forwards along all HTTP requests to other hostnames. This specifies the destination URL for these requests. If "" then forwarding will be disabled.

Default: "".

"hostname" CCS_HOSTNAME --hostname= Virtual hostname for Canopy Server. This option is required. HTTP requests to other hosts will be forwarded to the "forward-other-hosts".

Default: "".

"http-port" CCS_HTTP_PORT --http-port= HTTP port to listen on.

Default: 80.

"https-cert-file" CCS_HTTPS_CERT_FILE --https-cert-file= Filename of HTTPS certificate file on local file system. For example: "/etc/canopy/cert.pem". This is mandatory if "enable-https" is true.

Default: "".

"https-port" CCS_HTTPS_PORT --https-port= HTTPS port to listen on.

Default: 443.

"https-priv-key-file" CCS_HTTPS_PRIV_KEY_FILE --http-priv-key-file= Filename of HTTPS private key file on local file system. For example: "/etc/canopy/key.pem". This is mandatory if "enable-https" is true.

Default: "".

"js-client-path" CCS_JS_CLIENT_PATH --js-client-path= Local filesystem path to the canopy-js-client/ project. This must be configured to enable the Canopy Device Manager.

Default: "".

"log-file" CCS_LOG_FILE --log-file= Location of log file on local filesystem.

Default: "/var/log/canopy/canopy-server.log".

"password-hash-cost" CCS_PASSWORD_HASH_COST --password-hash-cost= Integer value between 4 and 31 specifying the bcrypt hash cost for newly generated passwords. Higher values result in it taking longer to verify a user's password, making it more difficult to perform a brute force attack. However, if this is too high then it will affect server responsiveness and consume too many resources on the server.

Default: 10.

"password-secret-salt" CCS_PASSWORD_SECRET_SALT --password-secret-salt= Secure secret string that must not be made public and should be different for every server. This is used as "salt" that is appended to passwords before they are hashed.

Warning: Changing this value will cause everyone's password to no longer work.

Default: "".

"production-secret" CCS_PRODUCTION_SECRET --production-secret= Secure secret string that must not be made public and should be different for every server. This is used to secure session cookies.

Default: "".

"sendgrid-secret-key" CCS_SENDGRID_SECRET_KEY --sendgrid-secret-key= Secure secret key provided by SendGrid. This must not be made public and should be different for every server. This is required if "email-service" is "sendgrid". Otherwise it is ignored.

Default: "".

"sendgrid-username" CCS_SENDGRID_USERNAME --sendgrid-secret-key= Username of SendGrid account, used for sending emails. This is required if "email-service" is "sendgrid". Otherwise it is ignored.

Default: "".

"web-manager-path" CCS_WEB_MANAGER_PATH --web-manager-path= Local filesystem path to the canopy-app/ project. This must be configured to enable the Canopy Device Manager.

Default: "".

Config File Example
{
    "allow-anon-devices" : false,
    "allow-origin" : "sandbox.canopy.link myiotapp.com",
    "email-service" : "sendgrid",
    "enable-http" : false,
    "enable-https" : true,
    "forward-other-hosts" : "http://canopy.link",
    "http-port" : 80,
    "https-cert-file" : "/etc/canopy/cert.pem",
    "https-port" : 443,
    "https-priv-key-file" : "/etc/canopy/key.pem",
    "hostname" : "sandbox.canopy.link",
    "js-client-path" : "/home/ubuntu/canopy-js-client",
    "password-hash-cost" : 10,
    "password-secret-salt" : "E2vOmLYAig7ER5uIoL9wSa",
    "production-secret" : "my production secret",
    "sendgrid-secret-key" : "ljP)48-1x*CL@0iA#",
    "sendgrid-username" : "mysendgridusername",
    "web-manager-path" : "/home/ubuntu/canopy-app",
}
Environment Override Example
For example, to run the Canopy Server as a non-root user, you might use:
CCS_HTTPS_PORT=8443 CCS_LOG_FILE="ccs.log" /usr/local/bin/canopy-server
Command-line Override Example
An alternative way to run the Canopy Server as a non-root user is:
/usr/local/bin/canopy-server --https-port=8443 --log-file="ccs.log"
Checking the Configuration

You can check the configuration of a running Canopy Server using the /api/info endpoint. For example:

https://sandbox.canopy.link/api/info

Additionally, the log file may contain a dump of all configuration settings (except passwords) when the server starts up:

tail -f /var/log/canopy-server.log
SSL Setup - Self-signed Certificate
This command creates a 2048-bit private key (key.pem) and a self-signed certificate (cert.pem)
openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out cert.pem

Place these files somewhere secure on your server. Perhaps:

sudo mv key.pem /etc/canopy/key.pem
sudo mv cert.pem /etc/canopy/cert.pem

Configure your /etc/canopy/server.conf file to point to these files:

"enable-https" : true,
"https-cert-file" : "/etc/canopy/cert.pem",
"https-priv-key-file" : "/etc/canopy/key.pem",
SSL Setup - CA-signed Certificate
This command creates a Certificate Signing Request (csr.pem) and a 2048-bit private key (key.pem).
openssl req -newkey rsa:2048 -nodes -keyout key.pem -out csr.pem

Follow the instructions provided by your Certificate Authority of choice (such as VeriSign, GoDaddy, or Gandi.net) to convert this CSR into an SSL Certificate.

Once you receive the certificate, copy the private key and certificate somewhere secure on your server, such as:

/etc/canopy/key.pem
/etc/canopy/cert.pem

Configure your /etc/canopy/server.conf file to point to these files:

"enable-https" : true,
"https-cert-file" : "/etc/canopy/cert.pem",
"https-priv-key-file" : "/etc/canopy/key.pem",
Operations
Starting & Stopping the Canopy Server

The Installation process installs an init.d script that has been tested on Ubuntu 14.04 LTS. This script can be used to start and stop the server.

Start the Canopy Server with:

sudo /etc/init.d/canopy-server start

Stop the Canopy Server with:

sudo /etc/init.d/canopy-server stop

Restart the Canopy Server with:

sudo /etc/init.d/canopy-server restart

Check whether or not the Canopy Server is currently running with:

sudo /etc/init.d/canopy-server status

Errors (including errors with startup) will be reported in the log file, typically:

tail /var/log/canopy/server.log
The canopy-ops Tool

The Installation process also installs the canopy-ops tool. Typically it can be found in the /usr/local/bin/ directory.

This tool supports a growing number of commands that are useful for operating a Canopy Server deployment. Run "canopy-ops help" for the list of commands supported by your version of the tool.

Command Line Description
canopy-ops help [<TOPIC_OR_CMD>] Display general help or get help for a topic or command
canopy-ops create-db Initialize database
canopy-ops erase-db Wipe entire database
canopy-ops migrate-db <START_VER> <END_VER> Migrate database from one release to another.
canopy-ops reset-db Wipe entire db then initialize a new db
canopy-ops workers List all registered canopy workers
Upgrading the Canopy Server

Upgrade instructions vary depending on which release of the Canopy Server you are running and which release you are moving to. If your desired upgrade spans several releases, the safest approach is to perform the upgrade in multiple stages by going through each intermediate release.

Upgrade instructions for all releases can be found in this document: upgrade_process.md