In linux distros running systemd, like Ubuntu 15.04, adding -H tcp://0.0.0.0:2375 to /etc/default/docker does not have the effect it used to.

Instead, create a file called /etc/systemd/system/docker-tcp.socket to make docker available on a TCP socket on port 2375:

[Unit]
Description=Docker Socket for the API
[Socket]
ListenStream=2375
BindIPv6Only=both
Service=docker.service
[Install]
WantedBy=sockets.target

Then enable this new socket:
systemctl enable docker-tcp.socket

systemctl enable docker.socket

systemctl stop docker

systemctl start docker-tcp.socket

systemctl start docker

Test that it's working:
docker -H tcp://127.0.0.1:2375 ps

Source: https://coreos.com/docs/launching-containers/building/customizing-docker/