Docker is the world leading software containerization platform. I tried using GitLab as versioncontrol system and Jenkings as continuos integration system but the system turned out as not completly useful.
In that way a colleague told me about a system he wants to create based on Docker.
What we want:
- a git system under ouer control
- a continuos integration that is flexible and customizable
- every system available per SSL connection
Git Service -> GoGs – Go Git Service
It’s a simple self-hosted Git servise.
- easy to install, cross-platform, lightweight and OpenSourse
It contains everything importent to develop something in collaboration.
Continuos Integration -> drone
It’s a continuous integration platform build on container technology. Every build run will be triggerd by a push to a repository if it’s linked to drone.
- flexible and customizable: by setting up a config file you tell drone what is to do
https://github.com/drone/drone
SSL -> NGINX as reverse proxy
It’s a fine powerfull tool and a nice reverse proxy. With it we are able to provide the GoGs and the drone to the internet more secure and with SSL encryted.
We will colleced the SSL/TLS certificates from a Let’s Encryt service.
https://www.nginx.com/ https://letsencrypt.org/
How does it fit together?
In the way of Docker everything will be a container.
- GoGs as container
- drone as container
- NGINX reverse proxy as container
- Let’s Encryt service as container
For docker it doesn’t matter if you build that system on a dedicated Linux server or on a Linux system at home.
This is a example for self-hosting at home and I will use it for the setup steps to build such system.
Build or own in Docker Git-CI-System
Setup domains and forward ports
- If you have a WAN access with a dynamic IP change, than you have to set up/configurate a dyn-dns-service on your router/WAN access
- Create the two subdomains by your domain hoster and redirekt them to the dyn-dns name with ‘CNAME’-records or to your fixed IP address with ‘A’-records
- Forward the Ports 80 and 443 on your Router to the host where the GoGs and dorne system will be hosted
Why this first: because we create the reverse proxy and this one will only forward request for the spezified subdomains. Alternatively you can set some host requords on you local maschine to redirect the url’s local to the right host. But when you create the containers with gogs and drone then they have to be publicly accessable to generate the SSL certificates.
Install Docker on the Linux
- Install and configure docker on your system
(https://docs.docker.com/engine/installation/)
Create Containers
Replace all <…> to fit the system in your environment.
- Create the NGINX reverse proxy:
(https://github.com/jwilder/nginx-proxy)docker run -d -p 80:80 -p 443:443 --name nginx-proxy \ -v </path/to/certs>:/etc/nginx/certs:ro \ -v /etc/nginx/vhost.d \ -v /usr/share/nginx/html \ -v /var/run/docker.sock:/tmp/docker.sock:ro \ jwilder/nginx-proxy
- Create the Let’s Encrypt container which creates and renews the certificates:
(https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion)docker run -d --name nginx-letsencrypt \ --volumes-from nginx-proxy \ -v </path/to/certs>:/etc/nginx/certs:rw \ -v /var/run/docker.sock:/var/run /docker.sock:ro \ jrcs/letsencrypt-nginx-proxy-companion
- Create the GoGs container and link it to the nginx-proxy container:
(https://github.com/gogits/gogs/tree/master/docker)docker run -d --name gogs -h gogs \ -e "VIRTUAL_HOST=<your-gogs-sub-domain>" \ -e "LETSENCRYPT_HOST=<your-gogs-sub-domain>" \ -e "LETSENCRYPT_EMAIL=<your-email-address>" \ -e "VIRTUAL_PORT=3000" \ --expose 3000 \ -v <gogs/data/host/path>:/data \ --restart=always \ gogs/gogs:latest
- Go to your “gogs sub domain” in a browser and make the base configuration
- Create a drone enviromental/configuration file and store it to dronerc
(http://readme.drone.io/setup/overview/)REMOTE_DRIVER=gogs REMOTE_CONFIG=https://<your-gogs-sub-domain>?open=false DATABASE_DRIVER=sqlite3 DATABASE_CONFIG=/var/lib/drone/drone.sqlite
- Create the drone container and link it to the nginx-proxy container:
(https://github.com/drone/drone)docker run --name drone -h drone \ -e "VIRTUAL_HOST=<your-drone-sub-domain>" \ -e "LETSENCRYPT_HOST=<your-drone-sub-domain>" \ -e "LETSENCRYPT_EMAIL=<your-email-address>" \ -e "VIRTUAL_PORT=8000" \ --volume <drone/data/host/path>lib:/var/lib/drone \ --volume /var/run/docker.sock:/var/run/docker.sock \ --env-file </path/to/>dronerc \ --restart=always \ --detach=true \ drone/drone:0.4
- Go to your “drone sub domain” and login with your GoGs user and password; if it ends up in an error than please go trough the “Found Issus” point “Unable to login to drone”
Enable Git Repository
- Login to your drone system and go to “Available Repositories”
- Select one and click “ACTIVATE NOW” – that will set the webhook in your gogs repository
- Go to the tap “SETTINGS” and set this repository as trusted because we need the Docker-Socket (docker.sock) for local deployments
The last two config files
In your git repository you have to provide a “.drone.yml”-file and a “Dockerfile”.
The drone.yml-file contains the instructions for the drone system. The Dockerfile contains the instructions how the final build should be or should look like.
The drone.yml-file as instructed here: http://readme.drone.io/usage/overview/
A exampel for a drone.yml-file – is not perfect jet
debug: true clone: depth: 50 recursive: true build: image: bemayr/alpine-docker volumes: - /var/run/docker.sock:/var/run/docker.sock commands: - docker stop xyz || true - docker rm xyz || true - docker build -t temp/xyz . - docker run -d --name xyz -p 9999:3002 temp/xyz
The Dockerfile as instructed here: https://docs.docker.com/engine/reference/builder/
Now you are able to push and to trigger the build.
Conclusion
Docker is awesome. It supports us (developers) very well. We can run different setup’s for a project on one machine and if we ship our project to production – it’s no problem – just update the build steps.
Found Issues
Unable to login to drone:
Please run “docker logs <your-drone-container-name>
“. If you find there an error like “certificate is not valid for the given domain but for another”, “cannot access WAN-IP” or “level=error msg=’cannot authenticate user. Get https://_gogs-address_/api/v1/users/_gogs-user_/tokens. …'” I had this issus because my internet provider and the router does not allow a connection out from my network and back on the WAN-port or the router does not know what todo if that happends.
My Fix
I fixed it by running a local dns-server for my network and my containers where I’m catching all the requests for gogs and drone and redircting them to the docker server and forward all the others to a public dns-server. I use a dnsmasq
dns server for it. It has all i need, a small lightweight dns server and dhcp server. In this case I didn’t containerized dns server but I will change this in future.
All following steps are to made in the dnsmasq.conf file after installing dnsmasq through your package-manager
- Set the interfaces where dnsmasq should responce to requests.
interface=<eth0/enp3s0>,docker0
The importend interface is thedocker0
interface because all container in the docker “bridge” network will request through this interface. - Set the web-servers which should be caught and redirected to the docker host.
address=/<your-gogs/drone-sub-domain>/<docker-host-ip> ...
- Set the dns-server to which dnsmasq should talk after not matching a address.
server=<public-dns-server>
In my opinion it fixed my issue with the drone to gogs and gogs to drone communication
this text was written with the neo-layout – it took me a lot of time to write it because I’m a neo beginner. 😉 (http://neo-layout.org/)