Microplugins with Docker
Microplugins with Docker

        
          
              
                Microplugins DOCKER ******************* WITH *******************
              
              
          

        
        
          
              
                Freelance « DevOps » JULIEN VEY ERIC BELLEMON ********************************************* at
              
              bywan @julienvey @haklop
          

        
        
          
              
                BY NOW, WE THINK YOU KNOW WHAT IS DOCKER WHAT
              
              ?
          

        
        
          
              
                DOCKER HOW ? Patterns *******************************************
              
              
          

        
        
          
              
                MICROSERVICES
              
              
          

        
        
          
              
                A MICROSERVICE IS USUALLY A PERSISTANT SERVICE YOU CAN ACCESS
              
              VIA AN API
          

        
        
          
              
                GO GO PYTHON RUBY NODE JAVA JAVA PHP ANY LANGUAGE
              
              
          

        
        
          
              
                UBUNTU REDHAT DEBIAN CENTOS FEDORA FEDORA DEBIAN CENTOS ANY DISTRIBUTION
              
              
          

        
        
          
              
                AWS GCE COREOS ANY WHERE
              
              
          

        
        
          
              
                WHY CONTAINERS AND NOT VMS? CONTAINERS ARE LIGHTWEIGHT SPAWN FAST
              
              USE LAYERS
          

        
        
          
              
                DOCKER Tips n’ Tricks ******************************************* HOW ?
              
              
          

        
        
          
              
                DOCKER IN DOCKER
              
              
          

        
        
          
              
                THE DOCKER MODEL DOCKER DAEMON UNIX SOCKET DOCKER REMOTE API
              
              DOCKER CLI creates an which exposes the that can be used by the
          

        
        
          
              
                FIRST APPROACH
              
              
          

        
        
          
              
                DOCKER IN DOCKER IN DOCKER …
              
              
          

        
        
          
              
                docker run --privileged -t -i jpetazzo/dind SPECIAL DOCKER IMAGE
              
              
          

        
        
          
              
                docker run --privileged -t -i jpetazzo/dind PRIVILEGED MODE
              
              
          

        
        
          
              
                docker run --privileged -t -i jpetazzo/dind PRIVILEGED MODE
              
              
          

        
        
          
              
                IT RAISES SOME PROBLEMS INIT SYSTEM DISK SPACE CUSTOM IMAGE
              
              
          

        
        
          
              
                SECOND APPROACH
              
              
          

        
        
          
              
                DOCKER IN DOCKER IN DOCKER … DOCKER HOST DOCKER SOCKET
              
              
          

        
        
          
              
                docker run -v /var/run/docker.sock:/var/run/docker.sock [your image] DOCKER SOCKET AS A
              
              DOCKER VOLUME
          

        
        
          
              
                docker run -v /var/run/docker.sock:/var/run/docker.sock [your image] DOCKER SOCKET AS A
              
              DOCKER VOLUME
          

        
        
          
              
                NO PRIVILEGED MODE ONLY ONE DOCKER DAEMON (INSTEAD OF ONE
              
              PER CONTAINER)
          

        
        
          
              
                What about ? MICRO PLUGINS *********************************************
              
              
          

        
        
          
              
                MICROSERVICE: A SMALL SERVICE RUNNING IN A CONTAINER
              
              
          

        
        
          
              
                MICROPLUGIN: A SMALL PLUGIN RUNNING IN A CONTAINER
              
              
          

        
        
          
              
                What is a ? Plugin Architecture *********************************************
              
              
          

        
        
          
              
                A STANDARD INTERFACE A LOADING MECHANISM A PACKAGING MECHANISM A
              
              DISTRIBUTION MECHANISM
          

        
        
          
              
                VAGRANT ******************* WITH ******************* A PLUGIN ARCHITECTURE RUBY EXAMPLE
              
              
          

        
        
          
              
                A STANDARD INTERFACE require 'vagrant' module VagrantPlugins module Openstack class
              
              Provider < Vagrant.plugin('2', :provider) RUBY CLASSES
          

        
        
          
              
                A LOADING MECHANISM RUBY DYNAMIC LOADING SOURCES INSTALLED IN VAGRANT
              
              BUNDLE DIR
          

        
        
          
              
                A PACKAGING MECHANISM RUBY GEM PACKAGE CODE AND DOCUMENTATION DESCRIBE
              
              DEPENDENCIES
          

        
        
          
              
                A DISTRIBUTION MECHANISM RUBYGEMS.ORG
              
              
          

        
        
          
              
                ****************** AND ****************** PROS CONS
              
              
          

        
        
          
              
                PLATFORM INDEPENDANT PACKAGING AND DISTRIBUTION ARE NOT SPECIFIC TO VAGRANT
              
              PROS
          

        
        
          
              
                RUBY ONLY DEPENDENCY MANAGEMENT CAN BE HARD CONS
              
              
          

        
        
          
              
                JENKINS ****************** WITH ****************** A PLUGIN ARCHITECTURE JAVA EXAMPLE
              
              
          

        
        
          
              
                A STANDARD INTERFACE package jenkins.plugins.git; public abstract class AbstractGitSCMSource extends
              
              SCMSource { protected SCMRevision retrieve(…) protected void retrieve(…) } JAVA INTERFACES & CLASSES
          

        
        
          
              
                A LOADING MECHANISM JAVA CLASSLOADING
              
              
          

        
        
          
              
                A PACKAGING MECHANISM JPI/HPI = JENKINS/HUDSON PLUG IN CUSTOM PACKAGING
              
              (JPI/HPI)
          

        
        
          
              
                UPDATES.JENKINS-CI.ORG A DISTRIBUTION MECHANISM JENKINS HAS ITS OWN
              
              
          

        
        
          
              
                ****************** AND ****************** PROS CONS
              
              
          

        
        
          
              
                PLATFORM INDEPENDANT COMPILED PROS
              
              
          

        
        
          
              
                JVM ONLY DISTRIBUTION CONS SPECIFIC MECHANISM PACKAGING
              
              
          

        
        
          
              
                Can we do DOCKER ? a Plugin Architecture *********************** WITH
              
              ***********************
          

        
        
          
              
                Docker ****************** WITH ****************** A PLUGIN ARCHITECTURE
              
              
          

        
        
          
              
                A STANDARD INTERFACE
              
              
          

        
        
          
              
                A STANDARD INTERFACE DOCKER VOLUMES ├ conﬁg │ ├ conﬁg.json
              
              │ ├ key ├ src │ ├ main.go │ ├ main_test.go │ ├ Makeﬁle DOCKER PLUGIN INPUT & OUPUT
          

        
        
          
              
                A STANDARD INTERFACE DOCKER VOLUMES ├ conﬁg │ ├ conﬁg.json
              
              │ ├ key ├ src │ ├ main.go │ ├ main_test.go │ ├ Makeﬁle DOCKER PLUGIN A DOCKER PLUGIN B ├ binaries │ ├ main_darwin_amd64 │ ├ main_linux_x86 INPUT & OUPUT
          

        
        
          
              
                A STANDARD INTERFACE ENVIRONMENT VARIABLES docker run my_plugin -e URL=github.com
              
              -e DEPTH=3 INPUT ONLY
          

        
        
          
              
                A STANDARD INTERFACE DOCKER EXIT CODE OUTPUT ONLY docker run
              
              my_image echo $?
          

        
        
          
              
                A LOADING MECHANISM
              
              
          

        
        
          
              
                A LOADING MECHANISM DOCKER RUN docker run my_plugin
              
              
          

        
        
          
              
                A PACKAGING MECHANISM
              
              
          

        
        
          
              
                A PACKAGING MECHANISM DOCKER BUILD DOCKER IMAGES DOCKERFILES
              
              
          

        
        
          
              
                A DISTRIBUTION MECHANISM
              
              
          

        
        
          
              
                A DISTRIBUTION MECHANISM DOCKER HUB HUB.DOCKER.COM FOR PUBLIC PLUGINS
              
              
          

        
        
          
              
                A DISTRIBUTION MECHANISM ANY DOCKER REGISTRY FOR PUBLIC AND PRIVATE
              
              PLUGINS (OR DOCKER DISTRIBUTION NOW)
          

        
        
          
              
                ****************** VS ****************** CLASSIC PLUGIN Docker ****************** VS ****************** ARCHITECTURE
              
              
          

        
        
          
              
                NEED TO LEARN ONLY ONE LANGUAGE BUT TIED TO THIS
              
              LANGUAGE LANGUAGE AGNOSTIC PROGRAMMING LANGUAGE CLASSIC DOCKER
          

        
        
          
              
                NEED TO BE WRAPPED CAN BE NATIVE LANGUAGE CAN BE
              
              CHOSEN ACCORDINGLY NATIVE TOOLS CLASSIC DOCKER
          

        
        
          
              
                VERY SPECIALIZED INTERFACES HARD TO MAINTAIN RETRO-COMPATIBILITY SIMPLE INTERFACES BUT
              
              ALSO PERMISSIVE COMPATIBILITY CLASSIC DOCKER
          

        
        
          
              
                MANUALLY IMPLEMENTED DOCKER ECOSYSTEM CAN BE TRICKY WITH VOLUMES SCALING
              
              CLASSIC DOCKER
          

        
        
          
              
                GIT PLUGIN ******************* WITH ******************* AN EXAMPLE COMPARISON
              
              
          

        
        
          
              
                THE COMPETITION RETRIEVE SOURCE CODE Git URL Git Branch SSH
              
              Private key Source Code ?
          

        
        
          
              
                JENKINS
              
              
          

        
        
          
              
                JENKINS ~600 LINES OF CODE TO DO ‘GIT CLONE’
              
              
          

        
        
          
              
                JENKINS ~600 LINES OF CODE TO DO ‘GIT CLONE’
              
              
          

        
        
          
              
                DOCKER ├ private-key ├ out │ ├ main.go │ ├
              
              main_test.go │ ├ Makeﬁle GIT PLUGIN CONTAINER $SCM_URL $SCM_BRANCH
          

        
        
          
              
                DOCKER FROM debian:jessie RUN apt-get update \ && apt-get install
              
              -y git ADD run.sh /run.sh RUN echo "IdentityFile /private-key" \ >> /etc/ssh/ssh_config CMD ["/run.sh"] #!/bin/bash git clone "$SCM_URL" --recursive /out cd /out && git checkout "$SCM_BRANCH" Dockerﬁle run.sh
          

        
        
          
              
                NOW THAT WE HAVE 1 PLUGIN, HOW CAN WE ORCHESTRATE
              
              MULTIPLE PLUGINS ?
          

        
        
          
              
                BUILD PIPELINE EXAMPLE 1. FETCH SOURCE CODE 2. COMPILE &
              
              PACKAGE 3. TEST IT
          

        
        
          
              
                BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER
              
              
          

        
        
          
              
                GIT PLUGIN CONTAINER ├ private-key $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE
              
              ORCHESTRATOR CONTAINER docker run git_plugin
          

        
        
          
              
                GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER docker run git_plugin
          

        
        
          
              
                GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin
          

        
        
          
              
                GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin
          

        
        
          
              
                GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin docker run selenium_plugin echo $?
          

        
        
          
              
                GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH MERCURIAL INSTEAD OF GIT ? MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin docker run selenium_plugin echo $?
          

        
        
          
              
                HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH MERCURIAL INSTEAD OF GIT ? MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run maven_plugin docker run selenium_plugin echo $?
          

        
        
          
              
                HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run maven_plugin docker run selenium_plugin echo $? GRADLE INSTEAD OF MAVEN ?
          

        
        
          
              
                HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH GRADLE INSTEAD OF MAVEN ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
          

        
        
          
              
                HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH ANOTHER BUILD WORKFLOW ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
          

        
        
          
              
                HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
              
              │ ├ pom.xml $SCM_URL $SCM_BRANCH ANOTHER BUILD WORKFLOW ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war YOUR OWN ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
          

        
        
          
              
                Our Experiment BAZOOKa ******************* WITH *******************
              
              
          

        
        
          
              
                Our Experiment BAZOOKa ******************* WITH *******************
              
              
          

        
        
          
              None
          

        
        
          
              
                CONCLUSION *************************************************
              
              
          

        
        
          
              
                DOCKER MICROPLUGINS EASY TO SETUP EXTENSIBLE PERMISSIVE EASY TO DISTRIBUTE
              
              LANGUAGE AGNOSTIC
          

        
        
          
              
                QUESTIONS?
              
              
          

        
    