Added CI/CD chapter

Author: fabianenardon

developer-tools/java/chapters/ch09-cicd.adoc

@@ -2,3 +2,119 @@
 
 = CI/CD using Docker
 
+*PURPOSE*: This chapter explains how to use Jenkins and Docker to run continuous integration and continuous delivery.
+
+There are several possible approaches to run Docker builds with Jenkins:
+
+. Install Jenkins on your host machine, where Docker is also installed, and run Docker commands from your build, either using one of the several Jenkins Docker plugins, or by running Docker commands from a build step
+. Install Jenkins on your host machine and have a Jenkins slave machine with Docker installed to run your Docker builds
+. Run Jenkins on Docker and use the underlying Docker installed on the host to run Docker commands.
+
+NOTE: Another option is running Jenkins on Docker and do a complete Docker installation inside the Jenkins Docker container. This technique is called Docker in Docker and it is usually a bad idea. There are several discussions about the problems with this approach, like this one: http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ . A better approach is using Docker outside of Docker, as explained here: http://container-solutions.com/running-docker-in-jenkins-in-docker/
+
+== Run Jenkins on docker
+
+In this example, we will run Jenkins on Docker and use the underlying Docker installed on the host to run Docker commands. This technique is known as Docker outside of Docker.
+
+First, clone the project at https://github.com/fabianenardon/jenkins-docker-demo
+
+Then, in the project folder, run:
+
+[source, text]
+----
+docker-compose up
+----
+
+Wait for jenkins to start and then go to the browser and open `http://localhost:8081`. Jenkins should be running.
+
+The Jenkins installation on this lab comes pre-configured. To login use the username `jenkins` and the password `jenkins`.
+
+== Running Integration Tests with Docker and Jenkins
+
+For this Continuous Integration demo, we will run the application described on the link:chapters/ch11-bigdata.adoc[Big Data Processing with Docker and Hadoop] chapter. 
+
+[NOTE]
+====
+When running integration tests, you want to test your application in an environment as close to production as possible, so you can test interactions between the several components, services, databases, network communication, etc. Fortunately, docker can help you a lot with integration tests. There are several strategies to run integration tests, but in this application we are going to use the following:
+
+. Start the services with a `docker-compose.yml` file created for testing purposes. This file won't have any volumes mapped, so when the test is over, no state will be saved. The test `docker-compose.yml` file won't publish any port on the host machine, so we can run simultaneous tests.
+. Run the application, using the services started with the `docker-compose.yml` test file.
+. Run Maven integration tests to check if the application execution produced the expected results. This will be done by checking what was saved on the MongoDB database.
+. Stop the services. No state will be stored, so next time you run the integration tests, you will have a clean environment.
+====
+
+Create a new job on jenkins:
+
+1 . Select Freestyle project
+
+image::docker-ci-cd-01.png[]
+
+2 . In Source Code Management, select Git and add the repository URL: `https://github.com/fabianenardon/hadoop-docker-demo`
+
+image::docker-ci-cd-02.png[]
+
+3 . In Build, select Add build step and select Execute shell
+
+image::docker-ci-cd-03.png[]
+
+4 . In the shell Command, add these instructions:
+
+[source, text]
+----
+cd sample
+
+# Generates the images
+sudo /var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/maven/bin/mvn clean install -Papp-docker-image
+
+
+# Starts all the services. The -p option allows multiple builds to run at the same time, 
+# since we can start multiple instances of the containers
+sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml up -d
+
+# Waits for containers to start
+sleep 30
+
+# Creates the folder we need on hdfs to test
+sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml exec -T yarn hdfs dfs -mkdir /files/
+
+# Put the file we are going to process on hdfs
+sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml run docker-hadoop-example hdfs dfs -put /maven/test-data/text_for_word_count.txt /files/
+
+# Run our application
+sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml run docker-hadoop-example hadoop jar /maven/jar/docker-hadoop-example-1.0-SNAPSHOT-mr.jar hdfs://namenode:9000 /files mongo yarn:8050
+
+# Run our integration tests
+sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml run docker-hadoop-example-tests mvn -f /maven/code/pom.xml -Dmaven.repo.local=/m2/repository -Pintegration-test verify 
+----
+
+5 . Click on Add post-build action and select Execute a set of scripts
+
+image::docker-ci-cd-04.png[]
+
+6 . In Post-build Actions, select Execute shell
+
+image::docker-ci-cd-05.png[]
+
+7 . In the Command box, add:
+
+[source, text]
+----
+cd sample
+sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml down
+----
+
+8 . Uncheck the `Execute script only if build succeeds` and `Execute script only if build fails` options, so this script will run always when the build ends. This way, we make sure to always stop the services.
+
+
+[NOTE]
+====
+. The `-p app-$BUILD_NUMBER` option allows multiple builds to run at the same time, since we can start multiple instances of the containers. We are using Jenkins $BUILD_NUMBER variable to isolate the containers. This way, each set of services will run on its own network.
+. We are running the commands with sudo because we are actually running the Docker socket on the host. Jenkins runs with the jenkins user and we added the jenkins user to the sudoers list in our image. Obviously, this can have security consequences in a production environment, since one could create a build that would access root level services on the host. You can avoid this by configuring the jenkins user on the host, so it will have access to the Docker socket and then run the commands without sudo.
+====
+
+== Continuous Delivery with Docker and Jenkins
+
+Continuous Delivery strategies depend greatly on the application architecture. With a dockerized application like the one in our demo, the continuous delivery strategy could be to publish a new version of the application image if the tests passed. This way, next time the application runs on production, the new image will be downloaded and automatically deployed. You can publish images with Jenkins just like you invoked all the other docker commands in the build.
+
+
+

developer-tools/java/chapters/ch11-bigdata.adoc

@@ -164,7 +164,7 @@ image::docker-bigdata-02.png[]
 
 When running integration tests, you want to test your application in an environment as close to production as possible, so you can test interactions between the several components, services, databases, network communication, etc. Fortunately, docker can help you a lot with integration tests.
 
-There are several strategies to run integration test, but in this application we are going to use the following:
+There are several strategies to run integration tests, but in this application we are going to use the following:
 
 . Start the services with a `docker-compose.yml` file created for testing purposes. This file won't have any volumes mapped, so when the test is over, no state will be saved. The test `docker-compose.yml` file won't publish any port on the host machine, so we can run simultaneous tests.
 . Run the application, using the services started with the `docker-compose.yml` test file.