Skip to content

Commit 0163912

Browse files
author
Mano Marks
authored
Merge pull request docker#228 from fabianenardon/ci-cd-bigdata-chapters
CI/CD and Bigdata chapters
2 parents 945ac68 + 2943f81 commit 0163912

19 files changed

+461
-5
lines changed

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
.vscode/
2+
.DS_Store

‎developer-tools/java/chapters/ch03-build-image.adoc

+7-3
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,13 @@ This image uses `ubuntu` as the base image. `CMD` command defines the command th
4343

4444
Build the image using the command:
4545

46-
docker image build -t helloworld .
46+
[source, text]
47+
----
48+
docker image build . -t helloworld
49+
----
50+
51+
`.` in this command is the context for `docker image build`. `-t` adds a tag to the image.
52+
4753

4854
The following output is shown:
4955

@@ -67,8 +73,6 @@ Removing intermediate container 7615d69d04ec
6773
Successfully built 61edf15e4cec
6874
----
6975

70-
`.` in this command is the context for `docker image build` command.
71-
7276
List the images available using `docker image ls`:
7377

7478
[source, text]

‎developer-tools/java/chapters/ch09-cicd.adoc

+170
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,173 @@
22

33
= CI/CD using Docker
44

5+
*PURPOSE*: This chapter explains how to use Jenkins and Docker to run continuous integration and continuous delivery.
6+
7+
There are several possible approaches to run Docker builds with Jenkins:
8+
9+
. Install Jenkins on your host machine, where Docker is also installed, and run Docker commands from your build, either using one of the several Jenkins Docker plugins, or by running Docker commands from a build step
10+
. Install Jenkins on your host machine and have a Jenkins slave machine with Docker installed to run your Docker builds
11+
. Run Jenkins on Docker and use the underlying Docker installed on the host to run Docker commands.
12+
13+
NOTE: Another option is running Jenkins on Docker and do a complete Docker installation inside the Jenkins Docker container. This technique is called Docker in Docker and it is usually a bad idea. There are several discussions about the problems with this approach, like this one: http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ . A better approach is using Docker outside of Docker, as explained here: http://container-solutions.com/running-docker-in-jenkins-in-docker/
14+
15+
== Run Jenkins on docker
16+
17+
In this example, we will run Jenkins on Docker and use the underlying Docker installed on the host to run Docker commands. This technique is known as Docker outside of Docker.
18+
19+
First, clone the project at https://github.com/fabianenardon/jenkins-docker-demo
20+
21+
Then, in the project folder, run:
22+
23+
[source, text]
24+
----
25+
docker-compose up
26+
----
27+
28+
Wait for jenkins to start and then go to the browser and open `http://localhost:8081`. Jenkins should be running.
29+
30+
The Jenkins installation on this lab comes pre-configured. To login use the username `jenkins` and the password `jenkins`.
31+
32+
== Running Integration Tests with Docker and Jenkins
33+
34+
For this Continuous Integration demo, we will run a simple application that saves data on MongoDB. We will then run integration tests to check if the data was correctly saved on the database.
35+
36+
When running integration tests, you want to test your application in an environment as close to production as possible, so you can test interactions between the several components, services, databases, network communication, etc. Fortunately, docker can help you a lot with integration tests. There are several strategies to run integration tests, but in this application we are going to use the following:
37+
38+
. Start the services with a `docker-compose.yml` file created for testing purposes. This file won't have any volumes mapped, so when the test is over, no state will be saved. The test `docker-compose.yml` file won't publish any port on the host machine, so we can run simultaneous tests.
39+
. Run the application, using the services started with the `docker-compose.yml` test file.
40+
. Run Maven integration tests to check if the application execution produced the expected results. This will be done by checking what was saved on the MongoDB database.
41+
. Stop the services. No state will be stored, so next time you run the integration tests, you will have a clean environment.
42+
43+
Create a new job on jenkins:
44+
45+
1 . Select Freestyle project
46+
47+
image::docker-ci-cd-01.png[]
48+
49+
2 . In Source Code Management, select Git and add the repository URL: `https://github.com/fabianenardon/mongo-docker-demo.git`
50+
51+
image::docker-ci-cd-02.png[]
52+
53+
3 . In Build, select Add build step and select Execute shell
54+
55+
image::docker-ci-cd-03.png[]
56+
57+
4 . In the shell Command, add these instructions:
58+
59+
[source, text]
60+
----
61+
cd sample
62+
63+
# Generates the images
64+
sudo /var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/maven/bin/mvn clean install -Papp-docker-image
65+
66+
# Starts the mongo service. The -p option allows multiple builds to run at the same time,
67+
# since we can start multiple instances of the containers
68+
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml up -d mongo
69+
70+
# Waits for containers to start
71+
sleep 30
72+
73+
# Run the application
74+
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml \
75+
run mongo-docker-demo \
76+
java -jar /maven/jar/mongo-docker-demo-1.0-SNAPSHOT-jar-with-dependencies.jar mongo
77+
78+
# Run the integration tests
79+
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml \
80+
run mongo-docker-demo-tests \
81+
mvn -f /maven/code/pom.xml -Dmaven.repo.local=/m2/repository \
82+
-Pintegration-test verify
83+
----
84+
85+
5 . Click on Add post-build action and select Execute a set of scripts
86+
87+
image::docker-ci-cd-04.png[]
88+
89+
6 . In Post-build Actions, select Execute shell
90+
91+
image::docker-ci-cd-05.png[]
92+
93+
7 . In the Command box, add:
94+
95+
[source, text]
96+
----
97+
cd sample
98+
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml down
99+
----
100+
101+
8 . Uncheck the `Execute script only if build succeeds` and `Execute script only if build fails` options, so this script will run always when the build ends. This way, we make sure to always stop the services.
102+
103+
104+
[NOTE]
105+
====
106+
. The `-p app-$BUILD_NUMBER` option allows multiple builds to run at the same time, since we can start multiple instances of the containers. We are using Jenkins $BUILD_NUMBER variable to isolate the containers. This way, each set of services will run on its own network.
107+
. We are running the commands with sudo because we are actually running the Docker socket on the host. Jenkins runs with the jenkins user and we added the jenkins user to the sudoers list in our image. Obviously, this can have security consequences in a production environment, since one could create a build that would access root level services on the host. You can avoid this by configuring the jenkins user on the host, so it will have access to the Docker socket and then run the commands without sudo.
108+
====
109+
110+
111+
9 . Save the build and then click on `Build now` to run it. You should see a progress bar when the build is running. You can click on the progress bar to see the build console output.
112+
113+
image::docker-ci-cd-06.png[]
114+
115+
116+
10 . If the build is successful, you should see this on the build console output:
117+
118+
image::docker-ci-cd-07.png[]
119+
120+
== Running and debugging integration tests outside Jenkins
121+
122+
When creating integration tests, it is useful to be able to run and debug them outside Jenkins. In order to do that, you can simply run the same commands you ran in the Jenkins build:
123+
124+
[source, text]
125+
----
126+
# Generates the images
127+
mvn clean install -Papp-docker-image
128+
129+
# Starts mongo service
130+
docker-compose --file src/test/resources/docker-compose.yml up -d mongo
131+
132+
# Waits for services do start
133+
sleep 30
134+
135+
# Run our application
136+
docker-compose --file src/test/resources/docker-compose.yml \
137+
run mongo-docker-demo \
138+
java -jar /maven/jar/mongo-docker-demo-1.0-SNAPSHOT-jar-with-dependencies.jar mongo
139+
140+
# Run our integration tests
141+
docker-compose --file src/test/resources/docker-compose.yml \
142+
run mongo-docker-demo-tests mvn -f /maven/code/pom.xml \
143+
-Dmaven.repo.local=/m2/repository -Pintegration-test verify
144+
145+
# Stop all the services
146+
docker-compose --file src/test/resources/docker-compose.yml down
147+
----
148+
149+
150+
If you want to debug your integration tests, run the tests with this command:
151+
152+
[source, text]
153+
----
154+
# Run integration tests in debug mode
155+
docker run -v ~/.m2/repository:/m2/repository \
156+
-p 5005:5005 --link mongo:mongo \
157+
--net resources_default mongo-docker-demo-tests \
158+
mvn -f /maven/code/pom.xml \
159+
-Dmaven.repo.local=/m2/repository \
160+
-Pintegration-test verify -Dmaven.failsafe.debug
161+
----
162+
163+
This will make your test wait for a connection on port 5005 for debugging. You can then attach your IDE to this port and debug. Here is how this is done on Netbeans:
164+
165+
image::docker-ci-cd-08.png[]
166+
167+
image::docker-ci-cd-09.png[]
168+
169+
== Continuous Delivery with Docker and Jenkins
170+
171+
Continuous Delivery strategies depend greatly on the application architecture. With a dockerized application like the one in our demo, the continuous delivery strategy could be to publish a new version of the application image if the tests passed. This way, next time the application runs on production, the new image will be downloaded and automatically deployed. You can publish images with Jenkins just like you invoked all the other docker commands in the build.
172+
173+
174+

0 commit comments

Comments
 (0)