Skip to content

Commit 8b2bd68

Browse files
arun-guptaMano Marks
arun-gupta
authored and
Mano Marks
committed
Adding Docker for AWS chapter (docker#165)
* starting work on Docker for AWS * adding more steps * adding more structure * adding app removal output * renaming * adding more steps/details * updating some images and adding last bits
1 parent 573d941 commit 8b2bd68

18 files changed

+367
-3
lines changed

‎developer-tools/java/chapters/ch08-aws.adoc

+360
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,363 @@
22

33
= Docker for AWS
44

5+
https://docs.docker.com/docker-for-aws/[Docker for AWS] is a CloudFormation template that configures Docker in swarm-mode, running on EC2 instances backed by custom AMIs. This allows to create a cluster of Docker Engine in swarm-mode with a single click. This workshop will take the https://github.com/docker/labs/blob/master/developer-tools/java/chapters/ch06-swarm.adoc#multi-container-application[multi-container application] and deploy it on multiple hosts.
6+
7+
=== Requirements
8+
9+
What is required for creating this CloudFormation template?
10+
11+
. https://docs.docker.com/docker-for-aws/iam-permissions/[Permissions]
12+
. SSH key in AWS in the region where you want to deploy (required to access the completed Docker install). http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html[Amazon EC2 Key Pair] explains how to add SSH key to your account
13+
. AWS account that support EC2-VPC
14+
15+
=== Create swarm-mode cluster
16+
17+
https://console.aws.amazon.com/cloudformation/home#/stacks/new?stackName=Docker&templateURL=https://editions-us-east-1.s3.amazonaws.com/aws/stable/Docker.tmpl[Launch Stack] to create the CloudFormation template.
18+
19+
.Select template
20+
image::docker-aws-1.png[]
21+
22+
Click on `Next`
23+
24+
.Swarm size
25+
image::docker-aws-2.png[]
26+
27+
Select the number of Swarm manager (1) and worker (3) nodes. This wll create a 4 node cluster. Select the SSH key that will be used to access the cluster.
28+
29+
By default, the template is configured to redirect all log statements to CloudWatch. Until https://github.com/docker/docker/issues/30691[#30691] is fixed, the logs will only be available using CloudWatch. Alternatively, you may select to not redirect logs to CloudWatch. In this case, the usual command to get the logs will work.
30+
31+
Scroll down to select manager and worker properties.
32+
33+
.Swarm manager/worker properties
34+
image::docker-aws-3.png[]
35+
36+
`m3.medium` (1 vCPU and 3.75 GB memory) is a good start for manager. `m3.large` (2 vCPU and 7.5 GB memory) is a good start for worker node. Make sure the EC2 instance size is chosen to accommodate the processing and memory needs of containers that will run there.
37+
38+
Click on `Next`
39+
40+
.Swarm options
41+
image::docker-aws-4.png[]
42+
43+
Take default for all the options and click on `Next`.
44+
45+
.Swarm review
46+
image::docker-aws-5.png[]
47+
48+
.Swarm IAM accept
49+
image::docker-aws-6.png[]
50+
51+
Accept the checkbox for CloudFormation to create IAM resources. Click on `Create` to create the Swarm cluster.
52+
53+
It will take a few minutes for the CloudFormation template to complete. For example, it took about 10-12 minutes for this cluster to be created in `us-west-2` region. The output will look like:
54+
55+
.Swarm CloudFormation complete
56+
image::docker-aws-7.png[]
57+
58+
https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Instances:search=docker;sort=instanceState[EC2 Console] will show the EC2 instances for manager and worker.
59+
60+
.EC2 console
61+
image::docker-aws-8.png[]
62+
63+
Select the manager node, copy the public IP address:
64+
65+
[[Swarm_manager]]
66+
.Swarm manager
67+
image::docker-aws-9.png[]
68+
69+
Create a SSH tunnel using the command `ssh -i ~/.ssh/arun-cb-west2.pem -NL localhost:2374:/var/run/docker.sock docker@ec2-52-37-65-169.us-west-2.compute.amazonaws.com &`
70+
71+
Get more details about the cluster using the command `docker -H localhost:2374 info`. This shows the output:
72+
73+
```
74+
Containers: 5
75+
Running: 4
76+
Paused: 0
77+
Stopped: 1
78+
Images: 5
79+
Server Version: 1.13.0
80+
Storage Driver: overlay2
81+
Backing Filesystem: extfs
82+
Supports d_type: true
83+
Native Overlay Diff: true
84+
Logging Driver: awslogs
85+
Cgroup Driver: cgroupfs
86+
Plugins:
87+
Volume: local
88+
Network: bridge host ipvlan macvlan null overlay
89+
Swarm: active
90+
NodeID: ep8668sq4y8n7qdkvm8l2lecf
91+
Is Manager: true
92+
ClusterID: mw186ukvx9rx5h87vxzkr0ic0
93+
Managers: 1
94+
Nodes: 4
95+
Orchestration:
96+
Task History Retention Limit: 5
97+
Raft:
98+
Snapshot Interval: 10000
99+
Number of Old Snapshots to Retain: 0
100+
Heartbeat Tick: 1
101+
Election Tick: 3
102+
Dispatcher:
103+
Heartbeat Period: 5 seconds
104+
CA Configuration:
105+
Expiry Duration: 3 months
106+
Node Address: 172.31.42.42
107+
Manager Addresses:
108+
172.31.42.42:2377
109+
Runtimes: runc
110+
Default Runtime: runc
111+
Init Binary: docker-init
112+
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
113+
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
114+
init version: 949e6fa
115+
Security Options:
116+
seccomp
117+
Profile: default
118+
Kernel Version: 4.9.4-moby
119+
Operating System: Alpine Linux v3.5
120+
OSType: linux
121+
Architecture: x86_64
122+
CPUs: 1
123+
Total Memory: 3.67 GiB
124+
Name: ip-172-31-42-42.us-west-2.compute.internal
125+
ID: NNAE:BGOF:DU6D:DE2V:TLEO:PBUL:CD5S:H5QB:MEA5:DBAW:DTIQ:ASVP
126+
Docker Root Dir: /var/lib/docker
127+
Debug Mode (client): false
128+
Debug Mode (server): true
129+
File Descriptors: 69
130+
Goroutines: 182
131+
System Time: 2017-02-02T19:35:33.882319271Z
132+
EventsListeners: 0
133+
Username: arungupta
134+
Registry: https://index.docker.io/v1/
135+
Experimental: true
136+
Insecure Registries:
137+
127.0.0.0/8
138+
Live Restore Enabled: false
139+
```
140+
141+
List of nodes in the cluster can be seen using `docker -H localhost:2374 node ls`:
142+
143+
```
144+
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
145+
4gj5tt6f2rtv9bmmtegn3sw7l ip-172-31-22-34.us-west-2.compute.internal Ready Active
146+
jul7u4x2yue1pz46lxb62n3lt * ip-172-31-45-44.us-west-2.compute.internal Ready Active Leader
147+
trg4x49872k5w178q306pljhz ip-172-31-36-119.us-west-2.compute.internal Ready Active
148+
zyg7i7pki0jqdq9kjzp92vq0j ip-172-31-7-184.us-west-2.compute.internal Ready Active
149+
```
150+
151+
=== Multi-container application to multi-host
152+
153+
Use the Compose file as defined at https://github.com/docker/labs/blob/master/developer-tools/java/chapters/ch06-swarm.adoc#multi-container-application[Multi-container Application] to deploy a multi-container application to this Docker cluster. This will deploy a multi-container application to multiple hosts. The command is:
154+
155+
```
156+
docker -H localhost:2374 stack deploy --compose-file=docker-compose.yml webapp
157+
```
158+
159+
The output is:
160+
161+
```
162+
Creating network webapp_default
163+
Creating service webapp_db
164+
Creating service webapp_web
165+
```
166+
167+
WildFly and Couchbase services are started on this cluster. Each service has a single container. A new overlay network is created. This allows multiple containers on different hosts to communicate with each other.
168+
169+
=== Verify service/containers in application
170+
171+
Verify that the WildFly and Couchbase services are running using `docker -H localhost:2374 service ls`:
172+
173+
```
174+
ID NAME MODE REPLICAS IMAGE
175+
bfi9s7t5sdjo webapp_db replicated 1/1 arungupta/couchbase:travel
176+
ij04s9di00xw webapp_web replicated 1/1 arungupta/couchbase-javaee:travel
177+
```
178+
179+
`REPLICAS` colum shows that one of one replica for the container is running for each service. It might take a few minutes for the service to be running as the image needs to be downloaded on the host where the container is started.
180+
181+
More details about the service can be obtained using `docker -H localhost:2374 service inspect webapp_web`:
182+
183+
```
184+
[
185+
{
186+
"ID": "ssf0kj0hagl7c1tcpw8bbsiue",
187+
"Version": {
188+
"Index": 29
189+
},
190+
"CreatedAt": "2017-02-02T22:38:20.424806786Z",
191+
"UpdatedAt": "2017-02-02T22:38:20.428265482Z",
192+
"Spec": {
193+
"Name": "webapp_web",
194+
"Labels": {
195+
"com.docker.stack.namespace": "webapp"
196+
},
197+
"TaskTemplate": {
198+
"ContainerSpec": {
199+
"Image": "arungupta/couchbase-javaee:travel@sha256:e48e05c0327e30e1d11f226b7b68e403e6c9c8d977bf09cb23188c6fff46bf39",
200+
"Labels": {
201+
"com.docker.stack.namespace": "webapp"
202+
},
203+
"Env": [
204+
"COUCHBASE_URI=db"
205+
]
206+
},
207+
"Resources": {},
208+
"Placement": {},
209+
"ForceUpdate": 0
210+
},
211+
"Mode": {
212+
"Replicated": {
213+
"Replicas": 1
214+
}
215+
},
216+
"Networks": [
217+
{
218+
"Target": "poh9n7fbrl3mlue6lkl6qwbst",
219+
"Aliases": [
220+
"web"
221+
]
222+
}
223+
],
224+
"EndpointSpec": {
225+
"Mode": "vip",
226+
"Ports": [
227+
{
228+
"Protocol": "tcp",
229+
"TargetPort": 8080,
230+
"PublishedPort": 8080,
231+
"PublishMode": "ingress"
232+
},
233+
{
234+
"Protocol": "tcp",
235+
"TargetPort": 9990,
236+
"PublishedPort": 9990,
237+
"PublishMode": "ingress"
238+
}
239+
]
240+
}
241+
},
242+
"Endpoint": {
243+
"Spec": {
244+
"Mode": "vip",
245+
"Ports": [
246+
{
247+
"Protocol": "tcp",
248+
"TargetPort": 8080,
249+
"PublishedPort": 8080,
250+
"PublishMode": "ingress"
251+
},
252+
{
253+
"Protocol": "tcp",
254+
"TargetPort": 9990,
255+
"PublishedPort": 9990,
256+
"PublishMode": "ingress"
257+
}
258+
]
259+
},
260+
"Ports": [
261+
{
262+
"Protocol": "tcp",
263+
"TargetPort": 8080,
264+
"PublishedPort": 8080,
265+
"PublishMode": "ingress"
266+
},
267+
{
268+
"Protocol": "tcp",
269+
"TargetPort": 9990,
270+
"PublishedPort": 9990,
271+
"PublishMode": "ingress"
272+
}
273+
],
274+
"VirtualIPs": [
275+
{
276+
"NetworkID": "vsr5otzk5gwz7afwafjmiiv40",
277+
"Addr": "10.255.0.7/16"
278+
},
279+
{
280+
"NetworkID": "poh9n7fbrl3mlue6lkl6qwbst",
281+
"Addr": "10.0.0.2/24"
282+
}
283+
]
284+
},
285+
"UpdateStatus": {
286+
"StartedAt": "0001-01-01T00:00:00Z",
287+
"CompletedAt": "0001-01-01T00:00:00Z"
288+
}
289+
}
290+
]
291+
```
292+
293+
Logs for the service cannot be seen using `docker service logs`. This will be fixed with https://github.com/docker/docker/issues/30691[#30691]. Instead they are visible using https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logs:prefix=Docker[CloudWatch Logs].
294+
295+
.CloudWatch log group
296+
image::docker-aws-10.png[]
297+
298+
Select the log group:
299+
300+
.CloudWatch log stream
301+
image::docker-aws-11.png[]
302+
303+
Pick `webapp_db.xxx` log stream to see log statements from the Couchbase image:
304+
305+
.CloudWatch database log stream
306+
image::docker-aws-12.png[]
307+
308+
Pick `webapp_db.xxx` log stream to see log statements from the WildFly application server:
309+
310+
.CloudWatch application log stream
311+
image::docker-aws-13.png[]
312+
313+
=== Access application
314+
315+
Application is accessed using manager's IP address and on port 8080. By default, the port 8080 is not open. In <<Swarm_manager>>, click on `Docker-Managerxxx` in `Security groups`. Click on `Inbound`, `Edit`, `Add Rule`, and create a rule to enable TCP traffic on port 8080.
316+
317+
.Open port 8080 in Docker manager
318+
image::docker-aws-14.png[]
319+
320+
Click on `Save` to save the rules.
321+
322+
Now, the application is accessible using the command `curl -v http://ec2-52-37-65-169.us-west-2.compute.amazonaws.com:8080/airlines/resources/airline` and shows output:
323+
324+
```
325+
* Trying 52.37.65.169...
326+
* Connected to ec2-52-37-65-169.us-west-2.compute.amazonaws.com (52.37.65.169) port 8080 (#0)
327+
> GET /airlines/resources/airline HTTP/1.1
328+
> Host: ec2-52-37-65-169.us-west-2.compute.amazonaws.com:8080
329+
> User-Agent: curl/7.43.0
330+
> Accept: */*
331+
>
332+
< HTTP/1.1 200 OK
333+
< Connection: keep-alive
334+
< X-Powered-By: Undertow/1
335+
< Server: WildFly/10
336+
< Content-Type: application/octet-stream
337+
< Content-Length: 1402
338+
< Date: Thu, 02 Feb 2017 23:42:41 GMT
339+
<
340+
* Connection #0 to host ec2-52-37-65-169.us-west-2.compute.amazonaws.com left intact
341+
[{"travel-sample":{"country":"United States","iata":"Q5","callsign":"MILE-AIR","name":"40-Mile Air","icao":"MLA","id":10,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"TQ","callsign":"TXW","name":"Texas Wings","icao":"TXW","id":10123,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"A1","callsign":"atifly","name":"Atifly","icao":"A1F","id":10226,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":null,"callsign":null,"name":"Jc royal.britannica","icao":"JRB","id":10642,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"ZQ","callsign":"LOCAIR","name":"Locair","icao":"LOC","id":10748,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"K5","callsign":"SASQUATCH","name":"SeaPort Airlines","icao":"SQH","id":10765,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"KO","callsign":"ACE AIR","name":"Alaska Central Express","icao":"AER","id":109,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"5W","callsign":"FLYSTAR","name":"Astraeus","icao":"AEU","id":112,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"UU","callsign":"REUNION","name":"Air Austral","icao":"REU","id":1191,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":1203,"type":"airline"}}]
342+
```
343+
344+
Complete set of commands are at https://github.com/docker/labs/blob/master/developer-tools/java/chapters/ch06-swarm.adoc#access-application.
345+
346+
=== Shutdown application
347+
348+
Shutdown the application using the command `docker -H localhost:2374 stack rm webapp`:
349+
350+
```
351+
Removing service webapp_db
352+
Removing service webapp_web
353+
Removing network webapp_default
354+
```
355+
356+
This stops the container in each service and removes the services. It also deletes any networks that were created as part of this application.
357+
358+
=== Shutdown cluster
359+
360+
Docker cluster can be shutdown by deleting the stack created by CloudFormataion:
361+
362+
.Delete CloudFormation template
363+
image::docker-aws-15.png[]
364+

‎developer-tools/java/chapters/ch10-cli.adoc

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
:imagesdir: images
2+
3+
= Monitoring Docker Container Using Docker CLI
4+

‎developer-tools/java/chapters/images/docker-aws-1.png

311 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-10.png

71 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-11.png

58.1 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-12.png

260 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-13.png

561 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-14.png

130 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-15.png

59.5 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-2.png

280 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-3.png

225 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-4.png

246 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-5.png

297 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-6.png

222 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-7.png

171 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-8.png

119 KB
Loading

‎developer-tools/java/chapters/images/docker-aws-9.png

175 KB
Loading

‎developer-tools/java/readme.adoc

+3-3
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,14 @@ This tutorial offers Java developers an intro-level and self-paced hands-on work
66
* link:chapters/ch02-basic-concepts.adoc[Docker Basic Concepts]
77
* link:chapters/ch03-build-image.adoc[Build a Docker Image]
88
* link:chapters/ch04-run-container.adoc[Run a Docker Container]
9-
* link:chapters/ch05-compose.adoc[Mult-container Application Using Compose]
10-
* link:chapters/ch06-swarm.adoc[Multi-container Using Compose and Swarm Mode]
9+
* link:chapters/ch05-compose.adoc[Mult-container application using Compose]
10+
* link:chapters/ch06-swarm.adoc[Multi-container application on multi-host using Compose and Swarm Mode]
1111
* Java IDEs
1212
** link:chapters/ch07-netbeans.adoc[Docker Tooling in NetBeans]
1313
** link:chapters/ch07-intellij.adoc[Docker Tooling in IntelliJ IDEA]
1414
** link:chapters/ch07-eclipse.adoc[Docker Tooling in Eclipse]
1515
* Docker in Cloud
16-
** link:chapters/ch08-aws.adoc[Docker for AWS] (coming)
16+
** link:chapters/ch08-aws.adoc[Docker for AWS]
1717
** link:chapters/ch08-azure.adoc[Docker for Azure] (coming)
1818
** link:chapters/ch08-cloud.adoc[Docker Cloud]
1919
* link:chapters/ch09-cicd.adoc[CI/CD using Docker] (coming)

0 commit comments

Comments
 (0)