1 00:00:00,280 --> 00:00:01,170 ‫Now let's talk about 2 00:00:01,170 --> 00:00:04,350 ‫Amazon ECS Task Definitions, but in depth. 3 00:00:04,350 --> 00:00:06,240 ‫So you define them in JSON form 4 00:00:06,240 --> 00:00:07,180 ‫but through the console 5 00:00:07,180 --> 00:00:11,000 ‫there is a UI to help you create the JSON, 6 00:00:11,000 --> 00:00:14,020 ‫and the task definition tells the ECS service, 7 00:00:14,020 --> 00:00:18,090 ‫how to run a or multiple Docker containers on ECS. 8 00:00:18,090 --> 00:00:20,030 ‫And there is crucial information 9 00:00:20,030 --> 00:00:23,170 ‫within your task definition, such as the Image Name, 10 00:00:23,170 --> 00:00:25,470 ‫the Port Binding for the Container 11 00:00:25,470 --> 00:00:28,210 ‫and the Host, if you're on EC2. 12 00:00:28,210 --> 00:00:31,880 ‫The memory and the CPU required for your container. 13 00:00:31,880 --> 00:00:35,090 ‫The environment variables, the networking information, 14 00:00:35,090 --> 00:00:38,540 ‫the IAM role attached to the task definition, 15 00:00:38,540 --> 00:00:39,880 ‫and the login configuration 16 00:00:39,880 --> 00:00:41,490 ‫such as, for example, CloudWatch. 17 00:00:41,490 --> 00:00:42,730 ‫So there's more information as well 18 00:00:42,730 --> 00:00:44,370 ‫but these are the most important 19 00:00:44,370 --> 00:00:46,330 ‫and the exam will test you on a few of these. 20 00:00:46,330 --> 00:00:49,860 ‫So, I will do a deep dive on some of that in this lecture. 21 00:00:49,860 --> 00:00:50,740 ‫So let's take an example. 22 00:00:50,740 --> 00:00:52,370 ‫We have an EC2 instance, 23 00:00:52,370 --> 00:00:54,970 ‫and because it is registered with an ECS cluster, 24 00:00:54,970 --> 00:00:57,930 ‫it has to be running the ECS agent. 25 00:00:57,930 --> 00:01:00,130 ‫Next, we're going to run a Docker container 26 00:01:00,130 --> 00:01:01,640 ‫through ECS test definition, 27 00:01:01,640 --> 00:01:05,260 ‫for example, an Apache HTP server. 28 00:01:05,260 --> 00:01:08,600 ‫And we have to expose that server to the internet. 29 00:01:08,600 --> 00:01:12,240 ‫Therefore, we are going to define a container port 30 00:01:12,240 --> 00:01:15,610 ‫on 80, meaning that on the container , 31 00:01:15,610 --> 00:01:20,610 ‫the port 80 is the one that is exposing the HTP server. 32 00:01:20,870 --> 00:01:22,680 ‫But then we have also the host port 33 00:01:22,680 --> 00:01:24,330 ‫because we are on EC2. 34 00:01:24,330 --> 00:01:26,200 ‫If we were on Fargate, that would not be relevant, 35 00:01:26,200 --> 00:01:27,810 ‫but we are on EC2. 36 00:01:27,810 --> 00:01:30,790 ‫And therefore we need to map this container port 37 00:01:30,790 --> 00:01:33,690 ‫to a host port, which is for example, 38 00:01:33,690 --> 00:01:35,840 ‫could be 80 but it could be also at 8080. 39 00:01:35,840 --> 00:01:37,720 ‫So they don't have to be the same. 40 00:01:37,720 --> 00:01:41,590 ‫And then thanks to the host port, then the internet 41 00:01:41,590 --> 00:01:44,590 ‫or an external network communication 42 00:01:44,590 --> 00:01:47,250 ‫is able to access the EC2 instance, 43 00:01:47,250 --> 00:01:49,390 ‫on port 8080, the host port, 44 00:01:49,390 --> 00:01:52,660 ‫which is going to be directed to the container port 80, 45 00:01:52,660 --> 00:01:55,053 ‫and then will get access to the HTP server. 46 00:01:56,190 --> 00:01:57,023 ‫Okay. 47 00:01:57,023 --> 00:01:58,610 ‫And you should know that you can define 48 00:01:58,610 --> 00:02:00,550 ‫more than one container per task definition. 49 00:02:00,550 --> 00:02:05,090 ‫You can define up to 10 containers per task definition. 50 00:02:05,090 --> 00:02:08,320 ‫So let's do a deep dive onto first the container port. 51 00:02:08,320 --> 00:02:09,590 ‫So if you have load balancing 52 00:02:09,590 --> 00:02:11,770 ‫and you're using the EC2 launch type, 53 00:02:11,770 --> 00:02:12,750 ‫then you're going to get 54 00:02:12,750 --> 00:02:15,700 ‫what's called a Dynamic Host Port Mapping. 55 00:02:15,700 --> 00:02:18,010 ‫If you define only the container port 56 00:02:18,010 --> 00:02:19,600 ‫and the task definition. 57 00:02:19,600 --> 00:02:20,620 ‫Let me explain. 58 00:02:20,620 --> 00:02:23,450 ‫So we are running for example, an ECS task, 59 00:02:23,450 --> 00:02:25,810 ‫and all of them have the container port 60 00:02:25,810 --> 00:02:30,170 ‫set to 80 but the host port set to zero, 61 00:02:30,170 --> 00:02:31,710 ‫meaning not set. 62 00:02:31,710 --> 00:02:34,000 ‫What's going to happen is that the host port only 63 00:02:34,000 --> 00:02:36,150 ‫is represented in this diagram, 64 00:02:36,150 --> 00:02:38,300 ‫but the host port is going to be random, 65 00:02:38,300 --> 00:02:39,620 ‫is going to be dynamic. 66 00:02:39,620 --> 00:02:41,580 ‫And so, each ECS task 67 00:02:41,580 --> 00:02:43,670 ‫from within the EC2 instance, 68 00:02:43,670 --> 00:02:47,200 ‫is going to be accessible from a different port on the host, 69 00:02:47,200 --> 00:02:48,810 ‫the EC2 instance. 70 00:02:48,810 --> 00:02:52,600 ‫And therefore, if you define an application of that answer 71 00:02:52,600 --> 00:02:53,450 ‫then you may say, well, 72 00:02:53,450 --> 00:02:56,160 ‫it is difficult for the ALB to connect 73 00:02:56,160 --> 00:02:59,280 ‫to the ECS test because the port is changing. 74 00:02:59,280 --> 00:03:03,560 ‫But the ALB when linked to an ECS service 75 00:03:03,560 --> 00:03:05,580 ‫knows how to find the right port, 76 00:03:05,580 --> 00:03:08,740 ‫thanks to the Dynamic Host Port Mapping feature. 77 00:03:08,740 --> 00:03:09,940 ‫And so, the ALB, 78 00:03:09,940 --> 00:03:12,760 ‫automatically thanks to the ECS service, 79 00:03:12,760 --> 00:03:15,290 ‫knows to connect to different ports 80 00:03:15,290 --> 00:03:18,630 ‫onto different instances automatically. 81 00:03:18,630 --> 00:03:19,463 ‫And so this setup works, 82 00:03:19,463 --> 00:03:22,510 ‫but it does not work with a classic load balancer 83 00:03:22,510 --> 00:03:24,320 ‫because it is older generation. 84 00:03:24,320 --> 00:03:28,211 ‫So this logic only happens with the ALB. 85 00:03:28,211 --> 00:03:30,710 ‫And so therefore from a security perspective, 86 00:03:30,710 --> 00:03:35,230 ‫well the EC2 instance security group, must allow any port, 87 00:03:35,230 --> 00:03:37,210 ‫from the ALB security group, 88 00:03:37,210 --> 00:03:38,820 ‫because we don't know in advance 89 00:03:38,820 --> 00:03:40,963 ‫what is going to be the host port mapping. 90 00:03:41,910 --> 00:03:43,750 ‫So that was for the EC2 launch type. 91 00:03:43,750 --> 00:03:46,860 ‫But now what happens when we have the Fargate launch type? 92 00:03:46,860 --> 00:03:48,960 ‫Well, each ECS task is going to get 93 00:03:48,960 --> 00:03:51,580 ‫a unique private IP this time. 94 00:03:51,580 --> 00:03:52,850 ‫And so, because this is Fargate, 95 00:03:52,850 --> 00:03:53,760 ‫there is no host, 96 00:03:53,760 --> 00:03:57,550 ‫and therefore we only have to define the container ports. 97 00:03:57,550 --> 00:03:59,940 ‫And so if you look at your ECS cluster, 98 00:03:59,940 --> 00:04:01,810 ‫then for example, with four tasks 99 00:04:01,810 --> 00:04:05,280 ‫each task is going to get its own private IP 100 00:04:05,280 --> 00:04:08,564 ‫through an Elastic Network Interface or ENI. 101 00:04:08,564 --> 00:04:10,350 ‫And then each ENI is going to get 102 00:04:10,350 --> 00:04:12,350 ‫the same container ports. 103 00:04:12,350 --> 00:04:15,230 ‫And so this is a setup you're going to get with Fargate. 104 00:04:15,230 --> 00:04:17,320 ‫And therefore, when you have an ALB, 105 00:04:17,320 --> 00:04:19,270 ‫then to connect to the Fargate task, 106 00:04:19,270 --> 00:04:21,200 ‫it's just going to connect to all all of them 107 00:04:21,200 --> 00:04:23,390 ‫on the same port on port 80. 108 00:04:23,390 --> 00:04:27,130 ‫So there is the ECS ENI Security Group 109 00:04:27,130 --> 00:04:31,410 ‫that needs to allow port 80, from the ALB security group. 110 00:04:31,410 --> 00:04:33,410 ‫And then the ALB security group 111 00:04:33,410 --> 00:04:36,160 ‫needs to allow just to port 80 or 443, 112 00:04:36,160 --> 00:04:39,033 ‫if you have SSL enabled, from the web. 113 00:04:40,090 --> 00:04:43,850 ‫Next, the exam will ask you about IAM roles within ECS. 114 00:04:43,850 --> 00:04:46,980 ‫And you should know that IAM roles are assigned 115 00:04:46,980 --> 00:04:48,950 ‫per task definition. 116 00:04:48,950 --> 00:04:50,660 ‫So you have a task definition 117 00:04:50,660 --> 00:04:53,940 ‫and then you assign an ECS task role. 118 00:04:53,940 --> 00:04:56,320 ‫And this will allow you, for example, 119 00:04:56,320 --> 00:05:00,000 ‫for your ECS tasks out of your task definition, 120 00:05:00,000 --> 00:05:02,630 ‫to access the Amazon S3 service. 121 00:05:02,630 --> 00:05:05,060 ‫And therefore when you create an ECS service 122 00:05:05,060 --> 00:05:07,010 ‫from this task definition 123 00:05:07,010 --> 00:05:09,800 ‫then each ECS task automatically 124 00:05:09,800 --> 00:05:14,630 ‫is going to assume and inherit this ECS task role. 125 00:05:14,630 --> 00:05:16,760 ‫But you should know that the role is defined 126 00:05:16,760 --> 00:05:20,410 ‫at the task definition level, not at this service level. 127 00:05:20,410 --> 00:05:22,100 ‫And so, therefore all the tasks 128 00:05:22,100 --> 00:05:25,750 ‫within your service, are going to get access to Amazon S3. 129 00:05:25,750 --> 00:05:28,090 ‫And if you define another task definition, 130 00:05:28,090 --> 00:05:30,170 ‫you can add another role on it. 131 00:05:30,170 --> 00:05:32,570 ‫And this role, for example, can access DynamoDB. 132 00:05:32,570 --> 00:05:34,680 ‫And if you were to create another service, 133 00:05:34,680 --> 00:05:37,090 ‫then that service would assume this other role 134 00:05:37,090 --> 00:05:38,460 ‫and you would be good to go. 135 00:05:38,460 --> 00:05:40,010 ‫So the exam will ask you, 136 00:05:40,010 --> 00:05:43,970 ‫where do you define an IAM role for ECS task? 137 00:05:43,970 --> 00:05:47,143 ‫And the answer is, on your task definition. 138 00:05:48,170 --> 00:05:50,130 ‫Next, you have environment variables. 139 00:05:50,130 --> 00:05:52,850 ‫So, your task definition, can have environment variables 140 00:05:52,850 --> 00:05:54,890 ‫and they can come from multiple places. 141 00:05:54,890 --> 00:05:56,310 ‫You can hard code them. 142 00:05:56,310 --> 00:05:58,500 ‫For example, you set them directly 143 00:05:58,500 --> 00:06:00,630 ‫from within the test definition. 144 00:06:00,630 --> 00:06:01,463 ‫And for example, 145 00:06:01,463 --> 00:06:05,570 ‫this is when you will have a fixed non-secret URL. 146 00:06:05,570 --> 00:06:07,890 ‫But then if you have sensitive variables, 147 00:06:07,890 --> 00:06:10,840 ‫such API keys or shared configs, 148 00:06:10,840 --> 00:06:12,830 ‫or for example, database passwords, 149 00:06:12,830 --> 00:06:16,240 ‫then you can use either the SSM Parameter Store, 150 00:06:16,240 --> 00:06:19,230 ‫or Secrets manager, to store these values. 151 00:06:19,230 --> 00:06:23,510 ‫And you reference them from within the ECS task definition 152 00:06:23,510 --> 00:06:26,000 ‫and upon launching an ECS task 153 00:06:26,000 --> 00:06:28,540 ‫then these values are going to be fetched 154 00:06:28,540 --> 00:06:30,960 ‫and resolved at run time 155 00:06:30,960 --> 00:06:35,220 ‫and injected as environment variables, within your ECS task. 156 00:06:35,220 --> 00:06:36,950 ‫Finally, there is a last option 157 00:06:36,950 --> 00:06:40,100 ‫where you load your ECS environment variables 158 00:06:40,100 --> 00:06:43,150 ‫directly from an Amazon S3 bucket. 159 00:06:43,150 --> 00:06:45,890 ‫And this is called a bulk environment 160 00:06:45,890 --> 00:06:47,593 ‫variables loading through a file. 161 00:06:48,910 --> 00:06:51,950 ‫Next we have, how do we share data between ECS tasks? 162 00:06:51,950 --> 00:06:55,500 ‫So, as I said, an ECS task can contain one container 163 00:06:55,500 --> 00:06:57,580 ‫but also you can define multiple containers 164 00:06:57,580 --> 00:06:59,710 ‫in the same task definition. 165 00:06:59,710 --> 00:07:01,780 ‫And you would do that because sometimes, 166 00:07:01,780 --> 00:07:04,520 ‫your side containers also called side cars 167 00:07:04,520 --> 00:07:07,307 ‫can help you with lugging, with tracing and so on. 168 00:07:07,307 --> 00:07:09,350 ‫And so, it's a common pattern. 169 00:07:09,350 --> 00:07:11,480 ‫But sometimes, well for example, for lugging 170 00:07:11,480 --> 00:07:12,930 ‫and for metrics and so on, 171 00:07:12,930 --> 00:07:16,540 ‫these containers need to share some files together. 172 00:07:16,540 --> 00:07:19,930 ‫And therefore, we must mount a data volume 173 00:07:19,930 --> 00:07:23,343 ‫onto both containers, 174 00:07:24,410 --> 00:07:26,110 ‫and then they will be able to share data. 175 00:07:26,110 --> 00:07:28,650 ‫And so, this data volume, bind them out, 176 00:07:28,650 --> 00:07:31,090 ‫works for both EC2 and Fargate tasks. 177 00:07:31,090 --> 00:07:32,870 ‫So let's imagine we have an ECS task 178 00:07:32,870 --> 00:07:34,900 ‫and we have the application containers. 179 00:07:34,900 --> 00:07:36,370 ‫It could be one, it could be many. 180 00:07:36,370 --> 00:07:38,470 ‫And then some side car containers, 181 00:07:38,470 --> 00:07:41,330 ‫for example the metrics and logs container. 182 00:07:41,330 --> 00:07:43,410 ‫You're going to create a bind mount, 183 00:07:43,410 --> 00:07:45,280 ‫and it's going to create a shared storage 184 00:07:45,280 --> 00:07:47,710 ‫that you have to define between your task. 185 00:07:47,710 --> 00:07:51,010 ‫And you say, for example, it is /var/logs 186 00:07:51,010 --> 00:07:52,930 ‫and so therefore your application containers 187 00:07:52,930 --> 00:07:54,640 ‫are going to be able to write 188 00:07:54,640 --> 00:07:56,540 ‫to this shared storage, 189 00:07:56,540 --> 00:07:58,720 ‫and your metrics and log container 190 00:07:58,720 --> 00:08:01,300 ‫can read from these shared storage. 191 00:08:01,300 --> 00:08:04,070 ‫And this is the whole idea behind the bind mount. 192 00:08:04,070 --> 00:08:05,810 ‫So if you use EC2 tasks 193 00:08:05,810 --> 00:08:09,590 ‫then the bind Mount itself is the EC2 instant storage. 194 00:08:09,590 --> 00:08:12,290 ‫And therefore the data of that mount is tied 195 00:08:12,290 --> 00:08:14,830 ‫to lifecycle of the EC2 instance. 196 00:08:14,830 --> 00:08:17,940 ‫Or, for Fargate task, then you use ephemeral storage, 197 00:08:17,940 --> 00:08:20,690 ‫and the data is tied to the container, 198 00:08:20,690 --> 00:08:21,523 ‫a lifecycle using them. 199 00:08:21,523 --> 00:08:24,140 ‫And so whenever your Fargate task disappears, 200 00:08:24,140 --> 00:08:26,270 ‫then your storage disappears as well. 201 00:08:26,270 --> 00:08:30,420 ‫Now, on Fargate you get from 20 gigabytes to 200 gigabytes 202 00:08:30,420 --> 00:08:31,420 ‫of shared storage, 203 00:08:31,420 --> 00:08:34,790 ‫so it gives you lot of space for different use cases. 204 00:08:34,790 --> 00:08:35,870 ‫So between them. 205 00:08:35,870 --> 00:08:38,910 ‫So the basic use case, 206 00:08:38,910 --> 00:08:40,760 ‫especially from the exam perspective 207 00:08:40,760 --> 00:08:43,420 ‫is to share data between multiple containers 208 00:08:43,420 --> 00:08:46,200 ‫or when you have a side car container 209 00:08:46,200 --> 00:08:48,510 ‫where the side car is used to, for example 210 00:08:48,510 --> 00:08:51,650 ‫send metrics or logs to other destinations. 211 00:08:51,650 --> 00:08:54,690 ‫And it needs to read from a shared storage. 212 00:08:54,690 --> 00:08:55,523 ‫Okay. 213 00:08:55,523 --> 00:08:59,010 ‫So that's it for the Amazon ECS task definition, deep dive. 214 00:08:59,010 --> 00:09:00,130 ‫I hope you liked it. 215 00:09:00,130 --> 00:09:02,080 ‫And I will see you in the next lecture.