1 00:00:00,490 --> 00:00:01,323 ‫Okay. 2 00:00:01,323 --> 00:00:02,470 ‫So it's very important that you understand 3 00:00:02,470 --> 00:00:06,630 ‫the difference between SQS, SNS and Kinesis. 4 00:00:06,630 --> 00:00:09,880 ‫So SQS has a model where consumers pull data 5 00:00:09,880 --> 00:00:13,730 ‫by requesting messages from the SQSQ. 6 00:00:13,730 --> 00:00:15,870 ‫And once the data is processed 7 00:00:15,870 --> 00:00:18,310 ‫then the consumer has to delete it from the queue 8 00:00:18,310 --> 00:00:21,460 ‫so that's no other consumers can read it ever again. 9 00:00:21,460 --> 00:00:23,510 ‫You can have as many workers as you want. 10 00:00:23,510 --> 00:00:24,710 ‫So as many consumers as you want 11 00:00:24,710 --> 00:00:27,210 ‫and they all work together to consume 12 00:00:27,210 --> 00:00:29,420 ‫and delete all the messages from the queue. 13 00:00:29,420 --> 00:00:31,240 ‫You don't need to provision throughput in advance 14 00:00:31,240 --> 00:00:32,780 ‫because it is a managed service 15 00:00:32,780 --> 00:00:35,160 ‫and it can scale to hundreds of thousands 16 00:00:35,160 --> 00:00:37,290 ‫of messages very quickly. 17 00:00:37,290 --> 00:00:39,670 ‫The ordering guarantees are only available 18 00:00:39,670 --> 00:00:40,880 ‫if you enable FIFO queues. 19 00:00:40,880 --> 00:00:43,100 ‫So first it in first out type of queues. 20 00:00:43,100 --> 00:00:46,120 ‫And you also have individual message delay capability 21 00:00:46,120 --> 00:00:49,690 ‫if you want a message to appear to a consumer in the queue 22 00:00:49,690 --> 00:00:52,840 ‫after a certain amount of time, for example, 30 seconds. 23 00:00:52,840 --> 00:00:55,140 ‫When you look at SNS, this is a different model. 24 00:00:55,140 --> 00:00:56,200 ‫This is a pup sub model. 25 00:00:56,200 --> 00:00:58,710 ‫So you push data to many subscribers 26 00:00:58,710 --> 00:01:00,980 ‫and they will all receive a copy 27 00:01:00,980 --> 00:01:02,760 ‫of the message you're sending. 28 00:01:02,760 --> 00:01:07,760 ‫You can get up to 12,500,000 subscribers per SNS topic. 29 00:01:07,960 --> 00:01:11,570 ‫And once the data is sent to SNS, it is not persistent. 30 00:01:11,570 --> 00:01:13,780 ‫That means that if it is not delivered 31 00:01:13,780 --> 00:01:15,910 ‫you have a chance of losing it. 32 00:01:15,910 --> 00:01:16,810 ‫So it's a pub, as I said 33 00:01:16,810 --> 00:01:20,180 ‫and you can scale to hundreds of thousands of topics. 34 00:01:20,180 --> 00:01:22,350 ‫You don't need to provision throughput as well. 35 00:01:22,350 --> 00:01:26,120 ‫And if you want to combine it with SQS, you can. 36 00:01:26,120 --> 00:01:28,550 ‫So using the fan out architecture pattern 37 00:01:28,550 --> 00:01:31,890 ‫you can combine SNS with SQS 38 00:01:31,890 --> 00:01:36,890 ‫or you can combine SNS FIFO topics with SQS FIFO queues. 39 00:01:37,560 --> 00:01:38,710 ‫Okay. 40 00:01:38,710 --> 00:01:42,040 ‫Now finally, Kinesis has two modes of consumption. 41 00:01:42,040 --> 00:01:43,380 ‫You have the standard mode 42 00:01:43,380 --> 00:01:46,770 ‫where consumers pull data from Kinesis. 43 00:01:46,770 --> 00:01:50,080 ‫In which case you get two megabytes per second, per shard 44 00:01:50,080 --> 00:01:52,280 ‫or if you have enhanced fan out type 45 00:01:52,280 --> 00:01:57,120 ‫of consumption mechanism, you have Kinesis, sorry, 46 00:01:57,120 --> 00:01:59,640 ‫that pushes data into your consumers. 47 00:01:59,640 --> 00:02:03,140 ‫In which case you get two megabytes per second, per shard 48 00:02:03,140 --> 00:02:06,147 ‫per consumer, which gives you a much higher throughput 49 00:02:06,147 --> 00:02:08,460 ‫and the ability to have much more applications reading 50 00:02:08,460 --> 00:02:11,170 ‫from your Kinesis stream. 51 00:02:11,170 --> 00:02:12,640 ‫You have the possibility to reply data 52 00:02:12,640 --> 00:02:15,510 ‫because the data is persisted with new Kinesis data stream. 53 00:02:15,510 --> 00:02:18,620 ‫And so Kinesis is going to be used for real time big data, 54 00:02:18,620 --> 00:02:20,660 ‫analytics and ETL. 55 00:02:20,660 --> 00:02:23,030 ‫You will get ordering at the shard level 56 00:02:23,030 --> 00:02:25,320 ‫and you have to specify how many shards you want 57 00:02:25,320 --> 00:02:27,360 ‫per Kinesis data stream in advance. 58 00:02:27,360 --> 00:02:30,530 ‫So you need to scale the shard yourself 59 00:02:30,530 --> 00:02:33,140 ‫and the data will expire after X days. 60 00:02:33,140 --> 00:02:34,200 ‫And at the time of recording 61 00:02:34,200 --> 00:02:38,740 ‫this is between one and 365 days of data retention. 62 00:02:38,740 --> 00:02:40,320 ‫In terms of capacity mode, 63 00:02:40,320 --> 00:02:42,420 ‫we have two modes, the provision mode 64 00:02:42,420 --> 00:02:45,700 ‫in which we specify in advance the amount of shards we want 65 00:02:45,700 --> 00:02:49,290 ‫for Kinesis streams or the on-demand capacity mode, 66 00:02:49,290 --> 00:02:51,630 ‫where the number of shard is adjusted 67 00:02:51,630 --> 00:02:54,530 ‫by Kinesis streams directly for us. 68 00:02:54,530 --> 00:02:55,363 ‫Okay. 69 00:02:55,363 --> 00:02:57,430 ‫So that's it for this summer lecture. 70 00:02:57,430 --> 00:02:58,263 ‫I hope you liked it. 71 00:02:58,263 --> 00:03:00,210 ‫And I will see you in the next lecture.