1 00:00:00,730 --> 00:00:01,770 Hey, what's up, gurus. 2 00:00:01,770 --> 00:00:05,050 Congratulations for finishing another section. 3 00:00:05,050 --> 00:00:06,730 As we move along through the course, 4 00:00:06,730 --> 00:00:08,940 you'll notice that our drive and view 5 00:00:08,940 --> 00:00:10,240 is getting better and better, 6 00:00:10,240 --> 00:00:12,390 because we are almost done 7 00:00:12,390 --> 00:00:16,290 and almost ready to take that DP-203 exam 8 00:00:16,290 --> 00:00:18,900 and grab a new certification. 9 00:00:18,900 --> 00:00:20,330 But first we gotta take just a minute 10 00:00:20,330 --> 00:00:21,660 to look in the rear view mirror 11 00:00:21,660 --> 00:00:25,460 and see what we learned in this section. 12 00:00:25,460 --> 00:00:28,920 So in this section, we are going to be talking about a 13 00:00:28,920 --> 00:00:31,083 review. Keep in mind not everything is reviewed, 14 00:00:31,083 --> 00:00:34,290 but we are going to review some of the core concepts 15 00:00:34,290 --> 00:00:35,460 that you need to know. 16 00:00:35,460 --> 00:00:37,360 It's all around the DP-203, 17 00:00:37,360 --> 00:00:38,740 so focus on that. 18 00:00:38,740 --> 00:00:41,560 There are other topics and deeper dives 19 00:00:41,560 --> 00:00:42,630 that we are not doing 20 00:00:42,630 --> 00:00:44,810 that you might need for a data engineer, 21 00:00:44,810 --> 00:00:49,730 simply because my goal is to get you to pass that DP-203. 22 00:00:49,730 --> 00:00:51,320 Finally, if you don't know something, 23 00:00:51,320 --> 00:00:52,990 hey, go back and review. 24 00:00:52,990 --> 00:00:57,070 And if it seems like you've been here before, fantastic! 25 00:00:57,070 --> 00:00:59,773 That means you are getting close to the finish. 26 00:01:00,720 --> 00:01:03,800 So let's talk about how we start collecting data. 27 00:01:03,800 --> 00:01:06,230 If you'll remember, there's great news, 28 00:01:06,230 --> 00:01:08,160 we already are collecting data. 29 00:01:08,160 --> 00:01:10,940 Through Azure Monitor, all Azure services 30 00:01:10,940 --> 00:01:13,620 are generating activity logs, platform metrics, 31 00:01:13,620 --> 00:01:17,370 resource logs, and VM guest metrics and logs. 32 00:01:17,370 --> 00:01:19,110 We just need to tap into that 33 00:01:19,110 --> 00:01:21,473 and pull the data that we need out of it. 34 00:01:23,470 --> 00:01:24,810 We also talked about 35 00:01:24,810 --> 00:01:27,380 how we measure performance of data movement. 36 00:01:27,380 --> 00:01:30,010 If you remember, we estimate what we should get, 37 00:01:30,010 --> 00:01:31,700 we compare that to what we are getting, 38 00:01:31,700 --> 00:01:34,150 and then we optimize as needed. 39 00:01:34,150 --> 00:01:38,260 And because of a cool concept known as full utilization, 40 00:01:38,260 --> 00:01:41,530 we can estimate our throughput because of that. 41 00:01:41,530 --> 00:01:43,710 If you remember that chart that we had to look at 42 00:01:43,710 --> 00:01:45,770 and figure out what kind of performance 43 00:01:45,770 --> 00:01:48,083 we should be getting from data movement. 44 00:01:48,083 --> 00:01:51,890 And finally, we use our monitor copy data activity 45 00:01:51,890 --> 00:01:54,140 to measure what we are actually getting. 46 00:01:54,140 --> 00:01:56,910 So don't forget about that as well. 47 00:01:56,910 --> 00:01:59,455 We also talked about the synapse query store. 48 00:01:59,455 --> 00:02:01,703 What is the synapse query store? 49 00:02:01,703 --> 00:02:05,180 Well, you should say 50 00:02:05,180 --> 00:02:06,880 it's a tool that provides insight 51 00:02:06,880 --> 00:02:08,980 on query plan choice and performance, 52 00:02:08,980 --> 00:02:11,260 and allows us to capture queries and plans, 53 00:02:11,260 --> 00:02:12,950 and runtime statistics. 54 00:02:12,950 --> 00:02:14,990 But, you gotta turn it on. 55 00:02:14,990 --> 00:02:16,790 If you remember, I threw that script in there 56 00:02:16,790 --> 00:02:18,510 to show you how to do that. 57 00:02:18,510 --> 00:02:20,430 And there were 3 different kinds of stores. 58 00:02:20,430 --> 00:02:21,700 There's our plan store, 59 00:02:21,700 --> 00:02:23,260 our runtime stats store, 60 00:02:23,260 --> 00:02:26,160 and our wait stats store as well. 61 00:02:26,160 --> 00:02:29,400 We also talked about measuring HDInsight clusters 62 00:02:29,400 --> 00:02:31,917 and remember, it all starts with Apache Ambari. 63 00:02:31,917 --> 00:02:33,674 We can use this to create alerts 64 00:02:33,674 --> 00:02:37,300 and manage and monitor HDInsight clusters. 65 00:02:37,300 --> 00:02:38,560 And here's a couple of pictures 66 00:02:38,560 --> 00:02:39,957 if you remember what it looks like, 67 00:02:39,957 --> 00:02:42,810 and we access it by going to 68 00:02:42,810 --> 00:02:47,380 whatever our cluster name is .azurehdinsight.net. 69 00:02:47,380 --> 00:02:50,240 So don't forget about Apache Ambari. 70 00:02:50,240 --> 00:02:52,810 So, summarizing all of this together. 71 00:02:52,810 --> 00:02:54,760 Hey, think scenarios. 72 00:02:54,760 --> 00:02:56,960 Scenarios will make you a better engineer. 73 00:02:56,960 --> 00:02:59,910 So be thinking about how these concepts tie into 74 00:02:59,910 --> 00:03:01,770 the things you're doing at work 75 00:03:01,770 --> 00:03:05,640 or other scenarios that might fit that application. 76 00:03:05,640 --> 00:03:07,440 Don't forget about the core services. 77 00:03:07,440 --> 00:03:09,130 In this section, there were quite 78 00:03:09,130 --> 00:03:11,490 a few concepts and services, 79 00:03:11,490 --> 00:03:13,670 so just keep those concepts in mind 80 00:03:13,670 --> 00:03:16,060 with which service they live. 81 00:03:16,060 --> 00:03:18,230 And as always, don't forget about the labs, 82 00:03:18,230 --> 00:03:21,400 you'll need those for your exam and your career. 83 00:03:21,400 --> 00:03:23,470 If you are enjoying these lessons 84 00:03:23,470 --> 00:03:24,600 make sure you do me a favor 85 00:03:24,600 --> 00:03:26,290 and smash that thumbs up button. 86 00:03:26,290 --> 00:03:28,120 I greatly appreciate it. 87 00:03:28,120 --> 00:03:31,037 Congratulations on finishing another section 88 00:03:31,037 --> 00:03:35,590 and I will see you in our last big section. 89 00:03:35,590 --> 00:03:37,090 All right, I'll see you there.