1
00:00:01,290 --> 00:00:10,740
Extracting directory structure, crawling, so mapping the application layout and its structure is another

2
00:00:10,740 --> 00:00:11,820
very important task.

3
00:00:12,780 --> 00:00:18,900
Although there are some single page applications, the applications you will test can generally consist

4
00:00:18,900 --> 00:00:20,940
of multiple Web pages.

5
00:00:21,920 --> 00:00:28,990
And by multiple, I mean a lot in these pages can be independent or they can be linked to one another.

6
00:00:29,990 --> 00:00:36,560
There's actually no magic to get the structure unless you're the actual developer of the application

7
00:00:37,040 --> 00:00:39,410
and it is hours and hours of work.

8
00:00:41,960 --> 00:00:50,990
So the best way to extract the structure of the application is to visit every page and click every link

9
00:00:51,170 --> 00:01:01,670
and fill every form, then observe all the URL so you can manually walk through the application and

10
00:01:01,670 --> 00:01:07,670
identify Web pages from authenticated and unauthenticated users perspective.

11
00:01:09,090 --> 00:01:14,640
So this whole process is called crawling or spidering the application.

12
00:01:15,920 --> 00:01:21,560
Now, you might think and you would be right that it's not really possible to do everything manually.

13
00:01:22,190 --> 00:01:24,500
This really is a time consuming process.

14
00:01:25,550 --> 00:01:31,730
So, gosh darn it, wouldn't you just love it if somebody created a tool for crawling, and how about

15
00:01:31,730 --> 00:01:33,260
some scripts for good measure?

16
00:01:34,270 --> 00:01:36,010
Well, I'm glad you asked.

17
00:01:37,030 --> 00:01:46,270
Because now let's go over to Cali and open up your browser and berp, I know if raised like me, it's

18
00:01:46,270 --> 00:01:49,020
rude, but I'm going to ask you to do it again.

19
00:01:49,510 --> 00:01:50,940
Open your browser and burp.

20
00:01:51,440 --> 00:01:53,140
OK, so sorry.

21
00:01:53,860 --> 00:01:57,370
Go to Dashboard and make sure that capturing is active.

22
00:01:59,150 --> 00:02:02,180
Then disabled master interception from this button.

23
00:02:03,260 --> 00:02:04,460
OK, so za'atar.

24
00:02:05,390 --> 00:02:10,100
But in your browser, redirect the traffic to berp by enabling Foxe proxy.

25
00:02:11,560 --> 00:02:17,170
So now Berp will passively intercept all traffic coming from Firefox's.

26
00:02:18,780 --> 00:02:23,490
And what we're going to do here is visit pages and click links.

27
00:02:25,160 --> 00:02:32,450
So so that way, berp will create a structure of the advocation from the clicked you earles by intercepting

28
00:02:32,450 --> 00:02:33,230
passively.

29
00:02:35,050 --> 00:02:39,640
Right, so I'm going to click on some links here on the page.

30
00:02:41,310 --> 00:02:42,390
And then log in.

31
00:02:45,090 --> 00:02:47,610
Open up a few other pages and.

32
00:02:50,190 --> 00:02:51,990
OK, that's enough to show you.

33
00:02:53,770 --> 00:02:56,620
So, OK, look, the history in BIR.

34
00:02:57,820 --> 00:02:59,680
So all the requestor here.

35
00:03:01,310 --> 00:03:08,480
And so the aim here is to see a sort of a broad overview of the application layout.

36
00:03:09,680 --> 00:03:10,160
So.

37
00:03:11,870 --> 00:03:16,940
There are a few more things in this step to perform, but I want you to just have a look here.

38
00:03:18,080 --> 00:03:21,230
I'm sure that you've heard about robots that he asked.

39
00:03:22,980 --> 00:03:30,240
I know almost every website uses this file to allow or disallow directories to be called by bots, so

40
00:03:30,240 --> 00:03:38,610
robots dot text is a file that uses a specification protocol called robot exclusion protocol.

41
00:03:39,920 --> 00:03:45,500
Now, it's not really something that you need to consider, but you might want to have a look at a later.

42
00:03:47,810 --> 00:03:55,280
But, yeah, displaying this file is very handy if you want to see the sensitive pages and directories

43
00:03:55,280 --> 00:03:56,010
easily.

44
00:03:56,810 --> 00:03:59,180
So what am I saying?

45
00:03:59,400 --> 00:04:04,100
Yeah, go to the robots dot text file of Boab.

46
00:04:05,620 --> 00:04:13,390
And here are some directories that are not allowed, so I'm going to visit each of them admin directory.

47
00:04:14,730 --> 00:04:15,570
Documents.

48
00:04:16,930 --> 00:04:22,180
And images and then wait, what's this password's directory?

49
00:04:24,110 --> 00:04:25,010
Now go to berp.

50
00:04:26,220 --> 00:04:27,600
Click Target tab.

51
00:04:28,540 --> 00:04:29,560
Quick site map.

52
00:04:30,980 --> 00:04:33,800
On the left pane, you can see the site structure.

53
00:04:35,870 --> 00:04:43,670
And also, there are filter options above, you know, this is great, I hope you get excited, as I

54
00:04:43,670 --> 00:04:50,780
do so by clicking show all resources like access and images, they're all going to show up on the map

55
00:04:50,780 --> 00:04:51,300
as well.

56
00:04:53,760 --> 00:05:00,430
I think the commercial version of Berp will do this task automatically, but I really do want you to,

57
00:05:01,330 --> 00:05:04,030
you know, get your hands dirty, as I was saying before.

58
00:05:05,490 --> 00:05:12,690
I know it's not the most efficient way, but this is how you learn, so we need something a bit more

59
00:05:12,690 --> 00:05:13,690
intrusive.

60
00:05:13,830 --> 00:05:16,350
Now, I should think so, Kelly.

61
00:05:16,350 --> 00:05:19,590
Linux has a number of tools for this job.

62
00:05:20,960 --> 00:05:22,460
There, Buster is one of them.

63
00:05:23,700 --> 00:05:28,110
So open up the terminal and simply type there, buster.

64
00:05:29,420 --> 00:05:30,860
And again, we will come up.

65
00:05:32,310 --> 00:05:37,110
There, Buster, is the directory brute force here for Web applications.

66
00:05:39,080 --> 00:05:45,560
So now let's provide the target, you, Earl, which is HTP Cohen, so that says one nine two dot one

67
00:05:45,560 --> 00:05:49,640
six eight two zero four that one three zero B Web slash.

68
00:05:50,970 --> 00:05:58,710
And here's a starting point for door buster, and it will automate the tedious tasks of cataloging the

69
00:05:58,710 --> 00:06:00,610
pages within the application.

70
00:06:01,200 --> 00:06:02,480
That sounds good, huh?

71
00:06:03,770 --> 00:06:06,140
So it works by requesting a Web page.

72
00:06:07,040 --> 00:06:14,450
Parsing through it for links and then sending requests to these new links until all the Web pages are

73
00:06:14,450 --> 00:06:14,990
mapped.

74
00:06:16,380 --> 00:06:20,400
So then let's increase the number of threads here to 20.

75
00:06:21,660 --> 00:06:24,450
And then choose a list, Brouse.

76
00:06:25,920 --> 00:06:29,730
Now, there are several lists in der busser directory under Wordlist.

77
00:06:31,220 --> 00:06:33,920
And I'm going to choose the medium directory list.

78
00:06:35,430 --> 00:06:36,990
OK, and QuickStart.

79
00:06:38,770 --> 00:06:46,780
Oh, and another good thing is to identify administrative and test pages, these pages can contain sensitive

80
00:06:46,780 --> 00:06:52,180
information and provide entry points to perform attacks such as a brute force attack.

81
00:06:53,320 --> 00:06:57,580
And it's also possible to see old and backup files in the directory structure.

82
00:06:59,530 --> 00:07:03,940
Don't laugh, I've seen it many times in real world situations.

83
00:07:04,840 --> 00:07:14,380
So if the old version of the application functions and has any vulnerabilities, bingo, you can own

84
00:07:14,380 --> 00:07:15,550
the entire system.

85
00:07:17,120 --> 00:07:22,940
Besides that, the folders and files belong to the application, so there may be meta files and folders

86
00:07:23,240 --> 00:07:26,870
of the server software as well as the application framework.

87
00:07:28,160 --> 00:07:32,900
And I mean, no one files and folders such as my admin and so forth.

88
00:07:34,290 --> 00:07:36,710
So Derbez here also looks for this kind of stuff.

89
00:07:37,920 --> 00:07:40,620
And here, as you can see, it detects Drupal.

90
00:07:42,270 --> 00:07:44,400
My admin and escarole light.

91
00:07:45,750 --> 00:07:49,890
OK, so at this point, I'm going to stop this scan and you can take a report.

92
00:07:51,750 --> 00:07:53,130
Full text report.

93
00:07:54,220 --> 00:07:56,560
Browse for the location is safe and.

94
00:07:58,310 --> 00:07:59,120
Give it the name.

95
00:08:00,050 --> 00:08:03,830
I'm just going to type B rap and generate report.

96
00:08:06,800 --> 00:08:08,240
So now you can go to that folder.

97
00:08:10,050 --> 00:08:12,060
And here is the report we saved.

98
00:08:13,330 --> 00:08:14,560
So now you can analyze it.