WEBVTT

0:00:03.900000 --> 0:00:08.180000
 All right, so I am back within
 my Kali Linux system.

0:00:08.180000 --> 0:00:11.760000
 And before we get started with a demo,
 there's a couple of very, very

0:00:11.760000 --> 0:00:16.520000
 cool websites I want to show you that
 will help you understand character

0:00:16.520000 --> 0:00:20.860000
 sets much better like ASCII, for
 example, and also Unicode.

0:00:20.860000 --> 0:00:25.300000
 All right, so the first website is
 a site called ASCII- or minus code

0:00:25.300000 --> 0:00:30.380000
.com. This contains the ASCII table, the
 reference table that sort of highlights

0:00:30.380000 --> 0:00:40.580000
 the actual characters, or if you will,
 the symbols, the letters and the

0:00:40.580000 --> 0:00:44.540000
 number mapping to the
 actual control code.

0:00:44.540000 --> 0:00:49.960000
 So in this particular case, you can
 see that it's broken down really,

0:00:49.960000 --> 0:00:52.560000
 really well into the control characters.

0:00:52.560000 --> 0:00:55.300000
 So that's the character code 0 to 31.

0:00:55.300000 --> 0:00:58.700000
 And the character code is highlighted
 here, the decimal value, there we

0:00:58.700000 --> 0:01:02.400000
 are. And you can also see the binary
 representation for each.

0:01:02.400000 --> 0:01:06.000000
 And you then have printable characters,
 which is where you have your symbols,

0:01:06.000000 --> 0:01:11.140000
 your numbers, so 0 to 9, your
 colon, semicolon, etc.

0:01:11.140000 --> 0:01:13.420000
 And then uppercase letters.

0:01:13.420000 --> 0:01:15.700000
 You then have lowercase letters.

0:01:15.700000 --> 0:01:20.540000
 And then you have the extended codes,
 which are for specific symbols.

0:01:20.540000 --> 0:01:26.560000
 Like, for example, the Euro symbol,
 you have the ellipses, so on and so

0:01:26.560000 --> 0:01:30.540000
 forth. So you can probably already
 tell why this is important.

0:01:30.540000 --> 0:01:35.900000
 It's important so that, again, for all
 of these formats, so you can think

0:01:35.900000 --> 0:01:42.020000
 of these, you can think of these particular
 symbols or these values here

0:01:42.020000 --> 0:01:48.140000
 as values that the computer knows or
 can actually understand when you

0:01:48.140000 --> 0:01:52.880000
 type it in. So most systems have support
 for ASCII through Unicode.

0:01:52.880000 --> 0:01:58.720000
 Okay. And if we now take a look at the
 next site, which is the UTF-8-chartable

0:01:58.720000 --> 0:02:03.160000
.de, so it's a German website,
 but again, very, very useful.

0:02:03.160000 --> 0:02:08.180000
 You can see that it provides you with
 the UTF-8 encoding table and the

0:02:08.180000 --> 0:02:14.820000
 Unicode characters with code points
 from u plus 0 to u plus 0, 0, f, f.

0:02:14.820000 --> 0:02:19.160000
 Okay. Now the way this works is there's
 a couple of options that you can

0:02:19.160000 --> 0:02:19.960000
 play around with.

0:02:19.960000 --> 0:02:23.520000
 So the language will stick with English
 in terms of the page format.

0:02:23.520000 --> 0:02:25.780000
 Just keep it on standard.

0:02:25.780000 --> 0:02:28.960000
 And then you have the actual
 code blocks here.

0:02:28.960000 --> 0:02:33.760000
 So this is all of the languages or
 scripts that are supported by UTF-8

0:02:33.760000 --> 0:02:38.500000
 or Unicode. So you can see that we
 have support for even the most, you

0:02:38.500000 --> 0:02:43.480000
 know, esoteric, not esoteric, but pretty
 much all languages, like for

0:02:43.480000 --> 0:02:46.140000
 example, here, we'll go to the top.

0:02:46.140000 --> 0:02:50.720000
 We have basic Latin, we have Cyrillic,
 which is, you know, Russian and

0:02:50.720000 --> 0:02:52.460000
 Ukrainian languages.

0:02:52.460000 --> 0:02:55.900000
 We have Armenian, Hebrew, Arabic,
 so on and so forth.

0:02:55.900000 --> 0:03:01.080000
 So for example, if I wanted to see Hebrew
 and what those code points are,

0:03:01.080000 --> 0:03:05.020000
 you can see that firstly, my browser
 is not actually displaying those

0:03:05.020000 --> 0:03:06.880000
 symbols, which is very important.

0:03:06.880000 --> 0:03:08.280000
 And again, I'll explain why.

0:03:08.280000 --> 0:03:12.720000
 The reason this is the case is because
 this page, and this is one of the

0:03:12.720000 --> 0:03:16.360000
 reasons why I wanted to show you this
 example, this particular website

0:03:16.360000 --> 0:03:23.280000
 does not have the, within the actual
 header of the HTML code, it does

0:03:23.280000 --> 0:03:29.200000
 not have the meta tag to specify the
 UTF-8 encoding scheme that is to

0:03:29.200000 --> 0:03:33.580000
 be used. So you can see within the head
 here, we have meta tags, but these

0:03:33.580000 --> 0:03:39.380000
 are for, you know, the, the web server
 meta files, as well as additional

0:03:39.380000 --> 0:03:44.740000
 parameters, but nothing for the actual
 the character set specification.

0:03:44.740000 --> 0:03:48.580000
 So that means that, again, my browser
 cannot display those characters

0:03:48.580000 --> 0:03:53.060000
 because this particular web page does
 not specify the character set to

0:03:53.060000 --> 0:03:57.880000
 be used. Now it says at the top here,
 XML version one encoding UTF-8,

0:03:57.880000 --> 0:04:01.680000
 and you can still see that my, this is
 not being displayed in my browser.

0:04:01.680000 --> 0:04:05.680000
 Okay, so it might display this on your
 browser, but typically the way,

0:04:05.680000 --> 0:04:09.360000
 the way this works is modern browsers
 will set based on your language

0:04:09.360000 --> 0:04:15.820000
 specification, the encoding schema
 to use or the encoding schema that

0:04:15.820000 --> 0:04:20.680000
 it'll actually request in terms of the
 HTTP request headers, it'll actually

0:04:20.680000 --> 0:04:24.560000
 specify their what type of language
 or encoding it'll support.

0:04:24.560000 --> 0:04:28.340000
 And we've already taken a look at this,
 but you can see that we'll have

0:04:28.340000 --> 0:04:33.440000
 the code point and the character and then
 the UTF-8 hexadecimal representation

0:04:33.440000 --> 0:04:36.140000
 and the actual name here.

0:04:36.140000 --> 0:04:37.700000
 Now you can play around with this here.

0:04:37.700000 --> 0:04:42.240000
 So for example, the code positions
 per page and the display format for

0:04:42.240000 --> 0:04:46.500000
 UTF-8 encoding. So we can see the decimal
 value in terms of the UTF-8

0:04:46.500000 --> 0:04:52.640000
 encoding. We have the binary here, you
 can see that there, this is what

0:04:52.640000 --> 0:04:55.480000
 the computer, this is the code point.

0:04:55.480000 --> 0:04:59.000000
 So that means when you type in this
 particular Arabic, or in this case,

0:04:59.000000 --> 0:05:03.440000
 the Hebrew character, this is what the
 computer will treat it as, or this

0:05:03.440000 --> 0:05:06.280000
 is what the computer will
 actually denote it as.

0:05:06.280000 --> 0:05:10.340000
 So if we switch back over into, you
 know, a language that pretty much

0:05:10.340000 --> 0:05:16.680000
 most people can understand, like for
 example, basic Latin, you can see

0:05:16.680000 --> 0:05:20.720000
 that basic Latin again, has, will, you
 know, pretty much has the mappings

0:05:20.720000 --> 0:05:24.380000
 for Latin languages or languages
 based on Latin.

0:05:24.380000 --> 0:05:26.680000
 So we can see the alphabet here.

0:05:26.680000 --> 0:05:30.560000
 So when you type in, you know, the
 small letter A or lowercase A, this

0:05:30.560000 --> 0:05:31.700000
 is the mapping here.

0:05:31.700000 --> 0:05:35.580000
 So your computer will essentially,
 this is what your computer reads it

0:05:35.580000 --> 0:05:37.500000
 as. And that's very important.

0:05:37.500000 --> 0:05:44.780000
 And that means that for a file containing,
 containing letters for a particular

0:05:44.780000 --> 0:05:50.500000
 alphabet, this is how that data is stored,
 or how it's essentially tracked

0:05:50.500000 --> 0:05:51.780000
 down by the computer.

0:05:51.780000 --> 0:05:54.960000
 So when you type in A on your keyboard,
 this is what the computer reads

0:05:54.960000 --> 0:05:57.780000
 it as it does not know what A means.

0:05:57.780000 --> 0:06:01.940000
 But more so in the context of the web,
 it's more so ensuring that these

0:06:01.940000 --> 0:06:04.800000
 characters here are displayed correctly.

0:06:04.800000 --> 0:06:07.420000
 Now again, you might be a little bit
 confused still, but don't worry,

0:06:07.420000 --> 0:06:09.920000
 it'll make sense in one minute.

0:06:09.920000 --> 0:06:15.180000
 So you can also see the playing around
 with the options, you can also

0:06:15.180000 --> 0:06:18.960000
 view the actual Unicode character
 names, which are displayed.

0:06:18.960000 --> 0:06:20.740000
 This is this column here.

0:06:20.740000 --> 0:06:26.180000
 And then you can also just enable the
 links for adding the character to

0:06:26.180000 --> 0:06:31.420000
 text. So if I add that in there, you
 know, you can actually play around

0:06:31.420000 --> 0:06:34.200000
 with this website depending
 on your requirements.

0:06:34.200000 --> 0:06:38.320000
 And you can see that in this particular
 case, we click on this symbol

0:06:38.320000 --> 0:06:40.420000
 here, we can click on append.

0:06:40.420000 --> 0:06:42.820000
 And now you can sort of combine stuff.

0:06:42.820000 --> 0:06:51.720000
 So if I mark or point, and then a single
 quote, you can actually see the

0:06:51.720000 --> 0:06:58.140000
 value here. So UTF-8, the
 Unicode code points there.

0:06:58.140000 --> 0:07:02.520000
 And you know, that's a very,
 very useful website.

0:07:02.520000 --> 0:07:07.860000
 You then have another website called
 charset or character set dot org.

0:07:07.860000 --> 0:07:11.080000
 And this will this a fantastic website
 because it does the conversion

0:07:11.080000 --> 0:07:14.820000
 for you. So convert character to
 number the Unicode code points.

0:07:14.820000 --> 0:07:19.920000
 So this tool shows Unicode details about
 any character or letter, including

0:07:19.920000 --> 0:07:24.460000
 decimal hex code points and
 HTML URL encode syntax.

0:07:24.460000 --> 0:07:28.420000
 So if I say, for example, a, and I convert
 it, the character is uppercase

0:07:28.420000 --> 0:07:32.280000
 a, the Unicode code point is u plus 0041.


0:07:32.280000 --> 0:07:36.260000
 The Unicode decimal position
 is 65 Unicode hex position.

0:07:36.260000 --> 0:07:39.220000
 So hexadecimal is 41.

0:07:39.220000 --> 0:07:44.580000
 And the HTML encoded character
 is and or ampersand pound 65.

0:07:44.580000 --> 0:07:48.720000
 And the URL encoded character which
 URL encoding will take a look at is

0:07:48.720000 --> 0:07:54.160000
 very simple. We just have the percent,
 the percent symbol and 41.

0:07:54.160000 --> 0:08:00.180000
 So it just takes the, it just adds the
 prefix, the percentage sign always

0:08:00.180000 --> 0:08:03.420000
 for URL encoding is going
 to be the prefix.

0:08:03.420000 --> 0:08:08.220000
 And then what comes after that is the hex
 position, the Unicode hex position.

0:08:08.220000 --> 0:08:14.700000
 So you can see that this is part of
 the us ASCII first 128 characters.

0:08:14.700000 --> 0:08:17.880000
 And it's part of ISO
 basic Latin alphabet.

0:08:17.880000 --> 0:08:20.660000
 It's part of ISO 8, 8, 5, 9, 1.

0:08:20.660000 --> 0:08:26.080000
 And part of UTF 8 page one, number 65,
 the letter is uppercase in lowercase.

0:08:26.080000 --> 0:08:28.260000
 It also provides you with
 that value there.

0:08:28.260000 --> 0:08:30.960000
 You can also, you know, convert
 number to a character.

0:08:30.960000 --> 0:08:36.240000
 So for example, if I say number one,
 convert that to a character, there

0:08:36.240000 --> 0:08:37.560000
 is no character mapping.

0:08:37.560000 --> 0:08:42.660000
 But if I go back in here, we can see
 that again, say 65 convert that that

0:08:42.660000 --> 0:08:46.640000
 is uppercase a so the point that I'm
 trying to make here is that these

0:08:46.640000 --> 0:08:53.820000
 character sets and the character set
 encoding schemas are very important

0:08:53.820000 --> 0:09:01.440000
 so that web pages or data is, you know,
 is encoded and more importantly

0:09:01.440000 --> 0:09:06.200000
 displayed in the format it was
 intended to be displayed in.

0:09:06.200000 --> 0:09:10.680000
 So let's put all of this into context
 now by taking a look at an example.

0:09:10.680000 --> 0:09:15.340000
 So let's say I want to develop a web
 page and I speak or the language

0:09:15.340000 --> 0:09:18.040000
 my lingo franca is Greek, right?

0:09:18.040000 --> 0:09:20.260000
 So I'm developing a simple HTML web page.


0:09:20.260000 --> 0:09:24.480000
 So I'm just gonna close my browser
 here and open up a text editor like

0:09:24.480000 --> 0:09:26.980000
 mousepad. And I'm just gonna zoom in.

0:09:26.980000 --> 0:09:30.380000
 And I'm just gonna copy over
 some basic HTML code.

0:09:30.380000 --> 0:09:34.580000
 And I'm going to show you the issue
 or why encoding is important.

0:09:34.580000 --> 0:09:38.720000
 All right, so I've just put in some,
 you know, very basic HTML page, you

0:09:38.720000 --> 0:09:40.740000
 can see that it has all
 the tags required.

0:09:40.740000 --> 0:09:44.340000
 And in this case, the language
 is in Greek, right?

0:09:44.340000 --> 0:09:47.880000
 And I can actually get rid of this
 translation, which is not required.

0:09:47.880000 --> 0:09:50.400000
 So again, encoding is
 not about translation.

0:09:50.400000 --> 0:09:55.740000
 It is ensuring that these characters here
 can be rendered by the web browser.

0:09:55.740000 --> 0:09:59.960000
 And how do you tell the web browser
 what encoding schema to use?

0:09:59.960000 --> 0:10:05.400000
 You specify that within the actual
 head tags by using the meta tag and

0:10:05.400000 --> 0:10:07.040000
 specifying the character set.

0:10:07.040000 --> 0:10:09.060000
 So in this case, that
 is not been included.

0:10:09.060000 --> 0:10:10.440000
 And that's by design.

0:10:10.440000 --> 0:10:12.720000
 Because again, I want to
 show you this example.

0:10:12.720000 --> 0:10:17.700000
 So what I'll do here is let
 me save this on my desktop.

0:10:17.700000 --> 0:10:21.480000
 And I'll save it as encoded dot HTML.

0:10:21.480000 --> 0:10:26.480000
 Okay. And you can see the encoding
 by default, when saving a file, for

0:10:26.480000 --> 0:10:28.580000
 example, in Kali is set to UTF-8.

0:10:28.580000 --> 0:10:31.780000
 But don't worry, that really
 doesn't affect anything here.

0:10:31.780000 --> 0:10:33.580000
 So I'm going to save that there.

0:10:33.580000 --> 0:10:35.960000
 And now I'm going to open up my terminal.


0:10:35.960000 --> 0:10:40.100000
 And I'm just going to set up a very
 simple web server with Python.

0:10:40.100000 --> 0:10:42.040000
 So I'll navigate to where
 I saved the file.

0:10:42.040000 --> 0:10:47.120000
 And I'll say pseudo Python, a
 three module HTTP dot server.

0:10:47.120000 --> 0:10:49.300000
 And I'll say port 80.

0:10:49.300000 --> 0:10:53.100000
 Okay. So I'm just I'm just setting up
 a web server on my Kali Linux system.

0:10:53.100000 --> 0:10:55.060000
 So I can show you this issue.

0:10:55.060000 --> 0:10:59.760000
 So I'll go back into Kali now, and I'll
 navigate to 127.0.0.1, where I'm

0:10:59.760000 --> 0:11:01.400000
 hosting the web server.

0:11:01.400000 --> 0:11:06.200000
 And you can see that we have
 the encoded HTML file here.

0:11:06.200000 --> 0:11:08.760000
 So watch what happens when I click on it.


0:11:08.760000 --> 0:11:13.260000
 Remember, in the actual code,
 I've written it in Greek here.

0:11:13.260000 --> 0:11:18.640000
 But what, you know, I expect my web
 server or I expect my clients to be

0:11:18.640000 --> 0:11:23.180000
 able to see what I've the actual, you
 know, what's supposed to be displayed

0:11:23.180000 --> 0:11:26.240000
 on the web page in the way
 that I typed it out.

0:11:26.240000 --> 0:11:29.520000
 Again, regardless of whatever
 language I'm typing it out in.

0:11:29.520000 --> 0:11:31.780000
 So I'll click on encoded.

0:11:31.780000 --> 0:11:34.500000
 And what in the world is this?

0:11:34.500000 --> 0:11:37.480000
 I mean, this, that's fine,
 because that's ASCII.

0:11:37.480000 --> 0:11:39.260000
 Again, this is the English alphabet.

0:11:39.260000 --> 0:11:40.900000
 So there is support for that.

0:11:40.900000 --> 0:11:44.400000
 But there's this set of
 very weird symbols.

0:11:44.400000 --> 0:11:46.620000
 So this is why encoding is important.

0:11:46.620000 --> 0:11:49.700000
 It's not about translating languages.

0:11:49.700000 --> 0:11:54.560000
 It's ensuring that those characters are
 displayed correctly in the client's

0:11:54.560000 --> 0:11:57.700000
 browser. And that's why
 encoding is important.

0:11:57.700000 --> 0:12:03.500000
 So you can see from this basic demo,
 the importance of what I just covered

0:12:03.500000 --> 0:12:05.100000
 in the theoretical section.

0:12:05.100000 --> 0:12:08.280000
 So up to this point, you might have
 been saying what's the importance

0:12:08.280000 --> 0:12:11.400000
 of all of this? Well, you actually
 see that right now.

0:12:11.400000 --> 0:12:13.800000
 So I'm not going to go into
 the actual source code here.

0:12:13.800000 --> 0:12:19.100000
 And within the head, I'm going to include
 another tag called, it's a meta

0:12:19.100000 --> 0:12:23.760000
 tags. And then to specify the encoding,
 you specify the character set.

0:12:23.760000 --> 0:12:27.940000
 And in here, I can say, for example,
 ASCII, and this will probably not

0:12:27.940000 --> 0:12:29.940000
 work. But let's type it in.

0:12:29.940000 --> 0:12:32.220000
 And let's see whether
 that changes anything.

0:12:32.220000 --> 0:12:33.220000
 So I reload this.

0:12:33.220000 --> 0:12:34.160000
 Okay, no change.

0:12:34.160000 --> 0:12:38.580000
 So in this particular case, if I was
 developing this web page, I'll need

0:12:38.580000 --> 0:12:48.320000
 to ensure that I specify the encoding,
 browsers around the world are aware

0:12:48.320000 --> 0:12:52.960000
 of this. And when they receive this HTML
 file, when your browser receives

0:12:52.960000 --> 0:12:58.080000
 it, it knows how to handle everything
 within the web page, again, regardless

0:12:58.080000 --> 0:12:58.900000
 of the language.

0:12:58.900000 --> 0:13:03.220000
 So we're essentially telling the browser
 that this content here is encoded

0:13:03.220000 --> 0:13:09.280000
 in UTF-8. So please use UTF-8
 to decode it and display it.

0:13:09.280000 --> 0:13:14.420000
 So if I now say save, and I reload
 here, voila, there you go.

0:13:14.420000 --> 0:13:20.440000
 Now it's being displayed correctly again,
 regardless of the actual translation,

0:13:20.440000 --> 0:13:24.640000
 you know, because that's how I've designed
 my web page, I intend for it

0:13:24.640000 --> 0:13:26.380000
 to be displayed in Greek.

0:13:26.380000 --> 0:13:28.540000
 And it should work on everyone's
 browser now.

0:13:28.540000 --> 0:13:31.460000
 So you're essentially telling your browser,
 or you're telling browsers,

0:13:31.460000 --> 0:13:36.120000
 or anyone using your website that when
 you receive this HTML file, the

0:13:36.120000 --> 0:13:39.140000
 content within it is encoded in UTF-8.

0:13:39.140000 --> 0:13:43.340000
 So ensure you decode it, or you convert
 it using, you know, this particular

0:13:43.340000 --> 0:13:44.980000
 conversion table.

0:13:44.980000 --> 0:13:50.260000
 And as a result, the characters are
 displayed correctly or as intended.

0:13:50.260000 --> 0:13:52.960000
 So that's the importance of encoding.

0:13:52.960000 --> 0:13:57.320000
 And that's why websites utilize this
 particular tag in, you know, in this

0:13:57.320000 --> 0:14:01.860000
 very basic example of HTML code, this
 also applies to other, you know,

0:14:01.860000 --> 0:14:04.040000
 even server-side scripting languages.

0:14:04.040000 --> 0:14:07.440000
 So again, has nothing to do with translation,
 because you can add translation

0:14:07.440000 --> 0:14:09.640000
 support. That's not an issue.

0:14:09.640000 --> 0:14:13.780000
 It's more so ensuring that these special
 characters here are displayed

0:14:13.780000 --> 0:14:19.600000
 correctly. So this is a very, very basic
 example, but hopefully it sets

0:14:19.600000 --> 0:14:22.260000
 the stage for everything
 we'll be covering next.

0:14:22.260000 --> 0:14:26.560000
 So that's going to conclude the practical
 demonstration side of this video.

0:14:26.560000 --> 0:14:31.640000
 All right, so that brings us to the
 end of the first video within the

0:14:31.640000 --> 0:14:34.560000
 encoding section of this course.

0:14:34.560000 --> 0:14:38.740000
 By this point, you should have a very
 good understanding now as to what

0:14:38.740000 --> 0:14:43.460000
 encoding is the role it plays in the
 web and the functionality of web

0:14:43.460000 --> 0:14:49.100000
 applications, as well as its importance
 in the life of a web application

0:14:49.100000 --> 0:14:52.520000
 penetration tester with regards to understanding
 how the web application

0:14:52.520000 --> 0:14:57.860000
 works. And in addition to that, we
 were further able to extrapolate or

0:14:57.860000 --> 0:15:02.120000
 get an understanding of, you know, character
 sets, where we've come from

0:15:02.120000 --> 0:15:06.100000
 historically, and how we have evolved
 in terms of going from, you know,

0:15:06.100000 --> 0:15:10.180000
 character sets like ASCII to Unicode,
 and then the encoding schemas that

0:15:10.180000 --> 0:15:14.260000
 are used by your browsers, or, you
 know, how they incorporate it into

0:15:14.260000 --> 0:15:18.420000
 the web. So now that you have this
 basic understanding, which I think

0:15:18.420000 --> 0:15:22.220000
 is a good, you know, starting point,
 we're now going to turn our attention

0:15:22.220000 --> 0:15:24.300000
 to HTML encoding.

0:15:24.300000 --> 0:15:27.840000
 And you may be thinking to yourself,
 didn't we just take a look at HTML

0:15:27.840000 --> 0:15:32.160000
 encoding? Well, no, we took a look
 at character set encoding.

0:15:32.160000 --> 0:15:36.900000
 So we're specifically talking about characters,
 but HTML has its own encoding

0:15:36.900000 --> 0:15:40.680000
 in place. And it's there for a
 very, very important reason.

0:15:40.680000 --> 0:15:46.260000
 And as a web application penetration
 tester, looking out for HTML encoding

0:15:46.260000 --> 0:15:51.580000
 errors or the lack therein of HTML encoding
 is going to be very important

0:15:51.580000 --> 0:15:55.120000
 in identifying, for example, cross
-site scripting vulnerabilities.

0:15:55.120000 --> 0:15:58.640000
 With that being said, that's going
 to be it for this video.

0:15:58.640000 --> 0:16:00.880000
 And I'll be seeing you in the next video.