WLAN Security is a special mechanism which allows only a specific person (machine) to get access to a specific WLAN Network. In more formal term, it is a mechanism to assure confidentiality and integrity of data flowing over WLAN network.
Since this can easily be very dry topic and I don't have expert level knowledge of these algorithm, I want to try to describe this in more practical and intuitive level.
- WLAN Security Types
- WLAN Authentication Types
- Open system authentication
- Shared Key Authentication
- WPA-Personal / WPA2-Personal / WPA3-Personal - Pre-shared Key
- WPA-Enterprise / WPA2-Enterprise / WPA3-Enterprise - 802.1X Authentication
- Captive Portal
- Authentication using SSID of AP
- MAC address filtering
- WPA in Authentication and Security
Before you jump into a bunch of boring documents about Security, I would recommend to get familiar with some terminologies you would come across in technical documents. One good way to get familiar with would be to have some time to take a little bit closer look at some tools that almost everybody have used at some point but never paid closer attention.
First example is from WLAN setting on my own PC on the network that I am allowed to use. My WLAN interface has following types of Security options. You may see different options on your PC and your WLAN network. Don't try to understand everything in details here. Just get familiar with some terms of WPA, WPA2 or AES etc.

The dialog above shows how Windows splits the settings into three parts. Security type chooses the authentication, from No authentication (Open) and Shared up to WPA2-Enterprise and 802.1X. Encryption type chooses the cipher, TKIP or AES. The network authentication method appears only for the Enterprise types, and here it selects Microsoft: Protected EAP (PEAP), which is an EAP method.
Here goes another example. Following is from one of WiFi options shown on my mobile phone. You would also see a lot of new terminologies here. Again, just try to get familiar with the terms ..

The Wi-Fi list above shows most of the authentication types on this page. SSID 1 says "Sign-in is required", which is a captive portal. SSID 2 and SSID 3 have no lock icon, so they are open networks. SSID 5 is secured with WPA/WPA2, which is the Personal mode with a pre-shared key. SSID 4 and SSID 6 are secured with 802.1x, which is the Enterprise mode. Selecting SSID 6 opens the dialog on the right, which asks for the EAP method, the Phase 2 authentication, the CA certificate and the Identity. The table below lists the values these fields can take.
|
EAP Method |
PEAP, TLS, TTLS, PWD, SIM, AKA, FAST |
|
Phase 2 Authentication |
PAP, MSCHAP, MSCHAPV2, GTC |
|
CA Certification |
|
|
User Certificate |
|
|
Identity |
|
|
Password |
|
Now you would be wondering exactly where and how these algorithms play roles during WiFi attach or Data communication process. For this, I would recommend you to go through WiFi Protocol page and overall understanding of WiFi Protocol. And then take closer look at step 4, 5, 8, 9, 10, 11 of Sample log of WiFi Aattach process. (The detailed procedure and parameters in each step would vary depending on each algorithm but you would get a big picture from the page).
Now with a little bit of familiarity with basic keywords and overall understanding on protocol, let's take a small step into some of commonly used security algorithm.
You can get a pretty good tutorial of overall concept of WLAN Security from the list of Video Tutorial at the bottom of the page.
WLAN Security Types
WLAN security is an essential aspect of wireless networking, as it helps protect your network from unauthorized access and keeps your data safe. Over the years, several security protocols have been developed for WLANs, each with different levels of protection. Here are the main WLAN security types
WEP
WEP was the first security protocol introduced for Wi-Fi networks. It uses the RC4 stream cipher for encryption and supports 64-bit and 128-bit encryption keys. However, WEP has significant security vulnerabilities and is considered outdated and insecure. It's not recommended for use in modern WLANs.
WEP stands for Wired Equivalent Privacy and it has following properties/functionalities.
- Performs Authentication using "shared key"
- Performs Confidentiality using RC4 stream cipher encryption
- Performs Integrity checking using CRC-32 integrity mechanism
- Does not do key management
- Does not do protection against replay attacks
Drawback of WEP : Since there is no key management and uses shared key, every wireless station and AP has the same "preshared" key that is used during authentication and encryption. It means you have to manually distribute the key to every device and every AP and it would be very cumbersome to change key because once you change it and you have to redistribute them all manually. So usually every device will use the same/static key for a long time giving more chance for hackers to break the system.
WPA
WPA stands for WiFi Protected Access. It is proposed as Interim solution that offers key management using the 802.1X authentication framework and performs improved encryption and integrity checking.
WPA was introduced as an interim solution to address WEP's security weaknesses. It uses Temporal Key Integrity Protocol (TKIP) for encryption, which is an improvement over WEP, and also incorporates Message Integrity Check (MIC) to protect against data tampering. However, WPA still uses the RC4 stream cipher, which has known vulnerabilities, making it less secure than more recent security protocols.
It has following properties/functionalities.
- Performs Authentication using the 802.1X framework
- Performs Confidentiality using TKIP encryption
- Performs Integrity checking using "Michael" protocol
- Performs Key management using the 802.1X framework(it is also possible to use preshared keys)
- Performs Protection against replay attacks.
WPA2
This is a kind of WPA evolution and is specified in IEEE 802.11i. In most part, it is same as WPA but it uses improved encryption mechanism called AES.
WPA2 is a more secure and widely used security protocol for WLANs. It replaces TKIP with the more robust Advanced Encryption Standard (AES) cipher for encryption, providing a higher level of security. WPA2 supports two authentication methods: WPA2-Personal (also known as WPA2-PSK) for home and small office networks, which uses a pre-shared key, and WPA2-Enterprise for larger networks, which relies on a RADIUS server for authentication.
WPA 3
WPA3 is the latest security protocol and provides several enhancements over WPA2. It offers stronger encryption through the Simultaneous Authentication of Equals (SAE) method, which replaces the pre-shared key in WPA2-Personal with a more secure key exchange. WPA3-Enterprise provides an even higher level of security with 192-bit encryption. WPA3 also improves security for public Wi-Fi networks with Opportunistic Wireless Encryption (OWE).
WLAN Authentication Types
WLAN authentication is the process of verifying the identity of devices trying to connect to a wireless network. Authentication helps to prevent unauthorized access and protect your network from potential threats.
There are many different types of WLAN Authentication methods and the list gets longer and longer. Followings are some of the common methods and I will keep update as I learn more.
- Open system authentication
- Shared Key Authentication
- WPA-Personal / WPA2-Personal / WPA3-Personal - Pre-shared Key
- WPA-Enterprise / WPA2-Enterprise / WPA3-Enterprise - 802.1X Authentication
- Captive Portal
- Authentication using SSID of AP
- MAC address filtering
Open system authentication
Open System Authentication doesn't require any authentication or credentials to connect to the Wi-Fi network. Devices can connect without providing any identification, making it the least secure authentication method. Open networks are typically found in public places like coffee shops, libraries, or airports, where the priority is easy access rather than security.
In reality 'Open System Authentication' means 'No Authentication'. You would ask 'Is there anybody who really use this ?'.
The answer is Yes, largely for two reason. One is just using an Access Point for special testing purpose and in other more common situation is where you don't do any Authentication on AP (Access Point) level so you would set 'Open' in AP Security, but you let the packet goes through additional security system sitting behind AP. You would see this kind of methods being used in many company.
|
Step |
Direction |
Message |
|
1 |
Device -> AP |
MAC Address |
|
2 |
Device <- AP |
Status Code |
Status Code carries various information defined in 802.11 and it includes the status of whether attach is successful or not. Refer to 802.11 Association Status, 802.11 Deauth Reason codes for the details of the code.
Shared Key Authentication
Shared Key Authentication is an older method that uses a pre-shared key (PSK) known to both the client and the access point. The client and access point exchange encrypted messages to authenticate each other. However, Shared Key Authentication is considered insecure due to its reliance on the flawed WEP security protocol and is not recommended for use.
|
Step |
Direction |
Message |
|
1 |
Device -> AP |
MAC Address |
|
2 |
Device <- AP |
Challenge text (128 octets, clear) |
|
3 |
Device -> AP |
Response text (WEP encrypted) |
|
4 |
Device <- AP |
Status Code |
WPA-Personal / WPA2-Personal / WPA3-Personal - Pre-shared Key
WPA-Personal, WPA2-Personal, and WPA3-Personal use a pre-shared key (PSK) for authentication. All devices connecting to the network need to provide the correct PSK to gain access. This method is straightforward to set up and is suitable for home and small office networks. The security of the PSK method depends on the strength of the password and the encryption protocol used (WPA, WPA2, or WPA3).
|
Step |
Direction |
Description |
|
1 |
Device -> AP |
Device sends a probe request with the SSID to the AP. |
|
2 |
Device <- AP |
AP sends a probe response containing network information. |
|
3 |
Device -> AP |
Device sends an authentication request to the AP. |
|
4 |
Device <- AP |
AP sends an authentication response to the device. |
|
5 |
Device -> AP |
Device sends an association request. The pre-shared key (password) itself is never sent over the air. |
|
6 |
Device <- AP |
AP sends an association response, allowing the device to continue. The pre-shared key is checked later, in the 4-way handshake of steps 7 to 10. |
|
7 |
Device <- AP |
AP sends the ANonce (AP Nonce) to the device. |
|
8 |
Device -> AP |
Device generates the SNonce (Supplicant Nonce), derives the PTK, and sends the SNonce to the AP. |
|
9 |
Device <- AP |
AP derives the same PTK using the SNonce, ANonce, and pre-shared key, then sends a message confirming the PTK is installed and ready for encryption. |
|
10 |
Device -> AP |
Device sends a message confirming the PTK is installed and ready for encryption. |
|
11 |
Device <- AP |
Encrypted data communication begins between the device and the AP using the derived PTK. |
Steps 7 to 10 are the EAPOL 4-way handshake, and this is where the pre-shared key is actually checked. The PMK comes from the passphrase and the SSID. The PTK then comes from the PMK, the ANonce, the SNonce and the two MAC addresses. The device protects its message in step 8 with a MIC computed from the PTK. If the passphrase is wrong, the AP computes a different PTK, the MIC does not match, and the handshake fails. The key itself never goes over the air.
WPA3-Personal keeps this 4-way handshake but changes steps 3 and 4. Instead of Open System authentication, the device and the AP run SAE, which exchanges Commit and Confirm messages and derives the PMK. An attacker who records the exchange cannot test passwords against it offline, which is the main weakness of WPA2-Personal. You can see the WPA version of steps 7 to 10 as rows 8 to 11 of the capture on the WiFi Protocol page.
WPA-Enterprise / WPA2-Enterprise / WPA3-Enterprise - 802.1X Authentication
WPA-Enterprise, WPA2-Enterprise, and WPA3-Enterprise use the 802.1X authentication framework, which provides a more robust and scalable authentication solution for larger networks. It relies on a RADIUS server or another authentication server to manage user credentials and access policies. This method allows for individual user authentication, making it suitable for corporate or enterprise networks that require a higher level of security and access control.
|
Step |
Direction |
Description |
|
1 |
Device -> AP |
Device sends a probe request with the SSID to the AP. |
|
2 |
Device <- AP |
AP sends a probe response containing network information. |
|
3 |
Device -> AP |
Device sends an authentication request to the AP. |
|
4 |
Device <- AP |
AP sends an authentication response to the device. |
|
5 |
Device -> AP |
Device sends an association request to the AP. |
|
6 |
Device <- AP |
AP sends an association response, allowing the device to join the network. |
|
7 |
Device -> AP |
Device sends an EAPOL (Extensible Authentication Protocol over LAN) start message to initiate the 802.1X authentication process. |
|
8 |
Device <- AP |
AP sends an EAPOL message requesting the device's identity. |
|
9 |
Device -> AP |
Device sends its identity (e.g., username) in an EAPOL message to the AP. |
|
10 |
AP <-> Authentication Server |
AP forwards the identity to the authentication server (e.g., RADIUS server), which sends a challenge to the AP. |
|
11 |
Device <- AP |
AP forwards the challenge to the device. |
|
12 |
Device -> AP |
Device sends a response to the challenge, typically by providing a password or certificate. |
|
13 |
AP <-> Authentication Server |
AP forwards the response to the authentication server, which verifies the response and sends the result to the AP. |
|
14 |
Device <- AP |
If the authentication server approves the response, the AP sends a success message to the device, granting access to the network. |
|
15 |
Device <- AP |
Device and AP perform a four-way handshake to derive encryption keys for secure communication. |
|
16 |
Device <- AP |
AP sends a random value called the ANonce (AP Nonce) to the device. |
|
17 |
Device -> AP |
Device generates its own random value, called the SNonce (Supplicant Nonce), and uses it along with the ANonce, the authentication keys, and other information to derive a Pairwise Transient Key (PTK). The device sends the SNonce back to the AP. |
|
18 |
Device <- AP |
AP uses the SNonce, ANonce, authentication keys, and other information to derive the same PTK. The AP sends a message to the device confirming that the PTK is installed and that the AP is ready to encrypt data. |
|
19 |
Device -> AP |
Device sends a message back to the AP, confirming that it has also installed the PTK and is ready to encrypt data. |
|
20 |
Device <- AP |
Once the four-way handshake is complete, the device and the AP use the derived PTK to encrypt and decrypt data transmitted between them. This ensures that the data is protected from eavesdropping and tampering. |
In Enterprise mode, the PMK comes from the EAP exchange in steps 7 to 14 rather than from a passphrase. The device and the authentication server both derive keying material during EAP, and the server sends its copy to the AP with the success result. So each user gets a different PMK. In Personal mode, every device on the network starts from the same passphrase. The 4-way handshake in steps 15 to 20 is the same in both modes. It only starts from a different PMK.
Captive Portal
A Captive Portal is an authentication method often used in public Wi-Fi networks, like hotels or cafes, where users need to provide additional information or agree to terms and conditions before accessing the network. When connecting to a captive portal, users are redirected to a web page where they need to enter their credentials, such as a username and password, or provide other details like an email addres
|
Step |
Direction |
Description |
|
1 |
Device -> AP |
Device sends a probe request with the SSID to the AP. |
|
2 |
Device <- AP |
AP sends a probe response containing network information. |
|
3 |
Device -> AP |
Device sends an authentication request to the AP. |
|
4 |
Device <- AP |
AP sends an authentication response to the device. |
|
5 |
Device -> AP |
Device sends an association request to the AP. |
|
6 |
Device <- AP |
AP sends an association response, allowing the device to join the network. |
|
7 |
Device -> AP |
Device tries to access an external website (e.g., www.example.com) or the device automatically sends a request to a predetermined URL to check for internet access. |
|
8 |
Device <- AP |
Instead of providing access to the requested website, the AP intercepts the request and redirects the device to the captive portal page, which may require authentication, payment, or acceptance of terms and conditions. |
|
9 |
Device -> AP |
Device submits the required information on the captive portal page (e.g., entering a username and password, making a payment, or agreeing to terms and conditions). |
|
10 |
Device <- AP |
Once the device provides the required information, the AP grants the device full access to the network and the internet. |
|
11 |
Device <->AP |
Device can now access the internet and other network resources as permitted by the network policies. |
Authentication using SSID of AP
This method is not a real authentication. The device only has to know the SSID, and the SSID is sent in clear text in Beacon and Probe frames. Hiding the SSID in the Beacon does not help much, because the SSID still appears in the Probe Request of every device that joins the network.
|
Step |
Direction |
Message |
|
1 |
Device -> AP |
Probe request that includes SSID of AP |
|
2 |
Device <- AP |
Access OK |
MAC address filtering
MAC address filtering lets the AP accept only devices whose MAC address is on a list set by the administrator. It is easy to set up, but it is weak protection. The MAC address travels unencrypted in every frame header, so an attacker can read an allowed address from the air and use it on another device. Many recent phones also use a random MAC address for each network, so the list must hold that address rather than the hardware address.
|
Step |
Direction |
Message |
|
1 |
Device -> AP |
MAC Address |
|
2 |
AP |
AP searches the MAC Address in predefined list |
|
3 |
Device <- AP |
Access OK |
WPA in Authentication and Security
You would notice WPA family is used both in Authentication process and Security Process. Any major differences between the two ? The short answer is that one WPA name covers two separate jobs. One job decides who may join the network, and the other job protects the frames after the device has joined.
WPA authentication is about verifying the identity of devices trying to connect to a Wi-Fi network, while WPA security algorithms focus on the encryption methods used to protect data transmitted over the network.
WPA Authentication:
WPA authentication refers to the process of verifying the identity of a device when it tries to connect to a Wi-Fi network. WPA authentication can be either "WPA-Personal" (also known as WPA-PSK) or "WPA-Enterprise" (also known as WPA-802.1X). WPA-Personal uses a pre-shared key (PSK) for authentication, which is a passphrase known to both the device and the access point. WPA-Enterprise uses an authentication server, such as a RADIUS server, to authenticate users based on their individual credentials, like usernames and passwords or digital certificates.
WPA Security Algorithms:
WPA security algorithms focus on the encryption methods used to protect data transmitted over the Wi-Fi network. For WPA, the primary security algorithm is the Temporal Key Integrity Protocol (TKIP). TKIP was designed to improve the security of the older Wired Equivalent Privacy (WEP) standard without requiring new hardware. However, TKIP has its own limitations and vulnerabilities.
The same split continues in WPA2 and WPA3. WPA2 replaces TKIP with CCMP, which is based on AES, and WPA3 keeps AES-based encryption. On the authentication side, WPA2 keeps PSK and 802.1X, and WPA3-Personal replaces PSK with SAE. This is why the Windows dialog near the top of the page has two separate menus. Security type names the authentication, such as WPA2-Personal, and Encryption type names the cipher, such as AES.
The WPA name alone does not say how data is encrypted : WPA-Personal or WPA2-Enterprise names the authentication, and TKIP or AES names the cipher.Authentication produces the key that encryption uses : PSK, SAE or 802.1X gives the PMK, and the 4-way handshake turns it into the PTK for TKIP or AES.TKIP belongs to WPA and should be avoided : it still uses RC4, while AES-based CCMP is the WPA2 choice.
Video Tutorial
The two videos below cover the same topics from a different angle. The first follows the ICND1 course material on WLAN security. The second shows a rogue AP and packet capture with AirPcap.