PyNet Labs- Network Automation Specialists

What is IPsec (Internet Protocol Security)?

Last Updated : August 12, 2023
What is IPsec Featured Image


Nowadays, most organizations rely heavily on the Internet for daily business; as a result, there has been a surge in cyberattacks. As cybercrime becomes more common, businesses find it difficult to protect sensitive data without compromising productivity. That’s why IPsec is widely used by businesses to allow secure transmission and access of data. But what is IPsec? It is a framework that protects IP traffic at the network layer. IPsec can secure a company’s precious data with characteristics like anti-replay, authenticity, integrity, and confidentiality.

In this blog, we will mainly focus on the IPsec protocol, its history, how does IPsec work, its mode, and various other crucial concepts. Let’s Begin!

What is IPsec?

IPsec stands for Internet Protocol Security. It is a collection of communication protocols that establish secure connections across a network. The Internet Protocol (IP) is a widely adopted standard that regulates data transmission across the Internet. IPsec enhances the protocol’s security by integrating encryption and authentication techniques.

The IPsec protocol suite may be utilized by anybody, from individuals to massive enterprises, and it also acts as the primary protocol for various Virtual Private Networks (VPNs).

History of IPsec

In the early 1980s, when the Internet Protocol was being developed, security was not a top priority. But as the number of internet users increased, it became clear that more security was needed.

In response to this need, the National Security Agency provided sponsorship for developing security protocols as part of the Secure Data Network Systems initiative. As a result, the emergence of a Security Protocol at Layer 3 occurred, which subsequently led to Network Layer Security Protocol. Various engineers were involved in the 1990s in the development of IPsec.

It is well known today because it can be easily adapted without making any modifications to the associated peers.

Uses of IPsec

IPsec can be used to perform the following functions:

  • Protect data transmissions at the router level when using the public Internet.
  • It helps in encrypting the application data.
  • Quickly verify the authenticity of data if it is sourced from a trusted sender.
  • Enhance network data security by establishing encrypted circuits, commonly referred to as IPsec tunnels, which effectively encrypt all transmitted data between two chosen endpoints.

Businesses use IPsec as a safeguard against replay attacks. Replay attacks, also known as man-in-the-middle attacks, involve altering the content of a transmission in real-time by redirecting it via a third machine. Each data packet is given a unique identifier and checked for any indications of duplication by the IPsec protocol.

We already understand what is IPsec, how it comes into action, and the uses of IPsec. Before getting into the working of IPsec, let’s first understand IPsec encryption.

What is IPsec encryption?

IPsec encryption protects the data by encrypting and authenticating each packet. Encryption keys are used to encode data, while decryption keys are used to restore the original data.

IPsec uses both symmetric and asymmetric encryption techniques to achieve this. Symmetric encryption uses the same key to encrypt and decrypt the data, while asymmetric encryption uses a pair of keys: public and private keys. The public key can be shared with anyone, but the owner keeps the private key secret. IPsec uses symmetric encryption for the data payload and asymmetric encryption for the key exchange.

Let’s move on to the working of IPsec.

Working of IPsec

With IPsec, a client may safely exchange data with another network. Remember that this kind of communication isn’t commonly used for communicating with other devices but rather for connecting a laptop to a private network across a public network. IPsec may also be used to link two private networks together.

Further, the working of IPsec is classified into five steps. These are shown below with a diagram.

Working of IPsec

1 – Host Recognition

The host system is capable of recognizing when a data packet requires security measures and should be transmitted using IPsec protocols. At this stage, the data packet undergoes encryption authentication processes, making it ready for transmission.

2 – Negotiation (IKE Phase 1)

The two host systems that interact via IPsec set up an understanding of the protocols to be utilized and together authenticate the identities to each other. Between the two of them, a secure connection is formed, through which negotiation may occur as to what algorithms and regulations will be in operation. Further, negotiations can be of two types, i.e., main (simple) and aggressive.

  • Main negotiation: The host system proposes encryption and authentication algorithms, and subsequent systems negotiate their acceptance with them.
  • Aggressive negotiation: The host system presents its preferred encryption and authentication methods without engaging in negotiation or adjusting its preferences. If the host system receives agreement from the other party, the process proceeds to the subsequent step. If the specified condition is not met, the subsequent process will not proceed.

3 – IPsec Circuit (IKE Phase 2)

After successfully establishing a secure connection, as mentioned above, an IPsec circuit is then established. The host systems establish a mutual agreement on the encryption and decryption keys to be utilized. Additionally, they exchange cryptographic nonces, which are randomized numbers utilized for authentication purposes.

4 – Transmission

The transmission of encrypted IP packets occurs between the host systems. Upon arrival, the data packet undergoes encryption using the encryption keys that were previously shared.

5 – Termination

The IPsec connection is terminated when either all of the data has been transmitted, or the session duration limit has been reached. The private keys used in the transaction are deleted upon completion of the packet transfer.

What are IPsec Protocols?

IPsec protocols ensure the safety of data transmissions. A data packet is a predefined data structure used to prepare data for transmission across a network. It mainly consists of three elements. These are:

  1. Header: A header is a segment that precedes and contains instructional information necessary for the proper routing of a data packet to its intended destination.
  2. Payload: The term “payload” refers to the actual information that can be contained inside a data packet.
  3. Trailer: The trailer is an extra piece of information that is attached to the end of the payload in a data packet, used to signify the end of the sent data.

Below we have explained some of the IPsec protocols.

Authentication Header (AH)

The authentication header (AH) protocol adds a header with sender authentication information to prevent unauthorized parties from compromising a packet’s data. It warns the receiver of any modification that may have occurred with the original data packet. The computer checks the header and the result of the cryptographic hash computation performed on the data payload to ensure they are identical.

Authentication Header (AH)

Encapsulating security payload (ESP)

Authentication Headers are utilized to verify the identities of both the sender and receiver and to detect any tampering with the packets. On the other hand, Encapsulating Security Payloads (ESP) provides an additional layer of authentication along with encryption.

In the transport mode implementation of IPsec, the Encapsulating Security Payload (ESP) wraps the data payload while leaving the header intact and accessible. In tunnel mode, ESP envelops the whole data packet and adds an additional header on the outside. The outer header is the only visible component of the system without the need for authentication and the encryption key.

Encapsulating security payload (ESP)

Internet Key Exchange (IKE)

The Internet Key Exchange (IKE) protocol creates a secure connection between two devices operating on the Internet. Both devices establish a security association (SA) by engaging in the process of negotiating encryption keys and algorithms that will be used for the transmission and reception of future data packets.

IPsec Modes

IPsec is a network protocol that allows secure communication by offering two modes of operation: tunnel mode and transport mode. The modes are closely linked to the choice of protocols used, specifically AH or ESP.

Let’s understand both these Modes in detail.

  • Tunnel Mode: Every packet is kept safe when operating in tunnel mode. IPsec encapsulates the data packet within a new packet, applies encryption to it, and adds a new IP header. It is widely utilized for Site-to-site VPN setups.
  • Transport Mode: The IPsec transport mode encrypts the data packet’s payload and keeps the IP header in its original state. The unencrypted packet header helps routers identify the destination address for each data packet.

IPsec transport is frequently used within a secure and trusted network environment to safeguard a direct and uninterrupted communication link between two computer systems.

Note: When it comes to virtual private networks (VPNs), ESP with authentication in tunnel mode is the most typical IPsec setup. This setup allows for encrypted and anonymous data transfer over unsecured networks inside a VPN tunnel.

What is IPsec VPN?

An IPsec VPN, also known as a virtual private network, is a type of VPN that utilizes the IPsec protocol to establish a secure and encrypted tunnel over the internet.

A Virtual Private Network (VPN) is a technology that securely routes network traffic through an encrypted tunnel, ensuring the protection of data from unauthorized access. An IPsec VPN utilizes the IPsec protocol to establish a secure connection and encrypt data packets during transmission. This technology is especially beneficial for businesses and large organizations that have employees working remotely and require remote access to resources.

By configuring an IPsec VPN between a remote worker’s device and an internal server, an organization may provide the person with the same level of secure access to internal resources as if they were physically present in the office.

There are a variety of strategies for setting up an IPsec VPN. Some of these are Site-to-site, Client-to-site, and Client-to-client.

There’s always confusion over IPsec and SSL VPN, which one is better or which one is more secure. Below we have explained the basic difference between IPsec VPN and SSL VPN.

IPsec vs SSL

EncryptionIPsec VPN encrypts the entire IP packet at the network layer.SSL VPN encrypts only the application data at the transport layer.
AuthenticationIPsec VPN uses certificates or pre-shared keys for authentication.SSL VPN uses certificates or usernames/passwords for authentication.
DeploymentIPsec VPN requires client software installation and configuration on each device.SSL VPN can be accessed through a web browser without any client software installation.
PerformanceIPsec VPN has lower overhead and better throughput than SSL VPN.SSL VPN has higher overhead and lower throughput than IPsec VPN.
CompatibilityIPsec VPN supports any IP-based application and protocol.SSL VPN supports only web-based and some TCP-based applications and protocols.

Frequently Asked Questions

Q1 – What is IPsec and why it is used?

IPsec is a set of protocols that provide security for Internet Protocol (IP) networks. It is used to encrypt and authenticate the data packets that are exchanged between two or more parties. IPsec can protect the confidentiality, integrity, and availability of network communications.

Q2 – What are the 3 protocols used in IPsec?

The three main protocols used in IPsec are:

  • Authentication Header (AH): This protocol provides data integrity and authentication for IP packets but not encryption.
  • Encapsulating Security Payload (ESP): This protocol provides data integrity, authentication, and encryption for IP packets.
  • Internet Key Exchange (IKE): This protocol establishes and manages the cryptographic keys and parameters for AH and ESP.

Q3 – Why SSL is better than IPsec?

SSL allows for more specific permissions for users than IPsec does. Members of the remote team are given access to specific programs rather than being given complete network membership. This makes it easy to provide varying degrees of access to various users.

Q4 – What port is IPsec?

IP Security Virtual Private Network (IPsec VPN) uses IP Protocol 50 (Encapsulating Security Payload, or ESP) as its layer 3 communication protocol. Internet Key Exchange (IKE) may also be needed to maintain encryption keys, which can require UDP port 500.


IPsec consists of a standard set of protocols used to protect internet connections, ensuring authenticity, confidentiality, and integrity. It offers a transparent and secure route for upper-layer protocols without requiring any changes to these protocols or applications.

This Blog has covered all the basic concepts and different IPsec modes as well as different IPsec protocols. Still, if anyone wants to add something, feel free to comment below.

Leave a Reply

Your email address will not be published. Required fields are marked *

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram