How to Troubleshoot Palo Alto IPsec VPN Phase 1 and Phase 2 Problems

Troubleshooting site-to-site IPsec tunnels requires a structured approach. When a VPN connection fails, network engineers must isolate whether the issue stems from underlay IP connectivity, Phase 1 IKE negotiation, Phase 2 IPsec proposals, proxy ID misconfigurations, routing failures, or security policy blocks. This guide presents a systematic, workflow-driven method for Palo Alto IPsec VPN troubleshooting in enterprise PAN-OS environments.

Note: The incident, network parameters, and logs described in this tutorial represent a realistic lab troubleshooting scenario. All IP addresses, hostnames, and security parameters are example values and must be adapted to your production infrastructure.

Incident Scenario

An enterprise organization operates a central data center protected by a pair of Palo Alto Networks firewalls and a remote branch office running a third-party security appliance. During a scheduled off-hours network maintenance window, the site-to-site IPsec VPN tunnel connecting the data center to the branch office went offline.

Application traffic between the data center subnet (10.1.10.0/24) and the branch subnet (10.2.10.0/24) is failing completely. Network operations reported that users at the branch office cannot access critical database servers located in the primary data center. Your objective is to use PAN-OS operational tools, web interface metrics, system logs, and CLI utilities to isolate the exact cause, restore connectivity, and verify steady-state operation.

Network Topology

The site-to-site VPN uses a route-based design. The local Palo Alto Networks firewall terminates the IPsec tunnel on a dedicated logical interface (tunnel.1) bound to the internal VPN-Zone. Underlay connectivity routes across public internet addresses.

   +-----------------------+                         +-----------------------+
   |   Data Center FW      |                         |   Branch Appliance    |
   | Palo Alto Networks    |                         |  (Third-Party Peer)   |
   |                       |                         |                       |
   | Trust: 10.1.10.1/24   |                         | Trust: 10.2.10.1/24   |
   | External: 192.0.2.10  |======= IPSec Tunnel ====| External: 198.51.100.20|
   | Tunnel Interface:     |     (over Internet)     | Tunnel Interface:     |
   |   tunnel.1 (Unnumbered|                         |   10.255.0.2/30       |
   |   or 10.255.0.1/30)   |                         |                       |
   +-----------------------+                         +-----------------------+
               |                                                 |
      Data Center Network                                  Branch Network
         10.1.10.0/24                                       10.2.10.0/24

Symptoms and Evidence

Initial inspection reveals that the IPsec tunnel status indicator in the PAN-OS Web Interface is partially down. Phase 1 (IKE) displays a green status icon, but Phase 2 (IPsec) shows a red status icon, indicating that key exchange succeeds but security associations cannot be negotiated.

Parameter	Data Center (Local PA-FW)	Branch Office (Remote Peer)
Public IP (Underlay)	192.0.2.10 (LAB/EXAMPLE)	198.51.100.20 (LAB/EXAMPLE)
Local Protected Subnet	10.1.10.0/24	10.2.10.0/24
Remote Protected Subnet	10.2.10.0/24	10.1.10.0/24
Tunnel Interface	tunnel.1 (Zone: VPN-Zone)	vti.0 (Zone: VPNDTZ)
IKE Gateway Name	GW-Branch-Office	HQ-DataCenter-GW
IPsec Tunnel Name	Tunnel-Branch-Office	HQ-IPsec-Tunnel

Troubleshooting Strategy

Efficient Palo Alto IPsec VPN troubleshooting requires a structured, multi-layer methodology. Avoid guessing settings or changing parameters at random. Follow this logical sequence to isolate the failure layer:

1. Underlay & Physical Layer Check: Ensure external interfaces can route and reach the remote peer IP address over UDP 500 / UDP 4500.
2. IKE Phase 1 Negotiation: Check exchange mode, pre-shared keys, IKE Crypto Profiles (DH Groups, Encryption, Authentication), and IKE Gateway status.
3. IPsec Phase 2 Negotiation: Inspect IPsec Crypto Profiles, Transform sets, Perfect Forward Secrecy (PFS), and explicit Proxy IDs (Traffic Selectors).
4. Routing Architecture: Verify that traffic destined for the remote subnet points to the correct logical tunnel interface (e.g., tunnel.1).
5. Security Policy Enforcement: Validate that security rules permit traffic flow between the local zone (e.g., Trust) and the tunnel zone (e.g., VPN-Zone) in both directions.

Step-by-Step Investigation

1. Web Interface Status Checks

Navigate to Network > IPSec Tunnels in the PAN-OS Web Interface.

• Locate Tunnel-Branch-Office in the interface table.
• Observe the Status column. The status shows two light indicators:
  • IKE Gateway (Left Light): Green (Phase 1 active).
  • IPsec Tunnel (Right Light): Red (Phase 2 inactive).

Because Phase 1 is green, main mode or aggressive mode exchange succeeded. The authentication and Phase 1 crypto profile match. The failure occurs during Quick Mode (Phase 2) negotiation.

2. Checking Crypto Profile Configurations

Navigate to Network > Network Profiles > IPsec Crypto and check the settings assigned to the IPsec tunnel profile:

• IPsec Protocol: ESP
• Encryption: aes-256-cbc
• Authentication: sha256
• DH Group: no-pfs

Check the remote peer’s configured requirements. The remote peer administrator confirmed they recently updated their local compliance policy to enforce Perfect Forward Secrecy using Group 14 (DH14) and AES-256-GCM encryption.

CLI Investigation

The PAN-OS Command Line Interface (CLI) provides explicit operational details that help diagnose tunnel failures faster than the GUI.

Step 1: Check IKE Phase 1 Security Associations

Run the following operational command to inspect Phase 1 status:

show vpn ike-sa gateway GW-Branch-Office

Expected output confirms Phase 1 status is established:

IKE Gateway GW-Branch-Office:
Peer IP: 198.51.100.20
Role: Initiator
State: Established
Algorithm: AES256-CBC / SHA256 / DH Group 14
Lifetime: 28800 seconds

Step 2: Check IPsec Phase 2 Security Associations

Run the command to view active Phase 2 Security Associations (SAs):

show vpn ipsec-sa tunnel Tunnel-Branch-Office

If Phase 2 has failed, this command returns no active SAs or indicates an incomplete state:

No IPsec SA found for tunnel Tunnel-Branch-Office.
Total IPsec SAs shown: 0

Step 3: Analyze the IKE Manager Operational Log

To view real-time log messages generated by the IKE daemon (ikemgr), execute the following tail command:

tail follow yes lines 100 mp-log ikemgr.log

Look for Phase 2 error messages during negotiation. Common failure strings include:

2026-03-30 10:14:22.112 -0700 [WARN]: IKE protocol notification received: NO_PROPOSAL_CHOSEN (14)
2026-03-30 10:14:22.113 -0700 [ERROR]: IPsec Phase 2 negotiation failed for tunnel Tunnel-Branch-Office. Mismatch in IPsec crypto proposal or Proxy ID parameters.

The log line NO_PROPOSAL_CHOSEN indicates that the Phase 2 proposal parameters (Encryption, Authentication, PFS, or Proxy IDs) offered by the local firewall do not match the parameters required by the remote peer.

CAUTION: Debug level logging consumes management plane resources. Do not run advanced debug commands like debug ike global on debug on high-load production firewalls without explicit approval from technical support. Always remember to turn off debug logging immediately after capturing required events.

Traffic Log Analysis

Next, examine how traffic flows through the packet processing pipeline when internal hosts attempt to reach the remote branch.

1. Examine Traffic Logs

Navigate to Monitor > Logs > Traffic and filter by destination address:

(addr.dst in 10.2.10.0/24)

The logs show session entries with the action drop or deny, or sessions showing aged-out state with zero return bytes. This behavior occurs because the firewall cannot encapsulate packet payloads into an unestablished IPsec SA.

2. Session State Verification via CLI

Query the active session table to verify how outbound traffic is processed:

show session all filter source 10.1.10.15 destination 10.2.10.20

Example command output:

ID       Application    State   Type Flag  Src Zone    Dst Zone
-------------------------------------------------------------------------------
482101   web-browsing   DISCARD FLOW       Trust       VPN-Zone
  Inbound Interface: ethernet1/2
  Outbound Interface: tunnel.1
  Session Status: Discard due to encapsulated tunnel down

The session output confirms that routing correctly points to tunnel.1 and Security Policy allows traffic from Trust to VPN-Zone. However, the session drops at the flow layer because the IPsec SA is down.

Root Cause

Detailed analysis of ikemgr.log and configuration review isolates two distinct root causes for the Phase 2 failure:

1. Crypto Mismatch: The remote peer updated its local configuration to require AES-256-GCM with PFS Group 14. The local firewall’s IPsec Crypto Profile was configured for AES-256-CBC with no-pfs.
2. Proxy ID Mismatch: Route-based VPNs on Palo Alto Networks firewalls do not require explicit Proxy IDs when connecting to another PAN-OS device (defaulting to local 0.0.0.0/0 and remote 0.0.0.0/0). However, the remote policy-based security appliance requires explicit proxy subnets: local 10.1.10.0/24 and remote 10.2.10.0/24. Because Proxy IDs were left blank on the local firewall, the remote peer rejected the Quick Mode negotiation payload.

Resolution

To bring the IPsec Phase 2 tunnel online, apply the minimum necessary configuration changes on the local Palo Alto Networks firewall.

Step 1: Update the IPsec Crypto Profile

1. Navigate to Network > Network Profiles > IPsec Crypto.
2. Click on the profile attached to the branch tunnel (e.g., Crypto-Prof-Branch).
3. In the IPsec Protocol tab:
   • Set Encryption to aes-256-gcm.
   • Set Authentication to sha256 (or leave blank if using GCM authenticated encryption algorithms per peer design).
   • Set DH Group to group14.
4. Click OK.

Step 2: Configure Proxy IDs on the IPsec Tunnel

1. Navigate to Network > IPSec Tunnels.
2. Click on Tunnel-Branch-Office to open its properties.
3. Select the Proxy IDs tab.
4. Click Add and configure the explicit subnet match:
   • Proxy ID Name: Branch-Subnet-1
   • Local: 10.1.10.0/24
   • Remote: 10.2.10.0/24
   • Protocol: Any
5. Click OK twice to close the dialogue boxes.

Step 3: Commit Configuration Changes

Click Commit in the top right corner of the Web Interface to apply the changes to the running configuration.

Verification After the Fix

Once the commit operation completes successfully, force an explicit negotiation test using the operational CLI commands.

1. Force Phase 1 and Phase 2 Negotiation

Clear existing stale associations and re-initiate negotiation manually:

test vpn ike-sa gateway GW-Branch-Office

test vpn ipsec-sa tunnel Tunnel-Branch-Office:Branch-Subnet-1

2. Confirm Tunnel Status via CLI

Verify that Phase 2 SAs are successfully established:

show vpn ipsec-sa tunnel Tunnel-Branch-Office

Expected CLI output showing active SAs:

Tunnel Tunnel-Branch-Office:
  IPsec SA name: Tunnel-Branch-Office:Branch-Subnet-1
  Encryption algorithm: AES-256-GCM
  Authentication algorithm: SHA256
  PFS Group: DH Group 14
  Lifetime: 3600 seconds
  Inbound SPI:  0x9A8B7C6D
  Outbound SPI: 0x1F2E3D4C
  Status: Active

3. Verify Routing and Flow Capabilities

Initiate an operational ping test from the local firewall’s internal interface to the remote gateway host across the tunnel:

ping source 10.1.10.1 host 10.2.10.1 count 5

Output confirming full reachability:

PING 10.2.10.1 (10.2.10.1) from 10.1.10.1 : 56(84) bytes of data.
64 bytes from 10.2.10.1: icmp_seq=1 ttl=64 time=18.4 ms
64 bytes from 10.2.10.1: icmp_seq=2 ttl=64 time=17.9 ms
64 bytes from 10.2.10.1: icmp_seq=3 ttl=64 time=18.1 ms
64 bytes from 10.2.10.1: icmp_seq=4 ttl=64 time=17.8 ms
64 bytes from 10.2.10.1: icmp_seq=5 ttl=64 time=18.0 ms

--- 10.2.10.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms

4. Verify Web Interface Indicators

Return to Network > IPSec Tunnels. Both status indicators (IKE Gateway and IPsec Tunnel) should now show steady Green lights.

Why This Problem Happened

Palo Alto Networks uses a route-based IPsec VPN architecture. In a pure route-based environment where both peers are PAN-OS firewalls, Phase 2 negotiations pass generic 0.0.0.0/0 Proxy IDs. The actual traffic filtering and forwarding are deferred entirely to the routing engine and Security Policy rules.

However, when connecting a PAN-OS firewall to policy-based VPN endpoints or third-party devices, Phase 2 Quick Mode requires exact symmetry between local and remote traffic selectors (Proxy IDs). If host subnets on both sides do not match bit-for-bit (e.g., local firewall sending 0.0.0.0/0 while remote peer expects 10.1.10.0/24), the remote peer rejects the proposal payload and returns a NO_PROPOSAL_CHOSEN message.

Additionally, Phase 2 negotiations fail if cryptographic settings differ. When PFS (Perfect Forward Secrecy) is enabled on one endpoint but disabled on the other, Quick Mode key exchanges cannot derive shared keys, causing SA instantiation to fail.

Prevention and Monitoring

To ensure high availability and prevent future VPN outages, implement the following operational procedures:

1. Configure Tunnel Monitoring

Enable Tunnel Monitoring to automatically test path reachability and failover or restart SA negotiations if traffic stops flowing:

• Navigate to Network > IPSec Tunnels > [Tunnel Name] > Tunnel Monitor.
• Check Enable.
• Specify a target monitoring IP address located at the remote branch (e.g., 10.2.10.1).
• Select an action profile (e.g., fail-over or wait-recover).

2. Configure System Log Filters and Syslog Forwarding

Configure automated log notifications for VPN status changes. Navigate to Device > Log Settings > System and create a filter to trigger alerts when VPN components change status:

( eventid eq ikev2-ike-sa-failure ) or ( eventid eq ikev2-ipsec-sa-failure ) or ( object eq 'Tunnel-Branch-Office' )

3. Manage NAT Rules for Inter-Zone Traffic

Ensure that outbound source NAT rules matching external interfaces explicitly exclude tunnel-bound traffic. Traffic traversing route-based VPNs should not have its internal RFC 1918 addresses translated unless explicit overlapping-IP NAT topologies are required.

Related Palo Alto Guides

• Palo Alto GlobalProtect configuration
• Palo Alto certificate with Microsoft AD CS
• Palo Alto Syslog/SIEM configuration
• Palo Alto security profiles

Summary

Systematic Palo Alto IPsec VPN troubleshooting relies on isolating Phase 1 IKE issues from Phase 2 IPsec and routing failures. When performing maintenance or responding to outages:

• Verify underlay reachability over standard IPsec ports (UDP 500 / UDP 4500).
• Use show vpn ike-sa to confirm IKE Phase 1 authentication and gateway parameters.
• Use show vpn ipsec-sa and inspect ikemgr.log to identify Quick Mode proposal mismatches.
• Match cryptographic profiles (Encryption, Authentication, PFS) and explicit Proxy IDs exactly when interfacing with non-PAN-OS endpoints.
• Confirm that routing tables point destination traffic to the logical tunnel interface and that Security Policies permit cross-zone communication.