[lwip-users] IP forwarding to/from PPP working for one netif, but not another

[Andrew Pullin]

Hi folks,

I am stuck on an issue here where I am trying to use lwip's 
IP_FORWARDING feature.
I am trying to forward packets between a PPP server netif and a WAN 
interface. It is working on case, where the WAN-connected device is 
using WiFi, but then it fails to work in another case where the WAN 
device is configured to use an Ethernet interface.

This is to ultimately bring Ethernet connectivity to a device without 
Ethernet, but which can act as a PPP client (via UART).
Lwip is running on a device with Ethernet and using the lwip PPP server 
implementation.

I have read the other threads involving PPP and forwarding, but I don't 
believe I have seen quite this same problem arise.

Here is how the devices are arranged for each case:

Working:
   10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------>  
10.0.200.121 ----------> 10.0.1.1 ---> WAN
(PPP client)                    (PPP server, ESP32, lwip) (WiFi STA 
netif)               ( router )

In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1) 
and the netmask as 255.255.0.0

Not working:
   10.0.0.2 ------[UART]----------> 10.0.0.1 ------------------>  
10.0.200.113 ----------> 10.0.1.1 ---> WAN
(PPP client)                   (PPP server, ESP32, lwip)            
(Ethernet netif)               ( router )

In this case, the Ethernet netif reports the gateway IP correctly 
(10.0.1.1) and the netmask as 255.255.0.0 , which appears to be the same 
as for the working WiFi netif case.

The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually 
configured. They are hardcoded into the PPP server setup, and the client 
appears to be able to take it's 10.0.0.2 address via IPCP over PPP.


To test that the WAN connection "works" for the PPP client, I am issuing 
pings from PPP client device.
In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway), 
and 8.8.8.8. All work.
In the "not working" case on ethernet, I can ping 10.0.0.1 (just over 
the PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable.

Of course, I am building with IP_FORWARD defined to 1. Debug logs do 
reflect that forwarding is happening.
There are several shims in the Espressif SDK over the actual lwip netif 
sets/calls.
As far as I can see, the same setup is done for the ethernet netif as is 
for the wifi netif, but this is an obviously suspicious point.

In my case, I am using an ESP32 chip, and all of the lwip init and setup 
is done for me in the SDK.
As far as I can see, the init for each adapter/netif type looks the same.

I have logs for each case, too:
Working (wifi netif) - https://pastebin.com/RgqHa5CR
Not working (ethernet netif) - https://pastebin.com/L80raiRF

One note here is: the PPP client is not running lwip. For hardware & 
legacy reasons, it is running another network stack with a totally 
different PPP client implementation.
But, given that the working/not working is a function of the PPP server 
netif's, I did not suspect a mis-configuration issue in the PPP client.

Any insight here would be greatly appreciated. I am not too familiar 
with the deep under-the-hood details of network stacks.

Thanks,
Andrew Pullin