Allow response 404 and serving compressed files without triggering the captive portal

ESP32, Arduino Framework, using these libraries:

lib_deps= 
    ESP32Async/AsyncTCP
    ESP32Async/ESPAsyncWebServer
    https://github.com/guestisp/ESP32AsyncDNSServer.git

here the relevant code:

#include <ESPAsyncWebServer.h>
AsyncWebServer _server;

void serverBegin()
{

    _server.reset();

   // optional other _server.on() handlers for API, ecc...
   
    _server.onNotFound([this](AsyncWebServerRequest *request)
    { 
        if (handleStaticFile(request)) return;
        request->send(404);
    });

    _server.begin();
}

bool handleStaticFile(AsyncWebServerRequest *request) 
{
  // handle requests to root /
  String path = "/www" + request->url();   
  if (path == "/www/") path += "/index.html";

  // looking for compressed files
  String pathWithGz = path + ".gz";
  bool gzipped = LittleFS.exists(pathWithGz);

  if (gzipped)
  {
      AsyncWebServerResponse *response = request->beginResponse(LittleFS, pathWithGz, "");
      response->addHeader(F("Content-Encoding"), F("gzip"));
      request->send(response);
      return true;
  }
  else if (LittleFS.exists(path)) 
  {
      request->send(LittleFS, path, F(""), false, std::bind(&WebApp::processor, this, std::placeholders::_1)); 
      return true;
  }
  else return false;
}

I just left out the usual WiFi connection part. The code works as follow:

1. when an HTTP request is made from a browser the server will first look for a specific handler (usually for dynamic or API content) - if nothing is found it enters the onNotFound handler.

2. here it calls handleStaticFile() to serve files from the flash. If a compressed file (.gz) is found it serves this one adding the correct header

3. if neither the compressed nor the uncompressed file exist the function returns false and the response will be a 404

It works fine if I connect my ESP32 as station to an existing WiFi network. Instead, if I do the same using the ESP32 as an hotspot, initialized in this way:

void goHotspot()
{
    IPAddress local_ip(192, 168, 4, 1); 
    IPAddress gateway(192, 168, 4, 1); 
    IPAddress subnet(255, 255, 255, 0);

    WiFi.mode(WIFI_MODE_AP);
    WiFi.softAP(MY_SSID, MY_PASSWORD);
    WiFi.softAPConfig(local_ip, gateway, subnet);
    WiFi.softAPsetHostname(APP_NAME);
    esp_wifi_set_ps(WIFI_PS_NONE);

    serverBegin();
}

When I connect my smartphone or my PC to the hotspot a captive portal is triggered. As far as I understand this happens because the browser's request to check the connection (i.e. http://connectivitycheck.gstatic.com/generate_204) will fall through all the steps above and will end with a 404.

But even a 404 is enough for the browser to assume there is an active connection and then it opens a captive portal. So I tried to comment out that line providing no response:

_server.onNotFound([this](AsyncWebServerRequest *request)
{ 
    if (handleStaticFile(request)) return;
});

this actually work: no captive portal is triggered when I connect to the hotspot. The downside is if the user tries to reach a non-existing page the browser just shows the no response error:

Handler did not handle the request

This is not a good thing: the user should receive a 404 error (or better a 404 landing page).

So, how to provide a 404 error without triggering the captive portal if the request is done when in hotspot mode?