External polling (not wakeup) or deferred execution

[ldct-polemos]

I think the library is great in terms of offered features, but at the same time it is difficult to build abstractions on top of it. Let's take a look at two scenarios:

• application with an already established event loop framework (custom or popular like libuv) where all services reside in the main thread and use worker threads for long processing operations, while inter-service communication happens on the main thread, without the need to any locking mechanism;
• applications with multiple threads, where each service of the application resides (for simplicity) in a separate thread. Each service exposes methods to asynchronously get/put member variables or to invoke method and asynchronously publish the results in the caller thread. Again without locks.

Common to both scenarios is that each service must be reachable via an http endpoint (same address and port).

Non trivial applications fall in one of the two cases above. How can you integrate mongoose in such two scenarios? Wakeup() is just a way to exit the event loop to do something else, like spinning another event loop. But what if another library has the same way of handling interruptions? That is not scalable.

Popular library approaches: consider uwebsocket library: the defer() method allows to execute a lambda in the http server thread. Other libs define an interface to allow file descriptors to be polled from outside. It's a pity that this awesome library does not have any of these two. Such features would make this lib the most complete, extensible and portable I've seen till now in the area of http.

[scaprile]

I beg to differ. The vast majority of our customers are embedded systems and microcontrollers with no OS. Mongoose was never designed aiming to where you want to use it. Mongoose does not depend on other libraries, it must run in baremetal scenarios.
mg_wakeup() is not "a way to exit the event loop to do something else", it is a way to wake up the manager thread when other thread has data for it to deliver right now. Data is sent when the event manager runs; instead of waiting for external data or a timeout, you tell it to run now and hence deliver your data; thing that won't happen if using our lock-free queue.

[ldct-polemos]

Enabling an external loop is just a matter of registering a few callbacks (add_fd, remove_fd..) and maybe a macro to turn the feature on/off to add zero overhead if not used.
That would not break baremetal compatibility.

Wakeup() is good for freetos-like OS but that is suboptimal for any POSIX compliant platforms which mongoose do support. (I even saw a macro enabling epoll() for Linux). That's my point: support is there for advance concepts like poll/select/epoll but then the lib forces the user to spawn a dedicated thread (consuming memory and forcing context switches) to make things works. File descriptors could be polled/selected/epolled by a thread that runs mongoose and other N things. And everything would stay In a single thread; writing code would be faster and resource usage lower and bare metal people still happy.

Anyways.

[scaprile]

@cpq WDYT ?

[ldct-polemos]

Hi @cpq,
about resolution, I think that's up to the caller, the faster the external poll the better.
About the API for external polling, I would do something along these lines (note: not even compiled), assuming MG_EXT_POLL enables the external polling behavior:

/*
.h
*/

#ifdef MG_EXT_POLL
//external polling typedefs
typedef bool (*mg_add_fd)(int fd);
typedef bool (*mg_rm_fd)(int fd);
typedef bool (*mg_is_readable)(int fd);
typedef bool (*mg_is_writeable)(int fd);
typedef bool (*mg_has_errors)(int fd);
#endif

struct mg_mgr {
  struct mg_connection *conns;  // List of active connections
  struct mg_dns dns4;           // DNS for IPv4
  struct mg_dns dns6;           // DNS for IPv6
  int dnstimeout;               // DNS resolve timeout in milliseconds
  bool use_dns6;                // Use DNS6 server by default, see #1532
  unsigned long nextid;         // Next connection ID
  unsigned long timerid;        // Next timer ID
  void *userdata;               // Arbitrary user data pointer
  void *tls_ctx;                // TLS context shared by all TLS sessions
  uint16_t mqtt_id;             // MQTT IDs for pub/sub
  void *active_dns_requests;    // DNS requests in progress
  struct mg_timer *timers;      // Active timers
  int epoll_fd;                 // Used when MG_EPOLL_ENABLE=1
  void *priv;                   // Used by the MIP stack
  size_t extraconnsize;         // Used by the MIP stack
  MG_SOCKET_TYPE pipe;          // Socketpair end for mg_wakeup()
#if MG_ENABLE_FREERTOS_TCP
  SocketSet_t ss;  // NOTE(lsm): referenced from socket struct
#endif

#ifdef MG_EXT_POLL
//external polling callbacks
  mg_add_fd 	        cb_add_fd;
  mg_rm_fd	        cb_rm_fd;
  mg_is_readable	cb_is_readable;
  mg_is_writeable	cb_is_writeable;
  mg_has_errors	cb_has_errors;
#endif
};

And in the source file(s) I would append, after any occurrence of macro MG_EPOLL_ADD, the following:

/*
.c
*/
#if MG_EXT_POLL
      	if(!(*c->mgr->cb_add_fd)(fd)){
	    MG_ERROR(("cb_add_fd() failed"));	
	    //success = false;  OR return NULL; depending on the context
      	}
#endif      	
      }

The callbacks may have some default values that just log "function cb_add_fd cannot be NULL" or similar.

The mg_iotest() would be slightly changed as follows:

static void mg_iotest(struct mg_mgr *mgr, int ms) {
#if MG_ENABLE_FREERTOS_TCP
  ...
#elif MG_ENABLE_EPOLL
  ...
#elif MG_ENABLE_POLL
  ...
#elif MG_EXT_POLL
  (void)ms; //not used
  nfds_t n = 0;
  for (struct mg_connection *c = mgr->conns; c != NULL; c = c->next) {
    if (skip_iotest(c)) {
      // Socket not valid, ignore
    } else {
      bool readable = mgr->cb_is_readable(FD(c));
      bool writeable = mgr->cb_is_writeable(FD(c));
      bool has_errors = mgr->cb_has_errors(FD(c));
      if (has_errors) {
        mg_error(c, "socket error");
      } else {

        c->is_readable = readable;
        c->is_writable = writeable;
        if (c->rtls.len > 0 || mg_tls_pending(c) > 0) c->is_readable = 1;
      }
      n++;
    }
  }
#else
  ...
#endif
}

About cleaning up:

//closesocket
#ifndef closesocket

#if MG_EXT_POLL
#define closesocket(x) \
  do {\
    close(x);\
    if(!(c->mgr->cb_rm_fd)(x)){\
        MG_ERROR(("cb_rm_fd() failed"));\
    }\
  } while (0)

#else
#define closesocket(x) close(x)  

#endif

What do you think?

[cpq]

@ldct-polemos That looks quite complex to me.

If we take a look at mg_mgr_poll(), then it clearly has two parts:
a) multiplexing call, mg_iotest()
b) post-processing.

Here's the first part:

mongoose/src/sock.c

Lines 688 to 692 in 6a7129d

	void mg_mgr_poll(struct mg_mgr *mgr, int ms) {
	struct mg_connection *c, *tmp;
	uint64_t now;
	mg_iotest(mgr, ms);

So what we can do is to refactor the 2nd part of mg_mgr_poll() into a separate function, like mg_io(). Then the mg_mgr_poll() would look like this:

void mg_mgr_poll(struct mg_mgr *mgr, int ms) {
   mg_iotest(mgr, ms);  // Set c->is_readable and c->is_writable for all connections
   mg_io(mgr);   // Do IO, call handlers, etc.
}

Now, the external mutiplexing can easily take over Mongoose descriptors by implementing it's own mg_iotest. All connections and their descriptors are wide open, so we don't even need to invent any API. Here's how an external IO loop may look like:

   struct pollfd *fds = ....;

   add_my_descriptors(...);
   add_mongoose_descriptors(...);

   poll(fds, num_descriptors, timeout_ms);

   process_my_descriptors(...);
   
   // This is kind of mg_iotest
   for (struct mg_connection *c = mgr->conns; c != NULL; c = c->next) {
     // .. set c->is_readable and c->is_writable for all mongoose connections
   }
   mg_io();  // Process Mongoose descriptors

Said that, all we need to do, is to split mg_mgr_poll() in two, and make mg_io() public.

[ldct-polemos]

@cpq
I have a few observations:

1. A formal one: exposing mg_connection breaks encapsulation. The portion of code managing multiplexing should not be exposed to a container like that one, but only to stuff it really cares about (file descriptors). If one day mg_connection changes, the multiplexing logic should remain the same.
2. A performance one: After mg_io() there will be a O(N^2) loop not mentioned in your sample code to reconcile the external struct pollfd *fds array - or whatever container the external logic uses - with the mg_connection list's file descriptors. This is mandatory since lines like this remove a connection element from the list, so the post-mg_io() would need to scan all over struct pollfd *fds and for each element scan the mgr->conns to see if that has been removed. Having callbacks fired by mg_io() would instead give the multiplexing logic the possibility to remove the fd on the fly (e.g. EPOLL_CTL_DEL) or simmetrically, in case of new connections, add the corresponding FD on the fly. As an alternative, mongoose should handle pointers to FD but that would be risky and limiting for the multiplexing logic on the user side.

[cpq]

Yeah, both points are valid

However, it is a design decision of mongoose to expose all the guts to the user. So we are not worried about the encapsulation at all

About the performance - we're don't worry about it either. Mongoose is primarily used on embedded systems with moderate number of connections. If one is looking for a solution on a production server with many connections - like, hundreds of thousands, than Mongoose might not be a good choice anyway. We have prod servers running Mongoose code with dozens of thousands of connections with no noticeable performance issues. Usually, other things are the bottleneck, not the polling call.

[ldct-polemos]

@cpq ok got it. About the defer() mentioned above, I would say once external polling is in place it is quite easy to create C++ wrapper around it, living in the same thread, to honor requests even from other threads.

So is this feature approved? How does it work?

[scaprile]

#2939