Updated: 2022/Sep/29

Please read Privacy Policy. It's for your privacy.


ALTQ(9)                    Kernel Developer's Manual                   ALTQ(9)

NAME
     ALTQ - kernel interfaces for manipulating output queues on network
     interfaces

SYNOPSIS
     #include <sys/types.h>
     #include <sys/socket.h>
     #include <net/if.h>

     void
     IFQ_ENQUEUE(struct ifaltq *ifq, struct mbuf *m,
         struct altq_pktattr *pattr, int err);

     void
     IFQ_DEQUEUE(struct ifaltq *ifq, struct mbuf *m);

     void
     IFQ_POLL(struct ifaltq *ifq, struct mbuf *m);

     void
     IFQ_PURGE(struct ifaltq *ifq);

     void
     IFQ_CLASSIFY(struct ifaltq *ifq, struct mbuf *m, int af,
         struct altq_pktattr *pattr);

     void
     IFQ_IS_EMPTY(struct ifaltq *ifq);

     void
     IFQ_SET_MAXLEN(struct ifaltq *ifq, int len);

     void
     IFQ_INC_LEN(struct ifaltq *ifq);

     void
     IFQ_DEC_LEN(struct ifaltq *ifq);

     void
     IFQ_INC_DROPS(struct ifaltq *ifq);

     void
     IFQ_SET_READY(struct ifaltq *ifq);

DESCRIPTION
     The ALTQ system is a framework to manage queueing disciplines on network
     interfaces.  ALTQ introduces new macros to manipulate output queues.  The
     output queue macros are used to abstract queue operations and not to
     touch the internal fields of the output queue structure.  The macros are
     independent from the ALTQ implementation, and compatible with the
     traditional ifqueue macros for ease of transition.

     IFQ_ENQUEUE() enqueues a packet m to the queue ifq.  The underlying
     queueing discipline may discard the packet.  err is set to 0 on success,
     or ENOBUFS if the packet is discarded.  m will be freed by the device
     driver on success or by the queueing discipline on failure, so that the
     caller should not touch m after calling IFQ_ENQUEUE().

     IFQ_DEQUEUE() dequeues a packet from the queue.  The dequeued packet is
     returned in m, or m is set to NULL if no packet is dequeued.  The caller
     must always check m since a non-empty queue could return NULL under rate-
     limiting.

     IFQ_POLL() returns the next packet without removing it from the queue.
     It is guaranteed by the underlying queueing discipline that IFQ_DEQUEUE()
     immediately after IFQ_POLL() returns the same packet.

     IFQ_PURGE() discards all the packets in the queue.  The purge operation
     is needed since a non-work conserving queue cannot be emptied by a
     dequeue loop.

     IFQ_CLASSIFY() classifies a packet to a scheduling class, and returns the
     result in pattr.

     IFQ_IS_EMPTY() can be used to check if the queue is empty.  Note that
     IFQ_DEQUEUE() could still return NULL if the queueing discipline is non-
     work conserving.

     IFQ_SET_MAXLEN() sets the queue length limit to the default FIFO queue.

     IFQ_INC_LEN() and IFQ_DEC_LEN() increment or decrement the current queue
     length in packets.

     IFQ_INC_DROPS() increments the drop counter and is equal to IF_DROP().
     It is defined for naming consistency.

     IFQ_SET_READY() sets a flag to indicate this driver is converted to use
     the new macros.  ALTQ can be enabled only on interfaces with this flag.

COMPATIBILITY
   ifaltq structure
     In order to keep compatibility with the existing code, the new output
     queue structure ifaltq has the same fields.  The traditional IF_XXX()
     macros and the code directly referencing the fields within if_snd still
     work with ifaltq.  (Once we finish conversions of all the drivers, we no
     longer need these fields.)

                 ##old-style##                           ##new-style##
                                            |
      struct ifqueue {                      | struct ifaltq {
         struct mbuf *ifq_head;             |    struct mbuf *ifq_head;
         struct mbuf *ifq_tail;             |    struct mbuf *ifq_tail;
         int          ifq_len;              |    int          ifq_len;
         int          ifq_maxlen;           |    int          ifq_maxlen;
         uint64_t     ifq_drops;            |    uint64_t     ifq_drops;
      };                                    |    /* altq related fields */
                                            |    ......
                                            | };
                                            |
     The new structure replaces struct ifqueue in struct ifnet.

                 ##old-style##                           ##new-style##
                                            |
      struct ifnet {                        | struct ifnet {
          ....                              |     ....
                                            |
          struct ifqueue if_snd;            |     struct ifaltq if_snd;
                                            |
          ....                              |     ....
      };                                    | };
                                            |
     The (simplified) new IFQ_XXX() macros looks like:

             #ifdef ALTQ
             #define IFQ_DEQUEUE(ifq, m)                     \
                     if (ALTQ_IS_ENABLED((ifq))              \
                             ALTQ_DEQUEUE((ifq), (m));       \
                     else                                    \
                             IF_DEQUEUE((ifq), (m));
             #else
             #define IFQ_DEQUEUE(ifq, m)     IF_DEQUEUE((ifq), (m));
             #endif

   Enqueue operation
     The semantics of the enqueue operation are changed.  In the new style,
     enqueue and packet drop are combined since they cannot be easily
     separated in many queueing disciplines.  The new enqueue operation
     corresponds to the following macro that is written with the old macros.

     #define IFQ_ENQUEUE(ifq, m, pattr, err)                   \
     do {                                                      \
             if (ALTQ_IS_ENABLED((ifq)))                       \
                     ALTQ_ENQUEUE((ifq), (m), (pattr), (err)); \
             else {                                            \
                     if (IF_QFULL((ifq))) {                    \
                             m_freem((m));                     \
                             (err) = ENOBUFS;                  \
                     } else {                                  \
                             IF_ENQUEUE((ifq), (m));           \
                             (err) = 0;                        \
                     }                                         \
             }                                                 \
             if ((err))                                        \
                     (ifq)->ifq_drops++;                       \
     } while (false)

     IFQ_ENQUEUE() does the following:
     -   queue a packet
     -   drop (and free) a packet if the enqueue operation fails
     If the enqueue operation fails, err is set to ENOBUFS.  m is freed by the
     queueing discipline.  The caller should not touch mbuf after calling
     IFQ_ENQUEUE() so that the caller may need to copy m_pkthdr.len or m_flags
     field beforehand for statistics.  The caller should not use senderr()
     since mbuf was already freed.

     The new style if_output() looks as follows:

                 ##old-style##                           ##new-style##
                                            |
      int                                   | int
      ether_output(ifp, m0, dst, rt0)       | ether_output(ifp, m0, dst, rt0)
      {                                     | {
          ......                            |     ......
                                            |
                                            |     mflags = m->m_flags;
                                            |     len = m->m_pkthdr.len;
          s = splimp();                     |     s = splimp();
          if (IF_QFULL(&ifp->if_snd)) {     |     IFQ_ENQUEUE(&ifp->if_snd, m,
                                            |                 NULL, error);
              IF_DROP(&ifp->if_snd);        |     if (error != 0) {
              splx(s);                      |         splx(s);
              senderr(ENOBUFS);             |         return (error);
          }                                 |     }
          IF_ENQUEUE(&ifp->if_snd, m);      |
          ifp->if_obytes +=                 |     ifp->if_obytes += len;
                         m->m_pkthdr.len;   |
          if (m->m_flags & M_MCAST)         |     if (mflags & M_MCAST)
              ifp->if_omcasts++;            |         ifp->if_omcasts++;
                                            |
          if ((ifp->if_flags & IFF_OACTIVE) |     if ((ifp->if_flags & IFF_OACTIVE)
              == 0)                         |         == 0)
              (*ifp->if_start)(ifp);        |         (*ifp->if_start)(ifp);
          splx(s);                          |     splx(s);
          return (error);                   |     return (error);
                                            |
      bad:                                  | bad:
          if (m)                            |     if (m)
              m_freem(m);                   |         m_freem(m);
          return (error);                   |     return (error);
      }                                     | }
                                            |

   Classifier
     The classifier mechanism is currently implemented in if_output().  struct
     altq_pktattr is used to store the classifier result, and it is passed to
     the enqueue function.  (We will change the method to tag the classifier
     result to mbuf in the future.)

     int
     ether_output(ifp, m0, dst, rt0)
     {
             ......
             struct altq_pktattr pktattr;

             ......

             /* classify the packet before prepending link-headers */
             IFQ_CLASSIFY(&ifp->if_snd, m, dst->sa_family, &pktattr);

             /* prepend link-level headers */
             ......

             IFQ_ENQUEUE(&ifp->if_snd, m, &pktattr, error);

             ......
     }

HOW TO CONVERT THE EXISTING DRIVERS
     First, make sure the corresponding if_output() is already converted to
     the new style.

     Look for if_snd in the driver.  You will probably need to make changes to
     the lines that include if_snd.

   Empty check operation
     If the code checks ifq_head to see whether the queue is empty or not, use
     IFQ_IS_EMPTY().

                 ##old-style##                           ##new-style##
                                            |
      if (ifp->if_snd.ifq_head != NULL)     | if (IFQ_IS_EMPTY(&ifp->if_snd) == 0)
                                            |
     Note that IFQ_POLL() can be used for the same purpose, but IFQ_POLL()
     could be costly for a complex scheduling algorithm since IFQ_POLL() needs
     to run the scheduling algorithm to select the next packet.  On the other
     hand, IFQ_IS_EMPTY() checks only if there is any packet stored in the
     queue.  Another difference is that even when IFQ_IS_EMPTY() is false,
     IFQ_DEQUEUE() could still return NULL if the queue is under rate-
     limiting.

   Dequeue operation
     Replace IF_DEQUEUE() by IFQ_DEQUEUE().  Always check whether the dequeued
     mbuf is NULL or not.  Note that even when IFQ_IS_EMPTY() is false,
     IFQ_DEQUEUE() could return NULL due to rate-limiting.

                 ##old-style##                           ##new-style##
                                            |
      IF_DEQUEUE(&ifp->if_snd, m);          | IFQ_DEQUEUE(&ifp->if_snd, m);
                                            | if (m == NULL)
                                            |     return;
                                            |
     A driver is supposed to call if_start() from transmission complete
     interrupts in order to trigger the next dequeue.

   Poll-and-dequeue operation
     If the code polls the packet at the head of the queue and actually uses
     the packet before dequeueing it, use IFQ_POLL() and IFQ_DEQUEUE().

                 ##old-style##                           ##new-style##
                                            |
      m = ifp->if_snd.ifq_head;             | IFQ_POLL(&ifp->if_snd, m);
      if (m != NULL) {                      | if (m != NULL) {
                                            |
          /* use m to get resources */      |     /* use m to get resources */
          if (something goes wrong)         |     if (something goes wrong)
              return;                       |         return;
                                            |
          IF_DEQUEUE(&ifp->if_snd, m);      |     IFQ_DEQUEUE(&ifp->if_snd, m);
                                            |
          /* kick the hardware */           |     /* kick the hardware */
      }                                     | }
                                            |
     It is guaranteed that IFQ_DEQUEUE() immediately after IFQ_POLL() returns
     the same packet.  Note that they need to be guarded by splimp() if called
     from outside of if_start().

   Eliminating IF_PREPEND
     If the code uses IF_PREPEND(), you have to eliminate it since the prepend
     operation is not possible for many queueing disciplines.  A common use of
     IF_PREPEND() is to cancel the previous dequeue operation.  You have to
     convert the logic into poll-and-dequeue.

                 ##old-style##                           ##new-style##
                                            |
      IF_DEQUEUE(&ifp->if_snd, m);          | IFQ_POLL(&ifp->if_snd, m);
      if (m != NULL) {                      | if (m != NULL) {
                                            |
          if (something_goes_wrong) {       |     if (something_goes_wrong) {
              IF_PREPEND(&ifp->if_snd, m);  |
              return;                       |         return;
          }                                 |     }
                                            |
                                            |     /* at this point, the driver
                                            |      * is committed to send this
                                            |      * packet.
                                            |      */
                                            |     IFQ_DEQUEUE(&ifp->if_snd, m);
                                            |
          /* kick the hardware */           |     /* kick the hardware */
      }                                     | }
                                            |

   Purge operation
     Use IFQ_PURGE() to empty the queue.  Note that a non-work conserving
     queue cannot be emptied by a dequeue loop.

                 ##old-style##                           ##new-style##
                                            |
      while (ifp->if_snd.ifq_head != NULL) {|  IFQ_PURGE(&ifp->if_snd);
          IF_DEQUEUE(&ifp->if_snd, m);      |
          m_freem(m);                       |
      }                                     |
                                            |

   Attach routine
     Use IFQ_SET_MAXLEN() to set ifq_maxlen to len.  Add IFQ_SET_READY() to
     show this driver is converted to the new style.  (This is used to
     distinguish new-style drivers.)

                 ##old-style##                           ##new-style##
                                            |
      ifp->if_snd.ifq_maxlen = qsize;       | IFQ_SET_MAXLEN(&ifp->if_snd, qsize);
                                            | IFQ_SET_READY(&ifp->if_snd);
      if_attach(ifp);                       | if_attach(ifp);
                                            |

   Other issues
     The new macros for statistics:

                 ##old-style##                           ##new-style##
                                            |
      IF_DROP(&ifp->if_snd);                | IFQ_INC_DROPS(&ifp->if_snd);
                                            |
      ifp->if_snd.ifq_len++;                | IFQ_INC_LEN(&ifp->if_snd);
                                            |
      ifp->if_snd.ifq_len--;                | IFQ_DEC_LEN(&ifp->if_snd);
                                            |
     Some drivers instruct the hardware to invoke transmission complete
     interrupts only when it thinks necessary.  Rate-limiting breaks its
     assumption.

   How to convert drivers using multiple ifqueues
     Some (pseudo) devices (such as slip) have another ifqueue to prioritize
     packets.  It is possible to eliminate the second queue since ALTQ
     provides more flexible mechanisms but the following shows how to keep the
     original behavior.

     struct sl_softc {
             struct  ifnet sc_if;            /* network-visible interface */
             ...
             struct  ifqueue sc_fastq;       /* interactive output queue */
             ...
     };
     The driver doesn't compile in the new model since it has the following
     line (if_snd is no longer a type of struct ifqueue).

             struct ifqueue *ifq = &ifp->if_snd;
     A simple way is to use the original IF_XXX() macros for sc_fastq and use
     the new IFQ_XXX() macros for if_snd.  The enqueue operation looks like:

                 ##old-style##                           ##new-style##
                                            |
      struct ifqueue *ifq = &ifp->if_snd;   | struct ifqueue *ifq = NULL;
                                            |
      if (ip->ip_tos & IPTOS_LOWDELAY)      | if ((ip->ip_tos & IPTOS_LOWDELAY) &&
          ifq = &sc->sc_fastq;              | !ALTQ_IS_ENABLED(&sc->sc_if.if_snd)) {
                                            |     ifq = &sc->sc_fastq;
      if (IF_QFULL(ifq)) {                  |     if (IF_QFULL(ifq)) {
          IF_DROP(ifq);                     |         IF_DROP(ifq);
          m_freem(m);                       |         m_freem(m);
          splx(s);                          |         error = ENOBUFS;
          sc->sc_if.if_oerrors++;           |     } else {
          return (ENOBUFS);                 |         IF_ENQUEUE(ifq, m);
      }                                     |         error = 0;
      IF_ENQUEUE(ifq, m);                   |     }
                                            | } else
                                            |     IFQ_ENQUEUE(&sc->sc_if.if_snd,
                                            |                 m, NULL, error);
                                            |
                                            | if (error) {
                                            |     splx(s);
                                            |     sc->sc_if.if_oerrors++;
                                            |     return (error);
                                            | }
      if ((sc->sc_oqlen =                   | if ((sc->sc_oqlen =
           sc->sc_ttyp->t_outq.c_cc) == 0)  |      sc->sc_ttyp->t_outq.c_cc) == 0)
          slstart(sc->sc_ttyp);             |     slstart(sc->sc_ttyp);
      splx(s);                              | splx(s);
                                            |
     The dequeue operations looks like:

                 ##old-style##                           ##new-style##
                                            |
      s = splimp();                         | s = splimp();
      IF_DEQUEUE(&sc->sc_fastq, m);         | IF_DEQUEUE(&sc->sc_fastq, m);
      if (m == NULL)                        | if (m == NULL)
          IF_DEQUEUE(&sc->sc_if.if_snd, m); |     IFQ_DEQUEUE(&sc->sc_if.if_snd, m);
      splx(s);                              | splx(s);
                                            |

QUEUEING DISCIPLINES
     Queueing disciplines need to maintain ifq_len (used by IFQ_IS_EMPTY()).
     Queueing disciplines also need to guarantee the same mbuf is returned if
     IFQ_DEQUEUE() is called immediately after IFQ_POLL().

SEE ALSO
     pf(4), altq.conf(5), pf.conf(5), altqd(8), tbrconfig(8)

HISTORY
     The ALTQ system first appeared in March 1997 and found its home in the
     KAME project (http://www.kame.net). It was imported into NetBSD 1.6.

NetBSD 10.99                   October 24, 2022                   NetBSD 10.99