SCTP stands for Stream Control Transmission Protocol. As its name implies, it is a Transport Protocol like TCP. It is similar to TCP in terms of fundamentals but added some features which would compensate the weak spot of TCP.  Watch videos in Reference Section if you want to learn further details on SCTP protocol in general.

The short version of what makes it different is two words. Streams and paths. TCP gives you one ordered byte stream over one path. SCTP gives you many ordered message streams over possibly several paths, all inside a single connection. Most of what follows is a consequence of that.

There is a third difference that catches people out in a packet capture. SCTP is message oriented rather than byte oriented. What the sender handed down is what the receiver gets, with the boundaries still intact. TCP has no idea where one of your messages ends and the next begins. That is why almost every protocol running over TCP has to carry its own length field.

One piece of vocabulary before anything else, because it shows up everywhere including in Wireshark. SCTP does not call it a connection. It calls it an association, and the word is chosen deliberately, since an association can involve more than one IP address at each end. I will use that word for the rest of this note.

• SCTP in 4G/5G Network Architecture
• Why SCTP and not TCP ?
• Streams and Head of Line Blocking
• Multi Homing
• Packet Structure
• SCTP Establish
• Association Shutdown

SCTP in 4G/5G Network Architecture

This is not a choice left to implementers. Both S1-AP and NGAP name SCTP as their transport. Look at signalling between a base station and a core network node and you are looking at SCTP, whether you meant to or not.

The three figures below work from the outside in. Where these interfaces sit in the architecture, then how the protocol stack is layered on them, then what a single packet looks like on the wire.

In cellular communication system, SCTP is the protocol being used in the communication between eNB and MME(4G) and the protocol being used between gNB and AMF as illustrated below.

The yellow rings in the figure mark where that transport sits. On the 4G side they are on S1, between eNB and MME/S-GW. On the 5G side they are on NG, between gNB and AMF/UPF. The inter base station interfaces, X2 in 4G and Xn in 5G, run over SCTP as well, though the figure does not ring them.

Structure of the protocol stack for these interface is illustrated as below.

Two stacks side by side, 4G on the left and 5G on the right, and only the top box is different. S1-AP in one, NGAP in the other, and underneath both of them the same SCTP over IP. The horizontal line through the middle is the boundary 3GPP draws between the radio network layer and the transport network layer. SCTP is the top of the transport side of it.

If you take packet capture (e.g, Wireshark capture) over the interface, you would get the packet structure as below.

The same arrangement again, this time as bytes in the order they appear. IP header, then the SCTP header, then the application protocol, then the payload it is carrying. This is the nesting your display filter is matching against when you type sctp or ngap into Wireshark.

Why SCTP and not TCP ?

This is the question I would want answered first, so let me take it before the packet formats. TCP is the obvious choice for a control plane, it is everywhere, and every operating system has a good implementation of it. So why go to the trouble ?

The answer starts with the shape of the traffic. Signalling between a base station and a core network node is not one long transfer. It is thousands of small, independent transactions belonging to thousands of different subscribers. A registration for one UE has nothing to do with a handover for another. TCP has no way of knowing that, so it treats the whole lot as one ordered byte stream. One lost segment then holds up every subscriber behind it.

The other half of the answer is what happens when a link goes down. A TCP connection is pinned to one address at each end. Lose the path and you lose the connection, and everything above it has to notice and rebuild. For a link carrying the signalling of an entire cell site, that is an expensive way to find out about a cable fault.

Here is how the three transport protocols compare on the points that matter for this kind of traffic.

The three rows highlighted above are the ones that earned SCTP its place in 3GPP. Everything else on the list is a detail. The next two sections take the first two of them one at a time.

Streams and Head of Line Blocking

A stream in SCTP is not a separate connection and it is not a separate socket. It is a numbered lane inside one association, and the only thing a stream number does is define an ordering scope. Messages in the same stream arrive in the order they were sent. Messages in different streams have no relationship at all.

That sounds like a small thing until you see what it prevents.

In the upper half, one segment goes missing and packets 4, 5 and 6 arrive perfectly intact. The receiver cannot hand any of them up, because TCP promised to deliver bytes in order and it has a gap. Everything sits in a buffer until the retransmission of 3 arrives. That wait is head of line blocking. On a busy signalling link it means one dropped packet delays messages belonging to subscribers who have nothing to do with each other.

In the lower half the same loss happens, and the damage is contained. Stream 1 stalls, because ordering within a stream still has to be honoured. Streams 0 and 2 carry on as if nothing happened. That is the whole point of the design.

The number of streams is not fixed by the standard. Each side announces how many it wants at setup, and the smaller of the two offers wins. You can watch that negotiation happen in the capture further down this page.

Multi Homing

The second feature is easy enough to explain, and in my experience a good deal harder to actually get deployed. An SCTP endpoint can own more than one IP address, and it tells the other side about all of them during setup. The association is then between two sets of addresses rather than between two addresses.

One of the destination addresses is the primary, and it carries the traffic. The others sit idle. They are not forgotten though. SCTP sends HEARTBEAT chunks to every destination address it knows about. It always has a current opinion on which paths are usable. If the primary stops answering, the sender switches to one that is.

This is the part that separates SCTP from a TCP connection with a failover script bolted on. The association survives the path failure. There is no reconnect, no new handshake, and the application on top is never told, because from its point of view nothing broke.

It is worth saying that a lot of real deployments never use this. Multi homing needs genuinely independent paths to be worth anything, and if both addresses ride the same fibre you have gained very little. The capture on this page is a single homed setup, and you can tell from the INIT, which offers exactly one address family and no address list.

Packet Structure

An SCTP packet has two parts, and I found the rest of it much easier to read once I started keeping them apart. There is a twelve byte common header that appears exactly once, and after it one or more chunks.

The chunk is the unit that does all the work. Every kind of message SCTP has, from the setup handshake to your actual payload to the keepalives, is a chunk. They all share the same three field wrapper, and several unrelated ones can travel in the same packet. That last point is worth holding on to, because it is why a single SCTP packet can be doing more than one thing at once.

The figure below lays it out against a 32 bit ruler. IP header on top in white, SCTP common header in yellow, chunks in orange. Notice the vertical repetition at the bottom. Chunk 1 through Chunk N are all in the same packet.

The yellow band is the whole of the common header, and there is less to it than a TCP header. Four fields, and no sequence number, no window, no flags.

The verification tag is the field I would look at first when something odd is happening. It is how SCTP tells this association apart from a stale one that used the same four addresses and ports. It is also why a restarted peer cannot be mistaken for the old one.

Below the common header, every chunk is a plain type length value construction. One byte of type, one byte of flags whose meaning depends on the type, two bytes of length, then the content. Chunks are padded to a four byte boundary, and the length field does not count the padding. That detail catches people writing their own parsers.

Following is an example of IP header + SCTP packet.

The first screenshot is the IP header, the second is the SCTP packet sitting inside it. They come from a loopback capture rather than from a live interface, so a couple of the values look wrong at first glance and are not.

Only one line in that IP header really matters here, and it is Protocol: SCTP (132). SCTP is its own IP protocol number and does not sit on top of TCP or UDP. This is also why SCTP has such a hard time crossing consumer grade middleboxes, which tend to understand 6 and 17 and nothing else.

The green bracket on the right is the SCTP packet and the red box inside it is the chunk data of a single INIT chunk. Reading down from the top, the common header is the first six lines, and then everything from Chunk type onward belongs to the chunk.

• Verification tag: 0x00000000 - this is the one packet in the life of an association where that field is zero. The sender has not been told a tag yet, because it is asking to start.
• Checksum: 0x00000000 [unverified] - a zero checksum normally means a corrupt packet. Here it means the capture was taken before the checksum was filled in, which is what happens on loopback and on any NIC doing checksum offload. Wireshark is honest about it and says unverified.
• Outbound streams: 30, inbound streams: 65535 - what this side is offering. 65535 inbound is the way of saying it will accept anything the peer wants.
• Supported types: IPv6, IPv4 - the address families this endpoint is willing to use. Compare this with the capture further down the page, which offers IPv4 only.
• Dst Port: 3868 - worth noticing that this example is not NGAP at all. 3868 is Diameter. The SCTP layer looks identical whatever is riding on it, which is rather the point of a transport protocol.

The lines that read Bit: Skip parameter and continue processing of the chunk are not decoration. They are Wireshark spelling out the two most significant bits of the parameter type, and the same rule applies to chunk types. It is worth understanding before reading the table that follows.

Chunk Type

Followings are the list of values used as Chunk Type field.

Before the list itself, there is a rule hidden in the numbering that explains why the values are grouped the way they are. The two most significant bits of the chunk type are not part of the identity of the chunk. They are an instruction to a receiver that does not recognise it.

This is why the extensions added years after the original protocol live at the top of the range. FORWARD-TSN is 192 and ASCONF is 193, both in the skip and carry on band. An old implementation meeting one of them ignores it, and the association keeps working. Put the same extension at 20 and every old peer would drop the entire packet.

The same two bit rule governs parameter types inside a chunk, which is where those Wireshark lines in the screenshot above come from. ECN is 0x8000, so the top bits are 10, and Wireshark prints skip and continue plus do not report. Forward TSN supported is 0xc000, top bits 11, and the second line changes to do report. The decode is reading the bits straight off the type value.

Chunk Flags

Followings are some of the chunk flags. Which of these are used varies depending on Chunk Type.

•   U (Unordered): Indicates that the chunk is to be delivered unordered to the receiving endpoint.
•   B (Begin): Indicates that the chunk is the first of a multi-part message.
•   E (End): Indicates that the chunk is the last of a multi-part message.
•   I (Abort): Indicates that the sender has abandoned the transmission of the current message and is requesting an abort.
•   T (Forward TSN Supported): Indicates support for the "Forward TSN" feature of SCTP.
•   P (Padding): Indicates that the chunk contains padding to align the data on a 4-byte boundary.
•   R (Reserved): Indicates that the flag bit is reserved for future use and must be set to zero.
•   A (ASCONF Ack): Indicates that the chunk is an acknowledgment for an "Address Configuration" (ASCONF) chunk.

SCTP Establish

Like TCP, SCTP also requires the initial handshake to establish the connection between two end points. As shown below, SCTP goes through 4 step handshake whereas TCP goes through 3 way handshake. Following is SCTP initial setup process captured with Wireshark from Amarisoft Callbox.

The extra step is not clumsiness. It buys something specific, and it is easiest to see by asking what a TCP server does when a SYN arrives from a stranger. It allocates a control block and remembers the stranger. Send enough SYNs from addresses that do not exist and the server runs out of memory without you ever completing a single connection. That is a SYN flood, and it works because the server commits resources before the client has proved anything.

SCTP declines to play. When an INIT arrives, the server works out all the state it would need. It packs that into a cookie, signs it, and sends the cookie back inside INIT_ACK. Then it deliberately forgets the whole thing, and is holding nothing at all.

If the client is real it receives the INIT_ACK and returns the cookie in a COOKIE_ECHO. The server checks its own signature, unpacks the state, and only at that moment does an association come into existence. A spoofed INIT costs it one signature and one reply, because the INIT_ACK goes to the address that was forged and the cookie never comes back.

So the four steps are three steps of TCP with a proof of reachability wedged into the middle. The capture below is exactly that sequence.

Four frames, and nothing else between them. The Info column names the chunk in each one, which is the quickest way to confirm a handshake completed rather than stalling somewhere in the middle.

Two things in that summary are worth reading before opening the packets themselves.

• The lengths tell the story on their own - 82, 306, 278, 50. The two large ones are the two carrying the cookie. INIT_ACK is the larger of the pair, because it holds the cookie as well as everything INIT held. COOKIE_ACK at 50 bytes is the smallest thing SCTP sends, since it is a bare chunk header and nothing else.
• The whole exchange takes under five milliseconds - from 17.612827 to 17.617653. Almost all of that is one gap, between INIT_ACK and COOKIE_ECHO. The two server side turnarounds are tens of microseconds each. The server is doing less work than the client here, which is precisely what the cookie design intends.

The four dumps that follow are those same four frames opened up. Rather than reading them straight through, it is worth chasing one field across all four, because the verification tag is where the association actually gets its identity.

Two more things are visible if you go looking for them.

• The cookie really is echoed unchanged - INIT_ACK carries State cookie: 4124569d7e0548bd... and COOKIE_ECHO carries Cookie: 4124569d7e0548bd..., the same 228 bytes. The client does not interpret it. It could not, since only the server can check the signature on it.
• The stream count gets negotiated down - INIT asks for 8 outbound and will accept 65535 inbound. INIT_ACK comes back with 8 outbound and 8 inbound. Each direction settles on the smaller of what was offered and what was requested, and this association ends up with 8 streams each way.

(1) INIT

Internet Protocol Version 4, Src: 10.0.0.185, Dst: 10.0.0.162
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x02 (DSCP: CS0, ECN: ECT(0))
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..10 = Explicit Congestion Notification: ECN-Capable Transport codepoint '10' (2)
    Total Length: 68
    Identification: 0x0000 (0)
    Flags: 0x40, Don't fragment
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment Offset: 0
    Time to Live: 64
    Protocol: SCTP (132)
    Header Checksum: 0x24da [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 10.0.0.185
    Destination Address: 10.0.0.162
Stream Control Transmission Protocol, Src Port: 48041 (48041), Dst Port: 38412 (38412)
    Source port: 48041
    Destination port: 38412
    Verification tag: 0x00000000
    [Association index: 65535]
    Checksum: 0x0ffaab98 [unverified]
    [Checksum Status: Unverified]
    INIT chunk (Outbound streams: 8, inbound streams: 65535)
        Chunk type: INIT (1)
            0... .... = Bit: Stop processing of the packet
            .0.. .... = Bit: Do not report
        Chunk flags: 0x00
        Chunk length: 36
        Initiate tag: 0x8d10a6cf
        Advertised receiver window credit (a_rwnd): 106496
        Number of outbound streams: 8
        Number of inbound streams: 65535
        Initial TSN: 3757580128
        Supported address types parameter (Supported types: IPv4)
            Parameter type: Supported address types (0x000c)
                0... .... .... .... = Bit: Stop processing of chunk
                .0.. .... .... .... = Bit: Do not report
            Parameter length: 6
            Supported address type: IPv4 address (5)
            Parameter padding: 0000
        ECN parameter
            Parameter type: ECN (0x8000)
                1... .... .... .... = Bit: Skip parameter and continue processing of the chunk
                .0.. .... .... .... = Bit: Do not report
            Parameter length: 4
        Forward TSN supported parameter
            Parameter type: Forward TSN supported (0xc000)
                1... .... .... .... = Bit: Skip parameter and continue processing of the chunk
                .1.. .... .... .... = Bit: Do report
            Parameter length: 4

(2) INIT_ACK

Internet Protocol Version 4, Src: 10.0.0.162, Dst: 10.0.0.185
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x02 (DSCP: CS0, ECN: ECT(0))
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..10 = Explicit Congestion Notification: ECN-Capable Transport codepoint '10' (2)
    Total Length: 292
    Identification: 0x0000 (0)
    Flags: 0x40, Don't fragment
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment Offset: 0
    Time to Live: 64
    Protocol: SCTP (132)
    Header Checksum: 0x23fa [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 10.0.0.162
    Destination Address: 10.0.0.185
Stream Control Transmission Protocol, Src Port: 38412 (38412), Dst Port: 48041 (48041)
    Source port: 38412
    Destination port: 48041
    Verification tag: 0x8d10a6cf
    [Association index: 65535]
    Checksum: 0x30678470 [unverified]
    [Checksum Status: Unverified]
    INIT_ACK chunk (Outbound streams: 8, inbound streams: 8)
        Chunk type: INIT_ACK (2)
            0... .... = Bit: Stop processing of the packet
            .0.. .... = Bit: Do not report
        Chunk flags: 0x00
        Chunk length: 260
        Initiate tag: 0x790ad620
        Advertised receiver window credit (a_rwnd): 106496
        Number of outbound streams: 8
        Number of inbound streams: 8
        Initial TSN: 4169966867
        State cookie parameter (Cookie length: 228 bytes)
            Parameter type: State cookie (0x0007)
                0... .... .... .... = Bit: Stop processing of chunk
                .0.. .... .... .... = Bit: Do not report
            Parameter length: 232
            State cookie: 4124569d7e0548bdbfbc4c54e0af0c95e938d9f500000000000000000000000000000000
        ECN parameter
            Parameter type: ECN (0x8000)
                1... .... .... .... = Bit: Skip parameter and continue processing of the chunk
                .0.. .... .... .... = Bit: Do not report
            Parameter length: 4
        Forward TSN supported parameter
            Parameter type: Forward TSN supported (0xc000)
                1... .... .... .... = Bit: Skip parameter and continue processing of the chunk
                .1.. .... .... .... = Bit: Do report
            Parameter length: 4

(3) COOKIE_ECHO

Internet Protocol Version 4, Src: 10.0.0.185, Dst: 10.0.0.162
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x02 (DSCP: CS0, ECN: ECT(0))
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..10 = Explicit Congestion Notification: ECN-Capable Transport codepoint '10' (2)
    Total Length: 264
    Identification: 0x0000 (0)
    Flags: 0x40, Don't fragment
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment Offset: 0
    Time to Live: 64
    Protocol: SCTP (132)
    Header Checksum: 0x2416 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 10.0.0.185
    Destination Address: 10.0.0.162
Stream Control Transmission Protocol, Src Port: 48041 (48041), Dst Port: 38412 (38412)
    Source port: 48041
    Destination port: 38412
    Verification tag: 0x790ad620
    [Association index: 65535]
    Checksum: 0xa00fe0f7 [unverified]
    [Checksum Status: Unverified]
    COOKIE_ECHO chunk (Cookie length: 228 bytes)
        Chunk type: COOKIE_ECHO (10)
            0... .... = Bit: Stop processing of the packet
            .0.. .... = Bit: Do not report
        Chunk flags: 0x00
        Chunk length: 232
        Cookie: 4124569d7e0548bdbfbc4c54e0af0c95e938d9f500000000000000000000000000000000

(4) COOKIE_ACK

Internet Protocol Version 4, Src: 10.0.0.162, Dst: 10.0.0.185
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x02 (DSCP: CS0, ECN: ECT(0))
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..10 = Explicit Congestion Notification: ECN-Capable Transport codepoint '10' (2)
    Total Length: 36
    Identification: 0x0000 (0)
    Flags: 0x40, Don't fragment
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment Offset: 0
    Time to Live: 64
    Protocol: SCTP (132)
    Header Checksum: 0x24fa [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 10.0.0.162
    Destination Address: 10.0.0.185
Stream Control Transmission Protocol, Src Port: 38412 (38412), Dst Port: 48041 (48041)
    Source port: 38412
    Destination port: 48041
    Verification tag: 0x8d10a6cf
    [Association index: 65535]
    Checksum: 0xa018cd9c [unverified]
    [Checksum Status: Unverified]
    COOKIE_ACK chunk
        Chunk type: COOKIE_ACK (11)
            0... .... = Bit: Stop processing of the packet
            .0.. .... = Bit: Do not report
        Chunk flags: 0x00
        Chunk length: 4

Association Shutdown

Setting an association up takes four chunks. Taking it down takes three, and there are two quite different ways to do it depending on whether you are finished or something has gone wrong.

The polite version is SHUTDOWN, answered with SHUTDOWN_ACK, confirmed with SHUTDOWN_COMPLETE. The blunt version is a single ABORT chunk, with no reply expected and nothing to wait for.

There is one behaviour here that surprises people arriving from TCP, so it is worth stating plainly. SCTP has no half closed state. In TCP you can send a FIN, stop sending, and go on receiving from the other side for as long as it wants to keep talking. SCTP will not do that. The moment a shutdown starts, neither end accepts new data from its application. What is already queued is still delivered, so nothing in flight is lost. But the association is on its way out, and there is no lingering one way conversation.

In practice you will not see much of either on a working NG or S1 link. The association is meant to stay up for as long as the base station and the core node are both alive, which can be months. What fills the quiet periods instead is HEARTBEAT. If you are staring at a capture wondering why nothing is happening, that is usually what you are looking at.

An ABORT arriving out of nowhere on a link that was working is worth taking seriously. It is not a timeout and it is not congestion. Somebody sent it deliberately, and the chunk normally carries an error cause saying why.

Videos

• SCTP 1 (Oct 2013)
• SCTP 2 (Oct 2013)

Reference

• Securing GTP and SCTP Traffic User Guide for Security Devices - Juniper Networks