Common Profiles

There are several different approaches for throughput testing and followings are a couple of possible sectors (or profiles) I can think of for throughput testing. (It would be a little bit tricky to draw a line between profile (A) and (B) because it would be difficult to achieve very high data rate without meeting a certain degree of Real Time Requirement. So Very High Throughput would imply a certain degree of Real Time Requirement as well).

The profiles matter because each one stresses a different part of the device. So a result in one profile does not prove the result in another. The sections below place the profiles on two axes, look at the max throughput test, and then look at the low rate real time case.

What are the four test profiles ?

Let's first fix what the letters mean, because the rest of the page refers to them. The drawing below sorts throughput testing by two requirements, the data rate and the real time requirement. Each requirement is either low or high, so four profiles result.

Throughput test profiles by data rate and real time requirement

Data Rate runs up the vertical axis and Real Time Requirement runs along the horizontal axis. The four boxes are read as follows.

A - High Data Rate, Low RT Requirement : a bulk transfer such as a large file download, where the total rate matters and the delay of a single packet does not.
B - High Data Rate, High RT Requirement : a very high rate that also has to arrive on time. The max throughput test sits here, because a very high rate needs a steady flow of packets.
C - Low Data Rate, Low RT Requirement : browsing, e-mail and similar applications with small amounts of data and no strict delay target.
D - Low Data Rate, High RT Requirement : VoLTE, on-line gaming and other applications that send small packets with a strict delay target.

The QoS classes in 23.203 Table 6.1.7-A follow the same split. QCI 1, Conversational Voice, is a GBR class with a Packet Delay Budget of 100 ms. QCI 3, Real Time Gaming, is a GBR class with 50 ms. QCI 9, used for TCP-based services such as www and e-mail, is a Non-GBR class with 300 ms. So profile D maps to the GBR classes with short delay budgets, and profile C maps to the Non-GBR default class.

The two axes are independent : a low data rate does not mean a low real time requirement.
Profile D is the one a max throughput test does not reach : it needs guaranteed delay, not a high peak.
QCI gives each profile a number : GBR and the Packet Delay Budget turn the real time requirement into a test target.

Why test the max throughput ?

Now let's look at profile B, the one that most device tests have focused on. The question is simple. If no live network ever gives one UE the full peak, why test for it at all?

If you think about the time when HSDPA came out first and then evolve to higher and higher categories and think about the time when LTE device came out first. Almost everyone's interest have been focused on "Can the device achieve the maximum throughput specified in 3GPP spec ?"

On the other hand, some people say "Why we care about testing the max throughput ? In live air, there wouldn't be any condition that allows max throughput anyway. For example, if you have the LTE Cat 3 device, your max downlink throughput is around 100 Mbps. Would live network ever allocate this amount of resources for a single UE ? or can we have such good signal quality that can allows this kind of throughput ?". The answer would be "Probably Not". But this kind of testing would still be very meaningful not only for marketing reason but also for technical reason. The condition required to achieve the max throughput would be one of the most stressful condition for almost all the components which is in the ways of data traffic. So achieving a max throughput can be a very good criteria of the whole device functioning under a very stressful (fully loaded) condition and also be a good validation criteria for chipset's encoding/decoding process, L2, L3 and various device drivers.

Concretely, a max throughput test loads every block in the data path at once. The PHY decodes the largest TBS in every TTI, and HARQ has to keep up without a gap. L2 has to reorder and reassemble at the peak rate, and the drivers and the host interface carry the same rate. A weak point in any one of them shows up as a lower rate or as packet loss. The Throughput Overview page lists those nodes in order.

The max rate is rarely used in the field : but it is the most complete stress test of the data path.
A pass validates PHY, L2, L3 and drivers together : all of them run at their limit in the same test.

Does a max throughput pass cover the other profiles ?

A device that passes profile B has shown that it can carry a high rate. But profile D asks for something else, which is a guaranteed rate delivered on time. So the question is which of the other profiles the max throughput result also covers.

Let's assume that my device has passed the the max data throughput test (profile : B), does it automatic indicator saying "It would pass all the low throughput profile, like profile (C) or (D) ?". Probably "Yes, for profile (C)", but "No, for profile (D)". Until recently (as of Apr 2013), most of low throughput application would belong to profile (C), like browsing, email etc. But as LTE come out and become more widespread, we foresee more real time application based on data traffic which would belong to profile (D), e.g, Voice over LTE, on-line Gaming etc. In this case, just passing the max throughput would not guarantee the pass in this area. The key requirement on this category would be "It should 'guarantee the required Bit Rate (GBR)' and it should be delivered 'On-Time' (Real Time)'. Even though we can think of a couple of check points for this feature, there are not much practically known about implementing/testing this feature yet. I will get this updated as I get more hands-on experience/information on this. (On radio side, a lot of effort should be invested on optimizing the low layer scheduling parameters, e.g, Semi-persistant scheduling, some RLC parameters and optimal resource allocation based on Buffer Status Report etc).

Let's take VoLTE as the example for profile D. The voice codec produces one frame every 20 ms, so the eNB usually serves it with semi-persistent scheduling at a 20 ms interval. The bearer is QCI 1, with a 100 ms Packet Delay Budget and a Packet Error Loss Rate of 10^-2 (23.203 Table 6.1.7-A). A test for this profile therefore measures delay, jitter and packet loss at a low and steady rate. A peak rate figure says nothing about any of them.

Other settings also matter here and do not matter in profile B. For example, DRX saves UE power but adds delay, and TTI bundling helps the UL at the cell edge. The RLC mode matters too. VoLTE usually runs in RLC UM, where a lost packet is not retransmitted at RLC.

A max throughput pass usually covers profile C : low rate traffic without a delay target uses the same data path at a lower load.
It does not cover profile D : a GBR bearer with a short delay budget needs its own delay, jitter and loss test.
Scheduling settings decide profile D : SPS, DRX, TTI bundling and the RLC mode affect delay more than peak rate does.

Reference

3GPP TS 23.203 v20.0.0 - Table 6.1.7-A, standardized QCI characteristics