Q & A | Generator of Self-Similar Traffic

Q1. Traffic generator v.1 allowed a link rate to be explicitly set in the constructor. However, there is no mechanism to set the link rate in v.2. Why?

A1. Among other modifications, the version 2 was made link rate independent. Rather than using timestamp values, it uses bytestamps. In a bytestamp, one unit represents transmission time of one byte. Thus, transmission time of a 1000-byte long frame is exactly 1000 units, regardless of the link speed. To convert a bytestamp value B to timestamp value T, multiply B by the actual transmission time of one byte. For example, if you have 100 Mbps link and you want time resolution of 1 ns, then T = B * 80 [ns/byte].

Q2. I would like to use your version 3 s-s generator in my research, but I need to modify it slightly. I would like to enumerate packets per burst, i.e. first packet in a burst will have number 0, next packet in the same burst will have number 1, etc. Also, I need to know which packet is the last in particular burst (aka fin flag). I modified the structure of the packet:
struct Packet
{
    source_id_t  SourceId;
    pckt_size_t  PcktSize;
    pause_size_t Interval;
    int PcktNumber;
    bool PcktFlag;   // it should be equal to 1 if it's last  
                     // packet from burst and 0 otherwise.
};
But I cannot find a place where you generate packets from that burst. Could you explain what and where I should modify?


A2. First, we need to refine some definitions. Within the bursts, Ethernet packets are spaced 20 byte times apart to account for IPG and preamble. What spacing should be between packets to consider them belonging to different bursts? Let's assume that BURST_GAP value is defined somewhere. (If BURST_GAP == 1, then packets that are 21 byte times apart are considered to be in different bursts.)

Bursts are aggregated and filled with packets in procedure GetNextPacket(void). To modify packet structure, you would add code shown in blue:
////////////////////////////////////////////////////////////////////////
// FUNCTION:    Packet GetNextPacket(void)
// DESCRIPTION: Return the next packet from the aggregated traffic
// NOTES:
////////////////////////////////////////////////////////////////////////
Packet GetNextPacket(void)
{
    Stream*     pStrm;
    Packet      next_packet = NextPacket;
    pckt_size_t pckt_size   = pfPcktSize();
    bytestamp_t pckt_time   = Elapsed;

    // if the remaining burst size is less than the packet size,
    // aggregate additional bursts 
    while( Tokens < pckt_size && 
         ( pStrm = (Stream*)BusyPool->RemoveHead() ) != NULL ) 
    {
        if( pStrm->GetArrival() > pckt_time + Tokens )
        {
            pckt_time = pStrm->GetArrival() - Tokens;

            // if this is considered a new burst
            if( pckt_time - Elapsed >= BURST_GAP ) 
            {
                // mark previous packet as the last packet in a burst
                next_packet.PcktFlag  = TRUE;

                // next packet will be the first packet in a burst
                NextPacket.PcktNumber = -1;        
            }
        }

        Tokens += pStrm->GetBurstSize();
                
        pStrm->ExtractBurst();        // receive new burst 
        BusyPool->AddNode( pStrm );   // place the stream in BusyPool
    }

    Tokens -= pckt_size;
    pckt_time += pckt_size + MinIFG;  

    // update next packet 
    NextPacket.PcktSize = pckt_size;
    NextPacket.Interval = (int32s)( pckt_time - Elapsed );
    NextPacket.PcktNumber ++;

    Elapsed = pckt_time;
    return next_packet;
}
Also, add the following code to the PacketGenerator constructor:
NextPacket.PcktFlag  = FALSE;

Q3. The actual load produced by the traffic generator does not match the specified target load. Why?

A3. Individual streams generate on/off periods having heavy-tailed distribution. The heavy-tailness means that extremely large on or off periods will be present from time to time. If the number of packets is not sufficiently large, it could happen that the observed packet stream contained very large burst, but did not contain any large gaps. This would result in a higher value of load within the observed sequence.
Self-similar traffic requires a significantly larger number of observations for the actual load to approach the target load. In the Variance-Time experiment, the target load was 0.25. After generating ~246 million packets, the actual load was 0.26099.
Note: I still notice that with multiple sub-streams aggregated, the actual load exceeds the target load by about 3-4%, even if a large number of packets is generated. This does not happen if only one sub-stream is used. This is still under investigation.

Q4. What formula for the value of the load is used in file source.h of the v2 of the traffic generator?

A4. The Pareto distribution that is used to generate ON/OFF sub-streams requires two parameters: shape (α) and location (b). The αON, αOFF, as well as bON values are given. The bOFF value should be calculated to have each sub-stream producing desired load. The calculation is outlined here. In version 3 this is simplified because distribution of sizes for both ON and OFF periods use the same value of α. I found that distribution of OFF periods does not affect the Hurst parameter of the resultant traffic.

Q5. How does the generated stream relate to Hurst parameter H? Can I generate streams for different values of H?

A5. The following equation illustrates the relationship of the Hurst parameter H to the shape parameter α of the Pareto distribution: H = (3 - α)/2. Thus, H = 0.8 results in α = 1.4.

Q6. Version 3 of the generator outputs packet size PcktSize (in bytes) and Interval (in bytes). How to obtain absolute packet arrival times?

A6. Interval represents the time between the arrival of the previous packet and arrival of the current packet. The arrival time corresponds to the reception of the last bit of a packet. For example, if PcktSize = 100 and Interval = 250, then the inter-packet gap is 250-100 = 150 bytes.



To find absolute times (bytestamps) of each packet, set the bytestamp of the fist generated packet to zero (B0 = 0). Then, bytestamp Bk of packet k is Bk = Bk-1 + Intervalk. Refer to Q1 above to see how to convert bytestamps to timestamps.

Q7. I would like to identify by which stream a particular packet was generated.

A7. Streams in packet generator are not what typically can be referred as TCP stream or any other identifiable flow of packets in the network. A stream in packet generator is an internal abstraction, which is used to produce self-similar traffic. In version 3 of the traffic generator, these multiple streams are only responsible for generating bursts (ON periods) and gaps (OFF periods), but not packets. Packets with a specified size distribution are generated separately, such as to fill the aggregated burst. Therefore, it is not possible to associate a packet with a specific internal stream.

If it is desirable to study the behavior of multiple flows aggregated in a network, then multiple PacketGenerator objects must be created, one for each flow of packets. The constructor for PacketGenerator takes a parameter source_id, which would identify packets of each particular source or flow.

Q8. I found that the presented example of using traffic generator v3 consistently produced output where the second packet had a very large interval. Here is a typical output:

0;223;243
1;526;641170
2;250;270
3;1245;24246
4;1021;1041
5;986;1006
6;528;548
7;846;22429
8;724;744
9;1013;1033


A8.This phenomenon is related to a Reset() procedure for individual ON/OFF streams:
inline void Reset(void)
{
    BurstSize = 0;
    NodeKey   = static_cast(_uniform_real_0_1() * NextPauseSize() );
    ExtractBurst();
}
This procedure assumed that after a reset, each ON/OFF stream was in its OFF period. This allignment of OFF periods, which is an unwanted side effect, quickly disapears after a few bursts extracted from each stream. However, it resulted in the reported large interval for the second packet (the first packet is pre-generated to arrive at time zero, the second packet arrives after the shortest of the aligned OFF periods.) It is unlikely that such a short-lived anomaly would affect traffic or test results in any significant way, especially, if a warmup run is used, as it supposed to be in all simulations. At the same time, I am putting in a fix, which is really simple. The new Reset() function looks like:
inline void Reset(void)
{
    BurstSize = NextBurstSize();
    BurstTime = NextPauseSize() + BurstSize;

    // quick start: simulate start at random time during ON- or OFF-period
    bytestamp_t start_time = _uniform_int_( 0, (rnd_int_t)BurstTime );

    if( start_time < BurstSize )  // zero time fell on ON period 
    {
        BurstSize -= (burst_size_t)start_time;
        BurstTime = 0;
    }
    else  // zero time fell on OFF period
    {
        BurstSize = NextBurstSize(); 
        BurstTime -= start_time;
    }
}
The BurstTime is an alias for NodeKey, which is an inherited member of class AVLNode.

(The credit for finding this issue goes to Mr. Ajiow of National Taiwan University.)

Q9. What code changes are necessary to generate a self-similar stream with a constant packet size

A9. Class GEN::PacketGenerator requires a call-back function, that returns random packet sizes with a desired distribution. To generate a constant packet size simply define a call-back function that returns a constant.
#define CONSTANT_PACKET_SIZE  64

/////////////////////////////////////////////////////////////
// Callback function that generates packet sizes 
// with a desired distribution
/////////////////////////////////////////////////////////////
GEN::pckt_size_t PacketSize( void )
{
    // in this example, use constant packet size (in bytes)
    return (GEN::pckt_size_t) CONSTANT_PACKET_SIZE;
}

// the rest of the code as in the Quick Start  example.