TX RX Path

As we described in the hello world post, the madwifi driver sits between the kernel’s network layer and the physical hardware abstraction layer (HAL). When a user sends packets from his computer, using a socket connection, for example, the network layer invokes certain predefined functions of the MAC layer. The driver implements these entry points and processes the packets, adding MAC headers, deciding transmit rate and so on, and then calls the HAL code to physically transmit the packet. It is the HAL’s responsibility to enable the required registers on the hardware for sending the packet at the appropriate rate and timing.

When a packet is received by the hardware, it causes an interrupt at the driver level. The driver processes the interrupt and effectively receives the packet in software. The MAC header is stripped off the packet, and the packet is sent over to the network layer for further processing.

The Linux network layer calls the function pointed to by the function pointer hard_start_xmit in the net_device structure. One of the first things the madwifi driver does (in ath_attach()) is to initialize this function pointer to point to ath_hardstart(). The parameters to ath_hardstart() are the sk_buff structure and the net_device structure. This simplicity is an artifact of the layering structure that networking follows. The layer above passes to the lower layer only the cooked packet. The lower layer further cooks the packet and passes it to the next layer. The contents of the packet handed over by the network layer are accessible from the skb->data pointer. From this point forward, the packet is prepared for transmission on the wifi interface. It must be given a valid 802.11 header and the driver must decide the various values in this header. All this is done if the wifi device is in ap, sta, adhoc or ahdemo mode. If the device is in the monitor mode, no 802.11 header is attached to the packet and the packet is sent raw on air. Therefore, when in the monitor mode, ath_hardstart() quickly calls the ath_tx_startraw() function, whereas in other modes, the ath_tx_start() function is called. The monitor mode in madwifi thus actually allows one to transmit raw packets. A very important step performed by both the ath_start and ath_startraw functions is the call to ath_hal_setuptxdesc() HAL function. This call hands over the buffer to the hardware and sets up the parameters essential to send the packet on air. It sets the transmit power, the transmit rate, the antenna to use (most Atheros cards have more than one antenna), the type of packet and a host of other parameters through the flags. A lot of variations in the standard packet transmission style can be achieved by varying some of these parameters. For example, if you write some new auto-rate algorithm, you may just change the value of the txrate parameter to force the hardware to send packets at that rate. However, note that not all packets go through this function call. Some packets such as acknowledgments are produced by the hardware and are sent only at predefined rates. But all packets that are generated by the network layer go through this call, and therefore, it can be a good place to hack in. Another feature that we found useful, and had a very hard time figuring out, is to get all packets timestamped by the hardware while transmission. We were operating in the monitor mode and had our own MAC header attached to the network layer packet instead of the standard 802.11 header. Now, we wanted all our packets to be timestamped by the hardware to facilitate synchronization between nodes. The hardware always timestamps beacons, hence, if we can fool the hardware into believing that the packet being sent is a beacon, we can achieve timestamping. This fooling can be done by changing the packet type (atype) parameter to HAL_PKT_TYPE_BEACON in the call to ath_hal_setuptxdesc().

Both the ath_start() and the ath_startraw() functions call the ath_tx_txqaddbuf() function that actually inserts a buffer on a specific transmit queue (there are multiple hardware queues) and then calls the ath_hal_txstart() function. This function opens up a gate in the hardware that releases all the buffered packets buffered on the particular hardware queue.

When the hardware receives a packet, it causes the HAL_INT_RX interrupt. Interrupts are handled by the ath_intr() function. Since a lot of processing might be required on a received packet, this processing is deferred to a tasklet known as ath_rx_tasklet(). The tasklet is scheduled soon after the interrupt is fired, whenever the OS deems fit. Hence, it is inappropriate to assume the time of call to the rx_tasklet as being equivalent to the time of packet reception. Instead, a variable inside the ath_softc structure can give us the exact time of complete packet reception. The tasklet is a good place to check for physical errors, the reception rate, the per packet reception signal level and noise floor level. We have observed some odd packets that have length=0, frame type and subtype=0 in the ath_rx_tasklet() function. We believe that these frames are an artifact of the devices we used, but nonetheless, such frames must be ignored (all valid packets are received just fine even after ignoring these phantom entries). In ap, sta, adhoc and ahdemo mode, the tasklet goes on to process the packet further. In the monitor mode, the packet must be sent to the higher layer almost raw. A radiotap/prism2 header may be optionally attached to the raw received frame to preserve the PHY information regarding the rate of transmission etc. before passing it to the network layer. Applications such as tcpdump, wireshark and kismet benefit from the radiotap headers. Packets are passed to the ath_rx_capture() function which then calls the ieee80211_input_monitor() function. The ieee80211_input_monitor() function then calls the network layer function netif_rx() to complete its handling of the packet.

As a side note, the MAC layer does not free the skb it receives from the network layer or the hardware (in either direction). The skb is shared between all layers. Also, in the monitor mode the hardware sends all packets to the driver. In other modes, the driver receives only packets destined to it and broadcast packets. This fundamental difference between monitor mode and other modes, may cause a lot more interrupt handling at the driver for nodes in monitor mode than for other nodes. This may affect the performance of other applications if the monitor mode node is also expected to do other (time sensitive) tasks.

Advertisements

44 Responses to TX RX Path

  1. Aakash says:

    Hi Ashutosh, thanks for posting it! you cleared up a major doubt of mine in the last para of “side note” 🙂

    Keep up the good work!
    -Aakash

  2. Faramir says:

    Great starting point before facing the abrupt code; thank you. So,… can we consider that ath_hal_txstart defines the actual transmission start time? Or maybe there are hidden hal/hardware queues after the call to ath_hal_txstart and before the frame is actually ready for transmission…

    • Ashutosh Dhekne says:

      No, ath_hal_txstart is the last place in software. The packet may experience some delay before it is actually transmitted and that depends on the time taken by the HAL code, channel activity and the current number of backlogged packets in the queue. When beacon packets are sent by APs (as apposed to normal packets), they are timestamped by the hardware at the very instance when the first few bytes are leaving the radio. This time is the most accurate time of when the packet was transmitted. The timestamp is a 64bit value and never wraps around (unless a hardware reset occurs). On the receiver side, the hardware timestamps all packets with a 15bit value that has to be quickly extended (using ath_extend_tsf())in the rx_tasklet() to prevent it from becoming stale. This value is (of course) the receiver’s tsf time when the packet was received and is assumed to be accurate to 1usec. Thanks for reading our blog. Do write to us if something is not clear or if you have any other doubts.

  3. Faramir says:

    OoOooK,……as I suspected. The time I wanted to measure is actually MAC access time (from the moment at which a frame is at the head of its MAC queue ready for transmission until it is actually transmitted). From clause 11.1 of the 802.11 standard, I understand that when a beacon is generated, it is pushed to the head of the hardware queue. Therefore, in the case of beacons, the access delay should be TSF(ath_tx_capture)-TSF(ath_hal_txstart)…still wrong?

    By the way, any mac80211/ath5k blog in mind? 😉 Thank you!!

    • Ashutosh Dhekne says:

      Beacons generally use another hardware queue. I can’t say with full confidence, but I think you may be correct about TSF(ath_tx_capture)-TSF(ath_hal_txstart). You can measure the access time for other packets, but then you will need to fool the hardware into stamping the normal packet with the hardware-timestamp and also keeping the TSF(ath_hal_txstart) time. Then on the receiver side, when you get that packet you can use the timestamp inside the packet and the TSF(ath_hal_txstart) to compute the access time. Sorry, I am writing in a hurry, if you don’t get me, please write back and I will explain more.

      Currently we don’t have any other blogs on the chart! Will let you know if we write one, though 🙂

      • Faramir says:

        Nop,…I put some debug messages and it seems that after a beacon transmission there is no TX interrupt (ath_intr) and neither ath_tx_capture is called. So, in order to measure beacon access time (from the moment at which the beacon is ready for transmission until it is actually transmitted), TSF(ath_tx_capture)-TSF(ath_hal_txstart) doesn’t work. Am I doing something wrong? If no, is there any other way?
        How do I know when a beacon has been transmitted?

        I’m soOo sad…

      • Ashutosh Dhekne says:

        Beacons are created through an interrupt known as SWBA (Software Beacon Alert). And the interrupt is handled in the ath_intr() function for sure.

      • Faramir says:

        Yes, I can see periodic SWBAs according to the beacon interval. The SWBA is sent before a beacon is transmitted. However, I don’t see a ath_tx_capture call following those SWBAs. What I need to know is when the beacon has been transmitted. I thought that after a transmission the process is ath_intr -> ath_tx_tasklet -> ath_tx_processq -> ath_tx_capture. It seems this is not the case with beacons. Any clue?

      • Faramir says:

        FYI: beacon hw queue is set up with all interrupts disabled, that’s why I did’nt see ath_intr calls after a becaon was transmitted. These interrupts can be activated in ath_beaconq_configure()

        More details on

        http://sourceforge.net/mailarchive/message.php?msg_name=AANLkTikAnSN42RMr5UALxnSe%2BCPkiq98Vm%3Doghi6a2y5%40mail.gmail.com

  4. Faramir says:

    Got it. Actually, I’m only interested in beacons, so it should be easier. Thank you for your time!

  5. bikibcy says:

    hi
    I’m interested in accessing the physical layer data in order to process it in matlab and estimate the signal, so is ath_rx_tasklet() the right place to get the data from and how?

    • Ashutosh Dhekne says:

      Most data can be obtained from struct ath_rx_status in the ath_rx_tasklet(). If what you are looking for is not available in ath_rx_status, see the definition of all the variables created at the beginning of ath_rx_tasklet. The info you need will mostly be in one of those variables described there.

  6. Mohammad says:

    Hi,
    I have a question about turbo mode, as I know, MADWIFI only supports static turbo mode where two channels are reserved for transmission..And the last stage in transmitting is the ath_hal_txstart( in the software), so my question is:
    Which function (or part) is responsible to send each packet on a certain channel(In turbo mode)?

  7. Fernando says:

    Just curious how madwifi runs turbo mode. What transmission speeds could be expected in modes such as ahdemo without all the beacon’s and overhead of adhoc? I’m trying to fine tune madwifi with a ahdemo setup and turbo capable cards to communicate between devices. I’m trying to understand what the settings on this blog will help accomplish and if it’s needed in my case.

    Thanks

    • Ashutosh Dhekne says:

      The max transmit speed for 802.11 is 54Mbps. However, one could set certain modes so that frames are sent in a burst for an entire transmission opportunity. This is known as frame bursting. Also there is a possibility of sending two frames after only waiting for a SIFS interval.
      I guess the turbo mode can use 40MHz channels to transfer data instead of the usual 20MHz channels resulting in increased bandwidth.

      parameters ff=0 and burst=0 will disable fast frames and frame bursting. I am not very sure about turbo mode, however.

      • Prasanna Karthik says:

        “I guess the turbo mode can use 40MHz channels to transfer data instead of the usual 20MHz channels resulting in increased bandwidth.”

        You are right about this. And for operating in the turbo mode, we just have to tune into the channel that supports turbo operation. Ex: Channel 44 of 802.11a.

        However, is there a mean by which i can operate in channel 44 and still disable the turbo mode.
        using something like,
        “iwpriv ath0 turbo 0”. I tried this, but no luck so far.

  8. Dhiren says:

    Hi Ashutosh,

    Thanks for posting all the of information here madwifi. It has helped me lot with understanding madwifi operation.
    One question i have is that when packet is received, it gets 15 bit tsf. Then you have to use ath_extend_tsf() to convert it to 64 bit timestamp. if possible could you explain how that is achieved to 1 microsecond accuracy.

    Thank You!!!

    • Ashutosh Dhekne says:

      The 15 bits tsf that you get when the packet is received are the 15 lowest bits in a 64-bit number. So these 15 bits are already at 1 microsecond accurate. It is just that the bit space is so small that the number will wrap-around in 32.768 milliseconds (2^15us). That is why we have to extend it to 64 bit, which will “never” wrap-around. Since we have such a small time to extend the number, it is done immediately in the interrupt service routine/tasklet.

  9. Sara says:

    Hi Ashutosh:
    Thanks for sharing the tech info on madwifi. In one of your papers, you mentioned that you sent cwmin and cwmax to 1, and Q size to 1. I tried to use iwpriv cmd to set cwmin and cwmax values for ac BE. It accepts new values but never shows the new values. Is it really changing the values ? I do not know how to chech it. Could you comment on this behaviour ?

    thanks,

    • Ashutosh Dhekne says:

      We had changed it in the code and checked it using a CRO connected to the network card. But iwpriv should also work.

    • Mohammad says:

      Well, I also tried iwpriv to set the value of cwmin but it simply didn’t work.

      Well, you could try to configure the (Cwmin) and (cwmax) internally from the driver itself. In the module (if_ath.c) you will find functions like (ath_txq_update) and (ath_txq_setup). In those functions you will find parameters such as(qi.tqi_cwmin), try to configure them as desired.

      • Ashutosh Dhekne says:

        Thanks Mohammad,
        Sara: I think that was how we changed cwmin and cwmax in the code. I will write more about how we checked it on the CRO. Since the backoff becomes zero, a backlogged transmit queue will attempt to send immediately after a SIFS interval. When multiple packets are sent one after another, they all leave the exactly same gap between each other. If it is difficult to connect a spectrum analyzer / CRO, another card may be set in receive mode and receive timestamps of all the packets can be checked. If same sized packets are sent the receive timestamp should be separated by equal intervals.

  10. sara says:

    For my project, I’m running madwifi in monitor mode as you do. In monitor mode, we’re receiving raw pkts.
    How much work needs to be done to massage and pass the raw pkts to TCP/IP stack so that the stack can interpret the pkts are ligitimate and the usual communication going, like ping or any network communication in all the layers ?
    Are there any sample code available in this nature ?

    thanks,

  11. Aswin says:

    Do you know how we can change the channel width , 5, 10, 20MHZ ? Can you provide some pointers ……

    • jack says:

      I’m looking for the same, I need to change the channel width but I can’t deal with the command to do this. I think the solution round in the iwpriv command, but I can’t find a detailed description of all parameters that the atheros chipset accept.
      If you found how to change please contact me at ralbano at jack dot com dot ar

      regards.

  12. Jayto says:

    We have experienced a serious problem when using monitor mode – the received signal strength (RSSI) is around 6 dB lower compared to using adhoc or ap mode. Our setup was as follows:
    (1) TX: adhoc, RX:adhoc
    (2) TX: adhoc, RX:monitor
    (3) TX: monitor, RX:adhoc
    (4) TX: monitor, RX:monitor
    The RSSI was nearly the same for 1 and 3. However in case of 3 and 4 the RSSI was always around 6 dB lower. This is strange. Any ideas for the reason?
    We used madwifi 0.9.4, the phy bitrate was set to 1 Mbps.

    • Ashutosh Dhekne says:

      There is no reason that comes to the mind immediately. Here are a few things to check:
      (1) Use different vendor cards (i know the list is limited here since we have to use madwifi)
      (2) Check at various other frame rates
      (3) Check inside a faraday’s cage (use a lift (elevator) if a proper faraday cage is difficult to come by) – in monitor mode too many frames, both good frames as well as corrupt ones are incident on the card and come up to the driver. No reason why that would cause a reduction in RSSI, but just a good point to check.

      I will also communicate with my institute friends and check if any of them have seen a similar problem. Our own experiments were never in this direction, so I do not have first hand knowledge of the problem.
      If you come across any causes, resolutions or suggestions, keep populating this blog.

      • Jayto says:

        Hi,

        thank you for your fast reply.

        @1: We used cards from the same vendor (Rouerboard) for our experiments.

        @2: We used a constant rate of 1 Mbps. We increased the distance between sender and receiver in such a way that the RSSI was around 6 when using the adhoc mode on RX side. Here we had a packet delivery rate of nearly 1. So if we switched on using monitor mode for the RX side the RSSI was only around 0. This translates in a very high number of unsuccessful transmitted frames; i.e. low packet delivery rate. So the computation of the RSSI seem to be correct. Anyway, we can also make some measurements using a higher bitrate.

        @3: We will repeat this experiment in a more cleaner environment.

  13. Maysam says:

    First of I want to thank you for sharing useful information about MadWifi driver
    I’m working on my thesis, and I need to calculate the time interval that it takes for a packet to be sent
    I mean the time interval between a packet is put into transmission queue and the moment acknowledgment is received
    I added a field to ath_buf to fill it with a timestamp at ath_tx_start, and I put another function in ath_processq.
    Whenever an ack is received, I generate a timestamp at ath_processq, and subtract it from the timestamp recorded at ath_tx_start in order to calculate queuing and transmission delay.
    but it doesn’t work properly. because there are some extra generated timestamp.
    I want to know, Is there another way around it to force MadWifi to timestamp the packet before queuing and when the ack of packet is received.
    many thanks in advance.

  14. NewUser2011 says:

    Hello everybody, I’m working on my thesis exactly like Maysam, and I have a question that I hope can be easily answered. In the struct net_device_stats (file netdevice.h) there are rx_bytes and tx_bytes fields. Now, I know that rx_bytes is the number of CORRECTLY received bytes, but I need to know if the same holds for tx_bytes. Are those bytes the ones correctly transmitted or not? And in this second case, how could I modify the code to obtain just these ones? (In other words I have to compute the MAC-level goodput)

    Thanks in advance for who wants to help me!

    • Ashutosh Dhekne says:

      I know I am not answering your query, and you might have already thought about this, but must still ask the following: Do you really need the MAC level goodput of the system? Because, it might be a better idea to compare the bytes sent/received by the application layer versus the number of bytes sent/received by the MAC layer. Coming to your question, I am not very confident but I think tx_bytes is incremented even when a retransmission happens. But retransmission can happen directly from hardware too and these will not be recorded. As a result you may not be able to conclude what you want just from the tx_bytes. Does anyone know exactly how to get the number of retries? By the way, if you have a third machine, it might be possible to put it in monitor mode and record all packets seen. Then an analysis can be easily performed to find out the exact goodput.

      • NewUser2011 says:

        First of all, thank you for the quickness of your answer. Then, to tell you the truth, MAC level goodput is exactly what I’ve to compute in this very first step of my thesis (there will be other steps, but right now this is my task). Moreover, as I can see, you got the point, which is what I really have to do: compute MAC level goodput AND compare it to the application level throughput (using iperf, but this is the easy part). Now, I wrote a C program that computes tx_bytes and rx_bytes every x seconds (x is decided by the user) and prints them on stdout (together with RSSI, but this doesn’t matter) in STA mode, when I’m connected to my AP. The question now becomes: there is absolutely no way to compute this number, let’s say “tx_bytes_successful”, as an additional field of the struct net_device_stats or in any other way?

        Thank you for your help and for your time.

      • Ashutosh Dhekne says:

        Do you have the option of using a third machine in monitor mode is close vicinity of the other two? Of course this solution is no good if you need the mac goodput numbers in production environment or if it forms a part of some algo you are creating. But it is good enough if you want to check if something you made is giving you good useful throughput or not. Again, I am talking from the top of my head and have not looked into the code to get an answer to your question.

  15. NewUser2011 says:

    Actually now I’m working with: a pc with Ubuntu, an Ubiquity card that I use to get info through Madwifi and, obviously, my Access Point. I have another pc (close to the first one) that hasn’t the possibility to integrate the Ubiquity card, but on which I could install Ubuntu. Now, if I understand well, you are suggesting me to use this second machine in monitor mode, and this should record all packets seen. From this analysis I can compute goodput: am I right?

    • Ashutosh Dhekne says:

      Yes, you are correct. But if you cannot use Ubiquity/ atheros card on the second PC, I am not sure if you can start monitor mode properly on the second PC.

      • NewUser2011 says:

        I’m afraid you’re right, because I realized it just after sending previous message… In addiction to this, I don’t think I’m allowed to use 2 pc to solve this task, so we’re back to the problem… This means that I have to change/add some code in order to get what I want. Anyway, I really appreciated your help. Thanks

      • Ashutosh Dhekne says:

        If monitor mode is not an option, you will have to use the tx_bytes somehow. Think about the problem and understand if disabling hardware automatic retransmissions is allowed for your experiments. Disabling retransmissions is possible. If it is going to work for you, then go ahead and do it. After that, tx_bytes will be equal to the bytes sent by the MAC layer. There will be more MAC layer errors and the higher layers will have to handle it properly. We have seen the TCP throughput reduce drastically after turning off HW retransmissions on bad links.

  16. NewUser2011 says:

    This could be a solution, but doing so will result in the fact that I can’t compare tx_bytes and tx_bytes_succ, just to be sure that everything went correctly. I have to check if I’m allowed to disable HW retrasmissions (also with the person that is following me in this thesis) and I’ll let you know. Thanks again.

    • NewUser2011 says:

      Hello, unfortunately disabling retransmissions is not an option. I have been told that a solution can be found through the code, so I have to check all the code concerning functions called during transmission phase. Again, if there is someone who really knows this stuff, I would really appreciate his/her help.

      Regards

  17. Someone says:

    Hello everybody,
    It is so nice to know that this thread is still alive.
    Nowadays, I’m working on received beacons, especially I’m interested on beacon time stamps. I know, this time stamp is 8 bytes long (can be assumed as u_int64_t). But I faced some problem when I want to convert it into system time (struct timeval) – timeval is not in the same trend for certain AP. I tried to convert it through le64toh() -> ns_to_timeval() functions. Does anybody try to convert it? Or am I doing smth wrong?

  18. Mighty says:

    Hi,
    I’m working on master thesis. I want to timestamp the packet whenever it’s want to sends out, and timestamp the Ack whenever it’s received.
    I timestamp the packet whenever it’s in ath_tx_start, and timestamp the Acks in ath_processq.
    But, I couldn’t find any proper identification for Acks packets to match them with packets that I timestamped before sending them.

    My question is what’s the format of Ack of sent packets?
    Is there any way other way to timestamp the packets and Acks and associate them together?

  19. shashank says:

    Thank you very helpful 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: