Running hostapd crashes when WPA2 encryption is used (8723AU module as Access Point)

[maggieroxas]

I am currently having problems with my wireless device (with its own microprocessor) installed with 8723au module.

• rtl8723au module: 8723au with 2.4Ghz
• rtl8723au driver: Version as of Dec.16, 2013 commit
• Linux Kernel: v3.10.24
• File System: Ubuntu 13.04 (Raring Ringtail)
• hostapd driver: NOT the one via "apt-get install", got my wpa_supplicant_hostapd source codes packaged within the 8723au module supplier's driver v4.1.6_7336.20130426.

I cross-compile the driver (with my device's platform's compiler) using my Ubuntu 12.04 PC as host and across kernel v3.10.24 source codes - NO problems in compilation (make clean && make all),

I copy and install the created 8723au.ko in my device's file system, via "insmod 8723au.ko debug=4" - NO problems in installation.

For 8723au as Client or STA - NO problems in connection (connects fine, signal OK, link quality OK).

For 8723au as Access Point (without encryption) - NO problem in connection (connects fine, signal OK, link quality OK).

For 8723au as Access Point (with WPA2 encryption) - HAS problem in connection (Kernel panic).

Please check these files you might need as reference:

• https://gist.github.com/maggieroxas/9225765 - crash logs (when I run "wifiap start" which is just a script that runs hostapd in the background)
• https://gist.github.com/maggieroxas/9225791 - Hostapd configuration file
• https://gist.github.com/maggieroxas/9225777 - /var/log/messages of my device which contain rtl8723au logs

Some quick questions:

• I can see that this site offers the rtl8723au WiFi driver, and rtl8723au_bt for BT driver. How about hostapd driver? Is there and open source driver for this that works well with your rtl8723au/rtl8723au_bt?
  The default hostapd driver installed via "apt-get install hostapad" also crashes.
• This kernel crash happens only if the AP is set with encryption (tested only WPA2 so far), When there is no encryption, there are no problems at all.
  Does this give a clue?

[lwfinger]

I do not have much experience with ARM kernels, but the crash dump you supplied does not seem to contain any backtrace to rtl8723au. It probably comes from hostapd. That old version of hostapd is supplied because rtl8723au did not contain the necessary code to accept the cfg80211 commands used by the modern versions of hostapd. In the kernel_version branch of the repo is a new version of rtl8723au that does speak cfg80211 and should work with a new hostapd. That branch of the repo is the code that will be submitted for inclusion in the kernel. Because of that, all the conditional code to handle kernel API changes has been removed. To use it, you will need a 3.13 or 3.14 kernel.

I have a question. Is the ARM-based RTL8723AU device commercially available? I have been trying to get a sample device, but so far I only knew about it being in a Lenovo Yoga 13, and that is too expensive for me.

[maggieroxas]

First of all - thank you for the feedback! :)

I do not have much experience with ARM kernels, but the crash dump you supplied does not seem to contain any backtrace to rtl8723au.

Yes, I actually noticed that as well, as I'm seeing iptables errors.
However, the crash being triggered by running hostapd do make me wonder..
I do assume that it's either the rtl8723au or the hostapd that triggers these errors.

It probably comes from hostapd.
That old version of hostapd is supplied because rtl8723au did not contain the necessary code to accept the cfg80211 commands used by the modern versions of hostapd.

I see. So the current hostapd that I'm using (provided by the supplier) is the "old version" which is compatible to the current rtl8723au (master branch).

On the other hand, the "modern version" of hostapd (I'm assuming this is the one available at http://w1.fi/releases/hostapd-x.y.z.tar.gz, correct me if I'm wrong) uses cfg80211 commands NOT supported by rtl8723au (master branch).

In the kernel_version branch of the repo is a new version of rtl8723au that does speak cfg80211 and should work with a new hostapd.
That branch of the repo is the code that will be submitted for inclusion in the kernel.
Because of that, all the conditional code to handle kernel API changes has been removed.
To use it, you will need a 3.13 or 3.14 kernel.

Wow. I never noticed that there's a rtl8723au (kernel_version branch)!
Okay.
So this kernel_version already supports the modern version of hostapd (again, I'm assuming this is the one available at http://w1.fi/releases/hostapd-x.y.z.tar.gz) and will be usable on kernels 3.13 and up.

I will try this new combination (ARM + modern hostapd + rtl8723au kernel_version branch) and give you an update whether I'll encounter problems.

I have a question. Is the ARM-based RTL8723AU device commercially available?
I have been trying to get a sample device, but so far I only knew about it being in a Lenovo Yoga 13, and that is too expensive for me.

Hmm. I'll get back to you on this.

[maggieroxas]

BTW, I'm seeing a lot of releases at http://w1.fi/releases/ for modern hostapd.
Any specific release you'd recommend as best suitable for the rtl8723au kernel_branch version?

Also, using the new combination (ARM + modern hostapd + rtl8723au kernel_version branch), what is the best suited Bluetooth driver I should use?
Is this combination compatible with rtl8723au_bt (master branch)?

Thanks!

[lwfinger]

The current hostapd code can be found at http://hostap.epitest.fi/releases/hostapd-2.1.tar.gz.

The BT driver is separate from the rest. As I have no rtl8723au device, I cannot test. A separate version from Realtek is in the new branch of that repo.

[maggieroxas]

The BT driver is separate from the rest.
As I have no rtl8723au device, I cannot test. A separate version from Realtek is in the new branch of that repo.

I see. Sorry, where is this "new branch" of what "repo"?

The current hostapd code can be found at http://hostap.epitest.fi/releases/hostapd-2.1.tar.gz.

Okay.
I will get back to you on the following:

• Check if there will be issues in this combination: ARM + Linux Kernel 3.13.5 + rtl8723au (kernel_version branch as of today) + hostapd at http://hostap.epitest.fi/releases/hostapd-2.1.tar.gz
• About the ARM-based RTL8723AU device being commercially available

[maggieroxas]

Hi Larry - Update on this:

Check if there will be issues in this combination: ARM + Linux Kernel 3.13.5 + rtl8723au (kernel_version branch as of today) + hostapd at http://hostap.epitest.fi/releases/hostapd-2.1.tar.gz

I had no problems cross-compiling my platform in the kernel.
Also no problems cross-compiling the rtl8723au and hostapd.
Lastly, no problems installing the rtl8723au via insmod.

Note: I copied the rtl8723aufw_*.bin fw files in my platform's /lib/firmware/rtlwifi.

Some results below, via quick tests.

[Configuring 8723au as Client]

Seems problematic.

I can execute "ifconfig wlan0 up", but I have problems trying to connect to AP.
I get errors such as the ff. when I try to connect to an external AP (without encryption):

root@localhost:~/.setup/wlan# iw dev wlan0 connect snakecentral
command failed: Operation not supported (-95)

Are there other ways to connect using linux command-line other than the following which both failed:

• "iwconfig wlan0 essid snakecentral"
• "iw dev wlan0 connect snakecentral"

... that is compatible to my current setup?

Note: I can properly do "iw dev wlan0 scan" though.

[Configuring 8723au as Access Point (no encryption)]

Seems good.

Basic testing is OK.
I will thoroughly test tomorrow though and inform you the results.

[Configuring 8723au as Access Point (has WPA2 encryption)]

Seems problematic.

I can't connect at all, either using a PC or my phone.
I always get "Authentication problem" and eventually disconnected "Disconnected" message.

Logs and hostapd conf file are in the following links:
https://gist.github.com/maggieroxas/9249663
https://gist.github.com/maggieroxas/9249690

Please advise.
Thank you!

[lwfinger]

Please install the firmware. The device will never work without it!!!!

[maggieroxas]

Sorry for the confusion.
I removed my comments last night which discussed about my testing results without the firmware loaded since they're a lot and might flood the discussion.

But, in my comment #27 (comment), the firmware is already loaded in my platform (if you meant "install the firmware" as "copy the firmwre somewhere in /lib/ so the driver can load it"):

Note: I copied the rtl8723aufw_*.bin fw files in my platform's /lib/firmware/rtlwifi.

Firmware is already loaded, but I had problems in WiFi as Client and WiFi as AP with WPA2 encryption (kernel does not crash anymore, but I get authentication problems).

BTW, should I open another issue on the Client issue?).

Please check. Thank you.

[lwfinger]

As I do not have a device, it is impossible for me to help much with the type of issue you have. The conversion to a kernel-ready state is being done by someone at RedHat. AFAIK, he has not tested the device as an AP, but he has been using it as a client with encryption.

He is not available at the moment, but when he returns to his office, I'll ask him what encryption methods he has tested.

[maggieroxas]

Firmware is already loaded, but I had problems in WiFi as Client and WiFi as AP with WPA2 encryption (kernel does not crash anymore, but I get authentication problems).

I have filed a separate issue #28 on the WiFi in Client mode problem for proper tracking.

As for the WiFi as AP with WPA2 encryption mode problem:

The conversion to a kernel-ready state is being done by someone at RedHat.

I see. So this someone maintains the "kernel_version" branch?

AFAIK, he has not tested the device as an AP, but he has been using it as a client with encryption.
He is not available at the moment, but when he returns to his office, I'll ask him what encryption methods he has tested.

The "encryption methods he has tested" are for the "client with encryption" setup...?

Hmm. Not testing the device as an AP seems a bit weird to me... (if I'm correct in understanding that as you said, "That branch (kernel_version) of the repo is the code that will be submitted for inclusion in the kernel.").

I'm not really sure but shouldn't all 8723AU modules work as either Client or AP?
If it is, before it's included in mainline kernel, it should have been working in either, right?
Correct me if I'm wrong though...

[maggieroxas]

BTW, does these error snippets look familiar to you?
These logs kept reappearing during the time my phone (with MAC addr: e4:2d:02:13:18:09) tries to connect to 8723AU in a WPA2 encrypted AP mode.
My phone goes "Connecting" to "Authentication problem", retries, and eventually "Disconnected".

Jan 24 17:15:53 localhost kernel: RTL8723AU: OnDeAuth Reason code(3)
Jan 24 17:15:53 localhost kernel: RTL8723AU: ERROR ap recv deauth reason code(3) sta:e4:2d:02:13:18:09
Jan 24 17:15:53 localhost kernel: RTL8723AU: rtw_cfg80211_indicate_sta_disassoc(padapter=eb640440,wlan0)
Jan 24 17:15:53 localhost kernel: RTL8723AU: report_del_sta_event: delete STA, mac_id=2
Jan 24 17:15:53 localhost kernel: RTL8723AU: rtw_ht_operation_update current operation mode=0x13
Jan 24 17:15:53 localhost kernel: RTL8723AU: rtw_ht_operation_update new operation mode=0x0 changes=2

Jan 24 17:15:53 localhost kernel: free number_3 stainfo  with hwaddr = 0xe4 0x2d 0x02 0x13 0x18 0x09  
Jan 24 17:15:53 localhost kernel: RTL8723AU: Clean STA_(2) info
Jan 24 17:15:53 localhost kernel: RTL8723AU:  [0x00000004,4]
Jan 24 17:15:53 localhost kernel: recvbuf2recvframe: rtw_recv_entry(precvframe) != _SUCCESS
Jan 24 17:15:53 localhost kernel: RTL8723AU: ERROR set pairwise key to hw: alg:0(WEP40-1 WEP104-5 TKIP-2 4
Jan 24 17:15:53 localhost kernel: RTL8723AU:  [0x00000080,4]
Jan 24 17:15:53 localhost kernel: 
Jan 24 17:15:53 localhost kernel: ERROR: rtw_setstaKey_cmdrsp_callback => can't get sta_info 

Complete logs at:
https://gist.github.com/maggieroxas/9264733

[lwfinger]

You seem to not be comprehending the statement that "I have no device, and I cannot test anything."

If you want an analysis of the problems as an AP, then you need to use a 3rd computer to capture all of the on-the-air traffic. That is the best way to find out what is wrong with communication.

Testing as an AP is always the last thing that is tested. If that is important to you, then you will need to do some debugging.

[maggieroxas]

If you want an analysis of the problems as an AP, then you need to use a 3rd computer to capture all of the on-the-air traffic.
That is the best way to find out what is wrong with communication.
Testing as an AP is always the last thing that is tested. If that is important to you, then you will need to do some debugging.

If I capture details using a 3rd computer and posted pcap files here, can you help me decode it?
I have not that much knowledge on debugging these network captures, but I'll try my best.

[maggieroxas]

If you want an analysis of the problems as an AP, then you need to use a 3rd computer to capture all of the on-the-air traffic.

I tried to capture the on-the-air data via a 3rd computer (via pcaputils' pcapdump utility "pcapdump -i wlan0 -w sample.pcap", however - it seems that I can only capture these packets when I am already connected to the platform's AP.
Since the problem is, I can't connect to AP when I use WPA2 encryption, I can't capture any packets.

Anything I'm missing in my setup...?

To compensate with not being able to provide you packets, for now, I tried to debug the RTL8723AU logs when I try to connect to AP with WPA2 encryption. As I see it, this is what happens:

(1) My phone tries to connect to AP:

Jan  1 01:49:50 localhost kernel: RTL8723AU: +OnAuth
Jan  1 01:49:50 localhost kernel: RTL8723AU: auth alg=0, seq=1
Jan  1 01:49:50 localhost kernel: RTL8723AU: going to alloc stainfo for sa=e4:2d:02:13:18:09
Jan  1 01:49:50 localhost kernel: RTL8723AU: issue_auth

(2) My phone requests for connection/association to AP:

Jan  1 01:49:50 localhost kernel: RTL8723AU: OnAssocReq
Jan  1 01:49:50 localhost kernel: RTL8723AU: allocate new AID = (1)
Jan  1 01:49:50 localhost kernel: RTL8723AU: rtw_ht_operation_update current operation mode=0x0
Jan  1 01:49:50 localhost kernel: RTL8723AU: rtw_ht_operation_update new operation mode=0x13 changes=2
Jan  1 01:49:50 localhost kernel: RTL8723AU: update_bcn_htcap_ie
Jan  1 01:49:50 localhost kernel: RTL8723AU: update_bcn_htinfo_ie
Jan  1 01:49:50 localhost kernel: RTL8723AU: bss_cap_update_on_sta_join, updated=0
Jan  1 01:49:50 localhost kernel: RTL8723AU: update_sta_info_apmode
Jan  1 01:49:50 localhost kernel: RTL8723AU: Set STA_(2) info
Jan  1 01:49:50 localhost kernel: RTL8723AU: issue_asocrsp
Jan  1 01:49:50 localhost kernel: RTL8723AU: indicate_sta_join_event to upper layer - hostapd
Jan  1 01:49:50 localhost kernel: RTL8723AU: rtw_cfg80211_indicate_sta_assoc(padapter=ee9d8440,wlan0)
Jan  1 01:49:50 localhost kernel: RTL8723AU: report_add_sta_event: add STA
Jan  1 01:49:50 localhost kernel: RTL8723AU: add_RATid=> mac_id:2 , raid:4 , bitmap=0x40000fff, arg=0x82

(3) Signal seems to degrade suddenly:

Jan  1 01:49:50 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 34, rssi:-78, sig1
Jan  1 01:49:51 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 56, rssi:-67, sig1
Jan  1 01:49:52 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 38, rssi:-76, sig0
Jan  1 01:49:53 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 59, rssi:-65,
sig1
Jan  1 01:49:54 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 40, rssi:-75,
sig0
Jan  1 01:49:55 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 27, rssi:-81,
sig0
Jan  1 01:49:56 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 18, rssi:-86,
sig0
Jan  1 01:49:57 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 12, rssi:-89,
sig0
Jan  1 01:49:58 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength:  8, rssi:-91,
sig0
Jan  1 01:49:59 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength:  6, rssi:-92,
sig0

(4) Some null data issuing happened, possibly due to low signal(?):

Jan  1 01:49:59 localhost kernel: RTL8723AU: issue_nulldata(wlan0) to e4:2d:02:13:18:09, ch:1, 1/1 in 0ms
Jan  1 01:49:59 localhost kernel: RTL8723AU: ack check for asoc expire, keep_alive_trycnt=1
Jan  1 01:50:00 localhost kernel: RTL8723AU: validate_recv_ctrl_frame alive check-rx ps-poll
Jan  1 01:50:00 localhost kernel: RTL8723AU: no buffered packets to xmit

(5) My phone initiates disconnection from AP (cause: unknown).

Jan  1 01:50:00 localhost kernel: RTL8723AU: OnDeAuth Reason code(3)
Jan  1 01:50:00 localhost kernel: RTL8723AU: ERROR ap recv deauth reason code(3) sta:e4:2d:02:13:18:09
Jan  1 01:50:00 localhost kernel: RTL8723AU: rtw_cfg80211_indicate_sta_disassoc(padapter=ee9d8440,wlan0)
Jan  1 01:50:00 localhost kernel: RTL8723AU: report_del_sta_event: delete STA, mac_id=2
Jan  1 01:50:00 localhost kernel: RTL8723AU: rtw_ht_operation_update current operation mode=0x13
Jan  1 01:50:00 localhost kernel: RTL8723AU: rtw_ht_operation_update new operation mode=0x0 changes=2
Jan  1 01:50:00 localhost kernel: RTL8723AU: update_bcn_htcap_ie
Jan  1 01:50:00 localhost kernel: RTL8723AU: update_bcn_htinfo_ie
Jan  1 01:50:00 localhost kernel: RTL8723AU: bss_cap_update_on_sta_leave, updated=0
Jan  1 01:50:00 localhost kernel: free number_3 stainfo  with hwaddr = 0xe4 0x2d 0x02 0x13 0x18 0x09  
Jan  1 01:50:00 localhost kernel: RTL8723AU: Clean STA_(2) info
Jan  1 01:50:00 localhost kernel: ERROR: rtw_setstaKey_cmdrsp_callback => can't get sta_info 
Jan  1 01:50:00 localhost kernel: RTL8723AU: rtw_stadel_event_callback(mac_id=2)=e4:2d:02:13:18:09

Note: Complete AP with WPA2 logs are at: https://gist.github.com/maggieroxas/9322891

Based on above events during trying to connect to a WPA2 encrypted network (tried 3x and the same log patterns occur), the client connecting (in this case, my phone) disconnects itself after receive signal did a sudden degrade after the client initiated association.
I read that the deauthentication code 3 is that the Client initiates disconnection from the AP.

Given the case that the Client initiates disconnection, I tried to verify if manually disconnecting from the AP indeed has the same logs as above.
So using the AP without encryption, I tried to connect my phone to AP (successful/connected), then disconnect after a while my connection is established.
I had the same logs as above, except:

• "updated"=1 unlike in part (2) above:

Jan  1 02:14:49 localhost kernel: RTL8723AU: STA did not include WPA/RSN IE in (Re)Association Request - e
Jan  1 02:14:49 localhost kernel: RTL8723AU: bss_cap_update_on_sta_join, updated=1

• Had DHCP logs unlike in part (2) above:

Jan  1 02:14:50 localhost kernel: ======================update_attrib: get DHCP Packet 
Jan  1 02:14:50 localhost kernel: ======================update_attrib: get DHCP Packet 

• Signal is not dropping unlike in part (3) above:

Jan  1 02:14:50 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 35, rssi:-77, sig2
Jan  1 02:14:51 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 57, rssi:-66, sig1
Jan  1 02:14:52 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 72, rssi:-59, sig3
Jan  1 02:14:53 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 82, rssi:-54, sig6
Jan  1 02:14:54 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 88, rssi:-51, sig7
Jan  1 02:14:55 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 92, rssi:-49, sig4
Jan  1 02:14:56 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 95, rssi:-47, sig9
Jan  1 02:14:57 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 97, rssi:-46, sig3
Jan  1 02:14:58 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 98, rssi:-46, sig2
Jan  1 02:14:59 localhost kernel: RTL8723AU: rtw_signal_stat_timer_hdl signal_strength: 99, rssi:-45, sig2

• "new operation mode"=0x13 and "changes"=0 unlike in part (5) above.

Jan  1 02:21:29 localhost kernel: RTL8723AU: rtw_ht_operation_update current operation mode=0x13
Jan  1 02:21:29 localhost kernel: RTL8723AU: rtw_ht_operation_update new operation mode=0x13 changes=0

Note: Complete AP without encryption logs are at: https://gist.github.com/maggieroxas/9322890

So my questions are:

1. Why does the working setup (AP without encryption) have DHCP logs and the non-working setup (AP with WPA2 encryption) does not?
2. Why does the working setup (AP without encryption) have sudden-dropping signal just after connection initiation and the non-working setup (AP with WPA2 encryption) does not?
3. Why does the working setup (AP without encryption) and the non-working setup (AP with WPA2 encryption) have difference in updating parameters?

Thank you very much and sorry for the long comment.

[lwfinger]

Unfortunately, neither pcapdump not tcpdump are useful for the kind of logging we want. Both pick the packets off at a relatively high level in the stack. You need a device/driver combo that can go into promiscuous mode, and use either kismet or wireshark to capture the packets as they appear on the air. That is the only way we can see the primitive packets, acks, etc.

I have not yet had a chance to look at your log data, but I doubt that they will be of much help.

[ghost]

hi, Iwfinger. I know a device which uses rtl8723au as wifi device. Please visit: www.taobao.com, and search "radxa". It's a board for developpment. It's only $100.