I’m Renjie Wu

在 Mikrotik RouterOS 6.48.2 上配置 DHCPv6-PD（前缀代理）

2021-06-05T01:05:17-07:00

最近这段时间一直在折腾 10G 内网，先是买了 HPE 的 MicroServer Gen10 Plus 来做 NAS，之后又买了威联通的交换机 QSW-M2108R-2C。路由器则选用的是 Mikrotik 的 hEX S (RB760iGS)，而之前作为主力使用的 AX88U 变成了专职 AP。

把它们堆在一起就是下面这个样子：

废话不多说，直接进入正题。

配置 DHCPv6-PD 客户端

启用 DHCPv6 客户端

登入 RouterOS 终端，输入以下命令启用 DHCPv6 客户端：

# 接受来自 ISP 的路由器通告 (RA)，以通过 SLAAC 获取 IPv6 WAN 地址
/ipv6 settings set accept-router-advertisements=yes
# DHCPv6 客户端配置:
# - 添加默认路由: yes
# - WAN 端口: ether1
# - IPv6 地址池名称: delegation（可以随意起名，之后的命令里对应修改）
# - IPv6 地址池前缀长度：60（这是关键，下文会解释）
# - IPv6 前缀提示: 56（和 ISP 挂钩，根据实际情况修改）
# - IPv6 请求：WAN 地址和 LAN 前缀
/ipv6 dhcp-client add add-default-route=yes interface=ether1   \
  pool-name=delegation pool-prefix-length=60 prefix-hint=::/56 \
  request=address,prefix

IPv6 前缀提示取决于你的 ISP，以我的 ISP 为例，Charter Spectrum 提供 /56 的前缀，所以这里我使用的是 ::/56。

要使 SLAAC（或者说无状态 DHCPv6）正常工作，你需要从 ISP 那拿到至少 /63 的前缀（这种情况下可以划分两个 /64 的子网）。如果 ISP 最多只给分配 /64 的前缀，那就和前缀代理无缘了 —— 能用的方案只有 NAT66 和 NDPv6 代理。

这条命令中的关键是 pool-prefix-length 参数：它决定了下文中 DHCPv6 服务器下发前缀的长度（或者说二级路由器拿到的前缀的长度）。

这里我设置的是 60，在 ISP 分配给我 /56 前缀的前提下，hEX S 可以划分出 16 个 /60 的前缀给二级路由器使用，二级路由器则可以提供另外 16 个 /64 的前缀给它们的设备使用。

运行 /ipv6 dhcp-client print 和 /ipv6 pool print 来查看 ISP 分配的前缀：

[admin@MikroTik] > /ipv6 dhcp-client print
Flags: D - dynamic, X - disabled, I - invalid
 #    INTERFACE STATUS  REQUEST   PREFIX
 0    ether1    bound   address   2600:6c51:fake:n00b::/56, 4d22h24m10s
                        prefix
[admin@MikroTik] > /ipv6 pool print
Flags: D - dynamic 
 #   NAME       PREFIX                   PREFIX-LENGTH EXPIRES-AFTER
 0 D delegation 2600:6c51:fake:n00b::/56            60 4d22h23m52s

添加必要的 IPv6 防火墙规则

如果上面命令的 STATUS 一栏一直显示的是 searching，那么有可能是 IPv6 防火墙阻止了 UDP 端口 546 的连接入站。

运行下面的命令添加允许 UDP 546 入站的规则：

# 允许 DHCPv6 客户端尝试前缀代理
/ipv6 firewall filter add action=accept chain=input \
  comment="Accept DHCPv6 client prefix delegation"  \ 
  dst-port=546 protocol=udp src-address=fe80::/10

为 LAN 端口分配 IPv6 地址

最后一步是从 IPv6 地址池中为默认网桥分配 IPv6 地址：

# RouterOS 默认启用了 RA，这样下级设备才能通过 SLAAC 自动配置 IPv6 地址。
# （原因是 RouterOS 的 DHCPv6 目前暂不支持有状态 IPv6 地址分配）
# 如果有单独运行的有状态 DHCPv6 服务器，则应该在后面加上 `advertise=no`。
/ipv6 address add address=::1 from-pool=delegation interface=bridge

如果有多个网桥或端口，则需要对每个网桥和端口单独运行上面的命令。运行 /ipv6 address print 来查看分配到 LAN 端口的地址：

[admin@MikroTik] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local 
 #    ADDRESS                                 FROM-POOL  INTERFACE ADVERTISE
 0  G 2600:6c51:fake:n00b::1/64               delegation bridge    yes
 1 DL fe80::a55:31ff:fe00:fake/64                        bridge    no
 2 DL fe80::a55:31ff:fe00:fake/64                        ether1    no
 3 DG 2600:6c51:some:fake:addr:why:so:long/64            ether1    no

配置 DHCPv6-PD 服务器

注意 1：要使用 DHCPv6-PD 服务器，你需要从 ISP 那拿到至少 /63 的 IPv6 前缀。

注意 2： 为了能让二级路由器通过 DHCPv6-PD 拿到前缀，上一节 DHCPv6 客户端中的 pool-prefix-length 参数必须正确设置。

开始之前，先重申一下 Mikrotik 文档中的说明：

Note: RouterOS DHCPv6 server can only delegate IPv6 prefixes, not addresses.

Mikrotik Documentation — Manual:IPv6/DHCP Server

换句话说，RouterOS 的 DHCPv6 服务器目前只支持 SLAAC（无状态 DHCPv6）。

启用 DHCPv6 服务器

下面的命令用于创建 DHCPv6 服务器：

# 从地址池 `delegation` 中代理 IPv6 地址
# 租约有效期为 1 天
/ipv6 dhcp-server add address-pool=delegation \
  interface=bridge lease-time=1d name=default

如果有多个网桥或端口，则需要对每个网桥和端口单独运行上面的命令。

现在就可以运行 /ipv6 dhcp-server binding print 来检查二级路由器有没有拿到前缀：

[admin@MikroTik] > /ipv6 dhcp-server binding print
Flags: I - invalid, X - disabled, R - radius, D - dynamic 
 #    ADDRESS                   DUID            SERVER   STATUS
 0  D 2600:6c51:fake:n01b::/60  0xa85e45fakeid  default  bound  

如果看到类似上面的输出，那就说明 DHCPv6 服务器已经正常工作了。

配置 DHCPv6 选项

如果想通过 DHCPv6 下发诸如 DNS (Option 23) 和域搜索列表 (Option 24) 等选项，则需要先在 IPv6 ND 中启用 Other Configuration：

/ipv6 nd set [ find default=yes ] other-configuration=yes

然后就可以设置 DHCPv6 选项了，这里我创建了 DNS 和域搜索列表两个选项：

/ipv6 dhcp-server option
# Option 23 (DNS): fd00::1
add code=23 name=dns value=0xfd000000000000000000000000000001
# Option 24 (域搜索列表): example.local
add code=24 name=domain-search value="0x07'example'0x05'local'0x00"

DNS (Option 23) 需要打上完整的 64 位十六进制 IPv6 地址，域搜索列表 (Option 24) 则可以使用这个工具生成：https://jjjordan.github.io/dhcp119/。

我这里的 DNS 设置的是 fd00::1，这样不需要额外的脚本就能让 hEX S 作为内网的默认 DNS （否则的话需要使用脚本来获取公网 IPv6 地址）。

这个方法来源于 dksoft。不过对于我的应用场景而言，只需要一条命令：

# 为 `bridge` 分配 `fd00::1`，不启用 RA
/ipv6 address add address=fd00::1 advertise=no interface=bridge

这里没有启用 RA，因为不想也没必要让设备拿到 fd00::1/8 前缀的地址。

最后一步就是把这些选项添加到 DHCPv6 服务器上：

# 根据你的设置对应修改服务器名称和选项名称
/ipv6 dhcp-server set default dhcp-options=dns,domain-search

总结

RouterOS 的学习曲线还是有些陡峭的，但是一旦搞明白怎么回事，属实比在 AX88U 上做相同的事来的轻松的多（感觉在讲废话，毕竟 AX88U 是针对消费市场的）。

无论如何，相对于 AX88U 这种多合一的设备，确实是发现还是专事专做比较好，尤其是有那种特殊或者专业需求的时候。

Configure DHCPv6-PD (Prefix Delegation) Client & Server on Mikrotik RouterOS 6.48.2

2021-05-24T23:19:58-07:00

For the past few months, I invested some bucks on my high-speed network project. I’ve got a HPE ProLiant MicroServer Gen10 Plus serving as the NAS, and a 10GbE switch QSW-M2108R-2C from QNAP as the network backbone. On top of them, I’m using a Mikrotik hEX S (RB760iGS) as my main router. And you might have guessed, my AX88U now serves as an AP.

In case of your curiosity, this is how my gears stack now:

Well enough talking, let’s go into the topic.

Set up DHCPv6-PD client

Enable DHCPv6 client

DHCPv6 client can be enabled with:

# Accept RA from ISP, so the router can get WAN address using SLAAC
/ipv6 settings set accept-router-advertisements=yes
# Configure DHCPv6 client:
# - Add default route: yes
# - WAN interface: ether1
# - IPv6 address pool: delegation (You can change to whatever you want)
# - Prefix length for DHCPv6 server's offering cascading routers: 60
# - Prefix-hint: 56 (This value may vary, depending on your ISP)
# - Request both WAN address and LAN prefix from ISP
/ipv6 dhcp-client add add-default-route=yes interface=ether1   \
  pool-name=delegation pool-prefix-length=60 prefix-hint=::/56 \
  request=address,prefix

Depending on your ISP, the prefix hint may vary. My ISP Charter Spectrum offers /56 prefixes, so I’m using ::/56 here.

Note that in order to make SLAAC (or DHCPv6 stateless) work, you’ll need a prefix at least shorter than /64. If your ISP won’t offer you a prefix shorter than /64, you can’t do subnetting (or more precisely, offering prefixes to cascading routers) without breaking IPv6, unless you wanna try the dirty NAT66.

The key parameter is the pool-prefix-length. This value determines the prefix length, or the subnet size, that the DHCPv6 server will offer for cascading routers.

For me, 60 is pretty reasonable: the main router is able to offer 16 /60 prefixes for cascading routers, and the cascading routers can offer another 16 /64 prefixes for their clients.

Now you can check if prefix is assigned with /ipv6 dhcp-client print and /ipv6 pool print:

[admin@MikroTik] > /ipv6 dhcp-client print
Flags: D - dynamic, X - disabled, I - invalid
 #    INTERFACE STATUS  REQUEST   PREFIX
 0    ether1    bound   address   2600:6c51:fake:n00b::/56, 4d22h24m10s
                        prefix
[admin@MikroTik] > /ipv6 pool print
Flags: D - dynamic 
 #   NAME       PREFIX                   PREFIX-LENGTH EXPIRES-AFTER
 0 D delegation 2600:6c51:fake:n00b::/56            60 4d22h23m52s

Add necessary IPv6 firewall rule

If the status keeps showing searching, check IPv6 firewall to see whether incoming packets to UDP port 546 (DHCPv6 client) is allowed. If not, this rule can be added via:

# Accept DHCPv6 client prefix delegation
/ipv6 firewall filter add action=accept chain=input \
  comment="Accept DHCPv6 client prefix delegation"  \ 
  dst-port=546 protocol=udp src-address=fe80::/10

Assign IPv6 address to interfaces

The last step is to assign IPv6 address on internal interface from address pool:

# RA is enabled by default. Clients connected to this interface 
# can get their IPv6 addresses via SLAAC. If you don't want clients
# configure themselves automatically or if you are running a separate 
# stateful DHCPv6 server, disable advertise with `advertise=no`
/ipv6 address add address=::1 from-pool=delegation interface=bridge

If you get multiple interfaces, run the above command on each interface separately. Use /ipv6 address print to check assigned addresses:

[admin@MikroTik] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local 
 #    ADDRESS                                 FROM-POOL  INTERFACE ADVERTISE
 0  G 2600:6c51:fake:n00b::1/64               delegation bridge    yes
 1 DL fe80::a55:31ff:fe00:fake/64                        bridge    no
 2 DL fe80::a55:31ff:fe00:fake/64                        ether1    no
 3 DG 2600:6c51:some:fake:addr:why:so:long/64            ether1    no

Set up DHCPv6-PD server

Note 1: Like I’ve mentioned before, you need a prefix at least shorter than /64 from your ISP to make DHCPv6-PD server works!

Note 2: In order to delegate prefixes to cascading routers, you have to assign a pool-prefix-length in RouterOS DHCPv6 client. Check the previous section for details.

Before started, I want to re-state the note (at least at the time of writing) from Mikrotik’s documentation:

Note: RouterOS DHCPv6 server can only delegate IPv6 prefixes, not addresses.

Mikrotik Documentation — Manual:IPv6/DHCP Server

In other words, DHCPv6 server in RouterOS is only capable of SLAAC (or stateless DHCPv6). There is no support for stateful DHCPv6.

Enable DHCPv6 server

The following command creates a DHCPv6 server:

# Delegate IPv6 addresses from pool `delegation`
# Lease is valid for 1 day
/ipv6 dhcp-server add address-pool=delegation \
  interface=bridge lease-time=1d name=default

Again, if you get multiple interfaces, run the above command on each interface separately.

Now you can check if cascading routers are assigned prefixes with /ipv6 dhcp-server binding print:

[admin@MikroTik] > /ipv6 dhcp-server binding print
Flags: I - invalid, X - disabled, R - radius, D - dynamic 
 #    ADDRESS                   DUID            SERVER   STATUS
 0  D 2600:6c51:fake:n01b::/60  0xa85e45fakeid  default  bound  

Congratulations! Your DHCPv6 services on RouterOS are working now.

Advertise DHCPv6 options

… though there could be one more question: what if I want to configure DHCPv6 options (DNS, search domains, etc.)?

First you need to allow Other Configuration in IPv6 ND (neighbor discovery):

/ipv6 nd set [ find default=yes ] other-configuration=yes

Then we can set up DHCPv6 options. Here I’m creating option 23 (DNS) and 24 (domain search list):

/ipv6 dhcp-server option
# Option 23 (DNS): fd00::1
add code=23 name=dns value=0xfd000000000000000000000000000001
# Option 24 (domain search list): example.local
add code=24 name=domain-search value="0x07'example'0x05'local'0x00"

For option 23 (DNS), you need to assign a full 64-bit address in hexadecimal.

To grab the value of DHCPv6 option 24 (domain search list), use this tool: https://jjjordan.github.io/dhcp119/.

You may have noticed I’m using a ULA address fd00::1 for option 23. This allows me to assign my Mikrotik router as the default DNS without scripting.

The idea comes from dksoft. But for my application, I only need one command (to assign fd00::1):

# Assign `fd00::1` to interface `bridge` without RA
/ipv6 address add address=fd00::1 advertise=no interface=bridge

Note that RA is disabled, because I don’t want clients get SLAAC addresses from fd00::1/8 prefix.

The last step is to assign these options to DHCPv6 server:

# Server name and dhcp-option names should match
# what you have set above
/ipv6 dhcp-server set default dhcp-options=dns,domain-search

Conclusion

I’d say once you figure out how RouterOS works, it is much less painful than doing the same things on AX88U, though it might be unfair to AX88U since it is designed for consumer market.

So what have I learned? Well, if you have special requirements to fullfil, instead of going for all-in-one products (for example AX88U in this case: you get a router, a switch, and an AP), buy products that focuses on specific applications.

Trust me, it’s much more painless!

Don’t use STL `regex` library in Microsoft Visual C++ (MSVC), at least for now

2021-02-28T23:08:12-08:00

I’ve been working on the new research projects for several days. The first task is pretty simple: load some datasets in UCR Time Series Archive into memory.

In our previous papers¹^,², since the original authors’ algorithms were done in Java (FastDTW³, TEASER⁴, etc.), I’ve mainly written Java for related experiments. I haven’t touched C++ for a while, so this time I decided to use C++, but I never expected to pay the “price” for my choice.

How could it go wrong with `\t+`?

The file format of UCR Time Series Archive is pretty straightforward: n rows of time series in each dataset, and for each time series, there are m columns separated by TAB (\t), representing m datapoints.

C++ <string> library didn’t come with a built-in splitting function, and I don’t really want to use 3rd-party libraries (like boost::split) in my little project. It could be very painful to use external libraries in C++, and when it comes to compatibility issues, oh man good luck to you.

I know I could make it with string::find, but hey it’s year 2021, C++ 20 was finalized last September, and I believe there must be fancier way than tedious string::find, so I came up with an idea: searching “How to split string in modern C++” on Stack Overflow.

And I did get a hit: Split a string using C++ 11, a question asked 9 years ago. The most voted answer showed an “elegant” way using std::regex_token_iterator.

So I wrote down this piece of code in Visual Studio 2019 (MSVC v14.28.29333, the latest version at the time of writing):

typedef std::vector<std::pair<int, std::vector<double>>> TimeSeriesDataset;

TimeSeriesDataset readTSV(const std::string& tsvPath) {
    TimeSeriesDataset dataset;

    std::ifstream stream{ tsvPath };
    std::string line;
    static const std::regex split(R"(\t+)");

    while (std::getline(stream, line)) {
      std::cregex_token_iterator first{ line.data(), line.data() + line.length(), split, -1 }, last;

      int label;
      std::from_chars(first->first, first->second, label);

      std::vector<double> datapoints;
      std::transform(++first, last, std::back_inserter(datapoints), [](auto subMatch) {
          double value;
          std::from_chars(subMatch.first, subMatch.second, value);
          return value;
      });

      dataset.push_back(std::make_pair(label, datapoints));
  }

  return dataset;
}

Note that I wrote a regular expression of \t+. The + sign is in fact not necessary in my case, but at that time I didn’t think too much about this simple expression. Consider the DFA only contains two states:

How could it possibly go wrong?

Bad `<regex>` implementation in MSVC

`++regex_token_iterator` takes too long

When I hit F5 to launch the program, I immediately noticed there is something wrong: it took way too long to load Mallat_TEST.tsv. I won’t consider this file to be big: essentially a 2345 x 1025 matrix of decimal, and my past experience in Java showed that a second should be long enough to load this file.

So I started the performance profiler to see what happened. To my surprise, it takes 6 full minutes (under Debug (Win32) configuration) to finish running the program on Mallat_TEST.tsv.

The hot path analysis reveals which operations hold the most CPU time (click picture to enlarge):

Apparently most of the time was spent in std::transform, which implicitly refers to std::regex_token_iterator::operator++.

Following the call tree reveals creating, deleting, and assigning object std::_Tgt_state_t in function std::_Matcher::_Do_if caused the problem:

Also if we sort used CPU time by functions, it reveals the same issue:

This means there must be something wrong with std::_Tgt_state_t.

What indeed happened?

So I looked into <regex>’s source code for _Matcher::_Do_if. The first glance looks pretty normal:

// IMPLEMENTATION OF _Matcher
template <class _BidIt, class _Elem, class _RxTraits, class _It>
bool _Matcher<_BidIt, _Elem, _RxTraits, _It>::_Do_if(_Node_if* _Node) { // apply if node
    _Tgt_state_t<_It> _St = _Tgt_state;

    // look for the first match
    for (; _Node; _Node = _Node->_Child) { // process one branch of if
        _Tgt_state = _St; // rewind to where the alternation starts in input
        if (_Match_pat(_Node->_Next)) { // try to match this branch
            break;
        }
    }

    // if none of the if branches matched, fail to match
    if (!_Node) {
        return false;
    }

    // if we aren't looking for the longest match, that's it
    if (!_Longest) {
        return true;
    }

    // see if there is a longer match
    _Tgt_state_t<_It> _Final = _Tgt_state;
    auto _Final_len          = _STD distance(_St._Cur, _Tgt_state._Cur);
    for (;;) { // process one branch of if
        _Node = _Node->_Child;
        if (!_Node) {
            break;
        }

        _Tgt_state = _St;
        if (_Match_pat(_Node->_Next)) { // record match if it is longer
            const auto _Len = _STD distance(_St._Cur, _Tgt_state._Cur);
            if (_Final_len < _Len) { // memorize longest so far
                _Final     = _Tgt_state;
                _Final_len = _Len;
            }
        }
    }

    // set the input end to the longest match
    _Tgt_state = _Final;
    return true;
}

The variable _Tgt_state is a member of class _Matcher.

However, if we take a closer look on how _Tgt_state is assigned and other temporary _Tgt_state_t variables _St and _Final are constructed:

_Tgt_state_t<_It> _St = _Tgt_state;
_Tgt_state = _St;
_Tgt_state_t<_It> _Final = _Tgt_state;
_Final     = _Tgt_state;
_Tgt_state = _Final;

All these assignments are implicitly invoking operator=, and the default operator= performs a member-wise copy, which could lead to deep copy if there are STL containers without special handling mechanism.

A later look at the definition of _Tgt_state_t proves my thought:

// CLASS TEMPLATE _Bt_state_t
template <class _BidIt>
class _Bt_state_t { // holds the state needed for backtracking
public:
    _BidIt _Cur;
    vector<bool> _Grp_valid;
};

// CLASS TEMPLATE _Tgt_state_t
template <class _BidIt>
class _Tgt_state_t : public _Bt_state_t<_BidIt> { // holds the current state of the match
public:
    struct _Grp_t { // stores a pair of iterators
        _BidIt _Begin;
        _BidIt _End;
    };

    vector<_Grp_t> _Grps;

    void operator=(const _Bt_state_t<_BidIt>& _Other) {
        static_cast<_Bt_state_t<_BidIt>&>(*this) = _Other;
    }
};

Class _Tgt_state_t has two member variables: _Grps and _Grp_valid (inherited from _Bt_state_t). Both of them are STL container vector, and by default, performing operator= over vector incurs deep copy.

Deep copy is already evil. To make it worse, the hot path reveals _Do_if would be called recursively (_Match_pat calls _Do_if and _Do_if calls _Match_pat). This means _Tgt_state_t variable _St and _Final are created and deleted every single time when _Do_if is called.

Remember class _Tgt_state_t has two vector member variables, and by default vector requires memory allocation/deallocation on heap. Normally this overhead seems neglectable, but under the circumstance of recursion, it could be a problem. The CPU time used by vector constructor/deconstructor proves:

So how can it be fixed? Well, in the very beginning, I thought how hard could it be? But after all the deep digging into source code, it could be very hard and might require a complete refactoring.

Speed under `Release` configuration

I decided to give MSVC one more chance, by compiling the code under Release configuration, or more precisely, with compiling options /O2 (maximum optimizations, favor speed) and /Ob2 (inline expansion level 2).

I wrote a small snippet to measure how long my readTSV function takes:

auto start = std::chrono::steady_clock::now();
auto dataset = readTSV(R"(../UCRArchive_2018/Mallat/Mallat_TEST.tsv)");
auto end = std::chrono::steady_clock::now();

std::cout << "Mallat_TEST[0], label: " << dataset.front().first << ", data[0, last]: " 
          << dataset.front().second.front() << ", " << dataset.front().second.back() << std::endl;
std::cout << "Mallat_TEST[last], label: " << dataset.back().first << ", data[0, last]: " 
          << dataset.back().second.front() << ", " << dataset.back().second.back() << std::endl;

std::cout << "Elapsed: " 
          << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() 
          << " ms" << std::endl;

The reason I added two more lines output is to avoid compiler’s optimization over potential dead code (clang does very well in this kind of elimination), which may remove the invocation to readTSV.

Now I can compile this little program:

# /nologo: Suppress copyright message
# /TP: Treat all files as C++
# /W3: Warning level 3
# /GR: Enable C++ RTTI
# /EHsc: Enable C++ EH
# /MD: Link with MSVCRT.LIB
# /O2: Maximum optimizations (favor speed)
# /Ob2: Inline expansion level 2
# /NDEBUG: Define NDEBUG marco
# /std:c++17: C++ 17 stardard
cl /nologo /TP /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG /std:c++17 RegexTokenIterator_C++17.cpp

Yet it still took 31.4 seconds to load Mallat_TEST.tsv:

# MSVC v14.28.29333 (/O2 /Ob2) with \t+
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 31461 ms

How about plain `\t`

I also tried the simplest regex for this case: \t. With the same compiler options, it still takes 4.6 seconds:

# MSVC v14.28.29333 (/O2 /Ob2) with \t
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 4641 ms

which is way less desirable and 5x slower than expected.

What about `clang` and `gcc`?

`clang` 11.0.1 + `libc++` 11.0

When I first tried to compile with:

clang++ -O3 -std=c++17 -stdlib=libc++ RegexTokenIterator_C++17.cpp -o clangRTI

clang spits out an error:

RegexTokenIterator_C++17.cpp:34:4: error: call to deleted function 'from_chars'
                        std::from_chars(subMatch.first, subMatch.second, value);
                        ^~~~~~~~~~~~~~~
/usr/local/clang-11.0.1/bin/../include/c++/v1/algorithm:1948:21: note: in instantiation of function template specialization 'readTSV(const std::string &)::(anonymous class)::operator()<std::__1::sub_match<const char *>>' requested here
        *__result = __op(*__first);
                    ^
RegexTokenIterator_C++17.cpp:32:8: note: in instantiation of function template specialization 'std::__1::transform<std::__1::regex_token_iterator<const char *, char, std::__1::regex_traits<char>>, std::__1::back_insert_iterator<std::__1::vector<double, std::__1::allocator<double>>>, (lambda at RegexTokenIterator_C++17.cpp:32:65)>' requested here
                std::transform(++first, last, std::back_inserter(datapoints), [](auto subMatch) {
                     ^
/usr/local/clang-11.0.1/bin/../include/c++/v1/charconv:123:6: note: candidate function has been explicitly deleted
void from_chars(const char*, const char*, bool, int = 10) = delete;
     ^
/usr/local/clang-11.0.1/bin/../include/c++/v1/charconv:606:1: note: candidate template ignored: requirement 'is_integral<double>::value' was not satisfied [with _Tp = double]
from_chars(const char* __first, const char* __last, _Tp& __value)
^
/usr/local/clang-11.0.1/bin/../include/c++/v1/charconv:613:1: note: candidate function template not viable: requires 4 arguments, but 3 were provided
from_chars(const char* __first, const char* __last, _Tp& __value, int __base)
^
1 error generated.

I was confused and “What?” is my first reaction. So I digged into libc++’s <charconv> and the only 3 definitions of std::from_chars I found are:

void from_chars(const char*, const char*, bool, int = 10) = delete;

template <typename _Tp, typename enable_if<is_integral<_Tp>::value, int>::type = 0>
inline _LIBCPP_INLINE_VISIBILITY from_chars_result
from_chars(const char* __first, const char* __last, _Tp& __value)
{
    return __from_chars_atoi(__first, __last, __value);
}

template <typename _Tp, typename enable_if<is_integral<_Tp>::value, int>::type = 0>
inline _LIBCPP_INLINE_VISIBILITY from_chars_result
from_chars(const char* __first, const char* __last, _Tp& __value, int __base)
{
    _LIBCPP_ASSERT(2 <= __base && __base <= 36, "base not in [2, 36]");
    return __from_chars_integral(__first, __last, __value, __base);
}

Meh. This means at the time of writing, libc++ doesn’t support floating point numbers in std::from_chars.

So I had to replace the std::from_chars in the std::transform lambda with the old school strtod, and it worked.

The result is much better than MSVC’s, but still slow:

# clang 11.0.1 (-O3) + libc++ 11.0 with \t+
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 3017 ms

# clang 11.0.1 (-O3) + libc++ 11.0 with \t
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 1848 ms

`gcc` 5.5.0 + `libstdc++` 3.4.21

I could have use the latest gcc 10.2, but I am just too lazy to compile it.

Fortunately gcc 5.5.0 + libstdc++ 3.4.21 has already fully supported C++ 14 features, so at least I can continue my experiments by simply replacing the last C++ 17 existence of std::from_chars to atoi.

The complier options are:

g++ -O3 -std=c++14 RegexTokenIterator_C++14.cpp -o g++RTI

I’m a little bit amazed by the result:

# gcc 5.5.0 (-O3) + libstdc++ 3.4.21 with \t+
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 992 ms

# gcc 5.5.0 (-O3) + libstdc++ 3.4.21 with \t
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 952 ms

Because both MSVC and clang + libc++ are in their latest stable version. For gcc + libstdc++, I was using an old version built on Oct. 2017, which I’d say it’s kind of unfair, but gcc + libstdc++ managed to win the game.

Tedious `string::find` did the trick

Nevertheless, I went back to the tedious way of string::find (let us also forget about whatever std::from_chars is):

typedef std::vector<std::pair<int, std::vector<double>>> TimeSeriesDataset;

TimeSeriesDataset readTSV(const std::string& tsvPath) {
    TimeSeriesDataset dataset;

    std::ifstream stream{ tsvPath };
    std::string line;

    while (std::getline(stream, line)) {
      auto tabPos = line.find('\t');
      auto label = atoi(line.data());
      
      std::vector<double> datapoints;
      for (size_t nextTabPos; 
           std::string::npos != (nextTabPos = line.find('\t', tabPos + 1)); 
           tabPos = nextTabPos) {
        datapoints.push_back(strtod(line.data() + tabPos + 1, nullptr));
      }
      datapoints.push_back(strtod(line.data() + tabPos + 1, nullptr));

      dataset.push_back(std::make_pair(label, datapoints));
  }

  return dataset;
}

I’m not surprised to see this piece of code is way more faster than its regular expression equivalent under all three libraries:

# MSVC v14.28.29333 (/O2 /Ob2)
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 892 ms

# clang 11.0.1 (-O3) + libc++ 11.0
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 431 ms

# gcc 5.5.0 (-O3) + libstdc++ 3.4.21
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 294 ms

Final speed comparison

With all the experiment data collected, we can finally compare the performance:

Compiler (Options)	Time (ms) (`rti`^ with `\t+`)	Time (ms) (`rti`^ with `\t`)	Time (ms) (`string::find`)
MSVC v14.28.29333 (`/O2 /Ob2`)	31461	4641	892
clang 11.0.1 (`-O3`) libc++ 11.0	3017	1848	431
gcc 5.5.0 (`-O3`) libstdc++ 3.4.21	992	952	294

^ rti is std::regex_token_iterator.

The result shows that at least for <regex>, libstdc++’s implementation is so far the best among all three libraries. But it still can’t beat the plain and tedious string::find, in terms of string splitting.

I tried to figure out why <regex> has been introduced for a decade and yet still so slow and badly implemented. A thread on Reddit points out something about ABI compatibility:

The fast engines add a zillion special cases for common patterns their engines recognize. But we can’t ever do that. And given that our engines were somewhat stupid initially now we can’t replace the engine with something better because that breaks ABI.

BillyONeal — std::regex_replace/std::chrono::high_resolution_clock::now() speed

Well, good luck to all of us C++ users.

Conclusion

It’s great to see the C++ standard getting periodical updates every 3 years since 2011: new STL components, syntax sugars, and a lot of other stuffs (range-based for loops, auto type deduction, variadic template (or parameter pack), and concepts finally, etc.) — all of which I (and probably many other people) have been waiting for years.

However, <regex> library is unfortunately an exception: apparently there is something wrong with MSVC’s underlying implementations, and the performance is surprisingly bad. Also not to mention <regex> library was introduced in C++ 11 since 2011 — a decade ago.

The lesson I learnt is that for simple tasks like splitting strings, it might be better just to stick with simple tools (string::find in this case, even though I have to reinvent the wheels every single time), instead of using something “fancy” like std::regex_token_iterator, at least for now.

Actually this is NOT the first time that I found issues in the particular STL implementation: several years ago, I noticed that the order of items from std::make_heap is slightly different between MSVC’s and GCC’s (or libstdc++). My later digging into source code revealed it was caused by the default comparison function: one used < and the other used <=.

Another issue is that specific feature mights not be fully implemented. Besides aforementioned libc++’s std::from_chars lacking support on floating point numbers, when I was trying to compile the cppwin32 project with Visual Studio 2017 (MSVC v14.16.27023), I found that MSVC’s std::to_chars has no support on floating-point in terms of std::chars_format::hex⁵. This made me upgrade to Visual Studio 2019.

All these chaos remind me the famous Russian proverb my advisor Dr. Keogh ever cited: “Доверя́й, но проверя́й” or “Trust, but verify”. Even standards like STL are not exceptions.

My experiences in C++ in the past decade has taught me one thing:

C++ is very hard, and to make it correct is even harder.

References

R. Wu and E.J. Keogh, “FastDTW is approximate and Generally Slower than the Algorithm it Approximates,” IEEE Transactions on Knowledge and Data Engineering (TKDE), in press, 2020. ↩
R. Wu, A. Der, and E.J. Keogh, “When is Early Classification of Time Series Meaningful?,” arXiv, preprint, 2021. ↩
S. Salvador and P. Chan, “FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space,” Intelligent Data Analysis, vol. 11, no. 5, 2007, pp. 561-580. ↩
P. Schäfer and U. Leser, “TEASER: Early and Accurate Time Series Classification,” Data Mining and Knowledge Discovery, vol. 34, no. 5, pp. 1336-1362, 2020. ↩
Microsoft, Microsoft C++ language conformance table: charconv. ↩

在 AX88U + Merlin 384.19 上配置基于 `RTL8156B` 的 2.5GbE 网卡

2021-01-09T22:32:34-08:00

主路由升级到 AX88U 已经有接近一年的时间，整体来说这台路由器非常不错：性能强悍、功能齐全。不过缺少板载的 2.5GbE 网卡属实为一大遗憾，尤其是考虑到其 CPU BCM49408 内建了对 2.5GbE 的支持¹（华硕后来的 2.5GbE 产品 AX86U 就是基于这款 CPU 的）。

不过好消息是螃蟹家（Realtek）提供了一个经济的解决方案 —— RTL8156²，基于 USB 3.0 的 2.5GbE 控制器。相比于贵的多的 5GbE 和 10GbE 的设备，它可以说是“相当廉价”了：30 美金左右就能买到基于 RTL8156 的 USB 网卡。

但是有一点需要特别注意，这个芯片有两个版本：RTL8156 和 RTL8156B。我推荐购买后者这个 B 修订版的 RTL8156B，因为前者 RTL8156 在用上一段时间后会有些奇怪的稳定性问题³。

此外如果买的是 USB-C 的网卡，我建议买一个“双面” USB-C 转 USB-A 的转接器（里面应该有个多路复用芯片，比如 VL160⁴ 之类的）。不然使用非“双面”的转接器，必须搞清楚把 USB-C 头的哪一面插入到转接器里，因为只有一面能跑 USB 3.0，另一面只能跑 USB 2.0。

不能即插即用？

一切正常的话，AX88U 应该是能识别插入的 USB 网卡（在路由器管理界面的网络地图里会显示）。另外运行 ifconfig -a，也能发现网卡已经被注册为 usb0。

然而一旦尝试启用网卡（ifconfig usb0 up），就会发现日志会被 cdc_ncm 这个模块刷屏：

cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink

造成这个问题的原因是 Linux 内核自带的驱动程序暂时还不支持 RTL8156 ⁵，所以必须得自己编译驱动。

准备工作

编译 Linux 内核

有关如何配置梅林固件编译环境，请参阅这篇教程：Compiling under WSL2（英文）。虽然标题写的是在 WSL (Windows Subsystem for Linux) 2 里编译，但是总体步骤也适用于 WSL 1（也称 legacy WSL）和真·Linux 环境。

如果你跟我一样用的是 WSL 1，还需要额外配置 32 位运行环境（参考这个 issue），因为梅林固件提供的工具链都是 32 位的。

首先把梅林固件的源代码拖下来，因为用不到 git 的历史，只需要把 HEAD 拖下来即可：

# 只拖 HEAD
git clone --depth 1 https://github.com/RMerl/asuswrt-merlin.ng

教程里是把 brcm-arm-sdk 文件夹链接到了 src-rt-6.x-4708 里面：

# `src-rt-6.x.4708` 用于 AC56U, AC68U 和 AC87U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-6.x.4708/toolchains

而我们需要把它链接到 src-rt-5.02axhnd 里面，梅林固件的编译脚本⁶里有提到 AX88U 用的是 src-rt-5.02axhnd：

# `src-rt-5.02axhnd` 用于 AX88U, AX58U 和 AX56U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/toolchains

接着就可以编译 AX88U 的梅林固件了：

# 目标平台 AX88U
cd release/src-rt-5.02axhnd && make rt-ax88u

由于我们只需要用到内核编译生成的头文件（.h），因此没必要把固件整个都编译出来，待内核编译流程开始之后，就可以按键盘上的 CTRL+C 终止固件编译。

编译测试工具

`iperf3`

iperf3 源代码从这里下载：esnet/iperf，本文使用的 iperf3 版本是 3.9。

运行下面的命令来为 AX88U 编译 iperf3：

# 创建 build 文件夹，作为 iperf3 安装目录
mkdir build
# 静态链接
./configure --host=arm-linux --prefix=`pwd`/build --enable-static --disable-shared
# 编译并安装到 build 文件夹
make && make install

生成的 iperf3 二进制文件应该可以在 ./build/bin 文件夹下找到，预编译好的版本可以在这里下载：附录：预编译的二进制文件：预编译的测试工具。

`ethtool`

ethtool 源代码从这里下载：kernel/software/network/ethtool，本文使用的 ethtool 版本是 5.8。

运行下面的代码来为 AX88U 编译 ethtool：

# 创建 build 文件夹，作为 ethtool 安装目录
mkdir build
# 禁用 netlink，一是用不到，二是去除对 libmnl 的依赖
./configure --host=arm-linux --prefix=`pwd`/build --disable-netlink
# 编译并安装到 build 文件夹
make & make install

生成的 ethtool 二进制文件应该可以在 ./build/bin 文件夹下找到，预编译好的版本可以在这里下载：附录 1：预编译的二进制文件：预编译的测试工具。

编译 `r8152` 驱动模块

你没看错，RTL8156B 的驱动模块确实叫 r8152，源码从这里下载：Realtek USB FE / GBE / 2.5G / Gaming Ethernet Family Controller Software，本文使用的 r8152 驱动模块版本是 v2.13.0。

为了验证买来的网卡是 RTL8156B 芯片的，可以在 rtl8152_get_drvinfo 函数里加上下面一句代码，这样就可以通过 ethtool 来获取芯片的版本：

--- source-2.13.0/r8152-original.c     2020-04-20 01:36:49.000000000 -0700
+++ source-2.13.0/r8152-modified.c     2020-10-06 23:47:07.123659400 -0700
@@ -16369,6 +16369,7 @@
        strlcpy(info->driver, MODULENAME, sizeof(info->driver));
        strlcpy(info->version, DRIVER_VERSION, sizeof(info->version));
        usb_make_path(tp->udev, info->bus_info, sizeof(info->bus_info));
+       snprintf(info->fw_version, sizeof(info->fw_version), "0x%04x", tp->version);
 } 

 #if LINUX_VERSION_CODE < KERNEL_VERSION(4,20,0)

运行下面的命令来编译驱动：

# 架构: arm64
# 交叉编译器: aarch64-linux
# 内核源代码目录: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 M=`pwd` modules

如果你倾向用 Makefile 的话，可以在这里找到：附录 2：脚本：Makefile。

一切顺利的情况下，r8152.ko 这个文件会生成在当前目录下，预编译好的版本可以在这里下载：附录：预编译的二进制文件：预编译的驱动模块。

配置驱动

首先在路由器 /jffs 文件夹内创建 drivers 和 tools 两个文件夹，然后把 r8152.ko 上传到路由器 /jffs/drivers 文件夹内，把编译好的 iperf3 和 ethtool 上传到路由器 /jffs/tools 文件夹内。

加载驱动

前文“不能即插即用？”一节已经提到 cdc_ncm 会接管 USB 网卡，所以首先需要把 cdc_ncm 卸载掉。

运行 lsmod | grep cdc_ncm 可以看到依赖 cdc_ncm 的模块：

# admin@ax88u:/# lsmod | grep cdc_ncm
cdc_ncm                16787  1 cdc_mbim
usbnet                 21074  7 cdc_mbim,qmi_wwan,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix
usbcore               166572 23 uas,usb_storage,cdc_mbim,qmi_wwan,cdc_wdm,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix,cdc_acm,usbnet,usblp,ohci_pci,ohci_platform,ohci_hcd,ehci_pci,ehci_platform,ehci_hcd,xhci_pci,xhci_plat_hcd,xhci_hcd

所以得把 cdc_mbim 也一同卸载掉：

# 先卸载 cdc_mbim，因为它依赖于 cdc_ncm
rmmod cdc_mbim
# 卸载 cdc_ncm
rmmod cdc_ncm

现在就可以加载 r8152.ko 了，同时记得把 cdc_ncm 和 cdc_mbim 也加载回来：

# 加载 r8152 驱动模块
insmod /jffs/drivers/r8152.ko
# 加载 cdc_ncm
insmod cdc_ncm
# 加载 cdc_mbim
insmod cdc_mbim

如果一切正常的话，系统日志（tail /tmp/syslog.log）会显示驱动加载成功：

# admin@ax88u:/# tail /tmp/syslog.log
Oct  7 21:27:42 kernel: usbcore: deregistering interface driver cdc_mbim
Oct  7 21:27:42 kernel: usbcore: registered new interface driver r8152
Oct  7 21:27:42 kernel: usb 2-2: reset SuperSpeed USB device number 2 using xhci-hcd
Oct  7 21:27:42 kernel: netif_napi_add() called with weight 256 on device eth%d
Oct  7 21:27:42 kernel: r8152 2-2:1.0 eth8: v2.13.0 (2020/04/20)
Oct  7 21:27:42 kernel: r8152 2-2:1.0 eth8: This product is covered by one or more of the following patents:
Oct  7 21:27:42 kernel: 		US6,570,884, US6,115,776, and US6,327,625.
Oct  7 21:27:42 hotplug: add net eth8.
Oct  7 21:27:42 kernel: usbcore: registered new interface driver cdc_ncm
Oct  7 21:27:42 kernel: usbcore: registered new interface driver cdc_mbim

从上面的日志里可以看到 USB 网卡被注册为 eth8，运行 ifconfig eth8 可以验证这一点：

eth8      Link encap:Ethernet  HWaddr 00:E0:4C:00:FA:KE
          inet6 addr: fe80::2e0:4cff:fe00:fake/64 Scope:Link
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

在路由器管理界面的 网络地图 里也可以看到 USB 网卡：

接下来就可以更新 nvram 变量，启用 eth8 并把它加入网桥 br0 里：

# 更新 nvram 变量
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"

# 把 eth8 加入网桥 br0，然后启用 eth8
brctl addif br0 eth8
ifconfig eth8 allmulti up

最后插入网线，运行 /jffs/tools/ethtool eth8 来验证协商的速度为全双工 2500Mb/s：

# admin@ax88u:/# /jffs/tools/ethtool eth8
Settings for eth8:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                2500baseX/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                2500baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00007fff (32767)
                               drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
        Link detected: yes

提示：完整的 RTL8156B 配置脚本可以在这里找到：附录：脚本：RTL8156B 配置脚本。

验证 `RTL` 芯片版本

运行 /jffs/tools/ethtool --driver eth8：

# admin@ax88u:/# /jffs/tools/ethtool --driver eth8
driver: r8152
version: v2.13.0 (2020/04/20)
firmware-version: 0x000e
expansion-rom-version:
bus-info: usb-xhci-hcd.0-2
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

如果 firmware-version 为 0x000b 或 0x000c，则芯片为 RTL8156。
如果 firmware-version 为 0x000d 或 0x000e，则芯片为 RTL8156B。

这里给出的输出结果中的 firmware-version 为 0x000e，因此证明我的网卡芯片为 RTL8156B。

注意：为了显示 firmware-version, 必须在 rtl8152_get_drvinfo 函数中添加一句代码，详情请参阅准备工作：编译 r8152 驱动模块。

性能优化

完整的 RTL8156B 配置脚本可以在这里找到：附录：脚本：RTL8156B 配置脚本。

接收（RX）

默认情况下，AX88U 上只有 4 号 CPU 核心为 xhci-hcd:usb1 响应硬件中断（cat /proc/irq/28/smp_affinity_list）。而且由于 RTL8156B 只有一个接收队列，也没法通过改变 smp_affinity_list 的方式，把硬件中断请求分散到其他核心去处理。

因此必须启用 RPS (Receive Packet Steering) 来分配软件中断（softirq）到各个核心，以达到优化接收性能的目的。有关 RPS 的更多信息，可以参考 Red Hat 的一篇文档：Performance Tuning Guide: 8.7 Receive Packet Steering（英文）。

运行下面的命令在 eth8 上启用 RPS：

# 在 eth8 上启用 RPS (Receive Packet Steering)
# 因为 AX88U 有四个核心，所以这里设置为 f
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus

我之后进行的测试表明仅启用 RPS 已经足以在 iperf3 跑到理论峰值速度，但是这里还是给出一些别的优化方法：

增加 net.core.netdev_max_backlog。默认值为 1000，可以增加到 2500：

# 1000 是 1GbE 的默认值，因此对 2.5GbE 来说 2500 看起来很合理
echo 2500 > /proc/sys/net/core/netdev_max_backlog

增加 RX 中断合并窗口（从而减少产生的中断数量）。rx-usecs 的默认值为 15，运行以下命令以修改：
```
# <N> 是数据包达到后产生 RX 中断的延迟时间，单位为毫秒。
/jffs/tool/ethtool -C eth8 rx-usecs <N>
```
增加 RX 缓冲区大小。默认值为 100，可以增加到 4096（最大值）：
```
# 4096 是 RX 缓冲区大小的最大值
/jffs/tool/ethtool -G eth8 rx 4096
```

这三篇文章（英文）介绍了更多优化接收性能的方法：Performance Tuning on Linux — Ethernet、How to achieve low latency with 10Gbps Ethernet、Performance Tuning Guide: Chapter 8. Networking。

发送（TX）

不像 RPS，启用 XPS (Transmit Packet Steering) 对于提升 RTL8156B 的发送性能没有帮助，这是因为 RTL8156B 只有一个发送队列，并且：

For a network device with a single transmission queue, XPS configuration has no effect, since there is no choice in this case.

对于只有一个发送队列的网络设备，启用 XPS 不会起到任何效果，因为能用的发送队列就只有一个。

Linux 内核文档 — Scaling in the Linux Networking Stack

有关发送性能优化的方法并不多，其中一个是把 txqueuelen 增加到 2500：

# 1000 是 1GbE 的默认值，因此对 2.5GbE 来说 2500 看起来很合理
ifconfig eth8 txqueuelen 2500

不知道为什么 Realtek 在内核版本低于 5.2.3 时（梅林 384.19 的内核版本号为 4.1.51），默认不启用 RTL8156B 的 USB scatter/gather：

#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3)
    if (usb_device_no_sg_constraint(udev))
        tp->sg_use = true;
#endif /* LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3) */

然而不启用这个功能会严重影响发送性能，因此另外一个优化方法是启用 RTL8156B 的 USB scatter/gather：

# 在 eth8 上启用 USB scatter/gather
echo enable > /sys/class/net/eth8/rtl_adv/sg_en

注意： 该选项仅适用于 r8152 驱动模块支持的 RTL 芯片。

性能测试

`iperf3`

该项测试中 AX88U 是服务端，我的 Windows 台式机是客户端。

台式机到 AX88U，最快 2.35Gbits/sec:

PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1
Connecting to host 192.168.50.1, port 5201
[  5] local 192.168.50.X port 13610 connected to 192.168.50.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   280 MBytes  2.35 Gbits/sec
[  5]   1.00-2.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   2.00-3.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   3.00-4.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   4.00-5.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   5.00-6.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   6.00-7.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   7.00-8.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   8.00-9.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   9.00-10.00  sec   279 MBytes  2.34 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  2.73 GBytes  2.34 Gbits/sec              sender
[  5]   0.00-10.04  sec  2.73 GBytes  2.33 Gbits/sec              receiver

iperf Done.

AX88U 到台式机，最快 2.38Gbits/sec:

PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1 -R
Connecting to host 192.168.50.1, port 5201
Reverse mode, remote host 192.168.50.1 is sending
[  5] local 192.168.50.X port 13619 connected to 192.168.50.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   284 MBytes  2.38 Gbits/sec
[  5]   1.00-2.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   2.00-3.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   3.00-4.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   4.00-5.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   5.00-6.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   6.00-7.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   7.00-8.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   8.00-9.00   sec   282 MBytes  2.36 Gbits/sec
[  5]   9.00-10.00  sec   283 MBytes  2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec  2.77 GBytes  2.37 Gbits/sec    0         sender
[  5]   0.00-10.00  sec  2.76 GBytes  2.37 Gbits/sec              receiver

iperf Done.

SMB 文件传输

该项测试中我的 Windows 台式机作为服务端，我的 Surface Pro（通过千兆有线 USB 网卡连接到路由器）和 Macbook Pro（通过 802.11ac 无线连接到路由器，协商链路速度为 1300 Mbps）作为客户端，测试内容为两个客户端同时从服务端拷贝 20GB 的文件。

测试结果表明 BCM49408 的性能不足以双向跑满 2.5 Gbps，但是也还算可以了：最大传输速度约 195.7MB/s，平均传输速度约 177.3MB/s。

提示：在 Windows 上运行这个命令可以快速创建一个 20GB 的文件：fsutil file createNew 20GB.txt 21474836480。最后一个参数为要创建的文件的大小，单位为字节。

实现 5GbE

截至本文发布时，市面上基于 USB 的 5GbE 解决方案只有 Aquantia 家（已经被 Marvell⁷ 收购了）的 AQC111U。就像一开始说的，5GbE 网卡并不便宜，我找到的最便宜的也得要 60 美金。

而且因为 AQC111U 是 USB 3.1 Gen 1 的方案，哪怕是启用 9K 巨型帧，减去 USB 开销⁸和以太网开销⁹，理论上能达到的最大传输速度约 3.96 Gbps。

同时 BCM49408 的性能也不足以支撑这个速度，启用了“性能优化”一章中描述的所有性能优化方法后，在我的 iperf3 测试中也只能实现最大 3.35 Gbits/sec（AX88U 到台式机）和 1.75 Gbits/sec（台式机到 AX88U）。

除此之外 AQC111U 还有以下的问题：

驱动不稳定。在短暂运行 iperf3 之后，我的 AX88U 直接 kernel panic 然后重启了。
发热很猛，容易掉线。传输过程中网卡会变得非常热，然后不久就失去网络连接了。不太清楚是 AQC111U 的问题，还是我买的网卡（Sabrent NT-SS5G）的散热设计的问题。

编译 `aqc111` 驱动模块

如果还是想试一下 AQC111U，可以从这里下载驱动：Marvell Drivers。在下载页面的 PLATFORM 下拉框里选择 Linux Kernel 3.10 and Higher，搜索结果中的 Marvell AQtion USB 3.1 Linux Driver 即为所需驱动，本文使用的 aqc111 驱动模块版本是 v1.3.3.0。

运行下面的命令来编译驱动：

# 架构: arm64
# 交叉编译器: aarch64-linux
# 内核源代码目录: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 SUBDIRS=`pwd` modules

如果你倾向用 Makefile 的话，可以在这里找到：附录 2：脚本：Makefile。

一切顺利的情况下，aqc111.ko 这个文件会生成在当前目录下，预编译好的版本可以在这里下载：附录：预编译的二进制文件：预编译的驱动模块。

配置驱动 & 性能优化

把 aqc111.ko 上传到路由器 /jffs/drivers/ 文件夹内，如果该文件夹不存在，则首先在 /jffs 文件夹下创建 drivers 文件夹。

AQC111U 的配置流程和 RTL8156B 的有些许不同，这里需要卸载的是模块是 cdc_ether：

# 卸载 rndis_host，因为它依赖于 cdc_ether
rmmod rndis_host
# 卸载 cdc_ether
rmmod cdc_ether
# 加载 aqc111.ko
insmod /jffs/drivers/aqc111.ko
# 加载 cdc_ether
insmod cdc_ether
# 加在 rndis_host
insmod rndis_host

针对 AQC111U 的性能优化则和 RTL8156B 的差不多，只是没法调整 RX 中断合并窗口和 RX 缓冲区大小，因为 aqc111 驱动模块不支持修改这两个属性。

提示：完整的 AQC111U 配置脚本可以在这里找到：附录：脚本：AQC111U 配置脚本。

总结

其实在去年早些时候，我就已经开始在 AX88U 上折腾 2.5GbE 网卡。只是在那个时候，市面上能买到的网卡都是基于 RTL8156 的，上文也提到过这个最初版本的芯片是有缺陷的。最终几个月之后，Realtek 修复了这些缺陷，并发布了修订版的 RTL8156B，这才使得我完成了在 AX88U 上配置 2.5GbE 网卡的任务。

另外还有一个好消息：QNAP 最近发布了一款小型（相比其他至少要 1U 的产品）的被动散热 5 口 2.5GbE 交换机 QNAP QSW-1105-5T¹⁰，售价为 100 美金，算是物美价廉吧，适合家庭使用。

附录：预编译的二进制文件

预编译的驱动模块

以下驱动模块仅适用于 AX88U（上传到 /jffs/drivers/）:

r8152 驱动模块 v2.13.0：r8152.ko。SHA1: e3a9f7a868baae6756bde4665a45852fcd578bdd
aqc111 驱动模块 v1.3.3.0：aqc111.ko。SHA1: 9dc4768154e065d19a274dcc33235a6b58fd9fb9

预编译的测试工具

以下测试工具仅适用于 AX88U（上传到 /jffs/tools/）：

ethtool 5.8：ethtool。SHA1: 6936931589476b2001667c50a8f91d713be705fb
iperf3 3.9：iperf3。SHA1: ec19717ecb56a5b5bbfeda3316cd7e712cb6bde0

附录：脚本

`Makefile`

以下 Makefile 仅适用于 AX88U：

r8152：r8152/Makefile。SHA1: dc22346bdae40f5ea5bb2a0c193dc922fa3e4b60
aqc111：acq111/Makefile。SHA1: 37dfc73f2e4b6676e5dd712ad38b389f0140fb76

`RTL8156B` 配置脚本

`/jffs/scripts/init-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/001-init-start
logger -t "rtl8156" "init-start: loading RTL8156 driver..."

# It's safe to `insmod` here, since `usbcore` has already been 
# installed before `init-start` was called. 
# init.c: sysinit() -> init_wl() -> 
#         init-broadcom.c: eval("insmod", "usbcore");
insmod /jffs/drivers/r8152.ko

# Note `insmod r8152.ko` must be executed before `cdc_ncm` module 
# is loaded into kernel. Otherwise `cdc_ncm` will take control 
# of the USB 2.5GbE adapter.
# If `r8152.ko` can't be loaded before `cdc_ncm`, then you should 
# `rmmod cdc_ncm`,` insmod r8152.ko` and `insmod cdc_ncm`
# (remember to rm/insmod modules depending on `cdc_ncm` first)

logger -t "rtl8156" "init-start: all done"
date >> /tmp/001-init-start

`/jffs/scripts/services-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "rtl8156" "services-start: setting up eth8..."

# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"

# Add eth8 into br0
brctl addif br0 eth8

# Set TX queue length to 2500 on eth8 and bring it up
ifconfig eth8 txqueuelen 2500 allmulti up

# Enable USB scatter/gather on eth8
echo enable > /sys/class/net/eth8/rtl_adv/sg_en

# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus

logger -t "rtl8156" "services-start: all done"
date >> /tmp/001-services-start

`AQC111U` 配置脚本

尽管能在 AX88U 上使用 AQC111U，但是发挥不出来 5Gbps 的性能，详情请参阅章节实现 5GbE。

`/jffs/scripts/services-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "aqc111" "services-start: loading AQC111 driver..."

# No need to `rmmod cdc_ether` here, as `cdc_ncm` will be loaded 
# after `services-start` is called
insmod /jffs/drivers/aqc111.ko

# Note `insmod aqc111.ko` must be executed before `cdc_ether` 
# module is loaded into kernel. Otherwise `cdc_ether` will take 
# control of the USB 5GbE adapter.
# If `aqc111.ko` can't be loaded before `cdc_ether`, then you should 
# `rmmod cdc_ether`, `insmod aqc111.ko` and `insmod cdc_ether` 
# (remember to rm/insmod modules depending on cdc_ether first)
logger -t "aqc111" "services-start: AQC111 driver loaded"

# Now we can setup the new interface
logger -t "aqc111" "services-start: setting up eth8..."

# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"

# Add eth8 into br0 and bring eth8 up
brctl addif br0 eth8
ifconfig eth8 allmulti up

# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'.
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus

logger -t "aqc111" "services-start: all done"
date >> /tmp/001-services-start

参考文献

Broadcom, BCM49408: 64 bit Quad-Core ARM v8 compliant Processor for Enterprise Access Point Applications. ↩
Realtek, Realtek Launches World’s First Single-Chip 2.5G Ethernet Controller for Multiple Applications, including Gaming Solution. ↩
domih, [HOWTO] 2.5Gbe or 5Gbe with H2 and/or N2 and/or C4?. ↩
VIA Labs, Inc., VL160 - USB-C™ 2:4 Data Switch with CC Function for USB 3.1. ↩
Kai-Heng Feng, Comment 39 for bug 1832472: cdc_ncm floods syslog unneccessarily. ↩
RMerl and Adamm, asuswrt-merlin.ng/build-all at master · RMerl/asuswrt-merlin.ng. ↩
Marvell, Marvell to Acquire Aquantia - Accelerating Ethernet Technology Leadership. ↩
Andrew Ku, So, What Makes USB 3.0 Slower Than We Expect? - Faster USB 3.0 Performance: Examining UASP And Turbo Mode. ↩
Rickard Nobel, Actual throughput on Gigabit Ethernet. ↩
Ryan Smith, At Last, a 2.5Gbps Consumer Network Switch: QNAP Releases QSW-1105-5T 5-Port Switch. ↩

Towards 2.5GbE (NBASE-T) on ASUS RT-AX88U with `RTL8156B` and Asuswrt-Merlin 384.19

2020-10-11T17:24:54-07:00

It has been several months since I updated my main router to AX88U. It is undoubtedly an excellent router and I’m quite satisfied with its features.

However, the lack of integrated 2.5GbE port unfortunately shadows its power and ability, not to mention that its processor BCM49408 does natively support 2.5GbE¹. To make it worse, recently released AX86U comes with a similar processor BCM4908 and, yeah, has a built-in 2.5GbE port.

The good news is that there is an external solution from Realtek: RTL8156², a USB 3.0 based 2.5GbE controller. Comparing to much expensive 5 or 10GbE equipments, it is fairly “cheap”: you can easily find a RLT8156 based USB dongle for about $30.

But do note that there are two versions of this chip: RTL8156 and RTL8156B. You should look for the B revision (RTL8156B), as the original RTL8156 has a weird stability issue in long time tests³.

In addition, if you bought a USB Type-C dongle, you may also need an “active” or “double side” USB Type-C to USB-A adapter (which usually include a multiplexer chip, e.g. VL160⁴). Otherwise, you have to figure out which side of the USB Type-C port to insert, since one side will work at the speed of USB 3.0 and the other side will do USB 2.0 only.

No Plug-and-Play?

If you directly insert the USB dongle into AX88U, there is no problem to get it recognized (on Network Map page in router’s web management console). Also in ifconfig -a, you can find that the dongle has been registered as usb0.

However, if you try to bring it up with ifconfig usb0 up, the dmesg will be spammed by cdc_ncm:

cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink

The reason is that the built-in driver in Linux kernel has no support for RTL8156 yet⁵. So we have to build the driver on our own.

Preparation

Compile Linux Kernel

Follow this guide for setting up the compiling environment for Asuswrt-Merlin. Though it is titled for WSL (Windows Subsystem for Linux) 2, you can still follow these steps under WSL 1 (legacy WSL) or pure Linux.

If you’re using legacy WSL like me, you have to follow this comment to run 32-bit applications in legacy WSL, because the provided toolchains are 32-bit.

As we don’t really need the git history, we can just clone the HEAD:

# Clone the HEAD only
git clone --depth 1 https://github.com/RMerl/asuswrt-merlin.ng

Also note that instead of linking brcm-arm-sdk folder under src-rt-6.x.4708:

# `src-rt-6.x.4708` is for AC56U, AC68U and AC87U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-6.x.4708/toolchains

… in the guide, we link it under src-rt-5.02axhnd:

# `src-rt-5.02axhnd` is for AX88U, AX58U and AX56U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/toolchains

… since we’re targeting AX88U, according to Merlin’s build script⁶.

Once you got the environment done, run this to build the firmware for AX88U:

# Targeting AX88U
cd release/src-rt-5.02axhnd && make rt-ax88u

It is not necessary to complete the whole firmware compiling, as we only need the generated header files (.h) from the part of compiling Linux kernel.

Therefore once the kernel building starts for a while, you may interrupt the build process by pressing CTRL+C.

Compile Tools

`iperf3`

Source code for iperf3 can be found at here. At the time of writing, the latest version of iperf3 is 3.9.

Run the following lines to compile iperf3 for AX88U:

# Create `build` folder as installation prefix
mkdir build
# We need the static-linked binary
./configure --host=arm-linux --prefix=`pwd`/build --enable-static --disable-shared
# Build the binary and install it into `build` folder
make && make install

The iperf3 binary should be generated in ./build/bin/iperf3. Or you can get the pre-compiled one in section Appendix: Pre-compiled Binaries: Pre-compiled Tools.

For the iperf3 running on Windows, please check this post.

`ethtool`

Source code for ethtool can be found at here. At the time of writing, the latest version of ethtool is 5.8.

Run the following lines to compile ethtool for AX88U:

# Create `build` folder as installation prefix
mkdir build
# Disable netlink, as we don't need it at all in our case.
# This also helps eliminate the dependency on `libmnl`
./configure --host=arm-linux --prefix=`pwd`/build --disable-netlink
# Build the binary and install it into `build` folder
make & make install

The ethtool binary should be generated in ./build/bin/ethtool. Or you can get the pre-compiled one in section Appendix: Pre-compiled Binaries: Pre-compiled Tools.

Compile `r8152` Driver

It might be confusing at the beginning, but yes, the driver for RTL8156B is called r8152. The source code can be downloaded from here. At the time of writing, the r8152 driver version is v2.13.0.

I added a line to function rtl8152_get_drvinfo in r8152.c, so we can verify the chip version is RTL8156B via ethtool:

--- source-2.13.0/r8152-original.c     2020-04-20 01:36:49.000000000 -0700
+++ source-2.13.0/r8152-modified.c     2020-10-06 23:47:07.123659400 -0700
@@ -16369,6 +16369,7 @@
        strlcpy(info->driver, MODULENAME, sizeof(info->driver));
        strlcpy(info->version, DRIVER_VERSION, sizeof(info->version));
        usb_make_path(tp->udev, info->bus_info, sizeof(info->bus_info));
+       snprintf(info->fw_version, sizeof(info->fw_version), "0x%04x", tp->version);
 } 

 #if LINUX_VERSION_CODE < KERNEL_VERSION(4,20,0)

Since we’re targeting AX88U, we need to call aarch64-linux cross-compiler and set the architecture to arm64:

# Architecture: arm64
# Cross-compiler: aarch64-linux
# Kernel source code directory: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 M=`pwd` modules

If you prefer using Makefile instead, it can be found in section Appendix: Scripts: Makefile.

If everything goes well, the r8152.ko should be generated in the working directory. You can also get the pre-compiled one in section Appendix: Pre-compiled Binaries: Pre-compiled Drivers.

Installation

Upload r8152.ko into /jffs/drivers/, iperf3 and ethtool into /jffs/tools/. Remember to create drivers and tools folders in /jffs first.

Load Driver

As we discussed in section No Plug-and-Play?, cdc_ncm will try to take control of the USB dongle, so we need to unload it first.

We need to check which modules depends on cdc_ncm with lsmod | grep cdc_ncm:

# admin@ax88u:/# lsmod | grep cdc_ncm
cdc_ncm                16787  1 cdc_mbim
usbnet                 21074  7 cdc_mbim,qmi_wwan,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix
usbcore               166572 23 uas,usb_storage,cdc_mbim,qmi_wwan,cdc_wdm,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix,cdc_acm,usbnet,usblp,ohci_pci,ohci_platform,ohci_hcd,ehci_pci,ehci_platform,ehci_hcd,xhci_pci,xhci_plat_hcd,xhci_hcd

So we have to unload cdc_mbim first and cdc_ncm later:

# Unload `cdc_mbim`, as it depends on `cdc_ncm`
rmmod cdc_mbim
# Unload `cdc_ncm`
rmmod cdc_ncm

Now we can load r8152.ko and also load cdc_ncm and cdc_mbim back:

# Load `r8152` driver
insmod /jffs/drivers/r8152.ko
# Load `cdc_ncm`
insmod cdc_ncm
# Load `cdc_mbim`
insmod cdc_mbim

If everything goes well, system log (tail /tmp/syslog.log) should reflect this:

# admin@ax88u:/# tail /tmp/syslog.log
Oct  7 21:27:42 kernel: usbcore: deregistering interface driver cdc_mbim
Oct  7 21:27:42 kernel: usbcore: registered new interface driver r8152
Oct  7 21:27:42 kernel: usb 2-2: reset SuperSpeed USB device number 2 using xhci-hcd
Oct  7 21:27:42 kernel: netif_napi_add() called with weight 256 on device eth%d
Oct  7 21:27:42 kernel: r8152 2-2:1.0 eth8: v2.13.0 (2020/04/20)
Oct  7 21:27:42 kernel: r8152 2-2:1.0 eth8: This product is covered by one or more of the following patents:
Oct  7 21:27:42 kernel: 		US6,570,884, US6,115,776, and US6,327,625.
Oct  7 21:27:42 hotplug: add net eth8.
Oct  7 21:27:42 kernel: usbcore: registered new interface driver cdc_ncm
Oct  7 21:27:42 kernel: usbcore: registered new interface driver cdc_mbim

As you can see, the adapter is registered as eth8, thus if we run ifconfig eth8:

eth8      Link encap:Ethernet  HWaddr 00:E0:4C:00:FA:KE
          inet6 addr: fe80::2e0:4cff:fe00:fake/64 Scope:Link
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Also if you check the Network Map page in Web management UI, the adapter should show up there:

Then we can update nvram variables, add eth8 to br0 and bring eth8 up:

# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"

# Add eth8 into br0 and bring it up
brctl addif br0 eth8
ifconfig eth8 allmulti up

Finally we can connect the ethernet cable to the USB dongle.

Run /jffs/tools/ethtool eth8 to verify the driver is correctly loaded and the negotiated link speed is 2500Mb/s full-duplex:

# admin@ax88u:/# /jffs/tools/ethtool eth8
Settings for eth8:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                2500baseX/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                2500baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00007fff (32767)
                               drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
        Link detected: yes

Tip: Complete scripts for RTL8156B can be found in Appendix: Scripts: Scripts for RTL8156B.

Verify `RTL` Version

To tell if the dongle is really RTL8156B, you can run /jffs/tools/ethtool --driver eth8:

# admin@ax88u:/# /jffs/tools/ethtool --driver eth8
driver: r8152
version: v2.13.0 (2020/04/20)
firmware-version: 0x000e
expansion-rom-version:
bus-info: usb-xhci-hcd.0-2
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

If firmware-version is either 0x000b or 0x000c, the internal chip is RTL8156.
If firmware-version is either 0x000d or 0x000e, the internal chip is RTL8156B.

As you can see, mine firmware-version is 0x000e thus it is RTL8156B.

Note: In order to show the firmware-version, the added line to function rtl8152_get_drvinfo in r8152.c is necessary. Check deatils about it in section Preparation: Compile r8152 driver.

Performance Tuning

Complete scripts for RTL8156B can be found in Appendix: Scripts: Scripts for RTL8156B.

Receive (RX)

By default, hardware interrupts for xhci-hcd:usb1 on AX88U is only handled by CPU core 4 (cat /proc/irq/28/smp_affinity_list).

However, as RTL8156B controller only has 1 RX queue, we can’t spread those hardware interrupts to other cores via changing smp_affinity_list.

Thus we have to enable RPS (Receive Packet Steering). RPS helps to distribute software interrupts (softirq) to multiple cores. You can check here for more details about RPS.

To enable RPS on eth8:

# Enable RPS (Receive Packet Steering) on eth8
# Since AX88U has four cores, so we set it to 'f'
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus

At least in my test, enabling RPS alone allows the adapter to reach its maxmium performance in iperf3.

But if you are looking for more RX tweaks, here are some examples:

Increase net.core.netdev_max_backlog. The default value is 1000 and you may increase it to 2500 via:

# 1000 is the default for 1GbE, thus 2500 seems reasonable for 2.5GbE
echo 2500 > /proc/sys/net/core/netdev_max_backlog

Increase RX interrupt coalescing (to decrase # interrupts). The defuault rx-usecs is 15 and you may change it via:

# <N> is the microseconds to delay an RX interrupt after packet arrival
/jffs/tool/ethtool -C eth8 rx-usecs <N>

Increase RX ring buffer size. The default value is 100 and you may change it to 4096 (which is the maxmium):
```
# 4096 is the maxmium supported value for RX ring buffer
/jffs/tool/ethtool -G eth8 rx 4096
```

More information can be found here, here and here.

Transmit (TX)

Unlike RPS, XPS (Transmit Packet Steering) will not help increase performance in our case, because

For a network device with a single transmission queue, XPS configuration has no effect, since there is no choice in this case.

Linux Kernel Documentation — Scaling in the Linux Networking Stack

… and RTL8156B has only 1 TX queue.

But we can increase txqueuelen to 2500 on eth8 anyway:

# 1000 is the default for 1GbE, thus 2500 seems reasonable for 2.5GbE
ifconfig eth8 txqueuelen 2500

Somehow Realtek didn’t enable USB scatter/gather for kernel version below 5.2.3 (AX88U’s kernel version is 4.1.51), according to r8152.c:

#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3)
    if (usb_device_no_sg_constraint(udev))
        tp->sg_use = true;
#endif /* LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3) */

… and without USB scatter/gather enabled, the TX performance is really hurt.

Of course you can modify the driver, but I found a way to enable it externally:

# Enable USB scatter/gather on eth8
echo enable > /sys/class/net/eth8/rtl_adv/sg_en

Note: This option only works for r8152 driver supported RTL chips.

Performance Evaluation

`iperf3`

AX88U is the iperf3 server and my Windows PC is the client.

PC to AX88U, achieving ~2.35Gbits/sec:

PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1
Connecting to host 192.168.50.1, port 5201
[  5] local 192.168.50.X port 13610 connected to 192.168.50.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   280 MBytes  2.35 Gbits/sec
[  5]   1.00-2.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   2.00-3.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   3.00-4.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   4.00-5.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   5.00-6.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   6.00-7.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   7.00-8.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   8.00-9.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   9.00-10.00  sec   279 MBytes  2.34 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  2.73 GBytes  2.34 Gbits/sec              sender
[  5]   0.00-10.04  sec  2.73 GBytes  2.33 Gbits/sec              receiver

iperf Done.

AX88U to PC, achieving ~2.38Gbits/sec:

PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1 -R
Connecting to host 192.168.50.1, port 5201
Reverse mode, remote host 192.168.50.1 is sending
[  5] local 192.168.50.X port 13619 connected to 192.168.50.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   284 MBytes  2.38 Gbits/sec
[  5]   1.00-2.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   2.00-3.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   3.00-4.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   4.00-5.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   5.00-6.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   6.00-7.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   7.00-8.00   sec   283 MBytes  2.37 Gbits/sec
[  5]   8.00-9.00   sec   282 MBytes  2.36 Gbits/sec
[  5]   9.00-10.00  sec   283 MBytes  2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec  2.77 GBytes  2.37 Gbits/sec    0         sender
[  5]   0.00-10.00  sec  2.76 GBytes  2.37 Gbits/sec              receiver

iperf Done.

SMB File Transfer

Here we test a real-world application: SMB file transfer.

My Windows PC acts the SMB server, while my Surface Pro (connected via wire with USB 1GbE adapter) and my Macbook Pro (connected via 802.11ac WiFi, negotiated link speed: 1300Mbps) are trying to copy a 20GB file simultaneously.

Consider the overhead of SMB protocol and the relatively less powerful BCM49408 comparing to x86 CPUs, I’d say the result below is fairly good by the speed reaching ~195.7MB/s and averaging at around 177.3MB/s.

Tip: To quickly create a dummy file of 20GB on Windows: fsutil file createNew 20GB.txt 21474836480. The last parameter is the specified file size in bytes.

What about 5GbE?

At the time of writing, the only USB solution for 5GbE is Aquantia’s (acquired by Marvell⁷) AQC111U. Like I said before, 5GbE is not cheap: the cheapest 5GbE adapter I found costs $60.

Since AQC111U is a USB 3.1 Gen 1 based solution, even with 9K jumbo frame enabled, theoretically the maxmium speed you can reach is about 3.96 Gbits/sec considering the overhead from both USB⁸ and Ethernet⁹.

Not to mention that BCM49408 seems not able to handle this speed: after applied all the tweaks described in section Performance Tuning, I can only get ~3.35 Gbits/sec from AX88U to PC, and ~1.75 Gbits/secs from PC to AX88U in iperf3 test.

There are also some other issues:

AQC111U driver seems unstable. Somehow my AX88U got kernel panic and suddenly rebooted after running iperf3 for a short while.
Connection lost after a while. It becomes very hot during transmission and will eventually lose ethernet connection. Not sure if the latter is caused by AQC111U or the thermal design of the particular adapter I have.

Compile `aqc111` Driver

If you still want to give it a try, you can download the driver code here.

Choose Linux Kernel 3.10 and Higher in the PLATFORM droplist and Marvell AQtion USB 3.1 Linux Driver is the one to look for.

At the time of writing, the least driver version is v1.3.3.0.

Since we’re targeting AX88U, we need to call aarch64-linux cross-compiler and set the architecture to arm64:

# Architecture: arm64
# Cross-compiler: aarch64-linux
# Kernel source code directory: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 SUBDIRS=`pwd` modules

If you prefer using Makefile instead, it can be found in section Appendix: Scripts: Makefile.

If everything goes well, the aqc111.ko should be generated in the working directory. You can also get the pre-compiled one in section Appendix: Pre-compiled Binaries: Pre-compiled Drivers.

Installation & Performance Tuning

First upload aqc111.ko into /jffs/drivers/. Remember to create drivers folder under /jffs if not exist.

The installation is a little bit different from RTL8156B. Instead of cdc_ncm, we need to unload cdc_ether:

# Unload `rndis_host`, as it depends on `cdc_ether`
rmmod rndis_host
# Unload `cdc_ether`
rmmod cdc_ether
# Load `aqc111.ko`
insmod /jffs/drivers/aqc111.ko
# Load `cdc_ether` back
insmod cdc_ether
# Load `rndis_host` back
insmod rndis_host

Tuning performance for AQC111U is similar to RTL8156B. However you can’t adjust interrupt coalescing or ring buffer size, since aqc111 driver doesn’t support them. You can refer to section Performance Tuning for more details.

Tip: Complete scripts for AQC111U can be found in Appendix: Scripts: Scripts for AQC111U.

Conclusion

Actually early this year, I’ve already started to try to bring 2.5GbE to AX88U. At that time, however, there was only the original RTL8156 chip and I couldn’t make it reach the maximum performance in iperf3 test.

Finally several months later, Realtek fixed the problem and released the RTL8156B revision. I’m so happy to see that I finally got the next generation of network and made my router future-proof.

Another good news is that QNAP recently announced an affordable ($100) 5-port 2.5GbE switch QNAP QSW-1105-5T¹⁰, which is passive-cooled and small in size - pretty suitable for home networking.

I hope you enjoy this post and upgrade your network to the next generation :)

Appendix: Pre-compiled Binaries

Pre-compiled Drivers

Pre-compiled drivers for AX88U (store them into /jffs/drivers/):

r8152 driver v2.13.0: r8152.ko. SHA1: e3a9f7a868baae6756bde4665a45852fcd578bdd
aqc111 driver v1.3.3.0: aqc111.ko. SHA1: 9dc4768154e065d19a274dcc33235a6b58fd9fb9

Pre-compiled Tools

Pre-compiled tools for AX88U (store them into /jffs/tools):

ethtool 5.8: ethtool. SHA1: 6936931589476b2001667c50a8f91d713be705fb
iperf3 3.9: iperf3. SHA1: ec19717ecb56a5b5bbfeda3316cd7e712cb6bde0

Appendix: Scripts

`Makefile`

Here are the Makefiles for drivers targeting AX88U.

Makefile for r8152: r8152/Makefile. SHA1: dc22346bdae40f5ea5bb2a0c193dc922fa3e4b60
Makefile for aqc111: acq111/Makefile. SHA1: 37dfc73f2e4b6676e5dd712ad38b389f0140fb76

Scripts for `RTL8156B`

`/jffs/scripts/init-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/001-init-start
logger -t "rtl8156" "init-start: loading RTL8156 driver..."

# It's safe to `insmod` here, since `usbcore` has already been 
# installed before `init-start` was called. 
# init.c: sysinit() -> init_wl() -> 
#         init-broadcom.c: eval("insmod", "usbcore");
insmod /jffs/drivers/r8152.ko

# Note `insmod r8152.ko` must be executed before `cdc_ncm` module 
# is loaded into kernel. Otherwise `cdc_ncm` will take control 
# of the USB 2.5GbE adapter.
# If `r8152.ko` can't be loaded before `cdc_ncm`, then you should 
# `rmmod cdc_ncm`,` insmod r8152.ko` and `insmod cdc_ncm`
# (remember to rm/insmod modules depending on `cdc_ncm` first)

logger -t "rtl8156" "init-start: all done"
date >> /tmp/001-init-start

`/jffs/scripts/services-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "rtl8156" "services-start: setting up eth8..."

# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"

# Add eth8 into br0
brctl addif br0 eth8

# Set TX queue length to 2500 on eth8 and bring it up
ifconfig eth8 txqueuelen 2500 allmulti up

# Enable USB scatter/gather on eth8
echo enable > /sys/class/net/eth8/rtl_adv/sg_en

# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus

logger -t "rtl8156" "services-start: all done"
date >> /tmp/001-services-start

Scripts for `AQC111U`

Though AQC111U works on AX88U, its maximum performance is unachievable.

Read section What about 5GbE? for more details.

`/jffs/scripts/services-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "aqc111" "services-start: loading AQC111 driver..."

# No need to `rmmod cdc_ether` here, as `cdc_ncm` will be loaded 
# after `services-start` is called
insmod /jffs/drivers/aqc111.ko

# Note `insmod aqc111.ko` must be executed before `cdc_ether` 
# module is loaded into kernel. Otherwise `cdc_ether` will take 
# control of the USB 5GbE adapter.
# If `aqc111.ko` can't be loaded before `cdc_ether`, then you should 
# `rmmod cdc_ether`, `insmod aqc111.ko` and `insmod cdc_ether` 
# (remember to rm/insmod modules depending on cdc_ether first)
logger -t "aqc111" "services-start: AQC111 driver loaded"

# Now we can setup the new interface
logger -t "aqc111" "services-start: setting up eth8..."

# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"

# Add eth8 into br0 and bring eth8 up
brctl addif br0 eth8
ifconfig eth8 allmulti up

# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'.
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus

logger -t "aqc111" "services-start: all done"
date >> /tmp/001-services-start

References

Broadcom, BCM49408: 64 bit Quad-Core ARM v8 compliant Processor for Enterprise Access Point Applications. ↩
Realtek, Realtek Launches World’s First Single-Chip 2.5G Ethernet Controller for Multiple Applications, including Gaming Solution. ↩
domih, [HOWTO] 2.5Gbe or 5Gbe with H2 and/or N2 and/or C4?. ↩
VIA Labs, Inc., VL160 - USB-C™ 2:4 Data Switch with CC Function for USB 3.1. ↩
Kai-Heng Feng, Comment 39 for bug 1832472: cdc_ncm floods syslog unneccessarily. ↩
RMerl and Adamm, asuswrt-merlin.ng/build-all at master · RMerl/asuswrt-merlin.ng. ↩
Marvell, Marvell to Acquire Aquantia - Accelerating Ethernet Technology Leadership. ↩
Andrew Ku, So, What Makes USB 3.0 Slower Than We Expect? - Faster USB 3.0 Performance: Examining UASP And Turbo Mode. ↩
Rickard Nobel, Actual throughput on Gigabit Ethernet. ↩
Ryan Smith, At Last, a 2.5Gbps Consumer Network Switch: QNAP Releases QSW-1105-5T 5-Port Switch. ↩

Parse MATLAB classification tree and Generate corresponding JSON

2020-09-16T23:17:39-07:00

MATLAB Classification Learner is a powerful tool for building and testing classification model. However, the generated classification tree is not very user-friendly to use outside of MATLAB.

Tip: Don’t care about details? Here is the final tool: MATLAB Classification Tree Parser.

Update: After I finished everything (the tool and this post), I somehow found that recently a MathWorks staff released a helper in File Exchange to export MATLAB models to PMML. What can I say? ¯\_(ツ)_/¯.

Background

An example output¹ (i.e. view(ctree)) of MATLAB classification tree looks like:

Decision tree for classification
if x3<2.45 then node 2 elseif x3>=2.45 then node 3 else setosa
class = setosa
if x4<1.75 then node 4 elseif x4>=1.75 then node 5 else versicolor
if x3<4.95 then node 6 elseif x3>=4.95 then node 7 else versicolor
class = virginica
if x4<1.65 then node 8 elseif x4>=1.65 then node 9 else versicolor
class = virginica
class = versicolor
class = virginica

This is quite difficult to use in other environments. Say if I want to make a classifier in C++, I need to convert the above tree output into C++ code by hand, which is a very tedious work and prone to error especially when the tree is dense.

Though MATLAB allows you to visually inspect tree in GUI via view(ctree, 'mode', 'graph'), it is still annoying to convert it manually.

Also in one of my projects, I’ve implemented a decision tree classifier in C++ and it allows me to dynamically load decision trees from JSON files. Since these trees have to be updated periodically, I have to manually write the JSON frequently for more than a year. It would definitely be nice if I can make a tool to automatically generate those JSON files.

In the beginning, I looked into MATLAB toolbox’s source code but found nothing useful about the view method in either class ClassificationTree or CompactClassificationTree. Thus I need to “decipher” the output by myself.

Though it is possible to parse the tree output with regular expressions, I’d prefer to solve it in a similar way of building compilers. Because we’re dealing with a tree, and a parser in compiler also build a tree (AST, or Abstract Syntax Tree).

Define the Grammar

I wrote the grammar in ANTLR 4. ANTLR can also generates lexer and parser for my later building the tool.

Here is the context-free grammar for MATLAB classification tree:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// MatlabCTree.g4
grammar MatlabCTree;

start           : 'Decision tree for classification'? stmtList;

stmtList        : stmt | stmtList stmt;
stmt            : INTEGER (ifStmt | classExpr);

ifStmt          : ifClause elseifClause? elseClause?;
ifClause        : 'if' condExpr 'then' nodeExpr;
elseifClause    : 'elseif' condExpr 'then' nodeExpr;
elseClause      : 'else' nodeExpr;

labelExpr       : INTEGER | IDENTIFIER;
classExpr       : 'class' '=' labelExpr;
nodeExpr        : 'node' INTEGER | labelExpr;

constExpr       : INTEGER | RATIONAL;
condExpr        : IDENTIFIER '<'  constExpr
                | IDENTIFIER '>'  constExpr
                | IDENTIFIER '<=' constExpr
                | IDENTIFIER '>=' constExpr
                | IDENTIFIER '==' constExpr
                | IDENTIFIER '~=' constExpr;

INTEGER         : '0' | [1-9] [0-9]*;
IDENTIFIER      : [a-zA-Z] [a-zA-Z0-9_]*;

// Adapted from https://git.io/JU4M9
RATIONAL        : NUMBER (E SIGN? NUMBER)?;
fragment NUMBER : INTEGER? '.' [0-9]+;
fragment E      : 'E' | 'e';
fragment SIGN   : '+' | '-';

WHITESPACE      : [ \t\r\n]+ -> skip;

Some key points in the above grammar:

Line 4: The prefix Decision tree for classification is optional.
Line 7: The following lines are either a branch or a leaf starting with a node number.
Line 9: At least from MATLAB’s example¹ and my own experience, the output of tree branch has a fixed pattern of
```
if feature<a then node x elseif feature>=a then node y else some_label
```
However I’m not sure if the elseif clause and else clause would be present in all scenarios. So I decided to leave them as optional.
Line 14: Depending on how Learner is configured, the label could be either a string (MATLAB’s example) or an integer (my case).
Line 19-24: Only 6 operators are supported: <, >, <=, >=, == and ~=.
Line 26: Preceding 0s are not allowed in integers.
Line 27: This follows the variable naming convention in MATLAB.

Now we have the grammar, we can test MATLAB’s example classification with ANTLR’s built-in GUI tester (click the figure to enlarge):

The above parsing tree looks like just what I want! It’s time to build the tool.

Build the Tool

Initially I was thinking to build a tool in Java or C#, but later I realized why not make a web-based tool as ANTLR supports outputting lexers and parsers in JavaScript. A online tool also makes everyone’s life easier: no need to worry in setting up environments and doing (potentially annoying) compiling job.

Also this tool will purely run in the browser, that is 100% of the code running on user’s device, there is no need to set up a server nor any interaction with remote server at all.

If you’re not familiar with using ANTLR on JavaScript target, here is the official documentation.

I used the below command to generate lexer and parser. Note that I’ll use visitor instead of listener in the final product.

java -jar antlr-4.8-complete.jar -Dlanguage=JavaScript -visitor -no-listener MatlabCTree.g4

Note: I’ll only cover the main points in the following sections. If you want to dive into details, source code is available in section Source code.

Traverse AST (Abstract Syntax Tree)

This part is pretty straightforward, just traverse the generated AST and store all classification tree nodes into an array.

Since it’s possible that node numbers are not in a increasing order, for example:

Decision tree for classification
if x<0 then node 2 elseif x>=0 then node 3 else foo
class = foo
class = bar

Note that the above order is 1, 3, 2 while we expect 1, 2, 3.

In order to detect this, we need to setup a counter, increase the counter every time we visit stmt and check if the node number matches the counter:

visitStmt(ctx) {
  // stmt: INTEGER (ifStmt | classExpr);
  let nodeNumber = parseInt(ctx.children[0].getText());

  // Check if the counter matches node number in stmt
  if (++this.nodeNumber !== nodeNumber) {
    let symbol = ctx.children[0].symbol;
    this.errors.push({
      line: symbol.line,
      message: `mismatched node number ${nodeNumber} ` +
        `expecting ${this.nodeNumber}`
    });
  }

  let result = this.visitChildren(ctx);
  this.jsonNodes.push(result[1]);

  // Actually it doesn't matter whether result is returned
  return result;
}

Because the generated tree from fitctree is a binary tree², we just need to check if the conditions in if and elseif clauses are complement. else clause can be safely ignored.

Here are the examples that conditions are not complement:

# Feature name doesn't match (x and y)
1  if x<0.5 then node 2 elseif y>=0.5 then node 3 else foo

# Comparison value doesn't match (0.5 and -0.5)
1  if x<0.5 then node 2 elseif x>=-0.5 then node 3 else foo

# Operators are not complement (< and >, which supposed to be < and >=)
1  if x<0.5 then node 2 elseif x>0.5 then node 3 else foo

To handle this:

visitIfStmt(ctx) {
  // ifStmt: ifClause elseifClause? elseClause?;
  let result = this.visitChildren(ctx);

  let featureName = result[0].condition.identifer;
  let operator = result[0].condition.operator;
  let compareValue = result[0].condition.value;
  let trueBranch = result[0].branch;
  let falseBranch = null;

  // 2 or more clauses?
  if (result.length > 1) {
    // elseifClause
    if (result[1].hasOwnProperty("condition")) {
      let elseifFN = result[1].condition.identifer;
      let elseifOP = result[1].condition.operator;
      let elseifCV = result[1].condition.value;

      // Does conditions match?
      if (elseifFN !== featureName ||
        elseifOP !== this.complementOperators[operator] ||
        elseifCV !== compareValue) {

        let elseIfExpr = ctx.children[1].children[1];
        this.errors.push({
          line: elseIfExpr.start.line,
          message: `mismatched elseif clause '${elseIfExpr.getText()}' ` +
            `expecting '${featureName}` + 
            `${this.complementOperators[operator]}` + 
            `${compareValue}'`
        });
      } else {
        falseBranch = result[1].branch;
      }
    } else {
      // elseClause
      falseBranch = result[1];
    }
  }

  // We ignored the elseClause here if there are 3 clauses present.
  // This should be safe since the generated classification tree from
  // MATLAB's fitctree function is supposed to be a binary tree.

  return { featureName, operator, compareValue, trueBranch, falseBranch };
}

Generate JSON

The output JSON follows the below format:

{
    "featureName": "{feature name}",
    "operator": "{operator for comparison}",
    "compareValue": {the value to be compared (in double)},
    "trueBranch": {true branch, if `fn op val` holds},
    "falseBranch": {false branch, otherwise}
}

trueBranch and falseBranch are either a string representing the predicted label, or an above object indicating a child tree node.

My classifier actually accepts a compact version of it. The key names are simplified into fn, op, val, tb, and fb for saving spaces.

Since we have already gathered all the nodes in the classification tree via AST visitor, we just need to perform a simple DFS (Depth-First Search):

dfs(node, index) {
  this.visitedNodes.push(index);

  return { 
    featureName: node.featureName,
    operator:  node.operator,
    compareValue: node.compareValue,
    trueBranch: this.visitBranch(node.trueBranch),
    falseBranch: this.visitBranch(node.falseBranch)
  };
}

Even though it is guaranteed to be a tree, it doesn’t prevent us to add support for forest:

// Perform DFS on all nodes so we can also handle forests
this.jsonNodes.forEach((node, index) => {
  // Only take care not yet visited ifStmt nodes
  if (typeof node !== "string" && !this.visitedNodes.includes(index)) {
    jsonOutput.push(this.dfs(node, index));
  }
});

We also need to handle a boundary condition where the tree consists of a label node only, for example:

Decision tree for classification
1  class = setosa

To handle this:

this.jsonNodes.forEach((node, index) => {
  // Boundary condition, the tree only has a label.
  if (typeof node === "string" && !this.visitedNodes.includes(index)) {
    jsonOutput.push(node);
  }
});

One more thing need to take care is branch target’s node number in if and elseif, since it might exceed the number of nodes. For example:

Decision tree for classification
1  if x3<2.45 then node 2 elseif x3>=2.45 then node 3 else setosa

Apparently, node 2 and 3 are missing in the input. So we need to detect this and report error:

if (branch.target > this.jsonNodes.length) {
  this.errors.push({
    line: branch.line,
    message: `invalid node index ${branch.target} ` +
      `expecting no greater than ${this.jsonNodes.length}`
  });
  return null;
}

Wrapper class

Now we can make a wrapper class MatlabCTreeConverter to call lexer and parser, and invoke AST visitor and JSON generator.

Here is the way to handle syntax errors, syntaxErrors is for later error reporting:

let syntaxErrors = [];

// Adapted from https://stackoverflow.com/a/60841205
parser.removeErrorListeners();
parser.addErrorListener({
  syntaxError: (recognizer, offendingSymbol, line, column, msg, err) => {
    syntaxErrors.push({ line: line, message: msg });
  }
});

Since our starting rule happened to be start, we can build AST by:

let ast = parser.start();

`webpack` everything

Setting up webpack is quite annoying and very painful. Fortunately, I found a working configuration file for webpack³. Hat tip to @nicolasdao!

Note: Since anyway the core code will be publicly accessible, uglifyjs-webpack-plugin is not used.

Once you got everything in that gist done, remember to add the following entry to webpack.config.js’s module.exports:

node: { module: 'empty' net: 'empty', fs: 'empty' }

This is VERY important (noted in ANTLR’s official documentation), otherwise ANTLR will try to import fs module which is not available in browsers.

User Interface

Big bruh moment. Needless to say how tedious is UI design and implementation. Anyway here is how it looks like:

If any error is detected in the input, it will report which line goes wrong:

In case you’re curious, I’m using CodeMirror for the input and output editors, and Viz.js for drawing the classification tree.

Finally the link to the tool: MATLAB Classification Tree Parser.

Source Code

Grammar definition has been covered here. Simply copy the code there and save it as MatlabCTree.g4.

Full JavaScript project for webpack: matlab-ctree-parser.zip. SHA1: 212f9647d35ad83f67a01a4bc825d72ac2ad21df

References

MathWorks Help Center, View Decision Tree - MATLAB & Simulink. ↩ ↩²
MathWorks Help Center, Fit binary decision tree for multiclass classification - MATLAB fitctree. ↩
Nicolas Dao, Basic damn Webpack config for simple transpilation ES6 to ES5. ↩

在 AX88U + Merlin 384.16 上实现 LAN 端口隔离（静态 VLAN）

2020-04-30T22:08:50-07:00

最近把手头用了四年多的 AC68P 升级成了支持 802.11ax (WiFi 6) 的 AX88U。不用多说，到手第一天就刷了 Merlin 梅林固件。

不过无论是原厂固件还是梅林固件，它们都没有提供原生的 GUI 划分 VLAN 的功能（相比之下，在 OpenWrt 上配置 VLAN 就很方便）。但是由于博通搞的闭源驱动¹，OpenWrt 在有生之年大概都不会支持 AX88U。所以要在 AX88U 上配置 VLAN，只能通过命令行的方式。

需求

AX88U 上一共有 8 个 LAN 口，我想把 LAN 2 隔离出来，用做访客网络。换句话说，LAN 1, 3 - 8 划入 VLAN 1, LAN 2 单独划入 VLAN 2。

理论上，真正的隔离应该是禁止两个 VLAN 互相访问。但是考虑到万一需要在我的电脑（VLAN 1）上重新配置二级路由器（VLAN 2），还是得允许 VLAN 1 到 VLAN 2 的单向通信。

此外，上述提到的隔离对客户端应当是完全透明的：VLAN 2 里的客户端不需要额外的配置，只需要插上网线即可上网。

有关 AX88U

没有 `robocfg`？

很不幸的是，AX88U（博通 HND 平台）移除了 robocfg 命令。搜索了一番之后，发现似乎 vlanctl² 可以用来配置 VLAN。但是玩了一阵子，目测 vlanctl 只能搞 Tagged VLAN，没法实现静态 VLAN。

不过以防有人有这方面的需求，推荐阅读这篇文章：上海电信 TL-EP110 + RT-AC86U 实现观看 4K IPTV 无卡顿 (2019-10)。这篇文章介绍了如何在 AC86U（也是 HND 平台的路由器）上配置 Tagged VLAN。

以太网接口到物理端口的映射

在网上我没搜到相关的信息，不过经过若干次网线插拔之后，总结出来这张表：

以太网接口	物理端口
eth0	WAN
eth1	LAN 4
eth2	LAN 3
eth3	LAN 2
eth4	LAN 1
eth5	LAN 5 - 8 的网桥
eth6	2.4 GHz 无线
eth7	5 GHz 无线

这里注意 eth5 应该是通过 BCM53134 组的硬件网桥（LAN 5 - 8）。使用 etchtl 和 ethswctl 或许能解散这个网桥，不过我没试验。所以，下文提到的解决方案只能用于隔离 LAN 1 - 4。

此外，AX88U 默认把 eth1 ~ eth7 组成了网桥 br0。

注意：我的 WAN 联机类型 是 动态 IP。如果你用的是 PPPoE 的话，可能需要看情况把下文提到的 eth0 换成 ppp0。

一条命令（`ebtables`）的解决方案

如果不需要给两个 VLAN 划分独立子网的话，只需要一条 ebtables 命令就可以了：

# 在 AX88U 上，eth3 是物理端口 LAN 2 
ebtables -A FORWARD -i eth3 -o br0 -j DROP

这里的 eth3 是物理端口 LAN 2。这条命令使用了二层防火墙 ebtables，用于阻隔 eth3 和 br0 之间的通信。由于 eth3 还在 br0 里，所以不需要额外配置 iptables 或者 DHCP。

基于网桥的解决方案

如果需要给两个 VLAN 划分独立的子网的话，就需要创建新的网桥了。下文以 VLAN 1 使用 192.168.50.0/24、VLAN 2 使用 192.168.150.0/24 为例。

新建网桥 `br1`

首先，从 br0 里移除 eth3，创建 br1 并把 eth3 加入 br1：

# 从 br0 中删除需要隔离的端口
brctl delif br0 eth3

# 把需要隔离的端口加入 br1
brctl addbr br1
brctl stp br1 on # 启用 STP 协议防止环路，避免网络风暴
brctl addif br1 eth3

这里新建网桥 br1 而不是直接使用 eth3 是为了方便管理：如果以后想把其他端口也隔离进 VLAN 2，只需要把对应的端口移出 br0 并加入 br1 即可。由于下文提到的所有规则都是针对 br1 的，所以新加入的端口不需要额外的设置。

执行 brctl show 验证 br1 已正确创建：

admin@ax88u:/# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.a85e45fakeid       yes             eth1
                                                        eth2
                                                        eth4
                                                        eth5
                                                        eth6
                                                        eth7
br1             8000.a85e45fakeid       yes             eth3

然后，给 br1 设置 IPv4 地址 192.168.150.1/24，并启用 br1：

# 设置 br1 的 IPv4 地址并启用 br1
# 这里使用的 IPv4 子网是 192.168.150.0/24
# 系统会自动设置 IPv6 链路本地地址
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up

由于 AX88U 默认给 br0 分配了 192.168.50.1/24，所以不需要设置 br0。

最后，执行 ifconfig br 验证 br1 的 IPv4 地址：

admin@ax88u:/# ifconfig br1
br1       Link encap:Ethernet  HWaddr A8:5E:45:00:FA:KE
          inet addr:192.168.150.1  Bcast:192.168.150.255  Mask:255.255.255.0
          inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

系统会使用 EUI-64 算法自动为 br1 生成 IPv6 链路本地地址。

提示：完整的脚本可以在脚本与配置文件：/jffs/scripts/services-start 中找到。

添加 `iptables` 规则

首先，允许来自 br1 的传入连接：

# 允许来自 br1 的传入连接
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT

但是禁止 br1 访问路由器 Web UI 和 SSH：

# 禁止 br1 访问路由器 Web UI 和 SSH
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP

然后，不转发来自 br1 的数据包：

# 不转发来自 br1 的数据包
iptables -I FORWARD -i br1 -j DROP

但是允许 br1 内部的转发：

# 但是允许 br1 内部的转发
iptables -I FORWARD -i br1 -o br1 -j ACCEPT

也允许 br1 的数据包转发至 eth0 (WAN)：

# 允许 br1 的数据包转发至 eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT

也允许 br0 到 br1 的单向通信：

# 允许 br0 到 br1 的单向通信
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT

最后，配置 br1 所属子网 192.168.150.0/24 的 NAT：

iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
  -o br1 -j MASQUERADE

分别执行 iptables -S INPUT，iptables -S FORWARD 和 iptables -t nat -S POSTROUTING 验证相应的规则已添加:

admin@ax88u:/# iptables -S INPUT
-P INPUT ACCEPT
-A INPUT -i br1 -p tcp -m tcp --dport 22 -j DROP
-A INPUT -i br1 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i br1 -m state --state NEW -j ACCEPT
# ... 省略多余输出 ...
-A INPUT -j DROP

admin@ax88u:/# iptables -S FORWARD
-P FORWARD DROP
-A FORWARD -i br1 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br0 -o br1 -j ACCEPT
-A FORWARD -i br1 -o eth0 -j ACCEPT
-A FORWARD -i br1 -o br1 -j ACCEPT
-A FORWARD -i br1 -j DROP
# ... 省略多余输出 ...

admin@ax88u:/# iptables -t nat -S POSTROUTING
-P POSTROUTING ACCEPT
# ... 省略多余输出 ...
-A POSTROUTING -s 192.168.50.0/24 -d 192.168.50.0/24 -o br0 -j MASQUERADE
-A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 -o br1 -j MASQUERADE

提示：

完整防火墙配置脚本：脚本与配置文件：/jffs/scripts/firewall-start
完整 NAT 配置脚本：脚本与配置文件：/jffs/scripts/nat-start

配置 DHCPv4 (`dnsmasq`)

在 /jffs/configs/ 中创建配置文件 dnsmasq.conf.add：

touch /jffs/configs/dnsmasq.conf.add

比照 br0，为 br1 设置 DHCPv4 的地址池和选项：

cat <<EOF >> /jffs/configs/dnsmasq.conf.add
interface=br1
# DHCPv4 地址池: 192.168.150.2 - 192.168.150.254, 子网掩码: 255.255.255.0
# DHCPv4 租约时间: 86400 秒 (1 天)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 路由器 (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
EOF

重启 dnsmasq 以应用设置：

service restart_dnsmasq

执行 tail /tmp/syslog.log -n 50 验证 dnsmasq.conf.add 已被正确加载：

admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... 省略多余输出 ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
# ... 省略多余输出 ...

配置 IPv6

本文使用的是无状态 DHCPv6 (DHCPv6 stateless)，主机的 IPv6 地址是通过 SLAAC³ 自动生成的。

由于 SLAAC 使用了 EUI-64 算法，这要求 IPv6 前缀长度必须小于 /64。所以要使用下文提到的解决方案，ISP 分配的 IPv6 LAN 前缀长度必须小于等于 /63。

通过前缀提示 (Prefix Hint) 获取更短的前缀

某些 ISP 允许通过发送前缀长度提示获取小于 /64 的前缀。例如，我这里的 ISP Charter Spectrum 就允许通过前缀提示获取 /56 的前缀。

我的 AX88U 上 IPv6 的设置是：

名称	值
基本设置
联机类型	Native
DHCP-PD	启用
IPv6 内部网络设置
自动配置设置	Stateless
IPv6 DNS 设置
自动接上 DNS 服务器	关闭
IPv6 DNS 服务器 1	2606:4700:4700::1111
IPv6 DNS 服务器 2	2606:4700:4700::1001
IPv6 DNS 服务器 3	留空
自动配置设置
是否启动路由广播	启用

AX88U 上的 DHCPv6 客户端 odhcp6c 默认不会发送前缀提示。要强制 odhcp6c 发送前缀提示⁴，首先执行：

ps | sed -e 's/^.*\odhcp6c \(.*\)$/\1/;t;d' | head -n 1

获取系统运行 odhcp6c 的命令：

-df -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0

<duid> 和 <iaid> 与路由器的 MAC 地址有关。取决于 IPv6 设置，可能还有类似 -r23、-r24 的参数，在下面的命令里也要加上这些参数。

对应替换 <duid> 和 <iaid>，执行下面的两条命令：

# 结束现有的 odhcp6c
killall odhcp6c
# 重新运行 odhcp6c，前缀提示长度为 56
# 对应替换 -c 和 -FP 的参数 <duid> 和 <iaid>
odhcp6c -df -P56 -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0

这里 -P56 中的 56 即是要发送的前缀提示长度。

执行 ifconfig br0 验证 br0 已被分配到 /56 的 IPv6 全局地址：

admin@ax88u:/# ifconfig br0
br0       Link encap:Ethernet  HWaddr A8:5E:45:00:FA:KE
          inet addr:192.168.50.1  Bcast:192.168.50.255  Mask:255.255.255.0
          inet6 addr: 2600:6c51:fake:n00b::1/56 Scope:Global
          inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:4241 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3714 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:829133 (809.7 KiB)  TX bytes:1787334 (1.7 MiB)

如果没看到类似的输出，那下文的解决方案并不适合你，但你可以在 br1 上设置有状态 DHCPv6 (DHCPv6 stateful)。

提示：完整的脚本可以在脚本与配置文件：/jffs/scripts/wan-event 中找到。

添加 `ip6tables` 规则

ip6tables 的规则类似上文 iptables 的规则，但是不需要设置 NAT。

INPUT 链：

# 允许来自 br1 的传入连接
ip6tables -I INPUT -i br1 -j ACCEPT # br0 的默认规则
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT

# 禁止 br1 访问路由器 Web UI 和 SSH
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP

FORWARD 链：

# 不转发来自 br1 的数据包
ip6tables -I FORWARD -i br1 -j DROP

# 但是允许 br1 内部的转发
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT

# 也允许 `br1` 的数据包转发至 `eth0` (WAN)
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT

# 也允许 `br0` 到 `br1` 的单向通信
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT

提示：完整的脚本可以在脚本与配置文件：/jffs/scripts/firewall-start 中找到。

配置 DHCPv6

默认情况下，br0 会使用整个 /56 的子网。如果要为 br1 划分子网，则需要改变 br0 的前缀长度。

首先，执行 ip -6 addr show br0 scope global 获取 ISP 分配的 IPv6 LAN 前缀：

admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
    inet6 2600:6c51:fake:n00b::1/56 scope global
       valid_lft forever preferred_lft forever

你拿到的前缀肯定跟这里的是不一样的，所以执行下面的命令时别忘了对应替换。

然后，为 br0 重新设置 /64 的前缀长度：

# 删除 /56 前缀，把 2600:6c51:fake:n00b::1 换成你的
ip -6 addr del 2600:6c51:fake:n00b::1/56 dev br0
# 设置 /64 前缀，把 2600:6c51:fake:n00b::1 换成你的
ip -6 addr add 2600:6c51:fake:n00b::1/64 dev br0

当然这里可以为 br0 设置更短的前缀长度，但是我觉得 /64 已经够用了。

再次执行 ip -6 addr show br0 scope global，检查 br0 的新前缀：

admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
    inet6 2600:6c51:fake:n00b::1/64 scope global
       valid_lft forever preferred_lft forever

由于 dnsmasq 并不支持 DHCPv6 前缀代理（DHCPv6 Prefix Delegation, DHCPv6-PD）⁵ ⁶ 所需的 option 25 和 26 ⁷，下面分两种情况讨论。

不使用 DHCPv6-PD

如果不接二级路由器的话，倒是可以用 dnsmasq 当 DHCPv6 服务器。

首先，为 br1 设置 br0 的邻接 /64 前缀：

# 为 br1 设置 br0 的邻接 /64 前缀
# 比如 br0 的前缀为 aaaa:bbbb:cccc:dddd::1/64,
# 则 br1 的前缀为 aaaa:bbbb:cccc:ddde::1/64.
# 把 2600:6c51:fake:n00c::1 换成你的
ip -6 addr add 2600:6c51:fake:n00c::1/64 dev br1

然后，为 br1 设置 DHCPv6 地址池和选项：

cat <<EOF >> /jffs/configs/dnsmasq.conf.add
# DHCPv6 路由广播间隔：10 秒, 路由器生命周期：600 秒
ra-param=br1,10,600
# DHCPv6 地址池：通过 br1 的前缀构造
# DHCPv6 前缀长度：64, 模式：无状态 DHCPv6
# DHCPv6 租约时间：600 秒 (10 分钟)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23)：从路由器继承
dhcp-option=br1,option6:23,[::]
EOF

重启 dnsmasq 以应用设置：

service restart_dnsmasq

执行 tail /tmp/syslog.log -n 50 验证 dnsmasq.conf.add 已被正确加载：

admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... 省略多余输出 ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00c::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: IPv6 router advertisement enabled
# ... 省略多余输出 ...

使用 DHCPv6-PD

如果需要在二级路由器上启用 DHCPv6-PD，则应该使用 6relayd 作为 DHCPv6-PD 服务器。

注意：如果使用 6relayd，不要在 /jffs/configs/dnsmasq.conf.add 中添加 DHCPv6 相关的设置。

在 6relayd⁸ 的服务器模式下，它会通过 DHCPv6-PD 把第二个 /64 前缀分配给二级路由器。所以，只需要为 br1 分配一个 /63 的前缀。

这里使用的是 br0 的邻接 /63 前缀：

# 为 br1 设置 br0 的邻接 /63 前缀
# 把 2600:6c51:fake:n00c::1 换成你的
ip -6 addr add 2600:6c51:fake:n00c::1/63 dev br1

然后，执行 6relayd -v -d -S . br1 启动 DHCPv6-PD 服务器：

# 监听于 br1 上的 DHCPv6-PD 服务器
6relayd -v -d -S . br1

如果收到 DHCPv6 请求的话，系统日志 (cat /tmp/syslog.log | grep 6relayd) 应该显示：

# admin@ax88u:/# cat /tmp/syslog.log | grep 6relayd
Apr 15 18:20:00 6relayd[1765]: Got DHCPv6 request
Apr 15 18:20:01 6relayd[1765]: Got DHCPv6 request

最后，在二级路由器上检查 IPv6 设置（系统记录 -> IPv6）：

IPv6 Connection Type	Native with DHCP-PD
WAN IPv6 Address	2600:6c51:fake:n00c::fake
WAN IPv6 Gateway	fe80::aa5e:45ff:fe00:fake
LAN IPv6 Address	2600:6c51:fake:n00d::1/64
LAN IPv6 Link-Local Address	fe80::a62:66ff:fe01:fake/64
DHCP-PD	Enabled
LAN IPv6 Prefix	2600:6c51:fake:n00d::/64
DNS Address	2600:6c51:fake:n00c::1

如果一切正常，LAN IPv6 Prefix 应该显示 2600:6c51:fake:n00d::/64。

提示： 自动配置 6relayd 的脚本可以在脚本与配置文件：/jffs/scripts/dhcpc-event 中找到。

脚本与配置文件

把下文中的脚本与配置文件上传到路由器对应的目录里，即可在路由器每次重启时自动配置 VLAN。

注意 1：路由器 系统管理 -> 系统设置 中的 Enable JFFS custom scripts and configs 需要设为是。

注意 2：上传完毕之后，运行 chmod +x /jffs/scripts/* 为这些脚本赋予可执行权限。

`/jffs/scripts/services-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/000-services-start

# Physical port to interface map:
# eth0   WAN
# eth1   LAN 4
# eth2   LAN 3
# eth3   LAN 2
# eth4   LAN 1
# eth5   Bridge of LAN 5, LAN 6, LAN 7, LAN 8
# eth6   2.4 GHz Radio
# eth7   5 GHz Radio

# Delete those interfaces that we want to isolate from br0
logger -t "isolate_port" "services-start: deleting LAN 2 (eth3) from br0"
brctl delif br0 eth3

# Create a new bridge br1 for isolated interfaces
logger -t "isolate_port" "services-start: creating br1 with LAN 2 (eth3)"
brctl addbr br1
brctl stp br1 on # STP to prevent bridge loops
brctl addif br1 eth3

# Set up the IPv4 address for br1
# Here we set the subnet to be 192.168.150.0/24
# IPv6 link local address will be assigned automatically
logger -t "isolate_port" "services-start: setting up IPv4 address for br1"
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up

logger -t "isolate_port" "services-start: all done"
date >> /tmp/000-services-start

`/jffs/scripts/firewall-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/000-firewall-start
logger -t "isolate_port" "firewall-start: applying INPUT rules for br1"

# Allow new incoming connections from br1
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT
ip6tables -I INPUT -i br1 -j ACCEPT # Same rule as br0 by default
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT

# Only forbid br1 access the web UI and SSH of the main router
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP

logger -t "isolate_port" "firewall-start: applying FORWARD rules for br1"

# Forbid packets from br1 to be forwarded to other interfaces
iptables -I FORWARD -i br1 -j DROP
ip6tables -I FORWARD -i br1 -j DROP

# But allow packet forwarding inside br1
iptables -I FORWARD -i br1 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT

# Allow packet forwarding between br1 and eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT

# Allow one-way traffic from br0 to br1
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT

logger -t "isolate_port" "firewall-start: all done"
date >> /tmp/000-firewall-start

`/jffs/scripts/nat-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/000-nat-start
logger -t "isolate_port" "nat-start: applying POSTROUTING rules for br1"

# NAT inside 192.168.150.0/24 on br1
iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
  -o br1 -j MASQUERADE

logger -t "isolate_port" "nat-start: all done"
date >> /tmp/000-nat-start

`/jffs/scripts/wan-event`

该脚本仅用于 IPv6。

#!/bin/sh

PD_PREFIX=56 # Change to which works for your ISP
RETRY=10

# Only run for connected event (the old wan-start)
if [ ! "$2" = "connected" ]; then
  exit 0
fi

# Make sure the script is indeed invoked
touch /tmp/000-wan-event-connected

# ipv6_service = dhcp6 means Connection Type: Native, 
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
   [ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then

  # Try to find odhcp6c
  i=0;
  while [ $i -lt $RETRY ]; do
    # Get odhcp6c's pid. If there are multiple instances (unlikely), 
    # we use the smallest one.
    PID=$(pidof odhcp6c | tr ' ' '\n' | head -n 1)
    CMDLINE=/proc/$PID/cmdline

    # Found odhcp6c?
    if [ ! -z "$PID" ] && [ -f "$CMDLINE" ]; then
      COMMAND="$(tr '\0' ' ' < /proc/$PID/cmdline)"
      logger -t "isolate_port" "wan-event[connected]:" \
        "found odhcp6c, PID: $PID, command: $COMMAND"

      # The first 11 chars should be "odhcp6c -df"
      PREFIX=$(echo $COMMAND | cut -c1-11)

      if [ "$PREFIX" = "odhcp6c -df" ]; then
        # There is a space between "odhcp6c -df" and remaining arguments.
        # So we start from the 13rd char.
        ARGS=$(echo $COMMAND | cut -c13-)

        # Check if arguments start with -P$PD_PREFIX
        ARG1=$(echo $ARGS | cut -c1-$(expr length "$PD_PREFIX" + 2))
        if [ ! "$ARG1" = "-P$PD_PREFIX" ]; then
          # Prefix length (-P$PD_PREFIX)
          COMMAND="odhcp6c -df -P$PD_PREFIX $ARGS"
          logger -t "isolate_port" "wan-event[connected]:" \
            "re-run odhcp6c with prefix hint $PD_PREFIX: $COMMAND"

          killall odhcp6c
          eval $COMMAND
        else
          logger -t "isolate_port" "wan-event[connected]:" \
            "odhcp6c already started with prefix hint $PD_PREFIX"
        fi
      else
        logger -t "isolate_port" "wan-event[connected]:" \
          "odhcp6c command prefix mismatch!" \
          "found '$PREFIX', expects 'odhcp6c -df'"
      fi

      # We break from here once found the `odhcp6c` process
      break
    else
      i=$(($i+1))
      logger -t "isolate_port" "wan-event[connected]:" \
        "odhcp6c not found ($i/$RETRY)"

      # Hope we're lucky next time
      sleep 1
    fi
  done
else
  logger -t "isolate_port" "wan-event[connected]: DHCPv6 PD not enabled"
fi

logger -t "isolate_port" "wan-event[connected]: all done"
date >> /tmp/000-wan-event-connected

`/jffs/scripts/dhcpc-event`

该脚本仅用于 IPv6。

#!/bin/sh

link_local_ipv6() {
  # Get IPv6 link local address from br0 (with the prefix length part)
  echo $(ip -6 addr show br0 scope link |\
    sed -e's/^.*inet6 \([^ ]*\).*$/\1/;t;d')
}

setup_br1() {
  # ipv6_service = dhcp6 means Connection Type: Native, 
  # also we require DHCP PD to be enabled
  if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
     [ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
    
    # Wait for udhcpc (udhcpc.c: bound6 -> add_ip6_lanaddr). 
    # We need to the updated ipv6 prefix from nvram.
    sleep 2

    # Allocated IPv6 prefix from ISP
    PREFIX=$(nvram get ipv6_prefix)

    if [ -z "$PREFIX" ]; then
      logger -t "isolate_port" "dhcpc-event: empty IPv6 prefix"
    else
      PD_PREFIX=$(nvram get ipv6_prefix_length)
      if [ "$PD_PREFIX" -ge 63 ]; then
        # We need at least 3 /64 subnets, /63 only gives 2 /64 subnet
        logger -t "isolate_port" "dhcpc-event: IPv6 prefix" \
          "(/$PD_PREFIX) is too long, at least /62 requried"
      else
        if [ "$PD_PREFIX" -le 48 ]; then
          # Shorter than /49? 
          # br1 will on a:b:c:2::/63 (br0 is on a:b:c:0::/64)
          BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):2::"
        else
          # /49 to /62? 
          # br1 will on a:b:c:dddf::/63 (br0 is on a:b:c:dddd::/64)
          BR0_NET_ID=$(echo $PREFIX | cut -d':' -f4)
          BR1_NET_ID=$(printf "%x" $((0x$BR0_NET_ID + 0x2)))
          BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):${BR1_NET_ID}::"
        fi

        # Clean up br0
        ip -6 route flush dev br0
        ip -6 addr flush dev br0 scope global

        # Assign br0 with the first /64 subnet (instead the /56 one)
        # ipv6_rtr_addr is the default router's IPv6 address 
        logger -t "isolate_port" "dhcpc-event: set prefix" \
          "$(nvram get ipv6_rtr_addr)/64 to br0"
        ip -6 addr add "$(nvram get ipv6_rtr_addr)/64" dev br0

        # Clean up br1 (note we also remove link local address)
        ip -6 route flush dev br1
        ip -6 addr flush dev br1

        # Re-add link local address
        # So route to fe80::/64 will be added back
        # Note this is VERY important for 6relayd
        # Otherwise 6relayd will throw network unreachable error
        # Because route to fe80::/64 doesn't exist
        ip -6 addr add $(link_local_ipv6) dev br1

        # Assign br1 with a /63 subnet, so the cascading router 
        # will be on the first /64 subnet and it will also get
        # the second /64 prefix via DHCPv6-PD by 6relayd
        logger -t "isolate_port" "dhcpc-event: set prefix" \
          "${BR1_PREFIX}1/63 to br1"
        ip -6 addr add "${BR1_PREFIX}1/63" dev br1

        # Re-run 6relayd
        logger -t "isolate_port" "dhcpc-event: re-run 6relayd on br1"
        killall 6relayd

        # Automatic DHCPv6 server to delegate prefix on br1 in daemon
        6relayd -v -d -S . br1
      fi
    fi
  fi
}

teardown_br1() {
  # ipv6_service = dhcp6 means Connection Type: Native,
  # also we require DHCP PD to be enabled
  if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
     [ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
     
    logger -t "isolate_port" "dhcpc-event:" \
      "flush IPv6 route and IPv6 address on br1"

    # br0 will be handled by udhcpc, 
    # we only need to care about br1
    # Note we also remove link local address
    ip -6 route flush dev br1
    ip -6 addr flush dev br1

    # Re-add link local address
    # So route to fe80::/64 will be added back
    # Note this is VERY important for 6relayd
    # Otherwise 6relayd will throw network unreachable error
    # Because route to fe80::/64 doesn't exist
    ip -6 addr add $(link_local_ipv6) dev br1
  fi
}

# Make sure the script is indeed invoked
touch /tmp/000-dhcpc-event

# Adapted from odhcp6c-example-script.sh
(
  flock 9
  case "$1" in
    bound)
      teardown_br1
      setup_br1
    ;;
    informed|updated|rebound|ra-updated)
      setup_br1
    ;;
    stopped|unbound)
      teardown_br1
    ;;
    started)
      teardown_br1
    ;;
  esac
) 9>/tmp/odhcp6c.lock.br1
rm -f /tmp/odhcp6c.lock.br1

date >> /tmp/000-dhcpc-event

`/jffs/configs/dnsmasq.conf.add`

不需要 DHCPv6：

interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1

需要 DHCPv6，并且需要为二级路由器启用 DHCPv6-PD：

interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1

（你没看错，和不用 DHCPv6 是一样的）

需要 DHCPv6，但是不需要 DHCPv6-PD：

interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
# DHCPv6 RA interval: 10s, router lifetime: 600s
ra-param=br1,10,600
# DHCPv6 range: whole subnet, constructing from br1's prefix
# DHCPv6 prefix length: 64, mode: Stateless DHCPv6
# DHCPv6 lease time: 600s (10 minutes)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23): inherit from the router
dhcp-option=br1,option6:23,[::]

参考文献

slh, Reply: ASUS RT-AX88U Router. ↩
u128393, RT-AC86U VLAN 配置 - vlanctl 篇. ↩
Wikipedia, Stateless address autoconfiguration. ↩
buddyp, Reply: IPv6 with Prefix Delegation. ↩
Simon Kelley, [Dnsmasq-discuss] IPV6 Prefix Delegation (IA_PD). ↩
Simon Kelley, dnsmasq: dhcp6-protocol.h. ↩
O. Troan, R. Droms, RFC 3633 - IPv6 Prefix Options for Dynamic Host Configuration Protocol (DHCP) version 6. ↩
Steven Barth, 6relayd - Embedded DHCPv6/RA Server & Relay. ↩

LAN port isolation (port-based VLAN) on ASUS RT-AX88U with Asuswrt-Merlin 384.16

2020-04-15T18:32:03-07:00

Recently I upgraded my 4-years-old AC68P to the 802.11ax (or WiFi 6, but I prefer the former) supported AX88U. Like my old AC68P, I flashed Asuswrt-Merlin on AX88U since day one.

However, one issue with Asuswrt (either the stock firmware or Merlin) is no built-in GUI support on VLANs. I really miss the good old days with OpenWrt, as you can configure VLANs in GUI. But OpenWrt won’t support AX88U in a foreseeable future (due to Broadcom’s closed source driver¹), CLI seems to be the only solution.

Requirement

Pretty straightforward. There are 8 LAN ports on AX88U and I want to isolate LAN 2 from the remaining 7 ports. LAN 1 and LAN 3 through 8 will be in VLAN 1, while LAN 2 will be isolated into VLAN 2 (guest network).

The traffic between two VLANs should be forbidden. But ideally, I want to allow one way traffic from VLAN 1 to VLAN 2. So I can still access the cascading router (in VLAN 2) from my computer (in VLAN 1), in case I need to reconfigure the secondary router.

In addition, I prefer everything to be configured automatically. So for clients in VLAN 2, the only thing is to plug in the ethernet cable and get connected to the Internet.

About AX88U

No `robocfg`?

The first problem is that robocfg is no more provided on AX88U (Broadcom’s HND platform). An alternative to robocfg on HND platform seems to be vlanctl². However, after several hours of searching, trying and error, I believe vlanctl can only create tagged VLAN, which unfortunately can’t satisfy my need.

But in case someone else has special demand on tagged VLAN, I recommend to read (in Chinese): 上海电信 TL-EP110 + RT-AC86U 实现观看 4K IPTV 无卡顿 (2019-10). This post describes how to set tagged VLANs up properly on AC86U (also a HND platform router).

Interface to Physical Port Mapping

Somehow I didn’t find it in the Internet. After playing with an ethernet cable on different ports, I figured out:

Interface	Physical Port
eth0	WAN
eth1	LAN 4
eth2	LAN 3
eth3	LAN 2
eth4	LAN 1
eth5	Bridge of LAN 5 - 8
eth6	2.4 GHz Radio
eth7	5 GHz Radio

Note that eth5 seems to be a hardware bridge (BCM53134) of LAN 5 - 8. You may be able to ungroup it with ethctl or ethswctl, but I didn’t spend much time on them. My solution can only isolate LAN ports 1 - 4.

By default, eth1 ~ eth7 are grouped in bridge br0.

Note: My WAN Connection Type is Automatic IP. If you are using other types like PPPoE, you may need to replace eth0 to ppp0 in the below accordingly.

One-liner with `ebtables`

If you don’t care about separating ports into different subnets, there actually exists an one-line solution:

# eth3 maps to LAN port 2 on AX88U 
ebtables -A FORWARD -i eth3 -o br0 -j DROP

Here eth3 maps to LAN 2. The layer 2 firewall ebtables will essentially block access between eth3 and remaining interfaces in br0.

Since eth3 is still in the same subnet as br0, there is no need to worry about iptables or DHCP. Everything will just be fine.

Note 1: If it doesn’t work, try ebtables -A FORWARD -i eth3 --logical-out br0 -j DROP. Thanks to @MarianMaciag!

Note 2: If it still doesn’t work, try iptables instead: iptables -A FORWARD -i eth3 -o br0 -j DROP. The downside with iptables is that only layer 3+ access is restricted.

Note 3: Also remember this only blocks access in one direction: eth3 to br0. You need -i br0 -o eth3 (or --logical-in br0 -o eth3) to block br0 to eth3.

Note 4: If it again doesn’t work, go separate bridge approach below.

Better Approach with Separate Bridge

However, my goal is to associate two VLANs with two subnets. Say VLAN 1 with 192.168.50.0/24 and VLAN 2 with 192.168.150.0/24.

Create Bridge `br1`

First, remove eth3 from br0 and create a new bridge br1 with eth3:

# Delete those interfaces that we want to isolate from br0
brctl delif br0 eth3

# Create a new bridge br1 for our isolated interfaces
brctl addbr br1
brctl stp br1 on # STP to prevent bridge loops
brctl addif br1 eth3

Here we use bridge br1 for easier management. If you want to add isolate other LAN ports, you can simply add the corresponding interface to br1. All rules on br1 will automatically apply to that port.

Now run brctl show to verify the settings for br1:

admin@ax88u:/# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.a85e45fakeid       yes             eth1
                                                        eth2
                                                        eth4
                                                        eth5
                                                        eth6
                                                        eth7
br1             8000.a85e45fakeid       yes             eth3

By default, br0 is assigned with 192.168.50.1/24 on AX88U, so we don’t have to worry about it

Then, assign 192.168.150.1/24 to br1 and bring it up:

# Set up the IPv4 address for br1 and bring it up
# Here we set the subnet to be 192.168.150.0/24
# IPv6 link local address will be assigned automatically
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up

Finally, run ifconfig br1 to see if br1 is up:

admin@ax88u:/# ifconfig br1
br1       Link encap:Ethernet  HWaddr A8:5E:45:00:FA:KE
          inet addr:192.168.150.1  Bcast:192.168.150.255  Mask:255.255.255.0
          inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

The IPv6 link local address is automatically generated by EUI-64.

Tip: Complete script can be found in Scripts & Configs: /jffs/scripts/services-start.

Add `iptables` Rules

First, allow new incoming connections from br1 to the router:

# Allow new incoming connections from br1
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT

But forbid accessing web UI and SSH from br1:

# Only forbid br1 access the web UI and SSH of the main router
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP

Then, drop all forwarding packets from br1:

# Forbid packets from br1 to be forwarded to other interfaces
iptables -I FORWARD -i br1 -j DROP

But allow packet forwarding inside br1:

# But allow packet forwarding inside br1
iptables -I FORWARD -i br1 -o br1 -j ACCEPT

… also allow packet forwarding between br1 and eth0 (WAN):

# Allow packet forwarding between br1 and eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT

… also allow one-way traffic from br0 to br1:

# Allow one-way traffic from br0 to br1
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT

Finally, set up NAT inside 192.168.150.0/24 on br1:

iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
  -o br1 -j MASQUERADE

Now run iptables -S INPUT, iptables -S FORWARD and iptables -t nat -S POSTROUTING to verify that rules have been added:

admin@ax88u:/# iptables -S INPUT
-P INPUT ACCEPT
-A INPUT -i br1 -p tcp -m tcp --dport 22 -j DROP
-A INPUT -i br1 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i br1 -m state --state NEW -j ACCEPT
# ... output omitted ...
-A INPUT -j DROP

admin@ax88u:/# iptables -S FORWARD
-P FORWARD DROP
-A FORWARD -i br1 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br0 -o br1 -j ACCEPT
-A FORWARD -i br1 -o eth0 -j ACCEPT
-A FORWARD -i br1 -o br1 -j ACCEPT
-A FORWARD -i br1 -j DROP
# ... output omitted ...

admin@ax88u:/# iptables -t nat -S POSTROUTING
-P POSTROUTING ACCEPT
# ... output omitted ...
-A POSTROUTING -s 192.168.50.0/24 -d 192.168.50.0/24 -o br0 -j MASQUERADE
-A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 -o br1 -j MASQUERADE

Tip: Complete scripts can be found below.

Script for firewall: Scripts & Configs: /jffs/scripts/firewall-start.
Script for NAT: Scripts & Configs: /jffs/scripts/nat-start.

Configure `dnsmasq` for DHCPv4

Create an additional configuration file dnsmasq.conf.add in /jffs/configs/:

touch /jffs/configs/dnsmasq.conf.add

Similar to br0, set up DHCPv4 ranges and options on br1:

cat <<EOF >> /jffs/configs/dnsmasq.conf.add
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
EOF

Restart dnsmasq to apply the config:

service restart_dnsmasq

Now run tail /tmp/syslog.log -n 50 to see if dnsmasq.conf.add is loaded:

admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... output omitted ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
# ... output omitted ...

What about IPv6?

Even in the scenario of stateless DHCPv6, hosts’ IPv6 addresses are still generated via SLAAC³ according to RFC 3736:

A node that uses stateless DHCP must have obtained its IPv6 addresses through some other mechanism, typically stateless address autoconfiguration (or SLAAC).

Since SLAAC uses EUI-64 algorithm, the subnet prefix is required to be shorter than /64. That means to do subnetting with stateless DHCPv6, the IPv6 LAN prefix allocated from your ISP must be at least of /63.

Prefix Hint for Shorter Subnet Prefix

Some ISPs allow you send prefix length hint to get a prefix shorter than /64 (fortunately Charter Spectrum does assign /56). If you’re not sure about this, you can try the following steps.

By the way, my AX88U’s IPv6 configuration is:

Name	Value
Basic Config
Connection type	Native
DHCP-PD	Enable
IPv6 LAN Setting
Auto Configuration Setting	Stateless
IPv6 DNS Setting
Connect to DNS Server automatically	Disable
IPv6 DNS Server 1	2606:4700:4700::1111
IPv6 DNS Server 2	2606:4700:4700::1001
IPv6 DNS Server 3	<leave blank>
Auto Configuration Setting
Enable Router Advertisement	Enable

By default, the DHCPv6 client odhcp6c on AX88U wouldn’t send prefix hint.
To force it do so⁴, first run

ps | sed -e 's/^.*\odhcp6c \(.*\)$/\1/;t;d' | head -n 1

to get the arguments of odhcp6c:

-df -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0

<duid> and <iaid> are linked to your router’s MAC address. Depends on your IPv6 configuration, there may be additional arguments like -r23 and -r24, don’t forget to append them too.

So in the next two commands, replace them with the output you got:

# Kill existing odhcp6c.
killall odhcp6c
# Re-run odhcp6c with prefix hint 56.
# Replace `-c` and `-FP` arguments with the `ps` output you got.
odhcp6c -df -P56 -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0

… and -P56 determines which prefix hint length to send.

Now you can run ifconfig br0 to verify if /56 subnet is assigned:

admin@ax88u:/# ifconfig br0
br0       Link encap:Ethernet  HWaddr A8:5E:45:00:FA:KE
          inet addr:192.168.50.1  Bcast:192.168.50.255  Mask:255.255.255.0
          inet6 addr: 2600:6c51:fake:n00b::1/56 Scope:Global
          inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:4241 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3714 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:829133 (809.7 KiB)  TX bytes:1787334 (1.7 MiB)

If something similar is printed to your screen, Congratulations! You can proceed to the next section. Otherwise, you may have to stop here and try to set up stateful DHCPv6 server on br1, which I didn’t try.

Tip: Complete script for re-running odhcp6c automatically after reboot can be found in Scripts & Configs: /jffs/scripts/wan-event.

Add `ip6tables` Rules

Similiar to IPv4 solution, some necessary ip6tables rules have to be added. But no need to deal with NAT (It’s IPv6!).

For INPUT chain:

# Allow new incoming connections from br1
ip6tables -I INPUT -i br1 -j ACCEPT # Same rule as br0 by default
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT

# Only forbid br1 access the web UI and SSH of the main router
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP

For FORWARD chain:

# Forbid packets from br1 to be forwarded to other interfaces
ip6tables -I FORWARD -i br1 -j DROP

# But allow packet forwarding inside br1
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT

# Allow packet forwarding between br1 and eth0 (WAN)
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT

# Allow one-way traffic from br0 to br1
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT

Tip: Complete script can be found in Scripts & Configs: /jffs/scripts/firewall-start.

Configure DHCPv6

By default, br0 will take the whole /56 subnet. We need to change it to /64 for subnetting on br1.

First, check what prefix br0 has gotten from ISP with ip -6 addr show br0 scope global:

admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
    inet6 2600:6c51:fake:n00b::1/56 scope global
       valid_lft forever preferred_lft forever

You IPv6 address is different so remember to replace 2600:6c51:fake:n00b::1 with yours in following commands.

Then, reassign br0 with a /64 subnet:

# Remove /56 subnet. Replace 2600:6c51:fake:n00b::1 with yours.
ip -6 addr del 2600:6c51:fake:n00b::1/56 dev br0
# Assign /64 subnet. Replace 2600:6c51:fake:n00b::1 with yours.
ip -6 addr add 2600:6c51:fake:n00b::1/64 dev br0

Of course you can make a shorter subnet prefix for br0, but I feel comfortable enough with /64.

Next, run ip -6 addr show br0 scope global again to check br0’s new perfix:

admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
    inet6 2600:6c51:fake:n00b::1/64 scope global
       valid_lft forever preferred_lft forever

The subnet for br1 is a little bit tricky, because dnsmasq doesn’t support DHCPv6 Prefix Delegation (PD), quoting from dnsmasq’s author⁵:

You’re right, though this (dnsmasq does not support replying to IA_PD (prefix delegation) requests) may change in the future.

The code of dnsmasq shipped in Merlin⁶ also proves no support of DHCPv6 prefix delegation (option 25 and 26⁷).

Without Prefix Delegation

If you have no cascading routers or don’t care about PD, dnsmasq can be used as DHCPv6 server.

First, assign br1 with the sibling /64 subnet of br0:

# Assign the sibling /64 subnet of br0 to br1.
# For example, if br0 is on aaaa:bbbb:cccc:dddd::1/64,
# then br1 should on aaaa:bbbb:cccc:ddde::1/64.
# Replace 2600:6c51:fake:n00c::1 with yours.
ip -6 addr add 2600:6c51:fake:n00c::1/64 dev br1

Then, set up DHCPv6 ranges and options on br1:

cat <<EOF >> /jffs/configs/dnsmasq.conf.add
# DHCPv6 RA interval: 10s, router lifetime: 600s
ra-param=br1,10,600
# DHCPv6 range: whole subnet, constructing from br1's prefix
# DHCPv6 prefix length: 64, mode: Stateless DHCPv6
# DHCPv6 lease time: 600s (10 minutes)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23): inherit from the router
dhcp-option=br1,option6:23,[::]
EOF

Restart dnsmasq to apply the config:

service restart_dnsmasq

Now run tail /tmp/syslog.log -n 50 to see if dnsmasq.conf.add is loaded:

admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... output omitted ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00c::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: IPv6 router advertisement enabled
# ... output omitted ...

With Prefix Delegation

Otherwise, if you want cascading router to get IPv6 prefix automatically through PD, 6relayd is required as a minimalistic DHCPv6 PD server.

Note: Don’t add DHCPv6 range and options to /jffs/configs/dnsmasq.conf.add if you wish to use 6relayd.

In the sever mode of 6realyd⁸:

If there are non-local addresses assigned to the slave interface when a router solicitation is received, said prefixes are announced automatically for stateless autoconfiguration and also offered via stateful DHCPv6. If all prefixes are bigger than /64 all but the first /64 of these prefixes is offered via DHCPv6-PD to downstream routers.

… that is what we want. We’ll assign a /63 subnet for br1, and 6relayd will delegate a /64 subnet for the cascading router.

First, assign a 63 subnet for br1 (here we use the sibling /63 subnet of br0):

# Assign the sibling /63 subnet of br0 to br1.
# Use a IPv6 subnet tool for the `/63` subnet range on `br0`.
# Replace 2600:6c51:fake:n00c::1 with yours.
ip -6 addr add 2600:6c51:fake:n00c::1/63 dev br1

Then, run 6relayd -v -d -S . br1 for DHCPv6 PD server:

# Automatic DHCPv6 server to delegate prefix on br1 in daemon
6relayd -v -d -S . br1

If there are incoming DHCPv6 requests, system log (cat /tmp/syslog.log | grep 6relayd) should show:

# admin@ax88u:/# cat /tmp/syslog.log | grep 6relayd
Apr 15 18:20:00 6relayd[1765]: Got DHCPv6 request
Apr 15 18:20:01 6relayd[1765]: Got DHCPv6 request

… which means 6relayd is working properly.

Now check the System Log -> IPv6 page in the cascading router (here my old AC68P is the cascading one):

IPv6 Connection Type	Native with DHCP-PD
WAN IPv6 Address	2600:6c51:fake:n00c::fake
WAN IPv6 Gateway	fe80::aa5e:45ff:fe00:fake
LAN IPv6 Address	2600:6c51:fake:n00d::1/64
LAN IPv6 Link-Local Address	fe80::a62:66ff:fe01:fake/64
DHCP-PD	Enabled
LAN IPv6 Prefix	2600:6c51:fake:n00d::/64
DNS Address	2600:6c51:fake:n00c::1

If everything is good, LAN IPv6 Prefix should show 2600:6c51:fake:n00d::/64.

Tip: Automatic script for setting up DHCPv6 PD can be found in Scripts & Configs: /jffs/scripts/dhcpc-event.

Scripts & Configs

In order to apply all settings after reboot, save the following scripts and configs into corresponding folders.

Note 1: Don’t forget to set Enable JFFS custom scripts and configs to Yes in Administration -> System.

Note 2: After uploading the following scripts, don’t forget to mark them as executable with chmod +x /jffs/scripts/*.

`/jffs/scripts/services-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/000-services-start

# Physical port to interface map:
# eth0   WAN
# eth1   LAN 4
# eth2   LAN 3
# eth3   LAN 2
# eth4   LAN 1
# eth5   Bridge of LAN 5, LAN 6, LAN 7, LAN 8
# eth6   2.4 GHz Radio
# eth7   5 GHz Radio

# Delete those interfaces that we want to isolate from br0
logger -t "isolate_port" "services-start: deleting LAN 2 (eth3) from br0"
brctl delif br0 eth3

# Create a new bridge br1 for isolated interfaces
logger -t "isolate_port" "services-start: creating br1 with LAN 2 (eth3)"
brctl addbr br1
brctl stp br1 on # STP to prevent bridge loops
brctl addif br1 eth3

# Set up the IPv4 address for br1
# Here we set the subnet to be 192.168.150.0/24
# IPv6 link local address will be assigned automatically
logger -t "isolate_port" "services-start: setting up IPv4 address for br1"
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up

logger -t "isolate_port" "services-start: all done"
date >> /tmp/000-services-start

`/jffs/scripts/firewall-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/000-firewall-start
logger -t "isolate_port" "firewall-start: applying INPUT rules for br1"

# Allow new incoming connections from br1
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT
ip6tables -I INPUT -i br1 -j ACCEPT # Same rule as br0 by default
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT

# Only forbid br1 access the web UI and SSH of the main router
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP

logger -t "isolate_port" "firewall-start: applying FORWARD rules for br1"

# Forbid packets from br1 to be forwarded to other interfaces
iptables -I FORWARD -i br1 -j DROP
ip6tables -I FORWARD -i br1 -j DROP

# But allow packet forwarding inside br1
iptables -I FORWARD -i br1 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT

# Allow packet forwarding between br1 and eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT

# Allow one-way traffic from br0 to br1
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT

logger -t "isolate_port" "firewall-start: all done"
date >> /tmp/000-firewall-start

`/jffs/scripts/nat-start`

#!/bin/sh

# Make sure the script is indeed invoked
touch /tmp/000-nat-start
logger -t "isolate_port" "nat-start: applying POSTROUTING rules for br1"

# NAT inside 192.168.150.0/24 on br1
iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
  -o br1 -j MASQUERADE

logger -t "isolate_port" "nat-start: all done"
date >> /tmp/000-nat-start

`/jffs/scripts/wan-event`

This script is only for IPv6.

#!/bin/sh

PD_PREFIX=56 # Change to which works for your ISP
RETRY=10

# Only run for connected event (the old wan-start)
if [ ! "$2" = "connected" ]; then
  exit 0
fi

# Make sure the script is indeed invoked
touch /tmp/000-wan-event-connected

# ipv6_service = dhcp6 means Connection Type: Native, 
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
   [ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then

  # Try to find odhcp6c
  i=0;
  while [ $i -lt $RETRY ]; do
    # Get odhcp6c's pid. If there are multiple instances (unlikely), 
    # we use the smallest one.
    PID=$(pidof odhcp6c | tr ' ' '\n' | head -n 1)
    CMDLINE=/proc/$PID/cmdline

    # Found odhcp6c?
    if [ ! -z "$PID" ] && [ -f "$CMDLINE" ]; then
      COMMAND="$(tr '\0' ' ' < /proc/$PID/cmdline)"
      logger -t "isolate_port" "wan-event[connected]:" \
        "found odhcp6c, PID: $PID, command: $COMMAND"

      # The first 11 chars should be "odhcp6c -df"
      PREFIX=$(echo $COMMAND | cut -c1-11)

      if [ "$PREFIX" = "odhcp6c -df" ]; then
        # There is a space between "odhcp6c -df" and remaining arguments.
        # So we start from the 13rd char.
        ARGS=$(echo $COMMAND | cut -c13-)

        # Check if arguments start with -P$PD_PREFIX
        ARG1=$(echo $ARGS | cut -c1-$(expr length "$PD_PREFIX" + 2))
        if [ ! "$ARG1" = "-P$PD_PREFIX" ]; then
          # Prefix length (-P$PD_PREFIX)
          COMMAND="odhcp6c -df -P$PD_PREFIX $ARGS"
          logger -t "isolate_port" "wan-event[connected]:" \
            "re-run odhcp6c with prefix hint $PD_PREFIX: $COMMAND"

          killall odhcp6c
          eval $COMMAND
        else
          logger -t "isolate_port" "wan-event[connected]:" \
            "odhcp6c already started with prefix hint $PD_PREFIX"
        fi
      else
        logger -t "isolate_port" "wan-event[connected]:" \
          "odhcp6c command prefix mismatch!" \
          "found '$PREFIX', expects 'odhcp6c -df'"
      fi

      # We break from here once found the `odhcp6c` process
      break
    else
      i=$(($i+1))
      logger -t "isolate_port" "wan-event[connected]:" \
        "odhcp6c not found ($i/$RETRY)"

      # Hope we're lucky next time
      sleep 1
    fi
  done
else
  logger -t "isolate_port" "wan-event[connected]: DHCPv6 PD not enabled"
fi

logger -t "isolate_port" "wan-event[connected]: all done"
date >> /tmp/000-wan-event-connected

`/jffs/scripts/dhcpc-event`

This script is only for IPv6.

#!/bin/sh

link_local_ipv6() {
  # Get IPv6 link local address from br0 (with the prefix length part)
  echo $(ip -6 addr show br0 scope link |\
    sed -e's/^.*inet6 \([^ ]*\).*$/\1/;t;d')
}

setup_br1() {
  # ipv6_service = dhcp6 means Connection Type: Native, 
  # also we require DHCP PD to be enabled
  if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
     [ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
    
    # Wait for udhcpc (udhcpc.c: bound6 -> add_ip6_lanaddr). 
    # We need to the updated ipv6 prefix from nvram.
    sleep 2

    # Allocated IPv6 prefix from ISP
    PREFIX=$(nvram get ipv6_prefix)

    if [ -z "$PREFIX" ]; then
      logger -t "isolate_port" "dhcpc-event: empty IPv6 prefix"
    else
      PD_PREFIX=$(nvram get ipv6_prefix_length)
      if [ "$PD_PREFIX" -ge 63 ]; then
        # We need at least 3 /64 subnets, /63 only gives 2 /64 subnet
        logger -t "isolate_port" "dhcpc-event: IPv6 prefix" \
          "(/$PD_PREFIX) is too long, at least /62 requried"
      else
        if [ "$PD_PREFIX" -le 48 ]; then
          # Shorter than /49? 
          # br1 will on a:b:c:2::/63 (br0 is on a:b:c:0::/64)
          BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):2::"
        else
          # /49 to /62? 
          # br1 will on a:b:c:dddf::/63 (br0 is on a:b:c:dddd::/64)
          BR0_NET_ID=$(echo $PREFIX | cut -d':' -f4)
          BR1_NET_ID=$(printf "%x" $((0x$BR0_NET_ID + 0x2)))
          BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):${BR1_NET_ID}::"
        fi

        # Clean up br0
        ip -6 route flush dev br0
        ip -6 addr flush dev br0 scope global

        # Assign br0 with the first /64 subnet (instead the /56 one)
        # ipv6_rtr_addr is the default router's IPv6 address 
        logger -t "isolate_port" "dhcpc-event: set prefix" \
          "$(nvram get ipv6_rtr_addr)/64 to br0"
        ip -6 addr add "$(nvram get ipv6_rtr_addr)/64" dev br0

        # Clean up br1 (note we also remove link local address)
        ip -6 route flush dev br1
        ip -6 addr flush dev br1

        # Re-add link local address
        # So route to fe80::/64 will be added back
        # Note this is VERY important for 6relayd
        # Otherwise 6relayd will throw network unreachable error
        # Because route to fe80::/64 doesn't exist
        ip -6 addr add $(link_local_ipv6) dev br1

        # Assign br1 with a /63 subnet, so the cascading router 
        # will be on the first /64 subnet and it will also get
        # the second /64 prefix via DHCPv6-PD by 6relayd
        logger -t "isolate_port" "dhcpc-event: set prefix" \
          "${BR1_PREFIX}1/63 to br1"
        ip -6 addr add "${BR1_PREFIX}1/63" dev br1

        # Re-run 6relayd
        logger -t "isolate_port" "dhcpc-event: re-run 6relayd on br1"
        killall 6relayd

        # Automatic DHCPv6 server to delegate prefix on br1 in daemon
        6relayd -v -d -S . br1
      fi
    fi
  fi
}

teardown_br1() {
  # ipv6_service = dhcp6 means Connection Type: Native,
  # also we require DHCP PD to be enabled
  if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
     [ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
     
    logger -t "isolate_port" "dhcpc-event:" \
      "flush IPv6 route and IPv6 address on br1"

    # br0 will be handled by udhcpc, 
    # we only need to care about br1
    # Note we also remove link local address
    ip -6 route flush dev br1
    ip -6 addr flush dev br1

    # Re-add link local address
    # So route to fe80::/64 will be added back
    # Note this is VERY important for 6relayd
    # Otherwise 6relayd will throw network unreachable error
    # Because route to fe80::/64 doesn't exist
    ip -6 addr add $(link_local_ipv6) dev br1
  fi
}

# Make sure the script is indeed invoked
touch /tmp/000-dhcpc-event

# Adapted from odhcp6c-example-script.sh
(
  flock 9
  case "$1" in
    bound)
      teardown_br1
      setup_br1
    ;;
    informed|updated|rebound|ra-updated)
      setup_br1
    ;;
    stopped|unbound)
      teardown_br1
    ;;
    started)
      teardown_br1
    ;;
  esac
) 9>/tmp/odhcp6c.lock.br1
rm -f /tmp/odhcp6c.lock.br1

date >> /tmp/000-dhcpc-event

`/jffs/configs/dnsmasq.conf.add`

If you need DHCPv6-PD (with 6relayd) for cascading router, or if you don’t care about IPv6:

interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1

Otherwise, DHCPv6 is done by dnsmasq on br1:

interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
# DHCPv6 RA interval: 10s, router lifetime: 600s
ra-param=br1,10,600
# DHCPv6 range: whole subnet, constructing from br1's prefix
# DHCPv6 prefix length: 64, mode: Stateless DHCPv6
# DHCPv6 lease time: 600s (10 minutes)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23): inherit from the router
dhcp-option=br1,option6:23,[::]

References

slh, Reply: ASUS RT-AX88U Router. ↩
u128393, RT-AC86U VLAN 配置 - vlanctl 篇. ↩
Wikipedia, Stateless address autoconfiguration. ↩
buddyp, Reply: IPv6 with Prefix Delegation. ↩
Simon Kelley, [Dnsmasq-discuss] IPV6 Prefix Delegation (IA_PD). ↩
Simon Kelley, dnsmasq: dhcp6-protocol.h. ↩
O. Troan, R. Droms, RFC 3633 - IPv6 Prefix Options for Dynamic Host Configuration Protocol (DHCP) version 6. ↩
Steven Barth, 6relayd - Embedded DHCPv6/RA Server & Relay. ↩

Hello, World!

2020-03-12T19:59:48-07:00

I’m glad you found this place from somewhere on the Internet!

Hope the following Java code snippet may give you the idea of this post ;)

System.out.print(
    IntStream.of(1214606444, 1865162839, 1869769828, 555753482)
            .mapToObj(n -> IntStream.iterate(24, i -> i >= 0, i -> i - 8)
                                    .boxed()
                                    .map(i -> (n & 255 << i) >> i)
                                    .collect(() -> ByteBuffer.allocate(4),
                                            (r, e) -> r.put(e.byteValue()),
                                            ByteBuffer::put))
            .map(b -> new String(b.array(), StandardCharsets.UTF_8))
            .collect(Collectors.joining())
);

Tip: You need JDK 9 or higher to compile the code above, or if you were lazy like me :P, click here to see the output.

I’m Renjie Wu

在 Mikrotik RouterOS 6.48.2 上配置 DHCPv6-PD（前缀代理）

配置 DHCPv6-PD 客户端

启用 DHCPv6 客户端

添加必要的 IPv6 防火墙规则

为 LAN 端口分配 IPv6 地址

配置 DHCPv6-PD 服务器

启用 DHCPv6 服务器

配置 DHCPv6 选项

总结

Configure DHCPv6-PD (Prefix Delegation) Client & Server on Mikrotik RouterOS 6.48.2

Set up DHCPv6-PD client

Enable DHCPv6 client

Add necessary IPv6 firewall rule

Assign IPv6 address to interfaces

Set up DHCPv6-PD server

Enable DHCPv6 server

Advertise DHCPv6 options

Conclusion

Don’t use STL `regex` library in Microsoft Visual C++ (MSVC), at least for now

How could it go wrong with \t+?

Bad <regex> implementation in MSVC

++regex_token_iterator takes too long

What indeed happened?

Speed under Release configuration

How about plain \t

What about clang and gcc?

clang 11.0.1 + libc++ 11.0

gcc 5.5.0 + libstdc++ 3.4.21

Tedious string::find did the trick

Final speed comparison

Conclusion

References

在 AX88U + Merlin 384.19 上配置基于 `RTL8156B` 的 2.5GbE 网卡

不能即插即用？

准备工作

编译 Linux 内核

编译测试工具

iperf3

ethtool

编译 r8152 驱动模块

配置驱动

加载驱动

验证 RTL 芯片版本

性能优化

接收（RX）

发送（TX）

性能测试

iperf3

SMB 文件传输

实现 5GbE

编译 aqc111 驱动模块

配置驱动 & 性能优化

总结

附录：预编译的二进制文件

预编译的驱动模块

预编译的测试工具

附录：脚本

Makefile

RTL8156B 配置脚本

/jffs/scripts/init-start

/jffs/scripts/services-start

AQC111U 配置脚本

/jffs/scripts/services-start

参考文献

Towards 2.5GbE (NBASE-T) on ASUS RT-AX88U with `RTL8156B` and Asuswrt-Merlin 384.19

No Plug-and-Play?

Preparation

Compile Linux Kernel

Compile Tools

iperf3

ethtool

Compile r8152 Driver

Installation

Load Driver

Verify RTL Version

Performance Tuning

Receive (RX)

Transmit (TX)

Performance Evaluation

How could it go wrong with `\t+`?

Bad `<regex>` implementation in MSVC

`++regex_token_iterator` takes too long

Speed under `Release` configuration

How about plain `\t`

What about `clang` and `gcc`?

`clang` 11.0.1 + `libc++` 11.0

`gcc` 5.5.0 + `libstdc++` 3.4.21

Tedious `string::find` did the trick

`iperf3`

`ethtool`

编译 `r8152` 驱动模块

验证 `RTL` 芯片版本

`iperf3`

编译 `aqc111` 驱动模块

`Makefile`

`RTL8156B` 配置脚本

`/jffs/scripts/init-start`

`/jffs/scripts/services-start`

`AQC111U` 配置脚本

`/jffs/scripts/services-start`

`iperf3`

`ethtool`

Compile `r8152` Driver

Verify `RTL` Version

`iperf3`

Compile `aqc111` Driver

`Makefile`

Scripts for `RTL8156B`

`/jffs/scripts/init-start`

`/jffs/scripts/services-start`

Scripts for `AQC111U`

`/jffs/scripts/services-start`

`webpack` everything

没有 `robocfg`？

一条命令（`ebtables`）的解决方案

新建网桥 `br1`

添加 `iptables` 规则

配置 DHCPv4 (`dnsmasq`)

添加 `ip6tables` 规则

`/jffs/scripts/services-start`

`/jffs/scripts/firewall-start`

`/jffs/scripts/nat-start`

`/jffs/scripts/wan-event`

`/jffs/scripts/dhcpc-event`

`/jffs/configs/dnsmasq.conf.add`

No `robocfg`?

One-liner with `ebtables`

Create Bridge `br1`

Add `iptables` Rules

Configure `dnsmasq` for DHCPv4

Add `ip6tables` Rules

`/jffs/scripts/services-start`