把它们堆在一起就是下面这个样子:
废话不多说,直接进入正题。
登入 RouterOS 终端,输入以下命令启用 DHCPv6 客户端:
IPv6 前缀提示取决于你的 ISP,以我的 ISP 为例,Charter Spectrum 提供 /56
的前缀,所以这里我使用的是 ::/56
。
要使 SLAAC(或者说无状态 DHCPv6)正常工作,你需要从 ISP 那拿到至少 /63
的前缀(这种情况下可以划分两个 /64
的子网)。如果 ISP
最多只给分配 /64
的前缀,那就和前缀代理无缘了 —— 能用的方案只有 NAT66
和 NDPv6 代理。
这条命令中的关键是 pool-prefix-length
参数:它决定了下文中 DHCPv6 服务器下发前缀的长度(或者说二级路由器拿到的前缀的长度)。
这里我设置的是 60
,在 ISP 分配给我 /56
前缀的前提下,hEX S 可以划分出 16 个 /60
的前缀给二级路由器使用,二级路由器则可以提供另外 16 个 /64
的前缀给它们的设备使用。
运行 /ipv6 dhcp-client print
和 /ipv6 pool print
来查看 ISP 分配的前缀:
[admin@MikroTik] > /ipv6 dhcp-client print
Flags: D - dynamic, X - disabled, I - invalid
# INTERFACE STATUS REQUEST PREFIX
0 ether1 bound address 2600:6c51:fake:n00b::/56, 4d22h24m10s
prefix
[admin@MikroTik] > /ipv6 pool print
Flags: D - dynamic
# NAME PREFIX PREFIX-LENGTH EXPIRES-AFTER
0 D delegation 2600:6c51:fake:n00b::/56 60 4d22h23m52s
如果上面命令的 STATUS
一栏一直显示的是 searching
,那么有可能是 IPv6 防火墙阻止了 UDP 端口 546
的连接入站。
运行下面的命令添加允许 UDP 546 入站的规则:
最后一步是从 IPv6 地址池中为默认网桥分配 IPv6 地址:
如果有多个网桥或端口,则需要对每个网桥和端口单独运行上面的命令。运行 /ipv6 address print
来查看分配到 LAN 端口的地址:
[admin@MikroTik] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local
# ADDRESS FROM-POOL INTERFACE ADVERTISE
0 G 2600:6c51:fake:n00b::1/64 delegation bridge yes
1 DL fe80::a55:31ff:fe00:fake/64 bridge no
2 DL fe80::a55:31ff:fe00:fake/64 ether1 no
3 DG 2600:6c51:some:fake:addr:why:so:long/64 ether1 no
注意 1:要使用 DHCPv6-PD 服务器,你需要从 ISP 那拿到至少 /63
的 IPv6 前缀。
注意 2: 为了能让二级路由器通过 DHCPv6-PD 拿到前缀,上一节 DHCPv6 客户端中的 pool-prefix-length
参数必须正确设置。
开始之前,先重申一下 Mikrotik 文档中的说明:
Note: RouterOS DHCPv6 server can only delegate IPv6 prefixes, not addresses.
Mikrotik Documentation — Manual:IPv6/DHCP Server
换句话说,RouterOS 的 DHCPv6 服务器目前只支持 SLAAC(无状态 DHCPv6)。
下面的命令用于创建 DHCPv6 服务器:
如果有多个网桥或端口,则需要对每个网桥和端口单独运行上面的命令。
现在就可以运行 /ipv6 dhcp-server binding print
来检查二级路由器有没有拿到前缀:
[admin@MikroTik] > /ipv6 dhcp-server binding print
Flags: I - invalid, X - disabled, R - radius, D - dynamic
# ADDRESS DUID SERVER STATUS
0 D 2600:6c51:fake:n01b::/60 0xa85e45fakeid default bound
如果看到类似上面的输出,那就说明 DHCPv6 服务器已经正常工作了。
如果想通过 DHCPv6 下发诸如 DNS (Option 23) 和域搜索列表 (Option 24) 等选项,则需要先在 IPv6 ND
中启用 Other Configuration
:
然后就可以设置 DHCPv6 选项了,这里我创建了 DNS 和域搜索列表两个选项:
DNS (Option 23) 需要打上完整的 64 位十六进制 IPv6 地址,域搜索列表 (Option 24) 则可以使用这个工具生成:https://jjjordan.github.io/dhcp119/。
我这里的 DNS 设置的是 fd00::1
,这样不需要额外的脚本就能让 hEX S 作为内网的默认 DNS (否则的话需要使用脚本来获取公网 IPv6 地址)。
这个方法来源于 dksoft
。不过对于我的应用场景而言,只需要一条命令:
这里没有启用 RA,因为不想也没必要让设备拿到 fd00::1/8
前缀的地址。
最后一步就是把这些选项添加到 DHCPv6 服务器上:
RouterOS 的学习曲线还是有些陡峭的,但是一旦搞明白怎么回事,属实比在 AX88U 上做相同的事来的轻松的多(感觉在讲废话,毕竟 AX88U 是针对消费市场的)。
无论如何,相对于 AX88U 这种多合一的设备,确实是发现还是专事专做比较好,尤其是有那种特殊或者专业需求的时候。
]]>In case of your curiosity, this is how my gears stack now:
Well enough talking, let’s go into the topic.
DHCPv6 client can be enabled with:
Depending on your ISP, the prefix hint may vary. My ISP Charter Spectrum offers /56
prefixes, so I’m using ::/56
here.
Note that in order to make SLAAC (or DHCPv6 stateless) work, you’ll need a prefix at least shorter than /64
. If your
ISP won’t offer you a prefix shorter than /64
, you can’t do subnetting (or more precisely, offering prefixes to cascading routers)
without breaking IPv6, unless you wanna try the dirty NAT66.
The key parameter is the pool-prefix-length
. This value determines the prefix length, or the subnet size, that the DHCPv6 server will offer for cascading routers.
For me, 60
is pretty reasonable: the main router is able to offer 16 /60
prefixes for cascading routers, and the
cascading routers can offer another 16 /64
prefixes for their clients.
Now you can check if prefix is assigned with /ipv6 dhcp-client print
and /ipv6 pool print
:
[admin@MikroTik] > /ipv6 dhcp-client print
Flags: D - dynamic, X - disabled, I - invalid
# INTERFACE STATUS REQUEST PREFIX
0 ether1 bound address 2600:6c51:fake:n00b::/56, 4d22h24m10s
prefix
[admin@MikroTik] > /ipv6 pool print
Flags: D - dynamic
# NAME PREFIX PREFIX-LENGTH EXPIRES-AFTER
0 D delegation 2600:6c51:fake:n00b::/56 60 4d22h23m52s
If the status keeps showing searching
, check IPv6 firewall to see whether incoming packets to UDP port 546
(DHCPv6 client) is allowed. If not, this rule can be added via:
The last step is to assign IPv6 address on internal interface from address pool:
If you get multiple interfaces, run the above command on each interface separately. Use /ipv6 address print
to check
assigned addresses:
[admin@MikroTik] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local
# ADDRESS FROM-POOL INTERFACE ADVERTISE
0 G 2600:6c51:fake:n00b::1/64 delegation bridge yes
1 DL fe80::a55:31ff:fe00:fake/64 bridge no
2 DL fe80::a55:31ff:fe00:fake/64 ether1 no
3 DG 2600:6c51:some:fake:addr:why:so:long/64 ether1 no
Note 1: Like I’ve mentioned before, you need a prefix at least shorter than /64
from your ISP to make DHCPv6-PD server works!
Note 2: In order to delegate prefixes to cascading routers, you have to assign a pool-prefix-length
in RouterOS DHCPv6 client.
Check the previous section for details.
Before started, I want to re-state the note (at least at the time of writing) from Mikrotik’s documentation:
Note: RouterOS DHCPv6 server can only delegate IPv6 prefixes, not addresses.
Mikrotik Documentation — Manual:IPv6/DHCP Server
In other words, DHCPv6 server in RouterOS is only capable of SLAAC (or stateless DHCPv6). There is no support for stateful DHCPv6.
The following command creates a DHCPv6 server:
Again, if you get multiple interfaces, run the above command on each interface separately.
Now you can check if cascading routers are assigned prefixes with /ipv6 dhcp-server binding print
:
[admin@MikroTik] > /ipv6 dhcp-server binding print
Flags: I - invalid, X - disabled, R - radius, D - dynamic
# ADDRESS DUID SERVER STATUS
0 D 2600:6c51:fake:n01b::/60 0xa85e45fakeid default bound
Congratulations! Your DHCPv6 services on RouterOS are working now.
… though there could be one more question: what if I want to configure DHCPv6 options (DNS, search domains, etc.)?
First you need to allow Other Configuration
in IPv6 ND
(neighbor discovery):
Then we can set up DHCPv6 options. Here I’m creating option 23 (DNS) and 24 (domain search list):
For option 23 (DNS), you need to assign a full 64-bit address in hexadecimal.
To grab the value of DHCPv6 option 24 (domain search list), use this tool: https://jjjordan.github.io/dhcp119/.
You may have noticed I’m using a ULA address fd00::1
for option 23. This allows me to assign my Mikrotik router as
the default DNS without scripting.
The idea comes from dksoft
. But for my application, I only
need one command (to assign fd00::1
):
Note that RA is disabled, because I don’t want clients get SLAAC addresses from fd00::1/8
prefix.
The last step is to assign these options to DHCPv6 server:
I’d say once you figure out how RouterOS works, it is much less painful than doing the same things on AX88U, though it might be unfair to AX88U since it is designed for consumer market.
So what have I learned? Well, if you have special requirements to fullfil, instead of going for all-in-one products (for example AX88U in this case: you get a router, a switch, and an AP), buy products that focuses on specific applications.
Trust me, it’s much more painless!
]]>In our previous papers1,2, since the original authors’ algorithms were done in Java (FastDTW3, TEASER4, etc.), I’ve mainly written Java for related experiments. I haven’t touched C++ for a while, so this time I decided to use C++, but I never expected to pay the “price” for my choice.
\t+
?The file format of UCR Time Series Archive is pretty straightforward: n rows of time series in each dataset, and for
each time series, there are m columns separated by TAB (\t
), representing m datapoints.
C++ <string>
library didn’t come with a built-in splitting function, and I don’t really want to use 3rd-party libraries
(like boost::split
) in my little project. It could be very painful to use external libraries in C++, and when it comes
to compatibility issues, oh man good luck to you.
I know I could make it with string::find
, but hey it’s year 2021, C++ 20 was finalized last September, and I believe
there must be fancier way than tedious string::find
, so I came up with an idea: searching “How to split string in modern
C++” on Stack Overflow.
And I did get a hit: Split a string using C++ 11,
a question asked 9 years ago. The most voted answer showed an “elegant” way
using std::regex_token_iterator
.
So I wrote down this piece of code in Visual Studio 2019 (MSVC v14.28.29333, the latest version at the time of writing):
typedef std::vector<std::pair<int, std::vector<double>>> TimeSeriesDataset;
TimeSeriesDataset readTSV(const std::string& tsvPath) {
TimeSeriesDataset dataset;
std::ifstream stream{ tsvPath };
std::string line;
static const std::regex split(R"(\t+)");
while (std::getline(stream, line)) {
std::cregex_token_iterator first{ line.data(), line.data() + line.length(), split, -1 }, last;
int label;
std::from_chars(first->first, first->second, label);
std::vector<double> datapoints;
std::transform(++first, last, std::back_inserter(datapoints), [](auto subMatch) {
double value;
std::from_chars(subMatch.first, subMatch.second, value);
return value;
});
dataset.push_back(std::make_pair(label, datapoints));
}
return dataset;
}
Note that I wrote a regular expression of \t+
. The +
sign is in fact not necessary in my case, but
at that time I didn’t think too much about this simple expression. Consider the DFA only contains two states:
How could it possibly go wrong?
<regex>
implementation in MSVC++regex_token_iterator
takes too longWhen I hit F5 to launch the program, I immediately noticed there is something wrong: it took way too long to load Mallat_TEST.tsv
.
I won’t consider this file to be big: essentially a 2345 x 1025 matrix of decimal, and my past experience in Java showed that
a second should be long enough to load this file.
So I started the performance profiler to see what happened. To my surprise, it takes 6 full minutes (under
Debug (Win32)
configuration) to finish running the program on Mallat_TEST.tsv
.
The hot path analysis reveals which operations hold the most CPU time (click picture to enlarge):
Apparently most of the time was spent in std::transform
, which implicitly refers to std::regex_token_iterator::operator++
.
Following the call tree reveals creating, deleting, and assigning object std::_Tgt_state_t
in function std::_Matcher::_Do_if
caused the problem:
Also if we sort used CPU time by functions, it reveals the same issue:
This means there must be something wrong with std::_Tgt_state_t
.
So I looked into <regex>
’s source code for _Matcher::_Do_if
. The first glance looks pretty normal:
// IMPLEMENTATION OF _Matcher
template <class _BidIt, class _Elem, class _RxTraits, class _It>
bool _Matcher<_BidIt, _Elem, _RxTraits, _It>::_Do_if(_Node_if* _Node) { // apply if node
_Tgt_state_t<_It> _St = _Tgt_state;
// look for the first match
for (; _Node; _Node = _Node->_Child) { // process one branch of if
_Tgt_state = _St; // rewind to where the alternation starts in input
if (_Match_pat(_Node->_Next)) { // try to match this branch
break;
}
}
// if none of the if branches matched, fail to match
if (!_Node) {
return false;
}
// if we aren't looking for the longest match, that's it
if (!_Longest) {
return true;
}
// see if there is a longer match
_Tgt_state_t<_It> _Final = _Tgt_state;
auto _Final_len = _STD distance(_St._Cur, _Tgt_state._Cur);
for (;;) { // process one branch of if
_Node = _Node->_Child;
if (!_Node) {
break;
}
_Tgt_state = _St;
if (_Match_pat(_Node->_Next)) { // record match if it is longer
const auto _Len = _STD distance(_St._Cur, _Tgt_state._Cur);
if (_Final_len < _Len) { // memorize longest so far
_Final = _Tgt_state;
_Final_len = _Len;
}
}
}
// set the input end to the longest match
_Tgt_state = _Final;
return true;
}
The variable _Tgt_state
is a member of class _Matcher
.
However, if we take a closer look on how _Tgt_state
is assigned and other temporary _Tgt_state_t
variables
_St
and _Final
are constructed:
_Tgt_state_t<_It> _St = _Tgt_state;
_Tgt_state = _St;
_Tgt_state_t<_It> _Final = _Tgt_state;
_Final = _Tgt_state;
_Tgt_state = _Final;
All these assignments are implicitly invoking operator=
, and the default operator=
performs a member-wise copy, which
could lead to deep copy if there are STL containers without special handling mechanism.
A later look at the definition of _Tgt_state_t
proves my thought:
// CLASS TEMPLATE _Bt_state_t
template <class _BidIt>
class _Bt_state_t { // holds the state needed for backtracking
public:
_BidIt _Cur;
vector<bool> _Grp_valid;
};
// CLASS TEMPLATE _Tgt_state_t
template <class _BidIt>
class _Tgt_state_t : public _Bt_state_t<_BidIt> { // holds the current state of the match
public:
struct _Grp_t { // stores a pair of iterators
_BidIt _Begin;
_BidIt _End;
};
vector<_Grp_t> _Grps;
void operator=(const _Bt_state_t<_BidIt>& _Other) {
static_cast<_Bt_state_t<_BidIt>&>(*this) = _Other;
}
};
Class _Tgt_state_t
has two member variables: _Grps
and _Grp_valid
(inherited from _Bt_state_t
). Both of them
are STL container vector
, and by default, performing operator=
over vector
incurs deep copy.
Deep copy is already evil. To make it worse, the hot path reveals _Do_if
would be called recursively
(_Match_pat
calls _Do_if
and _Do_if
calls _Match_pat
). This means _Tgt_state_t
variable _St
and _Final
are created and deleted every single time when _Do_if
is called.
Remember class _Tgt_state_t
has two vector
member variables, and by default vector
requires memory allocation/deallocation on heap.
Normally this overhead seems neglectable, but under the circumstance of recursion, it could be a problem. The CPU time
used by vector
constructor/deconstructor proves:
So how can it be fixed? Well, in the very beginning, I thought how hard could it be? But after all the deep digging into source code, it could be very hard and might require a complete refactoring.
Release
configurationI decided to give MSVC one more chance, by compiling the code under Release
configuration, or more precisely,
with compiling options /O2
(maximum optimizations, favor speed) and /Ob2
(inline expansion level 2).
I wrote a small snippet to measure how long my readTSV
function takes:
auto start = std::chrono::steady_clock::now();
auto dataset = readTSV(R"(../UCRArchive_2018/Mallat/Mallat_TEST.tsv)");
auto end = std::chrono::steady_clock::now();
std::cout << "Mallat_TEST[0], label: " << dataset.front().first << ", data[0, last]: "
<< dataset.front().second.front() << ", " << dataset.front().second.back() << std::endl;
std::cout << "Mallat_TEST[last], label: " << dataset.back().first << ", data[0, last]: "
<< dataset.back().second.front() << ", " << dataset.back().second.back() << std::endl;
std::cout << "Elapsed: "
<< std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count()
<< " ms" << std::endl;
The reason I added two more lines output is to avoid compiler’s optimization over potential dead code (clang
does very
well in this kind of elimination), which may remove the invocation to readTSV
.
Now I can compile this little program:
# /nologo: Suppress copyright message
# /TP: Treat all files as C++
# /W3: Warning level 3
# /GR: Enable C++ RTTI
# /EHsc: Enable C++ EH
# /MD: Link with MSVCRT.LIB
# /O2: Maximum optimizations (favor speed)
# /Ob2: Inline expansion level 2
# /NDEBUG: Define NDEBUG marco
# /std:c++17: C++ 17 stardard
cl /nologo /TP /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG /std:c++17 RegexTokenIterator_C++17.cpp
Yet it still took 31.4 seconds to load Mallat_TEST.tsv
:
# MSVC v14.28.29333 (/O2 /Ob2) with \t+
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 31461 ms
\t
I also tried the simplest regex for this case: \t
. With the same compiler options, it still takes 4.6 seconds:
# MSVC v14.28.29333 (/O2 /Ob2) with \t
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 4641 ms
which is way less desirable and 5x slower than expected.
clang
and gcc
?clang
11.0.1 + libc++
11.0When I first tried to compile with:
clang++ -O3 -std=c++17 -stdlib=libc++ RegexTokenIterator_C++17.cpp -o clangRTI
clang
spits out an error:
RegexTokenIterator_C++17.cpp:34:4: error: call to deleted function 'from_chars'
std::from_chars(subMatch.first, subMatch.second, value);
^~~~~~~~~~~~~~~
/usr/local/clang-11.0.1/bin/../include/c++/v1/algorithm:1948:21: note: in instantiation of function template specialization 'readTSV(const std::string &)::(anonymous class)::operator()<std::__1::sub_match<const char *>>' requested here
*__result = __op(*__first);
^
RegexTokenIterator_C++17.cpp:32:8: note: in instantiation of function template specialization 'std::__1::transform<std::__1::regex_token_iterator<const char *, char, std::__1::regex_traits<char>>, std::__1::back_insert_iterator<std::__1::vector<double, std::__1::allocator<double>>>, (lambda at RegexTokenIterator_C++17.cpp:32:65)>' requested here
std::transform(++first, last, std::back_inserter(datapoints), [](auto subMatch) {
^
/usr/local/clang-11.0.1/bin/../include/c++/v1/charconv:123:6: note: candidate function has been explicitly deleted
void from_chars(const char*, const char*, bool, int = 10) = delete;
^
/usr/local/clang-11.0.1/bin/../include/c++/v1/charconv:606:1: note: candidate template ignored: requirement 'is_integral<double>::value' was not satisfied [with _Tp = double]
from_chars(const char* __first, const char* __last, _Tp& __value)
^
/usr/local/clang-11.0.1/bin/../include/c++/v1/charconv:613:1: note: candidate function template not viable: requires 4 arguments, but 3 were provided
from_chars(const char* __first, const char* __last, _Tp& __value, int __base)
^
1 error generated.
I was confused and “What?” is my first reaction. So I digged into libc++
’s <charconv>
and the only 3 definitions
of std::from_chars
I found are:
void from_chars(const char*, const char*, bool, int = 10) = delete;
template <typename _Tp, typename enable_if<is_integral<_Tp>::value, int>::type = 0>
inline _LIBCPP_INLINE_VISIBILITY from_chars_result
from_chars(const char* __first, const char* __last, _Tp& __value)
{
return __from_chars_atoi(__first, __last, __value);
}
template <typename _Tp, typename enable_if<is_integral<_Tp>::value, int>::type = 0>
inline _LIBCPP_INLINE_VISIBILITY from_chars_result
from_chars(const char* __first, const char* __last, _Tp& __value, int __base)
{
_LIBCPP_ASSERT(2 <= __base && __base <= 36, "base not in [2, 36]");
return __from_chars_integral(__first, __last, __value, __base);
}
Meh. This means at the time of writing, libc++
doesn’t support floating point numbers in std::from_chars
.
So I had to replace the std::from_chars
in the std::transform
lambda with the old school strtod
, and it worked.
The result is much better than MSVC’s, but still slow:
# clang 11.0.1 (-O3) + libc++ 11.0 with \t+
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 3017 ms
# clang 11.0.1 (-O3) + libc++ 11.0 with \t
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 1848 ms
gcc
5.5.0 + libstdc++
3.4.21I could have use the latest gcc
10.2, but I am just too lazy to compile it.
Fortunately gcc
5.5.0 + libstdc++
3.4.21 has already fully supported C++ 14 features, so at least I can continue
my experiments by simply replacing the last C++ 17 existence of std::from_chars
to atoi
.
The complier options are:
g++ -O3 -std=c++14 RegexTokenIterator_C++14.cpp -o g++RTI
I’m a little bit amazed by the result:
# gcc 5.5.0 (-O3) + libstdc++ 3.4.21 with \t+
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 992 ms
# gcc 5.5.0 (-O3) + libstdc++ 3.4.21 with \t
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 952 ms
Because both MSVC and clang
+ libc++
are in their latest stable version. For gcc
+ libstdc++
, I was using an old
version built on Oct. 2017, which I’d say it’s kind of unfair, but gcc
+ libstdc++
managed to win the game.
string::find
did the trickNevertheless, I went back to the tedious way of string::find
(let us also forget about whatever std::from_chars
is):
typedef std::vector<std::pair<int, std::vector<double>>> TimeSeriesDataset;
TimeSeriesDataset readTSV(const std::string& tsvPath) {
TimeSeriesDataset dataset;
std::ifstream stream{ tsvPath };
std::string line;
while (std::getline(stream, line)) {
auto tabPos = line.find('\t');
auto label = atoi(line.data());
std::vector<double> datapoints;
for (size_t nextTabPos;
std::string::npos != (nextTabPos = line.find('\t', tabPos + 1));
tabPos = nextTabPos) {
datapoints.push_back(strtod(line.data() + tabPos + 1, nullptr));
}
datapoints.push_back(strtod(line.data() + tabPos + 1, nullptr));
dataset.push_back(std::make_pair(label, datapoints));
}
return dataset;
}
I’m not surprised to see this piece of code is way more faster than its regular expression equivalent under all three libraries:
# MSVC v14.28.29333 (/O2 /Ob2)
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 892 ms
# clang 11.0.1 (-O3) + libc++ 11.0
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 431 ms
# gcc 5.5.0 (-O3) + libstdc++ 3.4.21
Mallat_TEST[0], label: 5, data[0, last]: -1.06862, -1.01192
Mallat_TEST[last], label: 8, data[0, last]: -0.949403, -0.970955
Elapsed: 294 ms
With all the experiment data collected, we can finally compare the performance:
Compiler (Options) | Time (ms) ( rti ^ with \t+ ) |
Time (ms) ( rti ^ with \t ) |
Time (ms) ( string::find ) |
---|---|---|---|
MSVC v14.28.29333 (/O2 /Ob2 ) |
31461 | 4641 | 892 |
clang 11.0.1 (-O3 )libc++ 11.0 |
3017 | 1848 | 431 |
gcc 5.5.0 (-O3 )libstdc++ 3.4.21 |
992 | 952 | 294 |
^ rti
is std::regex_token_iterator
.
The result shows that at least for <regex>
, libstdc++
’s implementation is so far the best among all three libraries.
But it still can’t beat the plain and tedious string::find
, in terms of string splitting.
I tried to figure out why <regex>
has been introduced for a decade and yet still so slow and badly implemented.
A thread on Reddit points out something about ABI compatibility:
The fast engines add a zillion special cases for common patterns their engines recognize. But we can’t ever do that. And given that our engines were somewhat stupid initially now we can’t replace the engine with something better because that breaks ABI.
BillyONeal — std::regex_replace/std::chrono::high_resolution_clock::now() speed
Well, good luck to all of us C++ users.
It’s great to see the C++ standard getting periodical updates every 3 years since 2011: new STL components, syntax sugars,
and a lot of other stuffs (range-based for
loops,
auto
type deduction, variadic template (or parameter pack), and concepts
finally, etc.) — all of which
I (and probably many other people) have been waiting for years.
However, <regex>
library is unfortunately an exception: apparently there is something wrong with MSVC’s underlying
implementations, and the performance is surprisingly bad. Also not to mention <regex>
library was introduced in C++ 11
since 2011 — a decade ago.
The lesson I learnt is that for simple tasks like splitting strings, it might be better just to stick with simple tools (string::find
in
this case, even though I have to reinvent the wheels every single time), instead of using something “fancy” like
std::regex_token_iterator
, at least for now.
Actually this is NOT the first time that I found issues in the particular STL implementation: several years ago, I noticed
that the order of items from std::make_heap
is slightly different
between MSVC’s and GCC’s (or libstdc++
). My later digging into source code revealed it was caused by the default comparison
function: one used <
and the other used <=
.
Another issue is that specific feature mights not be fully implemented. Besides aforementioned libc++
’s std::from_chars
lacking support on floating point numbers, when I was trying to compile the cppwin32
project with Visual Studio 2017 (MSVC v14.16.27023), I found that MSVC’s std::to_chars
has no support on floating-point
in terms of std::chars_format::hex
5. This made me upgrade to Visual Studio 2019.
All these chaos remind me the famous Russian proverb my advisor Dr. Keogh ever cited: “Доверя́й, но проверя́й” or “Trust, but verify”. Even standards like STL are not exceptions.
My experiences in C++ in the past decade has taught me one thing:
R. Wu and E.J. Keogh, “FastDTW is approximate and Generally Slower than the Algorithm it Approximates,” IEEE Transactions on Knowledge and Data Engineering (TKDE), in press, 2020. ↩
R. Wu, A. Der, and E.J. Keogh, “When is Early Classification of Time Series Meaningful?,” arXiv, preprint, 2021. ↩
S. Salvador and P. Chan, “FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space,” Intelligent Data Analysis, vol. 11, no. 5, 2007, pp. 561-580. ↩
Microsoft, Microsoft C++ language conformance table: charconv. ↩
BCM49408
内建了对 2.5GbE 的支持1(华硕后来的 2.5GbE 产品 AX86U 就是基于这款 CPU 的)。
不过好消息是螃蟹家(Realtek)提供了一个经济的解决方案 —— RTL8156
2,基于 USB 3.0 的 2.5GbE 控制器。相比于贵的多的 5GbE 和 10GbE 的设备,它可以说是“相当廉价”了:30 美金左右就能买到基于 RTL8156
的 USB 网卡。
但是有一点需要特别注意,这个芯片有两个版本:RTL8156
和 RTL8156B
。我推荐购买后者这个 B 修订版的 RTL8156B
,因为前者 RTL8156
在用上一段时间后会有些奇怪的稳定性问题3。
此外如果买的是 USB-C 的网卡,我建议买一个“双面” USB-C 转 USB-A 的转接器(里面应该有个多路复用芯片,比如 VL160
4 之类的)。不然使用非“双面”的转接器,必须搞清楚把 USB-C 头的哪一面插入到转接器里,因为只有一面能跑 USB 3.0,另一面只能跑 USB 2.0。
一切正常的话,AX88U 应该是能识别插入的 USB 网卡(在路由器管理界面的网络地图里会显示)。另外运行 ifconfig -a
,也能发现网卡已经被注册为 usb0
。
然而一旦尝试启用网卡(ifconfig usb0 up
),就会发现日志会被 cdc_ncm
这个模块刷屏:
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
造成这个问题的原因是 Linux 内核自带的驱动程序暂时还不支持 RTL8156
5,所以必须得自己编译驱动。
有关如何配置梅林固件编译环境,请参阅这篇教程:Compiling under WSL2(英文)。虽然标题写的是在 WSL (Windows Subsystem for Linux) 2 里编译,但是总体步骤也适用于 WSL 1(也称 legacy WSL)和真·Linux 环境。
如果你跟我一样用的是 WSL 1,还需要额外配置 32 位运行环境(参考这个 issue),因为梅林固件提供的工具链都是 32 位的。
首先把梅林固件的源代码拖下来,因为用不到 git
的历史,只需要把 HEAD
拖下来即可:
# 只拖 HEAD
git clone --depth 1 https://github.com/RMerl/asuswrt-merlin.ng
教程里是把 brcm-arm-sdk
文件夹链接到了 src-rt-6.x-4708
里面:
# `src-rt-6.x.4708` 用于 AC56U, AC68U 和 AC87U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-6.x.4708/toolchains
而我们需要把它链接到 src-rt-5.02axhnd
里面,梅林固件的编译脚本6里有提到 AX88U 用的是 src-rt-5.02axhnd
:
# `src-rt-5.02axhnd` 用于 AX88U, AX58U 和 AX56U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/toolchains
接着就可以编译 AX88U 的梅林固件了:
# 目标平台 AX88U
cd release/src-rt-5.02axhnd && make rt-ax88u
由于我们只需要用到内核编译生成的头文件(.h
),因此没必要把固件整个都编译出来,待内核编译流程开始之后,就可以按键盘上的 CTRL+C
终止固件编译。
iperf3
iperf3
源代码从这里下载:esnet/iperf,本文使用的 iperf3
版本是 3.9。
运行下面的命令来为 AX88U 编译 iperf3
:
# 创建 build 文件夹,作为 iperf3 安装目录
mkdir build
# 静态链接
./configure --host=arm-linux --prefix=`pwd`/build --enable-static --disable-shared
# 编译并安装到 build 文件夹
make && make install
生成的 iperf3
二进制文件应该可以在 ./build/bin
文件夹下找到,预编译好的版本可以在这里下载:附录:预编译的二进制文件:预编译的测试工具。
ethtool
ethtool
源代码从这里下载:kernel/software/network/ethtool,本文使用的 ethtool
版本是 5.8。
运行下面的代码来为 AX88U 编译 ethtool
:
# 创建 build 文件夹,作为 ethtool 安装目录
mkdir build
# 禁用 netlink,一是用不到,二是去除对 libmnl 的依赖
./configure --host=arm-linux --prefix=`pwd`/build --disable-netlink
# 编译并安装到 build 文件夹
make & make install
生成的 ethtool
二进制文件应该可以在 ./build/bin
文件夹下找到,预编译好的版本可以在这里下载:附录 1:预编译的二进制文件:预编译的测试工具。
r8152
驱动模块你没看错,RTL8156B
的驱动模块确实叫 r8152
,源码从这里下载:Realtek USB FE / GBE / 2.5G / Gaming Ethernet Family Controller Software,本文使用的 r8152
驱动模块版本是 v2.13.0。
为了验证买来的网卡是 RTL8156B
芯片的,可以在 rtl8152_get_drvinfo
函数里加上下面一句代码,这样就可以通过 ethtool
来获取芯片的版本:
--- source-2.13.0/r8152-original.c 2020-04-20 01:36:49.000000000 -0700
+++ source-2.13.0/r8152-modified.c 2020-10-06 23:47:07.123659400 -0700
@@ -16369,6 +16369,7 @@
strlcpy(info->driver, MODULENAME, sizeof(info->driver));
strlcpy(info->version, DRIVER_VERSION, sizeof(info->version));
usb_make_path(tp->udev, info->bus_info, sizeof(info->bus_info));
+ snprintf(info->fw_version, sizeof(info->fw_version), "0x%04x", tp->version);
}
#if LINUX_VERSION_CODE < KERNEL_VERSION(4,20,0)
运行下面的命令来编译驱动:
# 架构: arm64
# 交叉编译器: aarch64-linux
# 内核源代码目录: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 M=`pwd` modules
如果你倾向用 Makefile
的话,可以在这里找到:附录 2:脚本:Makefile。
一切顺利的情况下,r8152.ko
这个文件会生成在当前目录下,预编译好的版本可以在这里下载:附录:预编译的二进制文件:预编译的驱动模块。
首先在路由器 /jffs
文件夹内创建 drivers
和 tools
两个文件夹,然后把 r8152.ko
上传到路由器 /jffs/drivers
文件夹内,把编译好的 iperf3
和 ethtool
上传到路由器 /jffs/tools
文件夹内。
前文“不能即插即用?”一节已经提到 cdc_ncm
会接管 USB 网卡,所以首先需要把 cdc_ncm
卸载掉。
运行 lsmod | grep cdc_ncm
可以看到依赖 cdc_ncm
的模块:
# admin@ax88u:/# lsmod | grep cdc_ncm
cdc_ncm 16787 1 cdc_mbim
usbnet 21074 7 cdc_mbim,qmi_wwan,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix
usbcore 166572 23 uas,usb_storage,cdc_mbim,qmi_wwan,cdc_wdm,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix,cdc_acm,usbnet,usblp,ohci_pci,ohci_platform,ohci_hcd,ehci_pci,ehci_platform,ehci_hcd,xhci_pci,xhci_plat_hcd,xhci_hcd
所以得把 cdc_mbim
也一同卸载掉:
# 先卸载 cdc_mbim,因为它依赖于 cdc_ncm
rmmod cdc_mbim
# 卸载 cdc_ncm
rmmod cdc_ncm
现在就可以加载 r8152.ko
了,同时记得把 cdc_ncm
和 cdc_mbim
也加载回来:
# 加载 r8152 驱动模块
insmod /jffs/drivers/r8152.ko
# 加载 cdc_ncm
insmod cdc_ncm
# 加载 cdc_mbim
insmod cdc_mbim
如果一切正常的话,系统日志(tail /tmp/syslog.log
)会显示驱动加载成功:
# admin@ax88u:/# tail /tmp/syslog.log
Oct 7 21:27:42 kernel: usbcore: deregistering interface driver cdc_mbim
Oct 7 21:27:42 kernel: usbcore: registered new interface driver r8152
Oct 7 21:27:42 kernel: usb 2-2: reset SuperSpeed USB device number 2 using xhci-hcd
Oct 7 21:27:42 kernel: netif_napi_add() called with weight 256 on device eth%d
Oct 7 21:27:42 kernel: r8152 2-2:1.0 eth8: v2.13.0 (2020/04/20)
Oct 7 21:27:42 kernel: r8152 2-2:1.0 eth8: This product is covered by one or more of the following patents:
Oct 7 21:27:42 kernel: US6,570,884, US6,115,776, and US6,327,625.
Oct 7 21:27:42 hotplug: add net eth8.
Oct 7 21:27:42 kernel: usbcore: registered new interface driver cdc_ncm
Oct 7 21:27:42 kernel: usbcore: registered new interface driver cdc_mbim
从上面的日志里可以看到 USB 网卡被注册为 eth8
,运行 ifconfig eth8
可以验证这一点:
eth8 Link encap:Ethernet HWaddr 00:E0:4C:00:FA:KE
inet6 addr: fe80::2e0:4cff:fe00:fake/64 Scope:Link
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
在路由器管理界面的 网络地图 里也可以看到 USB 网卡:
接下来就可以更新 nvram
变量,启用 eth8
并把它加入网桥 br0
里:
# 更新 nvram 变量
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"
# 把 eth8 加入网桥 br0,然后启用 eth8
brctl addif br0 eth8
ifconfig eth8 allmulti up
最后插入网线,运行 /jffs/tools/ethtool eth8
来验证协商的速度为全双工 2500Mb/s:
# admin@ax88u:/# /jffs/tools/ethtool eth8
Settings for eth8:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseX/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseX/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 2500Mb/s
Duplex: Full
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00007fff (32767)
drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
Link detected: yes
提示:完整的 RTL8156B
配置脚本可以在这里找到:附录:脚本:RTL8156B 配置脚本。
RTL
芯片版本运行 /jffs/tools/ethtool --driver eth8
:
# admin@ax88u:/# /jffs/tools/ethtool --driver eth8
driver: r8152
version: v2.13.0 (2020/04/20)
firmware-version: 0x000e
expansion-rom-version:
bus-info: usb-xhci-hcd.0-2
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
firmware-version
为 0x000b
或 0x000c
,则芯片为 RTL8156
。firmware-version
为 0x000d
或 0x000e
,则芯片为 RTL8156B
。这里给出的输出结果中的 firmware-version
为 0x000e
,因此证明我的网卡芯片为 RTL8156B
。
注意:为了显示 firmware-version
, 必须在 rtl8152_get_drvinfo
函数中添加一句代码,详情请参阅准备工作:编译 r8152
驱动模块。
完整的 RTL8156B
配置脚本可以在这里找到:附录:脚本:RTL8156B 配置脚本。
默认情况下,AX88U 上只有 4 号 CPU 核心为 xhci-hcd:usb1
响应硬件中断(cat /proc/irq/28/smp_affinity_list
)。而且由于 RTL8156B
只有一个接收队列,也没法通过改变 smp_affinity_list
的方式,把硬件中断请求分散到其他核心去处理。
因此必须启用 RPS (Receive Packet Steering) 来分配软件中断(softirq
)到各个核心,以达到优化接收性能的目的。有关 RPS 的更多信息,可以参考 Red Hat 的一篇文档:Performance Tuning Guide: 8.7 Receive Packet Steering(英文)。
运行下面的命令在 eth8
上启用 RPS:
# 在 eth8 上启用 RPS (Receive Packet Steering)
# 因为 AX88U 有四个核心,所以这里设置为 f
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus
我之后进行的测试表明仅启用 RPS 已经足以在 iperf3
跑到理论峰值速度,但是这里还是给出一些别的优化方法:
net.core.netdev_max_backlog
。默认值为 1000
,可以增加到 2500
:
# 1000 是 1GbE 的默认值,因此对 2.5GbE 来说 2500 看起来很合理
echo 2500 > /proc/sys/net/core/netdev_max_backlog
rx-usecs
的默认值为 15
,运行以下命令以修改:
# <N> 是数据包达到后产生 RX 中断的延迟时间,单位为毫秒。
/jffs/tool/ethtool -C eth8 rx-usecs <N>
100
,可以增加到 4096
(最大值):
# 4096 是 RX 缓冲区大小的最大值
/jffs/tool/ethtool -G eth8 rx 4096
这三篇文章(英文)介绍了更多优化接收性能的方法:Performance Tuning on Linux — Ethernet、How to achieve low latency with 10Gbps Ethernet、Performance Tuning Guide: Chapter 8. Networking。
不像 RPS,启用 XPS (Transmit Packet Steering) 对于提升 RTL8156B
的发送性能没有帮助,这是因为 RTL8156B
只有一个发送队列,并且:
For a network device with a single transmission queue, XPS configuration has no effect, since there is no choice in this case.
对于只有一个发送队列的网络设备,启用 XPS 不会起到任何效果,因为能用的发送队列就只有一个。
Linux 内核文档 — Scaling in the Linux Networking Stack
有关发送性能优化的方法并不多,其中一个是把 txqueuelen
增加到 2500
:
# 1000 是 1GbE 的默认值,因此对 2.5GbE 来说 2500 看起来很合理
ifconfig eth8 txqueuelen 2500
不知道为什么 Realtek 在内核版本低于 5.2.3 时(梅林 384.19 的内核版本号为 4.1.51),默认不启用 RTL8156B
的 USB scatter/gather:
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3)
if (usb_device_no_sg_constraint(udev))
tp->sg_use = true;
#endif /* LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3) */
然而不启用这个功能会严重影响发送性能,因此另外一个优化方法是启用 RTL8156B
的 USB scatter/gather:
# 在 eth8 上启用 USB scatter/gather
echo enable > /sys/class/net/eth8/rtl_adv/sg_en
注意: 该选项仅适用于 r8152
驱动模块支持的 RTL
芯片。
iperf3
该项测试中 AX88U 是服务端,我的 Windows 台式机是客户端。
台式机到 AX88U,最快 2.35Gbits/sec:
PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1
Connecting to host 192.168.50.1, port 5201
[ 5] local 192.168.50.X port 13610 connected to 192.168.50.1 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 280 MBytes 2.35 Gbits/sec
[ 5] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 3.00-4.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 4.00-5.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 5.00-6.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 6.00-7.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 8.00-9.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 9.00-10.00 sec 279 MBytes 2.34 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 2.73 GBytes 2.34 Gbits/sec sender
[ 5] 0.00-10.04 sec 2.73 GBytes 2.33 Gbits/sec receiver
iperf Done.
AX88U 到台式机,最快 2.38Gbits/sec:
PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1 -R
Connecting to host 192.168.50.1, port 5201
Reverse mode, remote host 192.168.50.1 is sending
[ 5] local 192.168.50.X port 13619 connected to 192.168.50.1 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 284 MBytes 2.38 Gbits/sec
[ 5] 1.00-2.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 2.00-3.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 3.00-4.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 4.00-5.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 5.00-6.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 6.00-7.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 7.00-8.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 8.00-9.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 9.00-10.00 sec 283 MBytes 2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.03 sec 2.77 GBytes 2.37 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec receiver
iperf Done.
该项测试中我的 Windows 台式机作为服务端,我的 Surface Pro(通过千兆有线 USB 网卡连接到路由器)和 Macbook Pro(通过 802.11ac 无线连接到路由器,协商链路速度为 1300 Mbps)作为客户端,测试内容为两个客户端同时从服务端拷贝 20GB 的文件。
测试结果表明 BCM49408
的性能不足以双向跑满 2.5 Gbps,但是也还算可以了:最大传输速度约 195.7MB/s,平均传输速度约 177.3MB/s。
提示:在 Windows 上运行这个命令可以快速创建一个 20GB 的文件:fsutil file createNew 20GB.txt 21474836480
。最后一个参数为要创建的文件的大小,单位为字节。
截至本文发布时,市面上基于 USB 的 5GbE 解决方案只有 Aquantia 家(已经被 Marvell7 收购了)的 AQC111U
。就像一开始说的,5GbE 网卡并不便宜,我找到的最便宜的也得要 60 美金。
而且因为 AQC111U
是 USB 3.1 Gen 1 的方案,哪怕是启用 9K 巨型帧,减去 USB 开销8和以太网开销9,理论上能达到的最大传输速度约 3.96 Gbps。
同时 BCM49408
的性能也不足以支撑这个速度,启用了“性能优化”一章中描述的所有性能优化方法后,在我的 iperf3
测试中也只能实现最大 3.35 Gbits/sec(AX88U 到台式机)和 1.75 Gbits/sec(台式机到 AX88U)。
除此之外 AQC111U
还有以下的问题:
iperf3
之后,我的 AX88U 直接 kernel panic 然后重启了。AQC111U
的问题,还是我买的网卡(Sabrent NT-SS5G)的散热设计的问题。aqc111
驱动模块如果还是想试一下 AQC111U
,可以从这里下载驱动:Marvell Drivers。在下载页面的 PLATFORM 下拉框里选择 Linux Kernel 3.10 and Higher
,搜索结果中的 Marvell AQtion USB 3.1 Linux Driver
即为所需驱动,本文使用的 aqc111
驱动模块版本是 v1.3.3.0。
运行下面的命令来编译驱动:
# 架构: arm64
# 交叉编译器: aarch64-linux
# 内核源代码目录: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 SUBDIRS=`pwd` modules
如果你倾向用 Makefile
的话,可以在这里找到:附录 2:脚本:Makefile。
一切顺利的情况下,aqc111.ko
这个文件会生成在当前目录下,预编译好的版本可以在这里下载:附录:预编译的二进制文件:预编译的驱动模块。
把 aqc111.ko
上传到路由器 /jffs/drivers/
文件夹内,如果该文件夹不存在,则首先在 /jffs
文件夹下创建 drivers
文件夹。
AQC111U
的配置流程和 RTL8156B
的有些许不同,这里需要卸载的是模块是 cdc_ether
:
# 卸载 rndis_host,因为它依赖于 cdc_ether
rmmod rndis_host
# 卸载 cdc_ether
rmmod cdc_ether
# 加载 aqc111.ko
insmod /jffs/drivers/aqc111.ko
# 加载 cdc_ether
insmod cdc_ether
# 加在 rndis_host
insmod rndis_host
针对 AQC111U
的性能优化则和 RTL8156B
的差不多,只是没法调整 RX 中断合并窗口和 RX 缓冲区大小,因为 aqc111
驱动模块不支持修改这两个属性。
提示:完整的 AQC111U
配置脚本可以在这里找到:附录:脚本:AQC111U 配置脚本。
其实在去年早些时候,我就已经开始在 AX88U 上折腾 2.5GbE 网卡。只是在那个时候,市面上能买到的网卡都是基于 RTL8156
的,上文也提到过这个最初版本的芯片是有缺陷的。最终几个月之后,Realtek 修复了这些缺陷,并发布了修订版的 RTL8156B
,这才使得我完成了在 AX88U 上配置 2.5GbE 网卡的任务。
另外还有一个好消息:QNAP 最近发布了一款小型(相比其他至少要 1U 的产品)的被动散热 5 口 2.5GbE 交换机 QNAP QSW-1105-5T10,售价为 100 美金,算是物美价廉吧,适合家庭使用。
以下驱动模块仅适用于 AX88U(上传到 /jffs/drivers/
):
r8152
驱动模块 v2.13.0:r8152.ko。SHA1: e3a9f7a868baae6756bde4665a45852fcd578bdd
aqc111
驱动模块 v1.3.3.0:aqc111.ko。SHA1: 9dc4768154e065d19a274dcc33235a6b58fd9fb9
以下测试工具仅适用于 AX88U(上传到 /jffs/tools/
):
ethtool
5.8:ethtool。SHA1: 6936931589476b2001667c50a8f91d713be705fb
iperf3
3.9:iperf3。SHA1: ec19717ecb56a5b5bbfeda3316cd7e712cb6bde0
Makefile
以下 Makefile
仅适用于 AX88U:
r8152
:r8152/Makefile。SHA1: dc22346bdae40f5ea5bb2a0c193dc922fa3e4b60
aqc111
:acq111/Makefile。SHA1: 37dfc73f2e4b6676e5dd712ad38b389f0140fb76
RTL8156B
配置脚本/jffs/scripts/init-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/001-init-start
logger -t "rtl8156" "init-start: loading RTL8156 driver..."
# It's safe to `insmod` here, since `usbcore` has already been
# installed before `init-start` was called.
# init.c: sysinit() -> init_wl() ->
# init-broadcom.c: eval("insmod", "usbcore");
insmod /jffs/drivers/r8152.ko
# Note `insmod r8152.ko` must be executed before `cdc_ncm` module
# is loaded into kernel. Otherwise `cdc_ncm` will take control
# of the USB 2.5GbE adapter.
# If `r8152.ko` can't be loaded before `cdc_ncm`, then you should
# `rmmod cdc_ncm`,` insmod r8152.ko` and `insmod cdc_ncm`
# (remember to rm/insmod modules depending on `cdc_ncm` first)
logger -t "rtl8156" "init-start: all done"
date >> /tmp/001-init-start
/jffs/scripts/services-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "rtl8156" "services-start: setting up eth8..."
# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"
# Add eth8 into br0
brctl addif br0 eth8
# Set TX queue length to 2500 on eth8 and bring it up
ifconfig eth8 txqueuelen 2500 allmulti up
# Enable USB scatter/gather on eth8
echo enable > /sys/class/net/eth8/rtl_adv/sg_en
# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus
logger -t "rtl8156" "services-start: all done"
date >> /tmp/001-services-start
AQC111U
配置脚本尽管能在 AX88U 上使用 AQC111U
,但是发挥不出来 5Gbps 的性能,详情请参阅章节 实现 5GbE。
/jffs/scripts/services-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "aqc111" "services-start: loading AQC111 driver..."
# No need to `rmmod cdc_ether` here, as `cdc_ncm` will be loaded
# after `services-start` is called
insmod /jffs/drivers/aqc111.ko
# Note `insmod aqc111.ko` must be executed before `cdc_ether`
# module is loaded into kernel. Otherwise `cdc_ether` will take
# control of the USB 5GbE adapter.
# If `aqc111.ko` can't be loaded before `cdc_ether`, then you should
# `rmmod cdc_ether`, `insmod aqc111.ko` and `insmod cdc_ether`
# (remember to rm/insmod modules depending on cdc_ether first)
logger -t "aqc111" "services-start: AQC111 driver loaded"
# Now we can setup the new interface
logger -t "aqc111" "services-start: setting up eth8..."
# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"
# Add eth8 into br0 and bring eth8 up
brctl addif br0 eth8
ifconfig eth8 allmulti up
# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'.
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus
logger -t "aqc111" "services-start: all done"
date >> /tmp/001-services-start
Broadcom, BCM49408: 64 bit Quad-Core ARM v8 compliant Processor for Enterprise Access Point Applications. ↩
Realtek, Realtek Launches World’s First Single-Chip 2.5G Ethernet Controller for Multiple Applications, including Gaming Solution. ↩
domih, [HOWTO] 2.5Gbe or 5Gbe with H2 and/or N2 and/or C4?. ↩
VIA Labs, Inc., VL160 - USB-C™ 2:4 Data Switch with CC Function for USB 3.1. ↩
Kai-Heng Feng, Comment 39 for bug 1832472: cdc_ncm floods syslog unneccessarily. ↩
RMerl and Adamm, asuswrt-merlin.ng/build-all at master · RMerl/asuswrt-merlin.ng. ↩
Marvell, Marvell to Acquire Aquantia - Accelerating Ethernet Technology Leadership. ↩
Andrew Ku, So, What Makes USB 3.0 Slower Than We Expect? - Faster USB 3.0 Performance: Examining UASP And Turbo Mode. ↩
Rickard Nobel, Actual throughput on Gigabit Ethernet. ↩
Ryan Smith, At Last, a 2.5Gbps Consumer Network Switch: QNAP Releases QSW-1105-5T 5-Port Switch. ↩
However, the lack of integrated 2.5GbE port unfortunately shadows its power and ability, not to mention that its
processor BCM49408
does natively support 2.5GbE1. To make it worse, recently released
AX86U comes with a similar processor BCM4908
and, yeah, has a
built-in 2.5GbE port.
The good news is that there is an external solution from Realtek: RTL8156
2, a USB 3.0 based 2.5GbE controller.
Comparing to much expensive 5 or 10GbE equipments, it is fairly “cheap”: you can easily find a RLT8156
based USB dongle
for about $30.
But do note that there are two versions of this chip: RTL8156
and RTL8156B
. You should look for the B revision
(RTL8156B
), as the original RTL8156
has a weird stability issue in long time tests3.
In addition, if you bought a USB Type-C dongle, you may also need an “active” or “double side” USB Type-C to USB-A adapter
(which usually include a multiplexer chip, e.g. VL160
4). Otherwise, you have to figure out which side of the USB
Type-C port to insert, since one side will work at the speed of USB 3.0 and the other side will do USB 2.0 only.
If you directly insert the USB dongle into AX88U, there is no problem to get it recognized (on Network Map page in router’s
web management console). Also in ifconfig -a
, you can find that the dongle has been registered as usb0
.
However, if you try to bring it up with ifconfig usb0 up
, the dmesg
will be spammed by cdc_ncm
:
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
cdc_ncm 2-2:1.0 usb0: network connection: connected
cdc_ncm 2-2:1.0 usb0: 2500 mbit/s downlink 2500 mbit/s uplink
The reason is that the built-in driver in Linux kernel has no support for RTL8156
yet5.
So we have to build the driver on our own.
Follow this guide for setting up the compiling environment for Asuswrt-Merlin. Though it is titled for WSL (Windows Subsystem for Linux) 2, you can still follow these steps under WSL 1 (legacy WSL) or pure Linux.
If you’re using legacy WSL like me, you have to follow this comment to run 32-bit applications in legacy WSL, because the provided toolchains are 32-bit.
As we don’t really need the git
history, we can just clone the HEAD
:
# Clone the HEAD only
git clone --depth 1 https://github.com/RMerl/asuswrt-merlin.ng
Also note that instead of linking brcm-arm-sdk
folder under src-rt-6.x.4708
:
# `src-rt-6.x.4708` is for AC56U, AC68U and AC87U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-6.x.4708/toolchains
… in the guide, we link it under src-rt-5.02axhnd
:
# `src-rt-5.02axhnd` is for AX88U, AX58U and AX56U
ln -s ~/am-toolchains/brcm-arm-sdk ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/toolchains
… since we’re targeting AX88U, according to Merlin’s build script6.
Once you got the environment done, run this to build the firmware for AX88U:
# Targeting AX88U
cd release/src-rt-5.02axhnd && make rt-ax88u
It is not necessary to complete the whole firmware compiling, as we only need the generated header files (.h
)
from the part of compiling Linux kernel.
Therefore once the kernel building starts for a while, you may interrupt the build process by pressing CTRL+C
.
iperf3
Source code for iperf3
can be found at here.
At the time of writing, the latest version of iperf3
is 3.9.
Run the following lines to compile iperf3
for AX88U:
# Create `build` folder as installation prefix
mkdir build
# We need the static-linked binary
./configure --host=arm-linux --prefix=`pwd`/build --enable-static --disable-shared
# Build the binary and install it into `build` folder
make && make install
The iperf3
binary should be generated in ./build/bin/iperf3
. Or you can get the pre-compiled one in section
Appendix: Pre-compiled Binaries: Pre-compiled Tools.
For the iperf3
running on Windows, please check this post.
ethtool
Source code for ethtool
can be found at here.
At the time of writing, the latest version of ethtool
is 5.8.
Run the following lines to compile ethtool
for AX88U:
# Create `build` folder as installation prefix
mkdir build
# Disable netlink, as we don't need it at all in our case.
# This also helps eliminate the dependency on `libmnl`
./configure --host=arm-linux --prefix=`pwd`/build --disable-netlink
# Build the binary and install it into `build` folder
make & make install
The ethtool
binary should be generated in ./build/bin/ethtool
. Or you can get the pre-compiled one in section
Appendix: Pre-compiled Binaries: Pre-compiled Tools.
r8152
DriverIt might be confusing at the beginning, but yes, the driver for RTL8156B
is called r8152
. The source code can be
downloaded from here.
At the time of writing, the r8152
driver version is v2.13.0.
I added a line to function rtl8152_get_drvinfo
in r8152.c
, so we can verify the chip version is RTL8156B
via ethtool
:
--- source-2.13.0/r8152-original.c 2020-04-20 01:36:49.000000000 -0700
+++ source-2.13.0/r8152-modified.c 2020-10-06 23:47:07.123659400 -0700
@@ -16369,6 +16369,7 @@
strlcpy(info->driver, MODULENAME, sizeof(info->driver));
strlcpy(info->version, DRIVER_VERSION, sizeof(info->version));
usb_make_path(tp->udev, info->bus_info, sizeof(info->bus_info));
+ snprintf(info->fw_version, sizeof(info->fw_version), "0x%04x", tp->version);
}
#if LINUX_VERSION_CODE < KERNEL_VERSION(4,20,0)
Since we’re targeting AX88U, we need to call aarch64-linux
cross-compiler and set the architecture to arm64
:
# Architecture: arm64
# Cross-compiler: aarch64-linux
# Kernel source code directory: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 M=`pwd` modules
If you prefer using Makefile
instead, it can be found in section Appendix: Scripts: Makefile.
If everything goes well, the r8152.ko
should be generated in the working directory. You can also get the
pre-compiled one in section Appendix: Pre-compiled Binaries: Pre-compiled Drivers.
Upload r8152.ko
into /jffs/drivers/
, iperf3
and ethtool
into /jffs/tools/
. Remember to create drivers
and tools
folders in /jffs
first.
As we discussed in section No Plug-and-Play?, cdc_ncm
will try to take control of the USB dongle,
so we need to unload it first.
We need to check which modules depends on cdc_ncm
with lsmod | grep cdc_ncm
:
# admin@ax88u:/# lsmod | grep cdc_ncm
cdc_ncm 16787 1 cdc_mbim
usbnet 21074 7 cdc_mbim,qmi_wwan,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix
usbcore 166572 23 uas,usb_storage,cdc_mbim,qmi_wwan,cdc_wdm,cdc_ncm,rndis_host,cdc_ether,ax88179_178a,asix,cdc_acm,usbnet,usblp,ohci_pci,ohci_platform,ohci_hcd,ehci_pci,ehci_platform,ehci_hcd,xhci_pci,xhci_plat_hcd,xhci_hcd
So we have to unload cdc_mbim
first and cdc_ncm
later:
# Unload `cdc_mbim`, as it depends on `cdc_ncm`
rmmod cdc_mbim
# Unload `cdc_ncm`
rmmod cdc_ncm
Now we can load r8152.ko
and also load cdc_ncm
and cdc_mbim
back:
# Load `r8152` driver
insmod /jffs/drivers/r8152.ko
# Load `cdc_ncm`
insmod cdc_ncm
# Load `cdc_mbim`
insmod cdc_mbim
If everything goes well, system log (tail /tmp/syslog.log
) should reflect this:
# admin@ax88u:/# tail /tmp/syslog.log
Oct 7 21:27:42 kernel: usbcore: deregistering interface driver cdc_mbim
Oct 7 21:27:42 kernel: usbcore: registered new interface driver r8152
Oct 7 21:27:42 kernel: usb 2-2: reset SuperSpeed USB device number 2 using xhci-hcd
Oct 7 21:27:42 kernel: netif_napi_add() called with weight 256 on device eth%d
Oct 7 21:27:42 kernel: r8152 2-2:1.0 eth8: v2.13.0 (2020/04/20)
Oct 7 21:27:42 kernel: r8152 2-2:1.0 eth8: This product is covered by one or more of the following patents:
Oct 7 21:27:42 kernel: US6,570,884, US6,115,776, and US6,327,625.
Oct 7 21:27:42 hotplug: add net eth8.
Oct 7 21:27:42 kernel: usbcore: registered new interface driver cdc_ncm
Oct 7 21:27:42 kernel: usbcore: registered new interface driver cdc_mbim
As you can see, the adapter is registered as eth8
, thus if we run ifconfig eth8
:
eth8 Link encap:Ethernet HWaddr 00:E0:4C:00:FA:KE
inet6 addr: fe80::2e0:4cff:fe00:fake/64 Scope:Link
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Also if you check the Network Map page in Web management UI, the adapter should show up there:
Then we can update nvram
variables, add eth8
to br0
and bring eth8
up:
# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"
# Add eth8 into br0 and bring it up
brctl addif br0 eth8
ifconfig eth8 allmulti up
Finally we can connect the ethernet cable to the USB dongle.
Run /jffs/tools/ethtool eth8
to verify the driver is correctly loaded and the negotiated link speed
is 2500Mb/s full-duplex:
# admin@ax88u:/# /jffs/tools/ethtool eth8
Settings for eth8:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseX/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseX/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 2500Mb/s
Duplex: Full
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00007fff (32767)
drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
Link detected: yes
Tip: Complete scripts for RTL8156B
can be found in Appendix: Scripts: Scripts for RTL8156B.
RTL
VersionTo tell if the dongle is really RTL8156B
, you can run /jffs/tools/ethtool --driver eth8
:
# admin@ax88u:/# /jffs/tools/ethtool --driver eth8
driver: r8152
version: v2.13.0 (2020/04/20)
firmware-version: 0x000e
expansion-rom-version:
bus-info: usb-xhci-hcd.0-2
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
firmware-version
is either 0x000b
or 0x000c
, the internal chip is RTL8156
.firmware-version
is either 0x000d
or 0x000e
, the internal chip is RTL8156B
.As you can see, mine firmware-version
is 0x000e
thus it is RTL8156B
.
Note: In order to show the firmware-version
, the added line to function rtl8152_get_drvinfo
in r8152.c
is necessary.
Check deatils about it in section Preparation: Compile r8152
driver.
Complete scripts for RTL8156B
can be found in Appendix: Scripts: Scripts for RTL8156B.
By default, hardware interrupts for xhci-hcd:usb1
on AX88U is only handled by CPU core 4 (cat /proc/irq/28/smp_affinity_list
).
However, as RTL8156B
controller only has 1 RX queue, we can’t spread those hardware interrupts to other cores
via changing smp_affinity_list
.
Thus we have to enable RPS (Receive Packet Steering). RPS helps to distribute software interrupts (softirq
) to multiple cores.
You can check here
for more details about RPS.
To enable RPS on eth8
:
# Enable RPS (Receive Packet Steering) on eth8
# Since AX88U has four cores, so we set it to 'f'
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus
At least in my test, enabling RPS alone allows the adapter to reach its maxmium performance in iperf3
.
But if you are looking for more RX tweaks, here are some examples:
net.core.netdev_max_backlog
. The default value is 1000
and you may increase it to 2500
via:
# 1000 is the default for 1GbE, thus 2500 seems reasonable for 2.5GbE
echo 2500 > /proc/sys/net/core/netdev_max_backlog
rx-usecs
is 15
and you may change it via:
# <N> is the microseconds to delay an RX interrupt after packet arrival
/jffs/tool/ethtool -C eth8 rx-usecs <N>
100
and you may change it to 4096
(which is the maxmium):
# 4096 is the maxmium supported value for RX ring buffer
/jffs/tool/ethtool -G eth8 rx 4096
More information can be found here, here and here.
Unlike RPS, XPS (Transmit Packet Steering) will not help increase performance in our case, because
For a network device with a single transmission queue, XPS configuration has no effect, since there is no choice in this case.
Linux Kernel Documentation — Scaling in the Linux Networking Stack
… and RTL8156B
has only 1 TX queue.
But we can increase txqueuelen
to 2500
on eth8
anyway:
# 1000 is the default for 1GbE, thus 2500 seems reasonable for 2.5GbE
ifconfig eth8 txqueuelen 2500
Somehow Realtek didn’t enable USB scatter/gather for kernel version below 5.2.3 (AX88U’s kernel version is 4.1.51),
according to r8152.c
:
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3)
if (usb_device_no_sg_constraint(udev))
tp->sg_use = true;
#endif /* LINUX_VERSION_CODE >= KERNEL_VERSION(5,2,3) */
… and without USB scatter/gather enabled, the TX performance is really hurt.
Of course you can modify the driver, but I found a way to enable it externally:
# Enable USB scatter/gather on eth8
echo enable > /sys/class/net/eth8/rtl_adv/sg_en
Note: This option only works for r8152
driver supported RTL
chips.
iperf3
AX88U is the iperf3
server and my Windows PC is the client.
PC to AX88U, achieving ~2.35Gbits/sec:
PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1
Connecting to host 192.168.50.1, port 5201
[ 5] local 192.168.50.X port 13610 connected to 192.168.50.1 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 280 MBytes 2.35 Gbits/sec
[ 5] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 3.00-4.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 4.00-5.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 5.00-6.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 6.00-7.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 8.00-9.00 sec 279 MBytes 2.34 Gbits/sec
[ 5] 9.00-10.00 sec 279 MBytes 2.34 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 2.73 GBytes 2.34 Gbits/sec sender
[ 5] 0.00-10.04 sec 2.73 GBytes 2.33 Gbits/sec receiver
iperf Done.
AX88U to PC, achieving ~2.38Gbits/sec:
PS E:\Tools\iperf-3.9-win64> .\iperf3.exe -c 192.168.50.1 -R
Connecting to host 192.168.50.1, port 5201
Reverse mode, remote host 192.168.50.1 is sending
[ 5] local 192.168.50.X port 13619 connected to 192.168.50.1 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 284 MBytes 2.38 Gbits/sec
[ 5] 1.00-2.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 2.00-3.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 3.00-4.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 4.00-5.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 5.00-6.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 6.00-7.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 7.00-8.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 8.00-9.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 9.00-10.00 sec 283 MBytes 2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.03 sec 2.77 GBytes 2.37 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec receiver
iperf Done.
Here we test a real-world application: SMB file transfer.
My Windows PC acts the SMB server, while my Surface Pro (connected via wire with USB 1GbE adapter) and my Macbook Pro (connected via 802.11ac WiFi, negotiated link speed: 1300Mbps) are trying to copy a 20GB file simultaneously.
Consider the overhead of SMB protocol and the relatively less powerful BCM49408
comparing to x86 CPUs, I’d say the
result below is fairly good by the speed reaching ~195.7MB/s and averaging at around 177.3MB/s.
Tip: To quickly create a dummy file of 20GB on Windows: fsutil file createNew 20GB.txt 21474836480
.
The last parameter is the specified file size in bytes.
At the time of writing, the only USB solution for 5GbE is Aquantia’s (acquired by Marvell7)
AQC111U
.
Like I said before, 5GbE is not cheap: the cheapest 5GbE adapter I found costs $60.
Since AQC111U
is a USB 3.1 Gen 1 based solution, even with 9K jumbo frame enabled, theoretically the
maxmium speed you can reach is about 3.96 Gbits/sec considering the overhead from both USB8 and
Ethernet9.
Not to mention that BCM49408
seems not able to handle this speed: after applied all the tweaks described in
section Performance Tuning, I can only get ~3.35 Gbits/sec from AX88U to PC, and ~1.75 Gbits/secs
from PC to AX88U in iperf3
test.
There are also some other issues:
AQC111U
driver seems unstable. Somehow my AX88U got kernel panic and suddenly rebooted after running iperf3
for a short while.AQC111U
or the thermal design of the particular adapter I have.aqc111
DriverIf you still want to give it a try, you can download the driver code here.
Choose Linux Kernel 3.10 and Higher
in the PLATFORM droplist and Marvell AQtion USB 3.1 Linux Driver
is the one to look for.
At the time of writing, the least driver version is v1.3.3.0.
Since we’re targeting AX88U, we need to call aarch64-linux
cross-compiler and set the architecture to arm64
:
# Architecture: arm64
# Cross-compiler: aarch64-linux
# Kernel source code directory: ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1
make ARCH=arm64 CROSS_COMPILE=aarch64-linux- -C ~/asuswrt-merlin.ng/release/src-rt-5.02axhnd/kernel/linux-4.1 SUBDIRS=`pwd` modules
If you prefer using Makefile
instead, it can be found in section Appendix: Scripts: Makefile.
If everything goes well, the aqc111.ko
should be generated in the working directory. You can also get the
pre-compiled one in section Appendix: Pre-compiled Binaries: Pre-compiled Drivers.
First upload aqc111.ko
into /jffs/drivers/
. Remember to create drivers
folder under /jffs
if not exist.
The installation is a little bit different from RTL8156B
. Instead of cdc_ncm
, we need to unload cdc_ether
:
# Unload `rndis_host`, as it depends on `cdc_ether`
rmmod rndis_host
# Unload `cdc_ether`
rmmod cdc_ether
# Load `aqc111.ko`
insmod /jffs/drivers/aqc111.ko
# Load `cdc_ether` back
insmod cdc_ether
# Load `rndis_host` back
insmod rndis_host
Tuning performance for AQC111U
is similar to RTL8156B
. However you can’t adjust interrupt coalescing or ring buffer size,
since aqc111
driver doesn’t support them. You can refer to section Performance Tuning for more details.
Tip: Complete scripts for AQC111U
can be found in Appendix: Scripts: Scripts for AQC111U.
Actually early this year, I’ve already started to try to bring 2.5GbE to AX88U. At that time, however, there was only
the original RTL8156
chip and I couldn’t make it reach the maximum performance in iperf3
test.
Finally several months later, Realtek fixed the problem and released the RTL8156B
revision. I’m so happy to see that
I finally got the next generation of network and made my router future-proof.
Another good news is that QNAP recently announced an affordable ($100) 5-port 2.5GbE switch QNAP QSW-1105-5T10, which is passive-cooled and small in size - pretty suitable for home networking.
I hope you enjoy this post and upgrade your network to the next generation :)
Pre-compiled drivers for AX88U (store them into /jffs/drivers/
):
r8152
driver v2.13.0: r8152.ko. SHA1: e3a9f7a868baae6756bde4665a45852fcd578bdd
aqc111
driver v1.3.3.0: aqc111.ko. SHA1: 9dc4768154e065d19a274dcc33235a6b58fd9fb9
Pre-compiled tools for AX88U (store them into /jffs/tools
):
ethtool
5.8: ethtool. SHA1: 6936931589476b2001667c50a8f91d713be705fb
iperf3
3.9: iperf3. SHA1: ec19717ecb56a5b5bbfeda3316cd7e712cb6bde0
Makefile
Here are the Makefile
s for drivers targeting AX88U.
Makefile
for r8152
: r8152/Makefile. SHA1: dc22346bdae40f5ea5bb2a0c193dc922fa3e4b60
Makefile
for aqc111
: acq111/Makefile. SHA1: 37dfc73f2e4b6676e5dd712ad38b389f0140fb76
RTL8156B
/jffs/scripts/init-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/001-init-start
logger -t "rtl8156" "init-start: loading RTL8156 driver..."
# It's safe to `insmod` here, since `usbcore` has already been
# installed before `init-start` was called.
# init.c: sysinit() -> init_wl() ->
# init-broadcom.c: eval("insmod", "usbcore");
insmod /jffs/drivers/r8152.ko
# Note `insmod r8152.ko` must be executed before `cdc_ncm` module
# is loaded into kernel. Otherwise `cdc_ncm` will take control
# of the USB 2.5GbE adapter.
# If `r8152.ko` can't be loaded before `cdc_ncm`, then you should
# `rmmod cdc_ncm`,` insmod r8152.ko` and `insmod cdc_ncm`
# (remember to rm/insmod modules depending on `cdc_ncm` first)
logger -t "rtl8156" "init-start: all done"
date >> /tmp/001-init-start
/jffs/scripts/services-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "rtl8156" "services-start: setting up eth8..."
# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"
# Add eth8 into br0
brctl addif br0 eth8
# Set TX queue length to 2500 on eth8 and bring it up
ifconfig eth8 txqueuelen 2500 allmulti up
# Enable USB scatter/gather on eth8
echo enable > /sys/class/net/eth8/rtl_adv/sg_en
# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus
logger -t "rtl8156" "services-start: all done"
date >> /tmp/001-services-start
AQC111U
Though AQC111U
works on AX88U, its maximum performance is unachievable.
Read section What about 5GbE? for more details.
/jffs/scripts/services-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/001-services-start
logger -t "aqc111" "services-start: loading AQC111 driver..."
# No need to `rmmod cdc_ether` here, as `cdc_ncm` will be loaded
# after `services-start` is called
insmod /jffs/drivers/aqc111.ko
# Note `insmod aqc111.ko` must be executed before `cdc_ether`
# module is loaded into kernel. Otherwise `cdc_ether` will take
# control of the USB 5GbE adapter.
# If `aqc111.ko` can't be loaded before `cdc_ether`, then you should
# `rmmod cdc_ether`, `insmod aqc111.ko` and `insmod cdc_ether`
# (remember to rm/insmod modules depending on cdc_ether first)
logger -t "aqc111" "services-start: AQC111 driver loaded"
# Now we can setup the new interface
logger -t "aqc111" "services-start: setting up eth8..."
# Update nvram variables
nvram set br0_ifnames="$(nvram get br0_ifnames) eth8"
nvram set lan_ifnames="$(nvram get lan_ifnames) eth8"
nvram set wired_ifnames="$(nvram get wired_ifnames) eth8"
# Add eth8 into br0 and bring eth8 up
brctl addif br0 eth8
ifconfig eth8 allmulti up
# Enable RPS (Receive Packet Steering) on eth8.
# Since AX88U has four cores, we set it to 'f'.
echo f > /sys/class/net/eth8/queues/rx-0/rps_cpus
logger -t "aqc111" "services-start: all done"
date >> /tmp/001-services-start
Broadcom, BCM49408: 64 bit Quad-Core ARM v8 compliant Processor for Enterprise Access Point Applications. ↩
Realtek, Realtek Launches World’s First Single-Chip 2.5G Ethernet Controller for Multiple Applications, including Gaming Solution. ↩
domih, [HOWTO] 2.5Gbe or 5Gbe with H2 and/or N2 and/or C4?. ↩
VIA Labs, Inc., VL160 - USB-C™ 2:4 Data Switch with CC Function for USB 3.1. ↩
Kai-Heng Feng, Comment 39 for bug 1832472: cdc_ncm floods syslog unneccessarily. ↩
RMerl and Adamm, asuswrt-merlin.ng/build-all at master · RMerl/asuswrt-merlin.ng. ↩
Marvell, Marvell to Acquire Aquantia - Accelerating Ethernet Technology Leadership. ↩
Andrew Ku, So, What Makes USB 3.0 Slower Than We Expect? - Faster USB 3.0 Performance: Examining UASP And Turbo Mode. ↩
Rickard Nobel, Actual throughput on Gigabit Ethernet. ↩
Ryan Smith, At Last, a 2.5Gbps Consumer Network Switch: QNAP Releases QSW-1105-5T 5-Port Switch. ↩
Tip: Don’t care about details? Here is the final tool: MATLAB Classification Tree Parser.
Update: After I finished everything (the tool and this post), I somehow found that recently a MathWorks staff released a helper in File Exchange to export MATLAB models to PMML. What can I say? ¯\_(ツ)_/¯.
An example output1 (i.e. view(ctree)
) of MATLAB classification tree looks like:
Decision tree for classification
1 if x3<2.45 then node 2 elseif x3>=2.45 then node 3 else setosa
2 class = setosa
3 if x4<1.75 then node 4 elseif x4>=1.75 then node 5 else versicolor
4 if x3<4.95 then node 6 elseif x3>=4.95 then node 7 else versicolor
5 class = virginica
6 if x4<1.65 then node 8 elseif x4>=1.65 then node 9 else versicolor
7 class = virginica
8 class = versicolor
9 class = virginica
This is quite difficult to use in other environments. Say if I want to make a classifier in C++, I need to convert the above tree output into C++ code by hand, which is a very tedious work and prone to error especially when the tree is dense.
Though MATLAB allows you to visually inspect tree in GUI via view(ctree, 'mode', 'graph')
, it is still annoying to convert it manually.
Also in one of my projects, I’ve implemented a decision tree classifier in C++ and it allows me to dynamically load decision trees from JSON files. Since these trees have to be updated periodically, I have to manually write the JSON frequently for more than a year. It would definitely be nice if I can make a tool to automatically generate those JSON files.
In the beginning, I looked into MATLAB toolbox’s source code but found nothing useful about the view
method in either
class ClassificationTree
or CompactClassificationTree
. Thus I need to “decipher” the output by myself.
Though it is possible to parse the tree output with regular expressions, I’d prefer to solve it in a similar way of building compilers. Because we’re dealing with a tree, and a parser in compiler also build a tree (AST, or Abstract Syntax Tree).
I wrote the grammar in ANTLR 4. ANTLR can also generates lexer and parser for my later building the tool.
Here is the context-free grammar for MATLAB classification tree:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// MatlabCTree.g4
grammar MatlabCTree;
start : 'Decision tree for classification'? stmtList;
stmtList : stmt | stmtList stmt;
stmt : INTEGER (ifStmt | classExpr);
ifStmt : ifClause elseifClause? elseClause?;
ifClause : 'if' condExpr 'then' nodeExpr;
elseifClause : 'elseif' condExpr 'then' nodeExpr;
elseClause : 'else' nodeExpr;
labelExpr : INTEGER | IDENTIFIER;
classExpr : 'class' '=' labelExpr;
nodeExpr : 'node' INTEGER | labelExpr;
constExpr : INTEGER | RATIONAL;
condExpr : IDENTIFIER '<' constExpr
| IDENTIFIER '>' constExpr
| IDENTIFIER '<=' constExpr
| IDENTIFIER '>=' constExpr
| IDENTIFIER '==' constExpr
| IDENTIFIER '~=' constExpr;
INTEGER : '0' | [1-9] [0-9]*;
IDENTIFIER : [a-zA-Z] [a-zA-Z0-9_]*;
// Adapted from https://git.io/JU4M9
RATIONAL : NUMBER (E SIGN? NUMBER)?;
fragment NUMBER : INTEGER? '.' [0-9]+;
fragment E : 'E' | 'e';
fragment SIGN : '+' | '-';
WHITESPACE : [ \t\r\n]+ -> skip;
Some key points in the above grammar:
Decision tree for classification
is optional.if feature<a then node x elseif feature>=a then node y else some_label
However I’m not sure if the elseif
clause and else
clause would be present in all scenarios. So I decided to leave
them as optional.
<
, >
, <=
, >=
, ==
and ~=
.0
s are not allowed in integers.Now we have the grammar, we can test MATLAB’s example classification with ANTLR’s built-in GUI tester (click the figure to enlarge):
The above parsing tree looks like just what I want! It’s time to build the tool.
Initially I was thinking to build a tool in Java or C#, but later I realized why not make a web-based tool as ANTLR supports outputting lexers and parsers in JavaScript. A online tool also makes everyone’s life easier: no need to worry in setting up environments and doing (potentially annoying) compiling job.
Also this tool will purely run in the browser, that is 100% of the code running on user’s device, there is no need to set up a server nor any interaction with remote server at all.
If you’re not familiar with using ANTLR on JavaScript target, here is the official documentation.
I used the below command to generate lexer and parser. Note that I’ll use visitor
instead of listener
in the final product.
java -jar antlr-4.8-complete.jar -Dlanguage=JavaScript -visitor -no-listener MatlabCTree.g4
Note: I’ll only cover the main points in the following sections. If you want to dive into details, source code is available in section Source code.
This part is pretty straightforward, just traverse the generated AST and store all classification tree nodes into an array.
Since it’s possible that node numbers are not in a increasing order, for example:
Decision tree for classification
1 if x<0 then node 2 elseif x>=0 then node 3 else foo
3 class = foo
2 class = bar
Note that the above order is 1
, 3
, 2
while we expect 1
, 2
, 3
.
In order to detect this, we need to setup a counter, increase the counter every time we visit stmt
and
check if the node number matches the counter:
visitStmt(ctx) {
// stmt: INTEGER (ifStmt | classExpr);
let nodeNumber = parseInt(ctx.children[0].getText());
// Check if the counter matches node number in stmt
if (++this.nodeNumber !== nodeNumber) {
let symbol = ctx.children[0].symbol;
this.errors.push({
line: symbol.line,
message: `mismatched node number ${nodeNumber} ` +
`expecting ${this.nodeNumber}`
});
}
let result = this.visitChildren(ctx);
this.jsonNodes.push(result[1]);
// Actually it doesn't matter whether result is returned
return result;
}
Because the generated tree from fitctree
is a binary tree2, we just need to check if the conditions in
if
and elseif
clauses are complement. else
clause can be safely ignored.
Here are the examples that conditions are not complement:
# Feature name doesn't match (x and y)
1 if x<0.5 then node 2 elseif y>=0.5 then node 3 else foo
# Comparison value doesn't match (0.5 and -0.5)
1 if x<0.5 then node 2 elseif x>=-0.5 then node 3 else foo
# Operators are not complement (< and >, which supposed to be < and >=)
1 if x<0.5 then node 2 elseif x>0.5 then node 3 else foo
To handle this:
visitIfStmt(ctx) {
// ifStmt: ifClause elseifClause? elseClause?;
let result = this.visitChildren(ctx);
let featureName = result[0].condition.identifer;
let operator = result[0].condition.operator;
let compareValue = result[0].condition.value;
let trueBranch = result[0].branch;
let falseBranch = null;
// 2 or more clauses?
if (result.length > 1) {
// elseifClause
if (result[1].hasOwnProperty("condition")) {
let elseifFN = result[1].condition.identifer;
let elseifOP = result[1].condition.operator;
let elseifCV = result[1].condition.value;
// Does conditions match?
if (elseifFN !== featureName ||
elseifOP !== this.complementOperators[operator] ||
elseifCV !== compareValue) {
let elseIfExpr = ctx.children[1].children[1];
this.errors.push({
line: elseIfExpr.start.line,
message: `mismatched elseif clause '${elseIfExpr.getText()}' ` +
`expecting '${featureName}` +
`${this.complementOperators[operator]}` +
`${compareValue}'`
});
} else {
falseBranch = result[1].branch;
}
} else {
// elseClause
falseBranch = result[1];
}
}
// We ignored the elseClause here if there are 3 clauses present.
// This should be safe since the generated classification tree from
// MATLAB's fitctree function is supposed to be a binary tree.
return { featureName, operator, compareValue, trueBranch, falseBranch };
}
The output JSON follows the below format:
{
"featureName": "{feature name}",
"operator": "{operator for comparison}",
"compareValue": {the value to be compared (in double)},
"trueBranch": {true branch, if `fn op val` holds},
"falseBranch": {false branch, otherwise}
}
trueBranch
and falseBranch
are either a string representing the predicted label, or an above object indicating a child tree node.
My classifier actually accepts a compact version of it. The key names are simplified into fn
, op
, val
, tb
, and fb
for saving spaces.
Since we have already gathered all the nodes in the classification tree via AST visitor, we just need to perform a simple DFS (Depth-First Search):
dfs(node, index) {
this.visitedNodes.push(index);
return {
featureName: node.featureName,
operator: node.operator,
compareValue: node.compareValue,
trueBranch: this.visitBranch(node.trueBranch),
falseBranch: this.visitBranch(node.falseBranch)
};
}
Even though it is guaranteed to be a tree, it doesn’t prevent us to add support for forest:
// Perform DFS on all nodes so we can also handle forests
this.jsonNodes.forEach((node, index) => {
// Only take care not yet visited ifStmt nodes
if (typeof node !== "string" && !this.visitedNodes.includes(index)) {
jsonOutput.push(this.dfs(node, index));
}
});
We also need to handle a boundary condition where the tree consists of a label node only, for example:
Decision tree for classification
1 class = setosa
To handle this:
this.jsonNodes.forEach((node, index) => {
// Boundary condition, the tree only has a label.
if (typeof node === "string" && !this.visitedNodes.includes(index)) {
jsonOutput.push(node);
}
});
One more thing need to take care is branch target’s node number in if
and elseif
, since it might exceed the number of nodes. For example:
Decision tree for classification
1 if x3<2.45 then node 2 elseif x3>=2.45 then node 3 else setosa
Apparently, node 2
and 3
are missing in the input. So we need to detect this and report error:
if (branch.target > this.jsonNodes.length) {
this.errors.push({
line: branch.line,
message: `invalid node index ${branch.target} ` +
`expecting no greater than ${this.jsonNodes.length}`
});
return null;
}
Now we can make a wrapper class MatlabCTreeConverter
to call lexer and parser,
and invoke AST visitor and JSON generator.
Here is the way to handle syntax errors, syntaxErrors
is for later error reporting:
let syntaxErrors = [];
// Adapted from https://stackoverflow.com/a/60841205
parser.removeErrorListeners();
parser.addErrorListener({
syntaxError: (recognizer, offendingSymbol, line, column, msg, err) => {
syntaxErrors.push({ line: line, message: msg });
}
});
Since our starting rule happened to be start
, we can build AST by:
let ast = parser.start();
webpack
everythingSetting up webpack
is quite annoying and very painful. Fortunately, I found a working configuration file for webpack
3.
Hat tip to @nicolasdao!
Note: Since anyway the core code will be publicly accessible, uglifyjs-webpack-plugin
is not used.
Once you got everything in that gist done,
remember to add the following entry to webpack.config.js
’s module.exports
:
node: { module: 'empty' net: 'empty', fs: 'empty' }
This is VERY important (noted in ANTLR’s official documentation),
otherwise ANTLR will try to import fs
module which is not available in browsers.
Big bruh moment. Needless to say how tedious is UI design and implementation. Anyway here is how it looks like:
If any error is detected in the input, it will report which line goes wrong:
In case you’re curious, I’m using CodeMirror for the input and output editors, and Viz.js for drawing the classification tree.
Finally the link to the tool: MATLAB Classification Tree Parser.
Grammar definition has been covered here. Simply copy the code there and save it as MatlabCTree.g4
.
Full JavaScript project for webpack
: matlab-ctree-parser.zip. SHA1: 212f9647d35ad83f67a01a4bc825d72ac2ad21df
MathWorks Help Center, View Decision Tree - MATLAB & Simulink. ↩ ↩2
MathWorks Help Center, Fit binary decision tree for multiclass classification - MATLAB fitctree. ↩
Nicolas Dao, Basic damn Webpack config for simple transpilation ES6 to ES5. ↩
不过无论是原厂固件还是梅林固件,它们都没有提供原生的 GUI 划分 VLAN 的功能(相比之下,在 OpenWrt 上配置 VLAN 就很方便)。但是由于博通搞的闭源驱动1,OpenWrt 在有生之年大概都不会支持 AX88U。所以要在 AX88U 上配置 VLAN,只能通过命令行的方式。
AX88U 上一共有 8 个 LAN 口,我想把 LAN 2 隔离出来,用做访客网络。换句话说,LAN 1, 3 - 8 划入 VLAN 1
, LAN 2 单独划入 VLAN 2
。
理论上,真正的隔离应该是禁止两个 VLAN 互相访问。但是考虑到万一需要在我的电脑(VLAN 1
)上重新配置二级路由器(VLAN 2
),还是得允许 VLAN 1
到 VLAN 2
的单向通信。
此外,上述提到的隔离对客户端应当是完全透明的:VLAN 2
里的客户端不需要额外的配置,只需要插上网线即可上网。
robocfg
?很不幸的是,AX88U(博通 HND 平台)移除了 robocfg
命令。搜索了一番之后,发现似乎 vlanctl
2 可以用来配置 VLAN。但是玩了一阵子,目测 vlanctl
只能搞 Tagged VLAN,没法实现静态 VLAN。
不过以防有人有这方面的需求,推荐阅读这篇文章:上海电信 TL-EP110 + RT-AC86U 实现观看 4K IPTV 无卡顿 (2019-10)。这篇文章介绍了如何在 AC86U(也是 HND 平台的路由器)上配置 Tagged VLAN。
在网上我没搜到相关的信息,不过经过若干次网线插拔之后,总结出来这张表:
以太网接口 | 物理端口 |
---|---|
eth0 | WAN |
eth1 | LAN 4 |
eth2 | LAN 3 |
eth3 | LAN 2 |
eth4 | LAN 1 |
eth5 | LAN 5 - 8 的网桥 |
eth6 | 2.4 GHz 无线 |
eth7 | 5 GHz 无线 |
这里注意 eth5
应该是通过 BCM53134 组的硬件网桥(LAN 5 - 8)。使用 etchtl
和 ethswctl
或许能解散这个网桥,不过我没试验。所以,下文提到的解决方案只能用于隔离 LAN 1 - 4。
此外,AX88U 默认把 eth1
~ eth7
组成了网桥 br0
。
注意:我的 WAN 联机类型 是 动态 IP。如果你用的是 PPPoE 的话,可能需要看情况把下文提到的 eth0
换成 ppp0
。
ebtables
)的解决方案如果不需要给两个 VLAN 划分独立子网的话,只需要一条 ebtables
命令就可以了:
# 在 AX88U 上,eth3 是物理端口 LAN 2
ebtables -A FORWARD -i eth3 -o br0 -j DROP
这里的 eth3
是物理端口 LAN 2。这条命令使用了二层防火墙 ebtables
,用于阻隔 eth3
和 br0
之间的通信。由于 eth3
还在 br0
里,所以不需要额外配置 iptables
或者 DHCP。
如果需要给两个 VLAN 划分独立的子网的话,就需要创建新的网桥了。下文以 VLAN 1
使用 192.168.50.0/24
、VLAN 2
使用 192.168.150.0/24
为例。
br1
首先,从 br0
里移除 eth3
,创建 br1
并把 eth3
加入 br1
:
# 从 br0 中删除需要隔离的端口
brctl delif br0 eth3
# 把需要隔离的端口加入 br1
brctl addbr br1
brctl stp br1 on # 启用 STP 协议防止环路,避免网络风暴
brctl addif br1 eth3
这里新建网桥 br1
而不是直接使用 eth3
是为了方便管理:如果以后想把其他端口也隔离进 VLAN 2
,只需要把对应的端口移出 br0
并加入 br1
即可。由于下文提到的所有规则都是针对 br1
的,所以新加入的端口不需要额外的设置。
执行 brctl show
验证 br1
已正确创建:
admin@ax88u:/# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.a85e45fakeid yes eth1
eth2
eth4
eth5
eth6
eth7
br1 8000.a85e45fakeid yes eth3
然后,给 br1
设置 IPv4 地址 192.168.150.1/24
,并启用 br1
:
# 设置 br1 的 IPv4 地址并启用 br1
# 这里使用的 IPv4 子网是 192.168.150.0/24
# 系统会自动设置 IPv6 链路本地地址
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up
由于 AX88U 默认给 br0
分配了 192.168.50.1/24
,所以不需要设置 br0
。
最后,执行 ifconfig br
验证 br1
的 IPv4 地址:
admin@ax88u:/# ifconfig br1
br1 Link encap:Ethernet HWaddr A8:5E:45:00:FA:KE
inet addr:192.168.150.1 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
系统会使用 EUI-64 算法自动为 br1
生成 IPv6 链路本地地址。
提示:完整的脚本可以在 脚本与配置文件:/jffs/scripts/services-start
中找到。
iptables
规则首先,允许来自 br1
的传入连接:
# 允许来自 br1 的传入连接
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT
但是禁止 br1
访问路由器 Web UI 和 SSH:
# 禁止 br1 访问路由器 Web UI 和 SSH
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP
然后,不转发来自 br1
的数据包:
# 不转发来自 br1 的数据包
iptables -I FORWARD -i br1 -j DROP
但是允许 br1
内部的转发:
# 但是允许 br1 内部的转发
iptables -I FORWARD -i br1 -o br1 -j ACCEPT
也允许 br1
的数据包转发至 eth0
(WAN):
# 允许 br1 的数据包转发至 eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT
也允许 br0
到 br1
的单向通信:
# 允许 br0 到 br1 的单向通信
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
最后,配置 br1
所属子网 192.168.150.0/24
的 NAT:
iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
-o br1 -j MASQUERADE
分别执行 iptables -S INPUT
,iptables -S FORWARD
和 iptables -t nat -S POSTROUTING
验证相应的规则已添加:
admin@ax88u:/# iptables -S INPUT
-P INPUT ACCEPT
-A INPUT -i br1 -p tcp -m tcp --dport 22 -j DROP
-A INPUT -i br1 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i br1 -m state --state NEW -j ACCEPT
# ... 省略多余输出 ...
-A INPUT -j DROP
admin@ax88u:/# iptables -S FORWARD
-P FORWARD DROP
-A FORWARD -i br1 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br0 -o br1 -j ACCEPT
-A FORWARD -i br1 -o eth0 -j ACCEPT
-A FORWARD -i br1 -o br1 -j ACCEPT
-A FORWARD -i br1 -j DROP
# ... 省略多余输出 ...
admin@ax88u:/# iptables -t nat -S POSTROUTING
-P POSTROUTING ACCEPT
# ... 省略多余输出 ...
-A POSTROUTING -s 192.168.50.0/24 -d 192.168.50.0/24 -o br0 -j MASQUERADE
-A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 -o br1 -j MASQUERADE
提示:
dnsmasq
)在 /jffs/configs/
中创建配置文件 dnsmasq.conf.add
:
touch /jffs/configs/dnsmasq.conf.add
比照 br0
,为 br1
设置 DHCPv4 的地址池和选项:
cat <<EOF >> /jffs/configs/dnsmasq.conf.add
interface=br1
# DHCPv4 地址池: 192.168.150.2 - 192.168.150.254, 子网掩码: 255.255.255.0
# DHCPv4 租约时间: 86400 秒 (1 天)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 路由器 (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
EOF
重启 dnsmasq
以应用设置:
service restart_dnsmasq
执行 tail /tmp/syslog.log -n 50
验证 dnsmasq.conf.add
已被正确加载:
admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... 省略多余输出 ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
# ... 省略多余输出 ...
本文使用的是无状态 DHCPv6 (DHCPv6 stateless),主机的 IPv6 地址是通过 SLAAC3 自动生成的。
由于 SLAAC 使用了 EUI-64 算法,这要求 IPv6 前缀长度必须小于 /64
。所以要使用下文提到的解决方案,ISP 分配的 IPv6 LAN 前缀长度必须小于等于 /63
。
某些 ISP 允许通过发送前缀长度提示获取小于 /64
的前缀。例如,我这里的 ISP Charter Spectrum 就允许通过前缀提示获取 /56
的前缀。
我的 AX88U 上 IPv6 的设置是:
名称 | 值 |
---|---|
基本设置 | |
联机类型 | Native |
DHCP-PD | 启用 |
IPv6 内部网络设置 | |
自动配置设置 | Stateless |
IPv6 DNS 设置 | |
自动接上 DNS 服务器 | 关闭 |
IPv6 DNS 服务器 1 | 2606:4700:4700::1111 |
IPv6 DNS 服务器 2 | 2606:4700:4700::1001 |
IPv6 DNS 服务器 3 | 留空 |
自动配置设置 | |
是否启动路由广播 | 启用 |
AX88U 上的 DHCPv6 客户端 odhcp6c
默认不会发送前缀提示。要强制 odhcp6c
发送前缀提示4,首先执行:
ps | sed -e 's/^.*\odhcp6c \(.*\)$/\1/;t;d' | head -n 1
获取系统运行 odhcp6c
的命令:
-df -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0
<duid>
和 <iaid>
与路由器的 MAC 地址有关。取决于 IPv6 设置,可能还有类似 -r23
、-r24
的参数,在下面的命令里也要加上这些参数。
对应替换 <duid>
和 <iaid>
,执行下面的两条命令:
# 结束现有的 odhcp6c
killall odhcp6c
# 重新运行 odhcp6c,前缀提示长度为 56
# 对应替换 -c 和 -FP 的参数 <duid> 和 <iaid>
odhcp6c -df -P56 -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0
这里 -P56
中的 56
即是要发送的前缀提示长度。
执行 ifconfig br0
验证 br0
已被分配到 /56
的 IPv6 全局地址:
admin@ax88u:/# ifconfig br0
br0 Link encap:Ethernet HWaddr A8:5E:45:00:FA:KE
inet addr:192.168.50.1 Bcast:192.168.50.255 Mask:255.255.255.0
inet6 addr: 2600:6c51:fake:n00b::1/56 Scope:Global
inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
RX packets:4241 errors:0 dropped:0 overruns:0 frame:0
TX packets:3714 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:829133 (809.7 KiB) TX bytes:1787334 (1.7 MiB)
如果没看到类似的输出,那下文的解决方案并不适合你,但你可以在 br1
上设置有状态 DHCPv6 (DHCPv6 stateful)。
提示:完整的脚本可以在 脚本与配置文件:/jffs/scripts/wan-event
中找到。
ip6tables
规则ip6tables
的规则类似上文 iptables
的规则,但是不需要设置 NAT。
INPUT
链:
# 允许来自 br1 的传入连接
ip6tables -I INPUT -i br1 -j ACCEPT # br0 的默认规则
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT
# 禁止 br1 访问路由器 Web UI 和 SSH
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP
FORWARD
链:
# 不转发来自 br1 的数据包
ip6tables -I FORWARD -i br1 -j DROP
# 但是允许 br1 内部的转发
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT
# 也允许 `br1` 的数据包转发至 `eth0` (WAN)
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT
# 也允许 `br0` 到 `br1` 的单向通信
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
提示:完整的脚本可以在 脚本与配置文件:/jffs/scripts/firewall-start
中找到。
默认情况下,br0
会使用整个 /56
的子网。如果要为 br1
划分子网,则需要改变 br0
的前缀长度。
首先,执行 ip -6 addr show br0 scope global
获取 ISP 分配的 IPv6 LAN 前缀:
admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
inet6 2600:6c51:fake:n00b::1/56 scope global
valid_lft forever preferred_lft forever
你拿到的前缀肯定跟这里的是不一样的,所以执行下面的命令时别忘了对应替换。
然后,为 br0
重新设置 /64
的前缀长度:
# 删除 /56 前缀,把 2600:6c51:fake:n00b::1 换成你的
ip -6 addr del 2600:6c51:fake:n00b::1/56 dev br0
# 设置 /64 前缀,把 2600:6c51:fake:n00b::1 换成你的
ip -6 addr add 2600:6c51:fake:n00b::1/64 dev br0
当然这里可以为 br0
设置更短的前缀长度,但是我觉得 /64
已经够用了。
再次执行 ip -6 addr show br0 scope global
,检查 br0
的新前缀:
admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
inet6 2600:6c51:fake:n00b::1/64 scope global
valid_lft forever preferred_lft forever
由于 dnsmasq
并不支持 DHCPv6 前缀代理(DHCPv6 Prefix Delegation, DHCPv6-PD)5 6 所需的 option 25 和 26 7,下面分两种情况讨论。
如果不接二级路由器的话,倒是可以用 dnsmasq
当 DHCPv6 服务器。
首先,为 br1
设置 br0
的邻接 /64
前缀:
# 为 br1 设置 br0 的邻接 /64 前缀
# 比如 br0 的前缀为 aaaa:bbbb:cccc:dddd::1/64,
# 则 br1 的前缀为 aaaa:bbbb:cccc:ddde::1/64.
# 把 2600:6c51:fake:n00c::1 换成你的
ip -6 addr add 2600:6c51:fake:n00c::1/64 dev br1
然后,为 br1
设置 DHCPv6 地址池和选项:
cat <<EOF >> /jffs/configs/dnsmasq.conf.add
# DHCPv6 路由广播间隔:10 秒, 路由器生命周期:600 秒
ra-param=br1,10,600
# DHCPv6 地址池:通过 br1 的前缀构造
# DHCPv6 前缀长度:64, 模式:无状态 DHCPv6
# DHCPv6 租约时间:600 秒 (10 分钟)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23):从路由器继承
dhcp-option=br1,option6:23,[::]
EOF
重启 dnsmasq 以应用设置:
service restart_dnsmasq
执行 tail /tmp/syslog.log -n 50 验证 dnsmasq.conf.add 已被正确加载:
admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... 省略多余输出 ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00c::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: IPv6 router advertisement enabled
# ... 省略多余输出 ...
如果需要在二级路由器上启用 DHCPv6-PD,则应该使用 6relayd
作为 DHCPv6-PD 服务器。
注意:如果使用 6relayd
,不要在 /jffs/configs/dnsmasq.conf.add
中添加 DHCPv6 相关的设置。
在 6relayd
8 的服务器模式下,它会通过 DHCPv6-PD 把第二个 /64
前缀分配给二级路由器。所以,只需要为 br1
分配一个 /63
的前缀。
这里使用的是 br0
的邻接 /63
前缀:
# 为 br1 设置 br0 的邻接 /63 前缀
# 把 2600:6c51:fake:n00c::1 换成你的
ip -6 addr add 2600:6c51:fake:n00c::1/63 dev br1
然后,执行 6relayd -v -d -S . br1
启动 DHCPv6-PD 服务器:
# 监听于 br1 上的 DHCPv6-PD 服务器
6relayd -v -d -S . br1
如果收到 DHCPv6 请求的话,系统日志 (cat /tmp/syslog.log | grep 6relayd
) 应该显示:
# admin@ax88u:/# cat /tmp/syslog.log | grep 6relayd
Apr 15 18:20:00 6relayd[1765]: Got DHCPv6 request
Apr 15 18:20:01 6relayd[1765]: Got DHCPv6 request
最后,在二级路由器上检查 IPv6 设置(系统记录 -> IPv6):
IPv6 Connection Type | Native with DHCP-PD |
---|---|
WAN IPv6 Address | 2600:6c51:fake:n00c::fake |
WAN IPv6 Gateway | fe80::aa5e:45ff:fe00:fake |
LAN IPv6 Address | 2600:6c51:fake:n00d::1/64 |
LAN IPv6 Link-Local Address | fe80::a62:66ff:fe01:fake/64 |
DHCP-PD | Enabled |
LAN IPv6 Prefix | 2600:6c51:fake:n00d::/64 |
DNS Address | 2600:6c51:fake:n00c::1 |
如果一切正常,LAN IPv6 Prefix 应该显示 2600:6c51:fake:n00d::/64
。
提示: 自动配置 6relayd
的脚本可以在 脚本与配置文件:/jffs/scripts/dhcpc-event 中找到。
把下文中的脚本与配置文件上传到路由器对应的目录里,即可在路由器每次重启时自动配置 VLAN。
注意 1:路由器 系统管理 -> 系统设置 中的 Enable JFFS custom scripts and configs 需要设为 是。
注意 2:上传完毕之后,运行 chmod +x /jffs/scripts/*
为这些脚本赋予可执行权限。
/jffs/scripts/services-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/000-services-start
# Physical port to interface map:
# eth0 WAN
# eth1 LAN 4
# eth2 LAN 3
# eth3 LAN 2
# eth4 LAN 1
# eth5 Bridge of LAN 5, LAN 6, LAN 7, LAN 8
# eth6 2.4 GHz Radio
# eth7 5 GHz Radio
# Delete those interfaces that we want to isolate from br0
logger -t "isolate_port" "services-start: deleting LAN 2 (eth3) from br0"
brctl delif br0 eth3
# Create a new bridge br1 for isolated interfaces
logger -t "isolate_port" "services-start: creating br1 with LAN 2 (eth3)"
brctl addbr br1
brctl stp br1 on # STP to prevent bridge loops
brctl addif br1 eth3
# Set up the IPv4 address for br1
# Here we set the subnet to be 192.168.150.0/24
# IPv6 link local address will be assigned automatically
logger -t "isolate_port" "services-start: setting up IPv4 address for br1"
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up
logger -t "isolate_port" "services-start: all done"
date >> /tmp/000-services-start
/jffs/scripts/firewall-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/000-firewall-start
logger -t "isolate_port" "firewall-start: applying INPUT rules for br1"
# Allow new incoming connections from br1
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT
ip6tables -I INPUT -i br1 -j ACCEPT # Same rule as br0 by default
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT
# Only forbid br1 access the web UI and SSH of the main router
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP
logger -t "isolate_port" "firewall-start: applying FORWARD rules for br1"
# Forbid packets from br1 to be forwarded to other interfaces
iptables -I FORWARD -i br1 -j DROP
ip6tables -I FORWARD -i br1 -j DROP
# But allow packet forwarding inside br1
iptables -I FORWARD -i br1 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT
# Allow packet forwarding between br1 and eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT
# Allow one-way traffic from br0 to br1
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
logger -t "isolate_port" "firewall-start: all done"
date >> /tmp/000-firewall-start
/jffs/scripts/nat-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/000-nat-start
logger -t "isolate_port" "nat-start: applying POSTROUTING rules for br1"
# NAT inside 192.168.150.0/24 on br1
iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
-o br1 -j MASQUERADE
logger -t "isolate_port" "nat-start: all done"
date >> /tmp/000-nat-start
/jffs/scripts/wan-event
该脚本仅用于 IPv6。
#!/bin/sh
PD_PREFIX=56 # Change to which works for your ISP
RETRY=10
# Only run for connected event (the old wan-start)
if [ ! "$2" = "connected" ]; then
exit 0
fi
# Make sure the script is indeed invoked
touch /tmp/000-wan-event-connected
# ipv6_service = dhcp6 means Connection Type: Native,
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
[ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
# Try to find odhcp6c
i=0;
while [ $i -lt $RETRY ]; do
# Get odhcp6c's pid. If there are multiple instances (unlikely),
# we use the smallest one.
PID=$(pidof odhcp6c | tr ' ' '\n' | head -n 1)
CMDLINE=/proc/$PID/cmdline
# Found odhcp6c?
if [ ! -z "$PID" ] && [ -f "$CMDLINE" ]; then
COMMAND="$(tr '\0' ' ' < /proc/$PID/cmdline)"
logger -t "isolate_port" "wan-event[connected]:" \
"found odhcp6c, PID: $PID, command: $COMMAND"
# The first 11 chars should be "odhcp6c -df"
PREFIX=$(echo $COMMAND | cut -c1-11)
if [ "$PREFIX" = "odhcp6c -df" ]; then
# There is a space between "odhcp6c -df" and remaining arguments.
# So we start from the 13rd char.
ARGS=$(echo $COMMAND | cut -c13-)
# Check if arguments start with -P$PD_PREFIX
ARG1=$(echo $ARGS | cut -c1-$(expr length "$PD_PREFIX" + 2))
if [ ! "$ARG1" = "-P$PD_PREFIX" ]; then
# Prefix length (-P$PD_PREFIX)
COMMAND="odhcp6c -df -P$PD_PREFIX $ARGS"
logger -t "isolate_port" "wan-event[connected]:" \
"re-run odhcp6c with prefix hint $PD_PREFIX: $COMMAND"
killall odhcp6c
eval $COMMAND
else
logger -t "isolate_port" "wan-event[connected]:" \
"odhcp6c already started with prefix hint $PD_PREFIX"
fi
else
logger -t "isolate_port" "wan-event[connected]:" \
"odhcp6c command prefix mismatch!" \
"found '$PREFIX', expects 'odhcp6c -df'"
fi
# We break from here once found the `odhcp6c` process
break
else
i=$(($i+1))
logger -t "isolate_port" "wan-event[connected]:" \
"odhcp6c not found ($i/$RETRY)"
# Hope we're lucky next time
sleep 1
fi
done
else
logger -t "isolate_port" "wan-event[connected]: DHCPv6 PD not enabled"
fi
logger -t "isolate_port" "wan-event[connected]: all done"
date >> /tmp/000-wan-event-connected
/jffs/scripts/dhcpc-event
该脚本仅用于 IPv6。
#!/bin/sh
link_local_ipv6() {
# Get IPv6 link local address from br0 (with the prefix length part)
echo $(ip -6 addr show br0 scope link |\
sed -e's/^.*inet6 \([^ ]*\).*$/\1/;t;d')
}
setup_br1() {
# ipv6_service = dhcp6 means Connection Type: Native,
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
[ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
# Wait for udhcpc (udhcpc.c: bound6 -> add_ip6_lanaddr).
# We need to the updated ipv6 prefix from nvram.
sleep 2
# Allocated IPv6 prefix from ISP
PREFIX=$(nvram get ipv6_prefix)
if [ -z "$PREFIX" ]; then
logger -t "isolate_port" "dhcpc-event: empty IPv6 prefix"
else
PD_PREFIX=$(nvram get ipv6_prefix_length)
if [ "$PD_PREFIX" -ge 63 ]; then
# We need at least 3 /64 subnets, /63 only gives 2 /64 subnet
logger -t "isolate_port" "dhcpc-event: IPv6 prefix" \
"(/$PD_PREFIX) is too long, at least /62 requried"
else
if [ "$PD_PREFIX" -le 48 ]; then
# Shorter than /49?
# br1 will on a:b:c:2::/63 (br0 is on a:b:c:0::/64)
BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):2::"
else
# /49 to /62?
# br1 will on a:b:c:dddf::/63 (br0 is on a:b:c:dddd::/64)
BR0_NET_ID=$(echo $PREFIX | cut -d':' -f4)
BR1_NET_ID=$(printf "%x" $((0x$BR0_NET_ID + 0x2)))
BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):${BR1_NET_ID}::"
fi
# Clean up br0
ip -6 route flush dev br0
ip -6 addr flush dev br0 scope global
# Assign br0 with the first /64 subnet (instead the /56 one)
# ipv6_rtr_addr is the default router's IPv6 address
logger -t "isolate_port" "dhcpc-event: set prefix" \
"$(nvram get ipv6_rtr_addr)/64 to br0"
ip -6 addr add "$(nvram get ipv6_rtr_addr)/64" dev br0
# Clean up br1 (note we also remove link local address)
ip -6 route flush dev br1
ip -6 addr flush dev br1
# Re-add link local address
# So route to fe80::/64 will be added back
# Note this is VERY important for 6relayd
# Otherwise 6relayd will throw network unreachable error
# Because route to fe80::/64 doesn't exist
ip -6 addr add $(link_local_ipv6) dev br1
# Assign br1 with a /63 subnet, so the cascading router
# will be on the first /64 subnet and it will also get
# the second /64 prefix via DHCPv6-PD by 6relayd
logger -t "isolate_port" "dhcpc-event: set prefix" \
"${BR1_PREFIX}1/63 to br1"
ip -6 addr add "${BR1_PREFIX}1/63" dev br1
# Re-run 6relayd
logger -t "isolate_port" "dhcpc-event: re-run 6relayd on br1"
killall 6relayd
# Automatic DHCPv6 server to delegate prefix on br1 in daemon
6relayd -v -d -S . br1
fi
fi
fi
}
teardown_br1() {
# ipv6_service = dhcp6 means Connection Type: Native,
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
[ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
logger -t "isolate_port" "dhcpc-event:" \
"flush IPv6 route and IPv6 address on br1"
# br0 will be handled by udhcpc,
# we only need to care about br1
# Note we also remove link local address
ip -6 route flush dev br1
ip -6 addr flush dev br1
# Re-add link local address
# So route to fe80::/64 will be added back
# Note this is VERY important for 6relayd
# Otherwise 6relayd will throw network unreachable error
# Because route to fe80::/64 doesn't exist
ip -6 addr add $(link_local_ipv6) dev br1
fi
}
# Make sure the script is indeed invoked
touch /tmp/000-dhcpc-event
# Adapted from odhcp6c-example-script.sh
(
flock 9
case "$1" in
bound)
teardown_br1
setup_br1
;;
informed|updated|rebound|ra-updated)
setup_br1
;;
stopped|unbound)
teardown_br1
;;
started)
teardown_br1
;;
esac
) 9>/tmp/odhcp6c.lock.br1
rm -f /tmp/odhcp6c.lock.br1
date >> /tmp/000-dhcpc-event
/jffs/configs/dnsmasq.conf.add
不需要 DHCPv6:
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
需要 DHCPv6,并且需要为二级路由器启用 DHCPv6-PD:
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
(你没看错,和不用 DHCPv6 是一样的)
需要 DHCPv6,但是不需要 DHCPv6-PD:
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
# DHCPv6 RA interval: 10s, router lifetime: 600s
ra-param=br1,10,600
# DHCPv6 range: whole subnet, constructing from br1's prefix
# DHCPv6 prefix length: 64, mode: Stateless DHCPv6
# DHCPv6 lease time: 600s (10 minutes)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23): inherit from the router
dhcp-option=br1,option6:23,[::]
slh, Reply: ASUS RT-AX88U Router. ↩
u128393, RT-AC86U VLAN 配置 - vlanctl 篇. ↩
Wikipedia, Stateless address autoconfiguration. ↩
buddyp, Reply: IPv6 with Prefix Delegation. ↩
Simon Kelley, dnsmasq: dhcp6-protocol.h. ↩
O. Troan, R. Droms, RFC 3633 - IPv6 Prefix Options for Dynamic Host Configuration Protocol (DHCP) version 6. ↩
Steven Barth, 6relayd - Embedded DHCPv6/RA Server & Relay. ↩
However, one issue with Asuswrt (either the stock firmware or Merlin) is no built-in GUI support on VLANs. I really miss the good old days with OpenWrt, as you can configure VLANs in GUI. But OpenWrt won’t support AX88U in a foreseeable future (due to Broadcom’s closed source driver1), CLI seems to be the only solution.
Pretty straightforward. There are 8 LAN ports on AX88U and I want to isolate LAN 2 from the remaining 7 ports.
LAN 1 and LAN 3 through 8 will be in VLAN 1
, while LAN 2 will be isolated into VLAN 2
(guest network).
The traffic between two VLANs should be forbidden. But ideally, I want to allow one way traffic from VLAN 1
to VLAN 2
. So I can still access the cascading router (in VLAN 2
) from my computer (in VLAN 1
), in case I need to
reconfigure the secondary router.
In addition, I prefer everything to be configured automatically. So for clients in VLAN 2
, the only thing is to
plug in the ethernet cable and get connected to the Internet.
robocfg
?The first problem is that robocfg
is no more provided on AX88U (Broadcom’s HND platform). An alternative to robocfg
on HND platform seems to be vlanctl
2. However, after several hours of searching, trying and error,
I believe vlanctl
can only create tagged VLAN, which unfortunately can’t satisfy my need.
But in case someone else has special demand on tagged VLAN, I recommend to read (in Chinese): 上海电信 TL-EP110 + RT-AC86U 实现观看 4K IPTV 无卡顿 (2019-10). This post describes how to set tagged VLANs up properly on AC86U (also a HND platform router).
Somehow I didn’t find it in the Internet. After playing with an ethernet cable on different ports, I figured out:
Interface | Physical Port |
---|---|
eth0 | WAN |
eth1 | LAN 4 |
eth2 | LAN 3 |
eth3 | LAN 2 |
eth4 | LAN 1 |
eth5 | Bridge of LAN 5 - 8 |
eth6 | 2.4 GHz Radio |
eth7 | 5 GHz Radio |
Note that eth5
seems to be a hardware bridge (BCM53134) of LAN 5 - 8. You may be able to ungroup it with ethctl
or ethswctl
, but I didn’t spend much time on them. My solution can only isolate LAN ports 1 - 4.
By default, eth1
~ eth7
are grouped in bridge br0
.
Note: My WAN Connection Type is Automatic IP. If you are using other types like PPPoE, you may need to replace eth0
to ppp0
in the below accordingly.
ebtables
If you don’t care about separating ports into different subnets, there actually exists an one-line solution:
# eth3 maps to LAN port 2 on AX88U
ebtables -A FORWARD -i eth3 -o br0 -j DROP
Here eth3
maps to LAN 2. The layer 2 firewall ebtables
will essentially block access between eth3
and remaining interfaces in br0
.
Since eth3
is still in the same subnet as br0
, there is no need to worry about iptables
or DHCP.
Everything will just be fine.
Note 1: If it doesn’t work, try ebtables -A FORWARD -i eth3 --logical-out br0 -j DROP
. Thanks to @MarianMaciag!
Note 2: If it still doesn’t work, try iptables
instead: iptables -A FORWARD -i eth3 -o br0 -j DROP
. The downside with iptables
is that only layer 3+ access is restricted.
Note 3: Also remember this only blocks access in one direction: eth3
to br0
. You need -i br0 -o eth3
(or --logical-in br0 -o eth3
) to block br0
to eth3
.
Note 4: If it again doesn’t work, go separate bridge approach below.
However, my goal is to associate two VLANs with two subnets. Say VLAN 1
with 192.168.50.0/24
and VLAN 2
with 192.168.150.0/24
.
br1
First, remove eth3
from br0
and create a new bridge br1
with eth3
:
# Delete those interfaces that we want to isolate from br0
brctl delif br0 eth3
# Create a new bridge br1 for our isolated interfaces
brctl addbr br1
brctl stp br1 on # STP to prevent bridge loops
brctl addif br1 eth3
Here we use bridge br1
for easier management. If you want to add isolate other LAN ports, you can simply
add the corresponding interface to br1
. All rules on br1
will automatically apply to that port.
Now run brctl show
to verify the settings for br1
:
admin@ax88u:/# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.a85e45fakeid yes eth1
eth2
eth4
eth5
eth6
eth7
br1 8000.a85e45fakeid yes eth3
By default, br0
is assigned with 192.168.50.1/24
on AX88U, so we don’t have to worry about it
Then, assign 192.168.150.1/24
to br1
and bring it up:
# Set up the IPv4 address for br1 and bring it up
# Here we set the subnet to be 192.168.150.0/24
# IPv6 link local address will be assigned automatically
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up
Finally, run ifconfig br1
to see if br1
is up:
admin@ax88u:/# ifconfig br1
br1 Link encap:Ethernet HWaddr A8:5E:45:00:FA:KE
inet addr:192.168.150.1 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
The IPv6 link local address is automatically generated by EUI-64.
Tip: Complete script can be found in Scripts & Configs: /jffs/scripts/services-start
.
iptables
RulesFirst, allow new incoming connections from br1
to the router:
# Allow new incoming connections from br1
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT
But forbid accessing web UI and SSH from br1
:
# Only forbid br1 access the web UI and SSH of the main router
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP
Then, drop all forwarding packets from br1
:
# Forbid packets from br1 to be forwarded to other interfaces
iptables -I FORWARD -i br1 -j DROP
But allow packet forwarding inside br1
:
# But allow packet forwarding inside br1
iptables -I FORWARD -i br1 -o br1 -j ACCEPT
… also allow packet forwarding between br1
and eth0
(WAN):
# Allow packet forwarding between br1 and eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT
… also allow one-way traffic from br0
to br1
:
# Allow one-way traffic from br0 to br1
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
Finally, set up NAT inside 192.168.150.0/24
on br1
:
iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
-o br1 -j MASQUERADE
Now run iptables -S INPUT
, iptables -S FORWARD
and iptables -t nat -S POSTROUTING
to verify that rules
have been added:
admin@ax88u:/# iptables -S INPUT
-P INPUT ACCEPT
-A INPUT -i br1 -p tcp -m tcp --dport 22 -j DROP
-A INPUT -i br1 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i br1 -m state --state NEW -j ACCEPT
# ... output omitted ...
-A INPUT -j DROP
admin@ax88u:/# iptables -S FORWARD
-P FORWARD DROP
-A FORWARD -i br1 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br0 -o br1 -j ACCEPT
-A FORWARD -i br1 -o eth0 -j ACCEPT
-A FORWARD -i br1 -o br1 -j ACCEPT
-A FORWARD -i br1 -j DROP
# ... output omitted ...
admin@ax88u:/# iptables -t nat -S POSTROUTING
-P POSTROUTING ACCEPT
# ... output omitted ...
-A POSTROUTING -s 192.168.50.0/24 -d 192.168.50.0/24 -o br0 -j MASQUERADE
-A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 -o br1 -j MASQUERADE
Tip: Complete scripts can be found below.
dnsmasq
for DHCPv4Create an additional configuration file dnsmasq.conf.add
in /jffs/configs/
:
touch /jffs/configs/dnsmasq.conf.add
Similar to br0
, set up DHCPv4 ranges and options on br1
:
cat <<EOF >> /jffs/configs/dnsmasq.conf.add
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
EOF
Restart dnsmasq
to apply the config:
service restart_dnsmasq
Now run tail /tmp/syslog.log -n 50
to see if dnsmasq.conf.add
is loaded:
admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... output omitted ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 14 20:49:22 dnsmasq-dhcp[15998]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
# ... output omitted ...
Even in the scenario of stateless DHCPv6, hosts’ IPv6 addresses are still generated via SLAAC3 according to RFC 3736:
A node that uses stateless DHCP must have obtained its IPv6 addresses through some other mechanism, typically stateless address autoconfiguration (or SLAAC).
Since SLAAC uses EUI-64 algorithm, the subnet prefix is required to be shorter than /64
. That means to do subnetting
with stateless DHCPv6, the IPv6 LAN prefix allocated from your ISP must be at least of /63
.
Some ISPs allow you send prefix length hint to get a prefix shorter
than /64
(fortunately Charter Spectrum does assign /56
). If you’re not sure about this, you can try the
following steps.
By the way, my AX88U’s IPv6 configuration is:
Name | Value |
---|---|
Basic Config | |
Connection type | Native |
DHCP-PD | Enable |
IPv6 LAN Setting | |
Auto Configuration Setting | Stateless |
IPv6 DNS Setting | |
Connect to DNS Server automatically | Disable |
IPv6 DNS Server 1 | 2606:4700:4700::1111 |
IPv6 DNS Server 2 | 2606:4700:4700::1001 |
IPv6 DNS Server 3 | <leave blank> |
Auto Configuration Setting | |
Enable Router Advertisement | Enable |
By default, the DHCPv6 client odhcp6c
on AX88U wouldn’t send prefix hint.
To force it do so4, first run
ps | sed -e 's/^.*\odhcp6c \(.*\)$/\1/;t;d' | head -n 1
to get the arguments of odhcp6c
:
-df -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0
<duid>
and <iaid>
are linked to your router’s MAC address. Depends on your IPv6 configuration, there may be
additional arguments like -r23
and -r24
, don’t forget to append them too.
So in the next two commands, replace them with the output you got:
# Kill existing odhcp6c.
killall odhcp6c
# Re-run odhcp6c with prefix hint 56.
# Replace `-c` and `-FP` arguments with the `ps` output you got.
odhcp6c -df -P56 -R -s /tmp/dhcp6c -N try -c <duid> -FP 0:<iaid> eth0
… and -P56
determines which prefix hint length to send.
Now you can run ifconfig br0
to verify if /56
subnet is assigned:
admin@ax88u:/# ifconfig br0
br0 Link encap:Ethernet HWaddr A8:5E:45:00:FA:KE
inet addr:192.168.50.1 Bcast:192.168.50.255 Mask:255.255.255.0
inet6 addr: 2600:6c51:fake:n00b::1/56 Scope:Global
inet6 addr: fe80::aa5e:45ff:fe00:fake/64 Scope:Link
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
RX packets:4241 errors:0 dropped:0 overruns:0 frame:0
TX packets:3714 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:829133 (809.7 KiB) TX bytes:1787334 (1.7 MiB)
If something similar is printed to your screen, Congratulations! You can proceed to the next section.
Otherwise, you may have to stop here and try to set up stateful DHCPv6 server on br1
, which I didn’t try.
Tip: Complete script for re-running odhcp6c
automatically after reboot can be found in Scripts & Configs: /jffs/scripts/wan-event
.
ip6tables
RulesSimiliar to IPv4 solution, some necessary ip6tables
rules have to be added. But no need to deal with NAT (It’s IPv6!).
For INPUT
chain:
# Allow new incoming connections from br1
ip6tables -I INPUT -i br1 -j ACCEPT # Same rule as br0 by default
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT
# Only forbid br1 access the web UI and SSH of the main router
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP
For FORWARD
chain:
# Forbid packets from br1 to be forwarded to other interfaces
ip6tables -I FORWARD -i br1 -j DROP
# But allow packet forwarding inside br1
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT
# Allow packet forwarding between br1 and eth0 (WAN)
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT
# Allow one-way traffic from br0 to br1
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
Tip: Complete script can be found in Scripts & Configs: /jffs/scripts/firewall-start
.
By default, br0
will take the whole /56
subnet. We need to change it to /64
for subnetting on br1
.
First, check what prefix br0
has gotten from ISP with ip -6 addr show br0 scope global
:
admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
inet6 2600:6c51:fake:n00b::1/56 scope global
valid_lft forever preferred_lft forever
You IPv6 address is different so remember to replace 2600:6c51:fake:n00b::1
with yours in following commands.
Then, reassign br0
with a /64
subnet:
# Remove /56 subnet. Replace 2600:6c51:fake:n00b::1 with yours.
ip -6 addr del 2600:6c51:fake:n00b::1/56 dev br0
# Assign /64 subnet. Replace 2600:6c51:fake:n00b::1 with yours.
ip -6 addr add 2600:6c51:fake:n00b::1/64 dev br0
Of course you can make a shorter subnet prefix for br0
, but I feel comfortable enough with /64
.
Next, run ip -6 addr show br0 scope global
again to check br0
’s new perfix:
admin@ax88u:/# ip -6 addr show br0 scope global
22: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500
inet6 2600:6c51:fake:n00b::1/64 scope global
valid_lft forever preferred_lft forever
The subnet for br1
is a little bit tricky, because dnsmasq
doesn’t support DHCPv6 Prefix Delegation (PD),
quoting from dnsmasq
’s author5:
You’re right, though this (dnsmasq does not support replying to IA_PD (prefix delegation) requests) may change in the future.
The code of dnsmasq
shipped in Merlin6 also proves no support of DHCPv6 prefix delegation
(option 25 and 267).
If you have no cascading routers or don’t care about PD, dnsmasq
can be used as DHCPv6 server.
First, assign br1
with the sibling /64
subnet of br0
:
# Assign the sibling /64 subnet of br0 to br1.
# For example, if br0 is on aaaa:bbbb:cccc:dddd::1/64,
# then br1 should on aaaa:bbbb:cccc:ddde::1/64.
# Replace 2600:6c51:fake:n00c::1 with yours.
ip -6 addr add 2600:6c51:fake:n00c::1/64 dev br1
Then, set up DHCPv6 ranges and options on br1
:
cat <<EOF >> /jffs/configs/dnsmasq.conf.add
# DHCPv6 RA interval: 10s, router lifetime: 600s
ra-param=br1,10,600
# DHCPv6 range: whole subnet, constructing from br1's prefix
# DHCPv6 prefix length: 64, mode: Stateless DHCPv6
# DHCPv6 lease time: 600s (10 minutes)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23): inherit from the router
dhcp-option=br1,option6:23,[::]
EOF
Restart dnsmasq
to apply the config:
service restart_dnsmasq
Now run tail /tmp/syslog.log -n 50
to see if dnsmasq.conf.add
is loaded:
admin@ax88u:/# tail /tmp/syslog.log -n 50
# ... output omitted ...
Apr 14 20:49:22 rc_service: service 15995:notify_rc restart_dnsmasq
Apr 14 20:49:22 dnsmasq[1149]: exiting on receipt of SIGTERM
Apr 14 20:49:22 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Apr 14 20:49:22 dnsmasq[15998]: started, version 2.81rc4-33-g7558f2b cachesize 1500
Apr 14 20:49:22 dnsmasq[15998]: asynchronous logging enabled, queue limit is 5 messages
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.150.2 -- 192.168.150.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCP, IP range 192.168.50.2 -- 192.168.50.254, lease time 1d
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00b::, constructed for br0
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: DHCPv6 stateless on 2600:6c51:fake:n00b::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: router advertisement on 2600:6c51:fake:n00c::, constructed for br1
Apr 15 18:09:01 dnsmasq-dhcp[18800]: IPv6 router advertisement enabled
# ... output omitted ...
Otherwise, if you want cascading router to get IPv6 prefix automatically through PD, 6relayd
is required as a minimalistic
DHCPv6 PD server.
Note: Don’t add DHCPv6 range and options to /jffs/configs/dnsmasq.conf.add
if you wish to use 6relayd
.
In the sever mode of 6realyd
8:
If there are non-local addresses assigned to the slave interface when a router solicitation is received, said prefixes are announced automatically for stateless autoconfiguration and also offered via stateful DHCPv6. If all prefixes are bigger than /64 all but the first /64 of these prefixes is offered via DHCPv6-PD to downstream routers.
… that is what we want. We’ll assign a /63
subnet for br1
, and 6relayd
will delegate a /64
subnet for the
cascading router.
First, assign a 63
subnet for br1
(here we use the sibling /63
subnet of br0
):
# Assign the sibling /63 subnet of br0 to br1.
# Use a IPv6 subnet tool for the `/63` subnet range on `br0`.
# Replace 2600:6c51:fake:n00c::1 with yours.
ip -6 addr add 2600:6c51:fake:n00c::1/63 dev br1
Then, run 6relayd -v -d -S . br1
for DHCPv6 PD server:
# Automatic DHCPv6 server to delegate prefix on br1 in daemon
6relayd -v -d -S . br1
If there are incoming DHCPv6 requests, system log (cat /tmp/syslog.log | grep 6relayd
) should show:
# admin@ax88u:/# cat /tmp/syslog.log | grep 6relayd
Apr 15 18:20:00 6relayd[1765]: Got DHCPv6 request
Apr 15 18:20:01 6relayd[1765]: Got DHCPv6 request
… which means 6relayd
is working properly.
Now check the System Log -> IPv6 page in the cascading router (here my old AC68P is the cascading one):
IPv6 Connection Type | Native with DHCP-PD |
---|---|
WAN IPv6 Address | 2600:6c51:fake:n00c::fake |
WAN IPv6 Gateway | fe80::aa5e:45ff:fe00:fake |
LAN IPv6 Address | 2600:6c51:fake:n00d::1/64 |
LAN IPv6 Link-Local Address | fe80::a62:66ff:fe01:fake/64 |
DHCP-PD | Enabled |
LAN IPv6 Prefix | 2600:6c51:fake:n00d::/64 |
DNS Address | 2600:6c51:fake:n00c::1 |
If everything is good, LAN IPv6 Prefix should show 2600:6c51:fake:n00d::/64
.
Tip: Automatic script for setting up DHCPv6 PD can be found in Scripts & Configs: /jffs/scripts/dhcpc-event.
In order to apply all settings after reboot, save the following scripts and configs into corresponding folders.
Note 1: Don’t forget to set Enable JFFS custom scripts and configs to Yes in Administration -> System.
Note 2: After uploading the following scripts, don’t forget to mark them as executable with chmod +x /jffs/scripts/*
.
/jffs/scripts/services-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/000-services-start
# Physical port to interface map:
# eth0 WAN
# eth1 LAN 4
# eth2 LAN 3
# eth3 LAN 2
# eth4 LAN 1
# eth5 Bridge of LAN 5, LAN 6, LAN 7, LAN 8
# eth6 2.4 GHz Radio
# eth7 5 GHz Radio
# Delete those interfaces that we want to isolate from br0
logger -t "isolate_port" "services-start: deleting LAN 2 (eth3) from br0"
brctl delif br0 eth3
# Create a new bridge br1 for isolated interfaces
logger -t "isolate_port" "services-start: creating br1 with LAN 2 (eth3)"
brctl addbr br1
brctl stp br1 on # STP to prevent bridge loops
brctl addif br1 eth3
# Set up the IPv4 address for br1
# Here we set the subnet to be 192.168.150.0/24
# IPv6 link local address will be assigned automatically
logger -t "isolate_port" "services-start: setting up IPv4 address for br1"
ifconfig br1 192.168.150.1 netmask 255.255.255.0
ifconfig br1 allmulti up
logger -t "isolate_port" "services-start: all done"
date >> /tmp/000-services-start
/jffs/scripts/firewall-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/000-firewall-start
logger -t "isolate_port" "firewall-start: applying INPUT rules for br1"
# Allow new incoming connections from br1
iptables -I INPUT -i br1 -m state --state NEW -j ACCEPT
ip6tables -I INPUT -i br1 -j ACCEPT # Same rule as br0 by default
ip6tables -I INPUT -i br1 -m state --state NEW -j ACCEPT
# Only forbid br1 access the web UI and SSH of the main router
iptables -I INPUT -i br1 -p tcp --dport 80 -j DROP
iptables -I INPUT -i br1 -p tcp --dport 22 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 80 -j DROP
ip6tables -I INPUT -i br1 -p tcp --dport 22 -j DROP
logger -t "isolate_port" "firewall-start: applying FORWARD rules for br1"
# Forbid packets from br1 to be forwarded to other interfaces
iptables -I FORWARD -i br1 -j DROP
ip6tables -I FORWARD -i br1 -j DROP
# But allow packet forwarding inside br1
iptables -I FORWARD -i br1 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br1 -j ACCEPT
# Allow packet forwarding between br1 and eth0 (WAN)
iptables -I FORWARD -i br1 -o eth0 -j ACCEPT
ip6tables -I FORWARD -i br1 -o eth0 -j ACCEPT
# Allow one-way traffic from br0 to br1
iptables -I FORWARD -i br0 -o br1 -j ACCEPT
iptables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
ip6tables -I FORWARD -i br0 -o br1 -j ACCEPT
ip6tables -I FORWARD -i br1 -o br0 -m state \
--state RELATED,ESTABLISHED -j ACCEPT
logger -t "isolate_port" "firewall-start: all done"
date >> /tmp/000-firewall-start
/jffs/scripts/nat-start
#!/bin/sh
# Make sure the script is indeed invoked
touch /tmp/000-nat-start
logger -t "isolate_port" "nat-start: applying POSTROUTING rules for br1"
# NAT inside 192.168.150.0/24 on br1
iptables -t nat -A POSTROUTING -s 192.168.150.0/24 -d 192.168.150.0/24 \
-o br1 -j MASQUERADE
logger -t "isolate_port" "nat-start: all done"
date >> /tmp/000-nat-start
/jffs/scripts/wan-event
This script is only for IPv6.
#!/bin/sh
PD_PREFIX=56 # Change to which works for your ISP
RETRY=10
# Only run for connected event (the old wan-start)
if [ ! "$2" = "connected" ]; then
exit 0
fi
# Make sure the script is indeed invoked
touch /tmp/000-wan-event-connected
# ipv6_service = dhcp6 means Connection Type: Native,
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
[ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
# Try to find odhcp6c
i=0;
while [ $i -lt $RETRY ]; do
# Get odhcp6c's pid. If there are multiple instances (unlikely),
# we use the smallest one.
PID=$(pidof odhcp6c | tr ' ' '\n' | head -n 1)
CMDLINE=/proc/$PID/cmdline
# Found odhcp6c?
if [ ! -z "$PID" ] && [ -f "$CMDLINE" ]; then
COMMAND="$(tr '\0' ' ' < /proc/$PID/cmdline)"
logger -t "isolate_port" "wan-event[connected]:" \
"found odhcp6c, PID: $PID, command: $COMMAND"
# The first 11 chars should be "odhcp6c -df"
PREFIX=$(echo $COMMAND | cut -c1-11)
if [ "$PREFIX" = "odhcp6c -df" ]; then
# There is a space between "odhcp6c -df" and remaining arguments.
# So we start from the 13rd char.
ARGS=$(echo $COMMAND | cut -c13-)
# Check if arguments start with -P$PD_PREFIX
ARG1=$(echo $ARGS | cut -c1-$(expr length "$PD_PREFIX" + 2))
if [ ! "$ARG1" = "-P$PD_PREFIX" ]; then
# Prefix length (-P$PD_PREFIX)
COMMAND="odhcp6c -df -P$PD_PREFIX $ARGS"
logger -t "isolate_port" "wan-event[connected]:" \
"re-run odhcp6c with prefix hint $PD_PREFIX: $COMMAND"
killall odhcp6c
eval $COMMAND
else
logger -t "isolate_port" "wan-event[connected]:" \
"odhcp6c already started with prefix hint $PD_PREFIX"
fi
else
logger -t "isolate_port" "wan-event[connected]:" \
"odhcp6c command prefix mismatch!" \
"found '$PREFIX', expects 'odhcp6c -df'"
fi
# We break from here once found the `odhcp6c` process
break
else
i=$(($i+1))
logger -t "isolate_port" "wan-event[connected]:" \
"odhcp6c not found ($i/$RETRY)"
# Hope we're lucky next time
sleep 1
fi
done
else
logger -t "isolate_port" "wan-event[connected]: DHCPv6 PD not enabled"
fi
logger -t "isolate_port" "wan-event[connected]: all done"
date >> /tmp/000-wan-event-connected
/jffs/scripts/dhcpc-event
This script is only for IPv6.
#!/bin/sh
link_local_ipv6() {
# Get IPv6 link local address from br0 (with the prefix length part)
echo $(ip -6 addr show br0 scope link |\
sed -e's/^.*inet6 \([^ ]*\).*$/\1/;t;d')
}
setup_br1() {
# ipv6_service = dhcp6 means Connection Type: Native,
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
[ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
# Wait for udhcpc (udhcpc.c: bound6 -> add_ip6_lanaddr).
# We need to the updated ipv6 prefix from nvram.
sleep 2
# Allocated IPv6 prefix from ISP
PREFIX=$(nvram get ipv6_prefix)
if [ -z "$PREFIX" ]; then
logger -t "isolate_port" "dhcpc-event: empty IPv6 prefix"
else
PD_PREFIX=$(nvram get ipv6_prefix_length)
if [ "$PD_PREFIX" -ge 63 ]; then
# We need at least 3 /64 subnets, /63 only gives 2 /64 subnet
logger -t "isolate_port" "dhcpc-event: IPv6 prefix" \
"(/$PD_PREFIX) is too long, at least /62 requried"
else
if [ "$PD_PREFIX" -le 48 ]; then
# Shorter than /49?
# br1 will on a:b:c:2::/63 (br0 is on a:b:c:0::/64)
BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):2::"
else
# /49 to /62?
# br1 will on a:b:c:dddf::/63 (br0 is on a:b:c:dddd::/64)
BR0_NET_ID=$(echo $PREFIX | cut -d':' -f4)
BR1_NET_ID=$(printf "%x" $((0x$BR0_NET_ID + 0x2)))
BR1_PREFIX="$(echo $PREFIX | cut -d':' -f1-3):${BR1_NET_ID}::"
fi
# Clean up br0
ip -6 route flush dev br0
ip -6 addr flush dev br0 scope global
# Assign br0 with the first /64 subnet (instead the /56 one)
# ipv6_rtr_addr is the default router's IPv6 address
logger -t "isolate_port" "dhcpc-event: set prefix" \
"$(nvram get ipv6_rtr_addr)/64 to br0"
ip -6 addr add "$(nvram get ipv6_rtr_addr)/64" dev br0
# Clean up br1 (note we also remove link local address)
ip -6 route flush dev br1
ip -6 addr flush dev br1
# Re-add link local address
# So route to fe80::/64 will be added back
# Note this is VERY important for 6relayd
# Otherwise 6relayd will throw network unreachable error
# Because route to fe80::/64 doesn't exist
ip -6 addr add $(link_local_ipv6) dev br1
# Assign br1 with a /63 subnet, so the cascading router
# will be on the first /64 subnet and it will also get
# the second /64 prefix via DHCPv6-PD by 6relayd
logger -t "isolate_port" "dhcpc-event: set prefix" \
"${BR1_PREFIX}1/63 to br1"
ip -6 addr add "${BR1_PREFIX}1/63" dev br1
# Re-run 6relayd
logger -t "isolate_port" "dhcpc-event: re-run 6relayd on br1"
killall 6relayd
# Automatic DHCPv6 server to delegate prefix on br1 in daemon
6relayd -v -d -S . br1
fi
fi
fi
}
teardown_br1() {
# ipv6_service = dhcp6 means Connection Type: Native,
# also we require DHCP PD to be enabled
if [ "$(nvram get ipv6_service)" = "dhcp6" ] && \
[ "$(nvram get ipv6_dhcp_pd)" = "1" ]; then
logger -t "isolate_port" "dhcpc-event:" \
"flush IPv6 route and IPv6 address on br1"
# br0 will be handled by udhcpc,
# we only need to care about br1
# Note we also remove link local address
ip -6 route flush dev br1
ip -6 addr flush dev br1
# Re-add link local address
# So route to fe80::/64 will be added back
# Note this is VERY important for 6relayd
# Otherwise 6relayd will throw network unreachable error
# Because route to fe80::/64 doesn't exist
ip -6 addr add $(link_local_ipv6) dev br1
fi
}
# Make sure the script is indeed invoked
touch /tmp/000-dhcpc-event
# Adapted from odhcp6c-example-script.sh
(
flock 9
case "$1" in
bound)
teardown_br1
setup_br1
;;
informed|updated|rebound|ra-updated)
setup_br1
;;
stopped|unbound)
teardown_br1
;;
started)
teardown_br1
;;
esac
) 9>/tmp/odhcp6c.lock.br1
rm -f /tmp/odhcp6c.lock.br1
date >> /tmp/000-dhcpc-event
/jffs/configs/dnsmasq.conf.add
If you need DHCPv6-PD (with 6relayd
) for cascading router, or if you don’t care about IPv6:
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
Otherwise, DHCPv6 is done by dnsmasq on br1
:
interface=br1
# DHCPv4 range: 192.168.150.2 - 192.168.150.254, netmask: 255.255.255.0
# DHCPv4 lease time: 86400s (1 day)
dhcp-range=br1,192.168.150.2,192.168.150.254,255.255.255.0,86400s
# DHCPv4 router (option 3): 192.168.150.1
dhcp-option=br1,3,192.168.150.1
# DHCPv6 RA interval: 10s, router lifetime: 600s
ra-param=br1,10,600
# DHCPv6 range: whole subnet, constructing from br1's prefix
# DHCPv6 prefix length: 64, mode: Stateless DHCPv6
# DHCPv6 lease time: 600s (10 minutes)
dhcp-range=br1,::,constructor:br1,ra-stateless,64,600
# DHCPv6 DNS (option 23): inherit from the router
dhcp-option=br1,option6:23,[::]
slh, Reply: ASUS RT-AX88U Router. ↩
u128393, RT-AC86U VLAN 配置 - vlanctl 篇. ↩
Wikipedia, Stateless address autoconfiguration. ↩
buddyp, Reply: IPv6 with Prefix Delegation. ↩
Simon Kelley, dnsmasq: dhcp6-protocol.h. ↩
O. Troan, R. Droms, RFC 3633 - IPv6 Prefix Options for Dynamic Host Configuration Protocol (DHCP) version 6. ↩
Steven Barth, 6relayd - Embedded DHCPv6/RA Server & Relay. ↩
Hope the following Java code snippet may give you the idea of this post ;)
System.out.print(
IntStream.of(1214606444, 1865162839, 1869769828, 555753482)
.mapToObj(n -> IntStream.iterate(24, i -> i >= 0, i -> i - 8)
.boxed()
.map(i -> (n & 255 << i) >> i)
.collect(() -> ByteBuffer.allocate(4),
(r, e) -> r.put(e.byteValue()),
ByteBuffer::put))
.map(b -> new String(b.array(), StandardCharsets.UTF_8))
.collect(Collectors.joining())
);
Tip: You need JDK 9 or higher to compile the code above, or if you were lazy like me :P, click here to see the output.
]]>