Personal projects go here
CTO of https://scie.nz
This is pretty much there now. At the moment it's just refreshing all files at the same time rather than having a separate timer for local files, but to be honest local file support is probably only going to mean
/etc/hostsanyway so I don't think there's much reason to complicate the refresh logic further.
I also implemented tidy teardown of the process upon sigkill/sigint(ctrl+c), so if we're in the middle of a filter update it will wait until the update has concluded before actually exiting.
So at this point there really isn't much left to do here. Additional work may be needed if the list of filters becomes dynamic at runtime, but we can deal with that if/when it becomes an issue.
Going with including an Additional TXT resource. For example:
$ dig @127.0.0.1 -p 5353 test-blocked.kapiti.io ; <<>> DiG 9.16.15 <<>> @127.0.0.1 -p 5353 test-blocked.kapiti.io ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 49683 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 2 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: 4c0bdcd19d757980 (echoed) ;; QUESTION SECTION: ;test-blocked.kapiti.io. IN A ;; AUTHORITY SECTION: test-blocked.kapiti.io. 300 IN SOA test-blocked.kapiti.io. kapiti.test-blocked.kapiti.io. 42069 300 3600 259200 300 ;; ADDITIONAL SECTION: test-blocked.kapiti.io. 300 IN TXT "hardcoded" # or could be e.g. "http://example.com/filter.txt:1234"
Need to test this with a couple clients and make sure that they aren't unhappy about the extra TXT record. If there are issues then this might just be a WONTDO.
An alternate solution may be to just abandon rustls entirely and use
async-native-tlsinstead. We were previously getting errors like
http2 error: protocol error: frame with invalid sizedue to lack of support for ALPN when using it in the hyper-based filter download client.
That support may be fixed as of rust-native-tls 0.2.7: https://github.com/sfackler/rust-native-tls/pull/194
However async-native-tls is still on rust-native-tls 0.2.3: https://github.com/async-email/async-native-tls/blob/master/Cargo.toml#L18
There is now a UDP client/UDP upstream test. In terms of seeing what overheads the service itself has, this is probably the most useful one to exercise. Could conceivably add another for TCP or HTTP upstream later, but that would likely just be bottlenecked on the client socket.
Assuming the numbers are right, it looks like we're getting about 12 kqps as of fbaed2a2, so now we've got a baseline before trying out some refactoring. Might also be able to hook up pprof or something while the benchmark is running as well.
At this point the code is in pretty good shape. Could potentially avoid copies in a few additional places, but its really in the territory where e.g. mutex overhead for reusing a buffer across contexts would outweigh any benefit from avoiding the malloc.
In other words - this is deep in early optimization territory, lets wait until a flame chart says its a problem.