Comment by ~nickbp on ~nickbp/kapiti
This is pretty much there now. At the moment it's just refreshing all files at the same time rather than having a separate timer for local files, but to be honest local file support is probably only going to mean
/etc/hostsanyway so I don't think there's much reason to complicate the refresh logic further.I also implemented tidy teardown of the process upon sigkill/sigint(ctrl+c), so if we're in the middle of a filter update it will wait until the update has concluded before actually exiting.
So at this point there really isn't much left to do here. Additional work may be needed if the list of filters becomes dynamic at runtime, but we can deal with that if/when it becomes an issue.
REPORTED RESOLVED FIXED
Comment by ~nickbp on ~nickbp/kapiti
Going with including an Additional TXT resource. For example:
$ dig @127.0.0.1 -p 5353 test-blocked.kapiti.io ; <<>> DiG 9.16.15 <<>> @127.0.0.1 -p 5353 test-blocked.kapiti.io ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 49683 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 2 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: 4c0bdcd19d757980 (echoed) ;; QUESTION SECTION: ;test-blocked.kapiti.io. IN A ;; AUTHORITY SECTION: test-blocked.kapiti.io. 300 IN SOA test-blocked.kapiti.io. kapiti.test-blocked.kapiti.io. 42069 300 3600 259200 300 ;; ADDITIONAL SECTION: test-blocked.kapiti.io. 300 IN TXT "hardcoded" # or could be e.g. "http://example.com/filter.txt:1234"Need to test this with a couple clients and make sure that they aren't unhappy about the extra TXT record. If there are issues then this might just be a WONTDO.
REPORTED RESOLVED FIXED
Comment by ~nickbp on ~nickbp/kapiti
An alternate solution may be to just abandon rustls entirely and use
async-native-tlsinstead. We were previously getting errors likehttp2 error: protocol error: frame with invalid sizedue to lack of support for ALPN when using it in the hyper-based filter download client.That support may be fixed as of rust-native-tls 0.2.7: https://github.com/sfackler/rust-native-tls/pull/194
However async-native-tls is still on rust-native-tls 0.2.3: https://github.com/async-email/async-native-tls/blob/master/Cargo.toml#L18
Security added by ~nickbp on ~nickbp/kapiti
Low Priority added by ~nickbp on ~nickbp/kapiti
Comment by ~nickbp on ~nickbp/kapiti
There is now a UDP client/UDP upstream test. In terms of seeing what overheads the service itself has, this is probably the most useful one to exercise. Could conceivably add another for TCP or HTTP upstream later, but that would likely just be bottlenecked on the client socket.
Assuming the numbers are right, it looks like we're getting about 12 kqps as of fbaed2a2, so now we've got a baseline before trying out some refactoring. Might also be able to hook up pprof or something while the benchmark is running as well.
REPORTED RESOLVED FIXED
Comment by ~nickbp on ~nickbp/kapiti
At this point the code is in pretty good shape. Could potentially avoid copies in a few additional places, but its really in the territory where e.g. mutex overhead for reusing a buffer across contexts would outweigh any benefit from avoiding the malloc.
In other words - this is deep in early optimization territory, lets wait until a flame chart says its a problem.
REPORTED RESOLVED FIXED