Comment by ~bptato on ~bptato/chawan
Thanks, added.
REPORTED RESOLVED CLOSED
Comment by ~bptato on ~bptato/chawan
- Add keep-alive connections [...]
- CGI is 1-shot only, so needs a new protocol:
Another option is to daemonize the CGI process, and then connect from new processes that come from the same loader client (e.g. using a session cookie). I doubt it makes things much easier, but who knows.
- Remove libcurl dependency?
We're also getting blocked by some sites (well, Cloudflare) because of curl's header order. There is no way to adjust this (apart from using curl-impersonate), so curl must go.
- need an alternative inflate: could use stbi or miniz. I think miniz is faster, but needs benchmarking. (miniz license also changed from PD to MIT, but the SDL2 version is still PD...)
stbi is out of question, it runs once per IDAT (i.e. can't stream). miniz it is.
For miniz, it seems 1.16 beta was the last release that touched the decompressor, and it is still PD. MIT miniz only has two bugfixes in tinfl. (SDL is still on 1.15 for some reason.)
Better yet, 1.16 beta also has a separate tinfl.h version, which is all we need. It only supports deflate, but it's easy to add gzip.
I've also seen servers send brotli despite not advertising it in Accept-Encoding (<- why we added it to w3m). So libbrotli is needed for best compatibility (and probably zstd too).
We might also want to support layered compression, but I don't know if any servers use it. (Combined with Transfer-Encoding, it gets quite difficult...)
- mbedtls?
- looks like it's smaller than OpenSSL, but idk how well it behaves with OS certs
You have to manually look them up with OpenSSL too, so the question is moot.
Anyway, the options are:
OpenSSL is the most convenient, as we already wrap it for Gemini. But it's is twice as large as the browser itself.
mbedTLS is fairly small, and seems complete enough for web browsing. In practice, TLS fingerprinters (again, Cloudflare) may block it.
NSS is another option, with an acceptable size too. Firefox uses it, so it may be easier to convince TLS fingerprinters that we are a "real" browser. However, it is MPL2 licensed, and its documentation is so bad that even curl dropped it.
I think for now it's best to stick with OpenSSL. We can switch to NSS if we still get blocked after dropping curl, or mbedTLS otherwise.
Comment by ~bptato on ~bptato/chawan
Fixed, thanks for reporting.
REPORTED RESOLVED CLOSED
Comment by ~bptato on ~bptato/chawan
Added a note about the issue to the readme, and included cutenice's cross compiling instructions in build.md (thanks!)
REPORTED RESOLVED CLOSED
Comment by ~bptato on ~bptato/chawan
Done in aa2e19429.
In the end, I've decided to go with curl's cookies.txt format despite its limitations, because it's at least vaguely compatible with other programs.
Cookie jar separation is implemented by prepending "jarname@" to the domain, but only if the domain is not the same as the cookie jar's name:
# cookie jar: a.example a.example FALSE / TRUE 1710846333 muffins 1234 # cookie jar: b.a.example b.a.example FALSE / TRUE 1710846333 muffins 1234 # cookie jar: a.example (but domain remains b.a.example) a.example@b.a.example FALSE / TRUE 1710846333 muffins 1234So it's always backwards-compatible, and usually (but not always) forwards-compatible. In practice, most cookies are specified on the right domain, so this should rarely be an issue (and even then, it's easy to sed it out).
As a test, I've tried taking some cookies from Firefox using https://addons.mozilla.org/en-US/firefox/addon/export-cookies-txt/, and it seems to be working as expected after setting "cookie" and "share-cookie-jar" appropriately. (I recommend ticking "Prefix HttpOnly cookies" for better security.)
REPORTED RESOLVED CLOSED
Comment by ~bptato on ~bptato/chawan
REPORTED RESOLVED CLOSED
Comment by ~bptato on ~bptato/chawan
REPORTED RESOLVED CLOSED
Comment by ~bptato on ~bptato/chawan
Correction: CFLAGS, etc. should be at the beginning, like
CFLAGS=... make. (Otherwise you override all other flags.)
Comment by ~bptato on ~bptato/chawan
~cutenice wrote:
Your hint of nightly pointed me to the right direction! I had been using gcc 14.2.1 and Nim 2.0.8. Updating Nim to 2.3.1 got me a little further, but now I get an error saying
/usr/bin/ld i386:x86-64 architecture of input file `/home/cn/code/chawan/src/utils/../../lib/chaseccomp/chaseccomp.o' is incompatible with i386 outputThis is from my attempt on my beefier machine. I am pretty green on the gcc stuff, so I may have done something wrong here!
I'm fairly clueless about cross-compiling, but I think you'd want something like
make CFLAGS=-m32 LDFLAGS=-m32 FLAGS=--cpu:i386....which chaseccomp's Makefile doesn't respect either for some reason. Apply this first:
--- a/lib/chaseccomp/Makefile +++ b/lib/chaseccomp/Makefile @@ -6,7 +6,7 @@ chasc_defs_%.c: %.chasc common.chasc gen_defs chaseccomp.h ./gen_defs <$< >$@ chasc_defs_%: chasc_defs_%.c - $(CC) $< -o $@ + $(CC) $(CFLAGS) $< -o $@ $(LDFLAGS) %.chasc.expanded: chasc_defs_% ./$< >$@I haven't tried a later version of nim on the Eee PC yet (where nim is v2.0.4 and gcc is v14.1.1), but perhaps that's the way to go.
Probably it's the easier way. (But it might take a while :)
~bptato wote:
The planned design so far [...]
That looks great! Indeed, no breaking changes in the slightest while at the same time offering new possibilities.
neither Netscape (used by curl) nor w3m allow multiple cookie jars in the same file
You mean multiple cookie jars for the same URL? But how would one chose between those then?