~lanodan/utils-std#5: 
base64/cat/…: pasting only takes 4096 bytes per line, redirection works okay

% base64 -w0 file_over_4KB > file_over_4KB.b64
% <file_over_4KB.b64 base64 -d > file_over_4KB.dec
% wc -c file_over_4KB.dec
3071 file_over_4KB.dec
% bc -l
3071/3*4
4094.66666666666666666664
Status
RESOLVED NOT_OUR_BUG
Submitter
~lanodan
Assigned to
No-one
Submitted
a month ago
Updated
a month ago
Labels
No labels applied.

~lanodan a month ago*

Seems to be related to whether the base64 data is pasted in or redirected in.

Reproduced in:

  • foot+tmux with wayland buffer
  • foot+tmux with tmux own buffer
  • pure foot
  • pure SHELL=/bin/dash foot
$ head -c 8K /dev/urandom >|random.raw
$ base64 -w0 random.raw >|random.b64
$ strace -o base64_decode_redir.strace base64 -d < random.b64 >|random.dec
$ strace -o base64_decode_tty_in.strace base64 -d >|random.dec
[paste base64 data in full]
[press newline; press ^D]
$ wc -c random.*
10924 random.b64
3071 random.dec
8192 random.raw
22187 total
$ pat base64_decode_tty_in.strace base64_decode_redir.strace
###     2 Files ###
###     File 1 << base64_decode_tty_in.strace >>        ###
execve("/usr/bin/base64", ["base64", "-d"], 0x7fff3da42f68 /* 111 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x704909a74898) = 0
set_tid_address(0x704909a74840)         = 20144
read(0, "mY8DxuAsTnTFVXl1BgLrgyKRTD2+qL+s"..., 1024) = 1024
ioctl(1, TIOCGWINSZ, 0x7ffd034af6c0)    = -1 ENOTTY (Not a tty)
writev(1, [{iov_base="\231\217\3\306\340,Nt\305Uyu\6\2\353\203\"\221L=\276\250\277\254\314vt\245{\332\24\331"..., iov_len=115}, {iov_base="\n", iov_len=1}], 2) = 116
read(0, "OtmmG7yvkGASo2iBWxpxFurHYKB2ULub"..., 1024) = 1024
writev(1, [{iov_base="\300k\306\21\244 \17\3008\214\224;)\303\2#\233s\16WKs\277\36\221\366R\r\217\340\242\21"..., iov_len=1024}, {iov_base="\33", iov_len=1}], 2) = 1025
read(0, "N61tgz5kA3zDoBgb2BZKMY8sDqUZft8h"..., 1024) = 1024
writev(1, [{iov_base="\362:\361\n\6\314cO\324\273\22qD\234w\2256\330\226\202M\233X\332\366/6+\fZ\325f"..., iov_len=1024}, {iov_base="\317", iov_len=1}], 2) = 1025
read(0, "JOQT3AnPqLj2KQciOh6O06IGymzFxkre"..., 1024) = 1024
read(0, "\n", 1024)                     = 1
read(0, "", 1024)                       = 0
writev(1, [{iov_base=".\310\212\260\333\325`\265S\20\336\243\374\345aDL\225\234\275\261\206&\32\374\234\336\205\205\20lp"..., iov_len=905}, {iov_base=NULL, iov_len=0}], 2) = 905
close(0)                                = 0
close(1)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

###     File 2 << base64_decode_redir.strace >> ###
execve("/usr/bin/base64", ["base64", "-d"], 0x7ffe718fcec8 /* 111 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x7557bcdd3898) = 0
set_tid_address(0x7557bcdd3840)         = 19858
read(0, "mY8DxuAsTnTFVXl1BgLrgyKRTD2+qL+s"..., 1024) = 1024
ioctl(1, TIOCGWINSZ, 0x7fff6190a6b0)    = -1 ENOTTY (Not a tty)
writev(1, [{iov_base="\231\217\3\306\340,Nt\305Uyu\6\2\353\203\"\221L=\276\250\277\254\314vt\245{\332\24\331"..., iov_len=115}, {iov_base="\n", iov_len=1}], 2) = 116
read(0, "OtmmG7yvkGASo2iBWxpxFurHYKB2ULub"..., 1024) = 1024
writev(1, [{iov_base="\300k\306\21\244 \17\3008\214\224;)\303\2#\233s\16WKs\277\36\221\366R\r\217\340\242\21"..., iov_len=1024}, {iov_base="\33", iov_len=1}], 2) = 1025
read(0, "N61tgz5kA3zDoBgb2BZKMY8sDqUZft8h"..., 1024) = 1024
writev(1, [{iov_base="\362:\361\n\6\314cO\324\273\22qD\234w\2256\330\226\202M\233X\332\366/6+\fZ\325f"..., iov_len=1024}, {iov_base="\317", iov_len=1}], 2) = 1025
read(0, "JOQT3AnPqLj2KQciOh6O06IGymzFxkre"..., 1024) = 1024
read(0, "Nzp1XXuzDU/+fLB6kKTepiM75l2lb591"..., 1024) = 1024
writev(1, [{iov_base=".\310\212\260\333\325`\265S\20\336\243\374\345aDL\225\234\275\261\206&\32\374\234\336\205\205\20lp"..., iov_len=1024}, {iov_base="\252", iov_len=1}], 2) = 1025
read(0, "WnrYHwCh3JFfTKGeKr6cRxQESwUH+b3l"..., 1024) = 1024
writev(1, [{iov_base="\206\246\333\322\257g!+6.\332K\237G\217\16\24\r\23\212\254\365\247\201\323\233N\\\365o\17\210"..., iov_len=1024}, {iov_base="\36", iov_len=1}], 2) = 1025
read(0, "2DB15tNVhwhWsqTcliKLsnBsFKV5rcn3"..., 1024) = 1024
writev(1, [{iov_base="\f\206\36V=\262\336\316$\261\325O\365k\245C\243\357)?\0\353\314\270R&ocSK~ "..., iov_len=1024}, {iov_base="\225", iov_len=1}], 2) = 1025
read(0, "YZ6YDWFOWWQreA/kdDJcwn4NJjYaM6Kb"..., 1024) = 1024
read(0, "SXjU9MnQFqmJn6hp8lHp54sbQIHo2K6B"..., 1024) = 1024
writev(1, [{iov_base="b\321fg\304\222L\26\333J\224j\203\214\211\301Gd\257\361\256\331\247\307K\310.\17\320h\256\273"..., iov_len=1024}, {iov_base="\265", iov_len=1}], 2) = 1025
read(0, "pKIfzhPkJMXidmN5rxaGhn5z+tAzLwlA"..., 1024) = 1024
writev(1, [{iov_base="\360\341\343`\372\36\3052W\0gJG\366\22\336>\254v\265\245t\20=>\363Fqt\\\206d"..., iov_len=1024}, {iov_base="\23", iov_len=1}], 2) = 1025
read(0, "+b8YeOb5DTUePjO2wOkQ09OST8es2OtA"..., 1024) = 684
read(0, "", 1024)                       = 0
close(0)                                = 0
writev(1, [{iov_base="X\267\206\353H\263\243?\215\241:\17\257\231W\250\257\321\264\230\2374J\357>LJC{\325\377\236"..., iov_len=901}, {iov_base=NULL, iov_len=0}], 2) = 901
close(1)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++
$ wl-copy < random.b64
$ wl-paste | base64 -d >|random.dec
$ wc -c random.*
10924 random.b64
8192 random.dec
8192 random.raw
27308 total
$ cmp -l random.dec random.raw
$ { wl-paste ; echo ; } | base64 -d >|random.dec
$ wc -c random.*
10924 random.b64
8192 random.dec
8192 random.raw
27308 total
$ { wl-paste ; printf $'\cD\n' ; } | base64 -d >|random.dec
base64: error: Invalid character '' (not EOF)
$ cat | base64 -d >|random.dec
[paste base64 data in full]
[press newline; press ^D]
$ wc -c random.*
10924 random.b64
3071 random.dec
8192 random.raw
22187 total
$ base64 random.dec | wc -c
4150
$ $ wc -c random.b64
10924 random.b64
$ wl-paste | wc -c
10925
$ cat | wc -c
[paste base64 data in full]
[press newline; press ^D]
4096
$

~lanodan REPORTED NOT_OUR_BUG a month ago*

Looking at tcsetattr(3) § Canonical and noncanonical mode:

In canonical mode:
[…]
The maximum line length is 4096 chars (including the terminating newline character); lines longer than 4096 chars are truncated.
[…]
In noncanonical mode input is available immediately (without the user having to type a line-delimiter character), no input processing is performed, and line editing is disabled. The read buffer will only accept 4095 chars;

So this is effectively an OS issue, meaning that either you need to use base -w4094 / base -w $(tput cols) instead of base -w0, or do something like wl-paste > file.base64.

Register here or Log in to comment, or comment via email.