~harmtemolder 7 months ago

Looks like you’re the first one with an MTP_DEVICE to try the plugin, so let’s see if we can add support for that. The thing is, I don’t have one of those, so I’ll have to rely on you for testing. Are you up for that?

~philips 7 months ago

I am happy to test it all out. I am a software engineer and comfortable with Python. However, I don’t have any experience debugging Calibre so I will need help with that. I am running a Xiaomi Inkpalm 5 ebook reader and plugging into a macOS device.

~harmtemolder 7 months ago

OK, great. I’ve enabled MTP_DEVICE so that you won’t get the error anymore: https://git.sr.ht/~harmtemolder/koreader-calibre-plugin/blob/main/releases/KOReader%20Sync%20v0.2.2-alpha.zip. Could you help my doing this:

1. Download that release
2. Add it to calibre by running this in your terminal: calibre-customize -a "KOReader Sync v0.2.2-alpha.zip"
3. Start calibre in debug mode with calibre-debug -g
4. Configure the KOReader plugin as described here
5. Connect your device
6. Run the sync by clicking the KOReader icon in your toolbar
7. Check the details of the message when it’s done if any/all books have been synced correctly
8. Check your (custom) columns for one of those books to see if their contents are what they should be
9. Check the output in your terminal for lines containing koreader to see what it did

If all checks are good, we’re good. If they’re not, we’ll dive in deeper.

~philips 7 months ago

Clicking the KOReader Sync I get "No metadata could be synced. See below for details."

The error message is repeated for all items in my Library but here is two snippets.

  {
    "result": "could not get sidecar contents",
    "book_uuid": null,
    "sidecar_path": "carroll, lewis - alice's adventures in wonderland.sdr/metadata.epub.lua"
  },

DEBUG:  137.7 koreader:action:KoreaderAction:get_paths:found 152 path(s) to sidecar Lua files
DEBUG:  137.7 koreader:action:KoreaderAction:get_sidecar:could not get carroll, lewis - alice's adventures in wonderland.sdr/metadata.epub.lua
DEBUG:  137.7 koreader:action:KoreaderAction:sync_to_calibre:skipping uuid = None

~philips 7 months ago

The first thing that pops out to me is that the paths are all lower case. Where as on my device the paths are like:

Books/Last, First - Title.sdr/metadata.epub.lua

~harmtemolder 7 months ago

You’re right, that lowercasing does seem strange. Not sure why the MTP driver (source) does that. I’ve changed the plugin to use a different property of book objects. Could you try again with https://git.sr.ht/~harmtemolder/koreader-calibre-plugin/blob/main/releases/KOReader%20Sync%20v0.2.2-alpha.zip ?

~philips 7 months ago

Well, something happened different this time. The debug log got up to 1Gigabyte and crashed my terminal.

Here is an excerpt:

MTP: Using cached metadata for Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.epub
DEBUG:   10.7 koreader:action:KoreaderAction:get_paths:found 152 path(s) to sidecar Lua files
DEBUG:   10.7 koreader:action:KoreaderAction:get_sidecar:could not decode PK^C^D^T^@^@^@^@^@�;�Roa�,^T^@^@^@^T^@^@^@^H^@^@^@mimetypeapplication/epub+zipP

~philips 7 months ago

So, it looks like it is able to find and open files on the MTP device now. But, it is opening the ebooks themselves not the sdr files?

~harmtemolder 7 months ago

Exactly. Not sure what the MTP_DEVICE driver reports as paths, so I’ve added three bits to the log:

• The path for each book
• The lpath of each book (what I originally used, but what seems to be lowercased on MTP_DEVICE)
• The path to the metadata file the plugin generates by replacing '\.(\w+)$' with '.sdr/metadata.\\1.lua'

Could you download and install it again and find Alice in Wonderland in all three of the printed lists for me?

~harmtemolder 7 months ago*

~philips 7 months ago

MTP: Using cached metadata for Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.epub
DEBUG:   23.0 koreader:action:KoreaderAction:get_paths:found these paths to books:
        mtp:::546:::Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.epub
MTP: Using cached metadata for Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.epub
DEBUG:   23.1 koreader:action:KoreaderAction:get_paths:found these lpaths to books:
        carroll, lewis - alice's adventures in wonderland.epub
MTP: Using cached metadata for Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.epub
DEBUG:   23.3 koreader:action:KoreaderAction:get_paths:generated 152 path(s) to sidecar Lua files:
        mtp:::546:::Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.sdr/metadata.epub.lua
DEBUG:   23.4 koreader:action:KoreaderAction:get_sidecar:could not decode PK^C^D^T^@^@^@^@^@�;�Roa�,^T^@

~harmtemolder 7 months ago

What does Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.sdr/metadata.epub.lua look like when you open it manually with a text editor?

~philips 7 months ago

-- we can read Lua syntax here!
return {
    ["smooth_scaling"] = false,
    ["doc_pages"] = 164,
    ["render_mode"] = 0,
    ["nightmode_images"] = true,
    ["copt_embedded_css"] = 1,
    ["copt_render_dpi"] = 96,
    ["copt_t_page_margin"] = 15,
    ["copt_b_page_margin"] = 15,
    ["copt_sync_t_b_page_margins"] = 0,
    ["copt_view_mode"] = 0,
    ["bookmarks_sorted"] = true,
    ["copt_smooth_scaling"] = 0,
    ["highlights_imported"] = true,
    ["gamma"] = 1,
    ["text_lang_embedded_langs"] = true,
    ["hyphenation"] = true,
    ["hyph_trust_soft_hyphens"] = false,
    ["hyph_soft_hyphens_only"] = false,
    ["hyph_force_algorithmic"] = false,
    ["floating_punctuation"] = 0,
    ["highlight"] = {},
    ["font_face"] = "Noto Serif",
    ["config_panel_index"] = 3,
    ["copt_status_line"] = 1,
    ["percent_finished"] = 0.10365853658537,
    ["gamma_index"] = 25,
    ["inverse_reading_order"] = false,
    ["readermenu_tab_index"] = 7,
    ["css"] = "./data/epub.css",
    ["page_overlap_style"] = "dim",
    ["rotation_mode"] = 0,
    ["render_dpi"] = 96,
    ["partial_md5_checksum"] = "baf759f0be2418a0e84e42cf35712fda",
    ["preferred_dictionaries"] = {},
    ["visible_pages"] = 1,
    ["copt_block_rendering_mode"] = 3,
    ["header_font_face"] = "Noto Sans",
    ["copt_font_size"] = 25.5,
    ["font_embolden"] = 0,
    ["copt_font_weight"] = 0,
    ["bookmarks"] = {},
    ["word_spacing"] = {
        [2] = 75,
        [1] = 95,
    },
    ["copt_word_spacing"] = {
        [2] = 75,
        [1] = 95,
    },
    ["copt_word_expansion"] = 0,
    ["copt_line_spacing"] = 100,
    ["copt_font_gamma"] = 25,
    ["word_expansion"] = 0,
    ["disable_fuzzy_search"] = false,
    ["cre_dom_version"] = 20200824,
    ["last_xpointer"] = "/body/DocFragment[4]/body/div/h2/text()[1].0",
    ["font_size"] = 25.5,
    ["toc_ticks_ignored_levels"] = {},
    ["font_hinting"] = 2,
    ["copt_visible_pages"] = 1,
    ["font_kerning"] = 3,
    ["highlight_disabled"] = false,
    ["text_lang"] = "en-US",
    ["copt_embedded_fonts"] = 1,
    ["copt_font_kerning"] = 3,
    ["doc_props"] = {
        ["authors"] = "Lewis Carroll",
        ["series"] = "",
        ["description"] = "",
        ["title"] = "Alice's Adventures in Wonderland",
        ["keywords"] = "Fantasy fiction\
Children's stories\
Imaginary places -- Juvenile fiction\
Alice (Fictitious character from Carroll) -- Juvenile fiction",
        ["language"] = "en",
    },
    ["copt_nightmode_images"] = 1,
    ["line_space_percent"] = 100,
    ["show_overlap_enable"] = false,
    ["embedded_css"] = true,
    ["copt_font_hinting"] = 2,
    ["copt_rotation_mode"] = 0,
    ["copt_h_page_margins"] = {
        [2] = 10,
        [1] = 10,
    },
    ["embedded_fonts"] = true,
    ["hide_nonlinear_flows"] = false,
    ["stats"] = {
        ["language"] = "en",
        ["series"] = "",
        ["authors"] = "Lewis Carroll",
        ["md5"] = "baf759f0be2418a0e84e42cf35712fda",
        ["title"] = "Alice's Adventures in Wonderland",
        ["notes"] = 0,
        ["pages"] = 164,
        ["highlights"] = 0,
    },
    ["highlight_drawer"] = "lighten",
}

~harmtemolder added device support 7 months ago

~harmtemolder assigned ~harmtemolder 7 months ago

~harmtemolder 7 months ago

OK, that’s exactly what it should look like. How weird. I’ll have to go find an MTP_DEVICE to test this for myself, hang on...

~harmtemolder 7 months ago

I’ve figured out what’s going on: The “546” in mtp:::546:::Internal shared storage/Carroll, Lewis - Alice's Adventures in Wonderland.epub is a file ID, so renaming the path to point to the metadata lua file doesn’t actually do anything. calibre will still ask for file 546, which is the actual epub.

I’m now digging through the MTP device’s driver.py, but it looks like calibre only builds an index (and thus IDs) of known ebook formats, hard-coded in src/calibre/ebooks/__init__.py. In other words, there doesn’t seem to be a way to get a lua file over MTP without changing the way calibre handles MTP devices.

I don’t see another option than to give up. Sorry.

~philips 7 months ago

Wow. that is unfortunate. Thank you for digging into it though! I really appreciate the effort.

~harmtemolder REPORTED WONT_FIX 6 months ago