Interactive annotation viewer/exporter for PocketBook e-readers, with additional tools.
We do:
We don’t:
Some highlights
PocketBook readers, with recent 6.x firmware.
Older firmware versions (5.x) are likely only partially supported - we are open to feedback.
Loading annotations directly from a device takes longer due to their slower storage mediums: the ‘local mirror’ feature improves this. See also performance improvements.
In general, performance is expected to be good to acceptable even on older hardware. Further optimizations are available under “Settings > Annotation Viewer”.
The data directory [C] may be synced across devices. The generic settings [A] are then configured locally, but this can be overridden using the CLI. However, these settings are few and easily reproduced.
Option (4) is the default situation. Any earlier condition takes precedence / overrides the default.
1. Portable mode (See below and "CLI Options")
2. CLI provided locations
3. GUI configured data location (settings dialog)
4. Default locations (Qt derived) if not provided by (2) and (3)Stores all settings and files in the application directory ‘<APP_DIR>’:
- avater.conf: <APPDIR>/avater.conf
- data: <APPDIR>/data/
- devices.conf: <APP_DIR>/data/devices.confShow available options.
Run application in portable mode.
Note: To convert a default ‘data’ directory for use with portable: copy/move it into the application directory. See also Settings and files.
Overrides the location of the generic avater.conf file.
Overrides the location of both the default/user-set data directory, and the therein stored devices.conf file.
Enable debug logging to stdout.
Note: CLI debug settings take precedence over preference debug settings (see “settings > advanced”).
Enable debug logging to the provided file. A text file will be created if necessary (write permission is not checked, and will fail silently). The log is accessible from the GUI help/debug menu.
Note: CLI debug settings take precedence over preference debug settings (see “settings > advanced”).
Scans system for new or removed devices. This includes:
Both USB devices, and local mirrors (0.9.8+) are removed if not found (removed from the program, not unmounted - for local mirrors no files are removed).
Removes all USB devices from the program - but does not unmount these from the OS.
If created, opens the ‘data’ directory using the OS’ file browser. See configuration files. Note: this directory is created when needed.
Manually check for a new program version. If an update is found, a prompt offers the choice of visiting the download page. Alternatively, you may enable/disable the automatic update check under settings.
Opens the settings dialog. Settings are undocumented, but hovering over labels/options will show ‘tooltip’ pop-ups with information.
Any found and supported annotation sources (i.e. PB profiles) are shown here. Selecting one will clear the viewer and will load any of that source its annotations.
Offers various links and the about window, but version information.
If either debug mode is enabled, the debug menu is shown.
For selected annotations, this outputs the ‘raw’ SQL data to the debug log. This can be used by the developers to solve certain problems.
Note: The export is currently hardcoded to use the local mirror when mirroring is enabled, else the device’s databases.
Accessible if debug messages are logged to a logfile. To change, see settings > advanced, or consult the CLI “–help” information.
Scans system for new or removed devices. See also Rescan Devices
Removes the device from the program. This does not unmount the device.
If the device has a ‘local db mirror’, the device entry is kept, is labeled as disconnected, and any device references are purged. Ejecting ‘local mirrors’ does nothing.
Select one of multiple devices or locally mirrored devices.
Switching between devices loads the annotations for the selected device (unless disabled under settings > Annotationsviewer). Previously loaded annotations are kept in memory until the device is removed from the program (for local DB mirrors this will not occur).
Connected devices are indicated by addition of an e-reader icon, and a “connected” text-label. Entries without these are so-called local mirrors.
You may add a custom label using “Device Tools > Set Devicelabel”. For example the model type.
Available tools may differ per device and vendor.
This tool copies system files, such as apps and fonts, to their appropriate folders on the reader memory (or card memory).
It’s mainly intended for (partially) restoring backups generated by the program.
- .acsm (adobe digital download file)
- .app
- .dic (dictionary)
- .pbi (dictionary/app)
- .ttf (font)
- .otf (font)
- .ttc (font)
- .bin (update file)Files stored in .zip files, such as backup archives, can be processed and -if supported- extracted by the program.
Note that marking such a file for deletion, will delete the parent archive, not the individual file.
Not implemented yet. ACSM downloading is a planned feature.
Creates a common .zip backup archive with selected device content.
Type of files to backup:
Databases: Copies the (e-book) metadata and annotation database(s). Databases are always archived, to prevent recovery problems at a later date.
Apps, Fonts, Dictionaries: No special remarks apply.
Other files (main/card): Back-up any other directories and files on the device, excluding a) the previously discussed types and b) the /system/ folder. The program will not remember these being checked to reduce device access.
Note: e-Reader transfer speeds average at about 1-2 minutes per GB
As e-Books and dictionary files tend to be already (maximally) compressed, these are ‘stored’ in the .zip archive, saving (processing) time.
Ideally the end-user (you) would tests back-ups, which is not practical. The program employs some precautions to ensure correct storage, and will (attempt to) warn on failure.
Note regarding data integrity: the .zip format’s internal checksums can (only) indicate data corruption after storage or transmission. Depending on the value of your data, consider a good backup strategy, and hardware/software integrity features (ECC, ZFS, etc.).
This tool checks both device and locally mirrored databases (DBs) for errors. It utilizes SQLite’s (‘official’) “integrity_check” feature.
Should any errors be indicated, click on “Show Details” for more information. Warnings regarding so-called ‘WAL’ files, or “database not found” are harmless; any other errors can indicate serious issues with the database or the device it’s (internal) memory.
Usually such errors are unrecoverable, but you may attempt to recover part or the whole DB using third party recovery tools (SQLite’s CLI tool is a free option).
Note: ‘WAL file’ related errors may be ignored, and may not persist across checks. Errors relating to opening databases may occur when the device monitor is disabled (by user or a failure to start), and a device was disconnected.
This allows merging of bookmarks/annotations for similar e-books.
After an e-book file is changed on the device, it may be treated as a new book. Examples are modifying metadata, or re-lending/downloading public library e-books. Afterwards, annotations from the previous file may no longer show for the ‘new’ file.
Should the two e-books be (nearly) similar, one can transfer annotations between files. To do so, this tool searches for duplicated titles, and (interactively) modifies associated annotations to point to the (highest) ID representing the ‘newest’ book entry.
In the tool GUI: - Parent items represent the ‘newest’ book version, with child items being older files. - The annotations count shows if any annotations can be transferred. - ‘DBSet/ProfileID’ refers to the annotation profile (this is usually 1, and can be ignored if so)
If you are sure the child items refer to the same book (see warnings), you may check these titles for merging. Clicking the parent checks all child items. Next, click ‘Ok’ to apply the merge/fix.
Warning:
Please backup your database file(s) first, for example using the ‘Manual backup’ menu option.
The assumption is that the book content was not changed between each version. If it has, transferred annotations may point to the wrong locations, as the tool does not correct for this (yet).
The PB database design tends towards adding or duplicating entries instead of modifying them. However, to avoid excessive duplication, this tool modifies data in-place.
Mirrors a device databases locally, for accessing annotations when a device is disconnected. It also provides a speed-up.
If enabled for a device, the local mirror is updated whenever that device connects to the program. Annotations are then reloaded (or do so manually using the Viewer menu option “Reload annotations”).
Note: this is a basic local caching solution, that leverages existing device support. A future centralized database is being considered.
Beware that a device reset (which resets the device DBs) will be carried over into the local mirror, thus erasing any previously locally stored annotations. Should you want to reset a device, store a manual backup before resetting. This backup can be imported to keep access to the annotations.
The label provided here, will be shown in the device selector.
Only visible when auto-loading annotations is disabled (see settings). If your device is slow, this may improve startup time.
The viewer displays annotation (meta)data in multiple columns.
Can be toggled by right-clicking in the ‘header’ bar (i.e. with “Title”, “Page”). By default, the “Date” and “Authors” columns are hidden.
Double clicking on the highlight or notes cell, allows selection and copying of text fragments. Note: Newlines are currently not shown.
Highlight colors can be shown (since 0.9.8), with an optional color text abbreviation. See Additional options..
The viewer context menu (right-click), offers short-cuts for filtering on that row’s fields.
Copy this cell copies the pointed to (single) cell’s content to the clipboard as raw text. For advanced export features, see the exporting section.
Copy selected row(s): short-cut for copy to clipboard. Follows the GUI sorting mode; for advanced export features, see the exporting section.
Selecting text segments If wanting to select text segments within a highlight/note, double click that cell to access the text. (Note: rich text formatting is to be implemented)
Clear Filters and show this row Clears all filters, and attempts to move the display to show this row, or the first selected one.
Allows interactive searching. Text searches are slightly delayed - this duration can be changed under “settings > Annotation Viewer”.
Case sensitivity can be toggled under the Viewer Tools menu (right-most button).
Currently implemented only for “Search highlights/notes” (but trivial to do so elsewhere). A general grasp of ‘common’ regular expression syntax will be beneficial. Some examples:
Note: This is still a basic feature. If you have a specific use-case, feel free to inform the authors.
The following sorting modes are available:
Note: Custom sorting is partially implemented, and does not persist across restarts.
Case sensitivity can be toggled under the Annotation Viewer menu.
Lastly, sorting of exported annotations can be configured separately.
Accessible via the right-most tools button.
Show notes inline: ‘Inlines’ the note texts into the highlight column, hiding the note column.
Show Bookmarks: Show or hide bookmarks in the viewer. These are hidden by default.
Note: If selected, hiding them using option removes their selection status.
Show Deleted annotations (0.10+): Show or hide deleted annotations in the viewer: if enabled, deleted annotation rows will be shown slightly faded and have a “D” mark in the delete column. Note that deleted annotations may include modified annotations (their previous version).
Note: If selected, hiding them using option removes their selection status.
Row color mode Sets the highlight color mode. Options include full row coloring (default), the ‘color column’ or disabled. Note: May require scrolling to update the view
Toggle color text: Shows or hides a text abbreviation of the highlight color, for use in bad light conditions, UI ‘nightlight’ modes or for visually impaired users. Note: Scrolling may be needed to update the view.
Resize Rows: Recomputes the row heights to show (nearly) all text. Should only be needed after resizing columns. Note: running this once, makes the auto-resize obsolete, until a (search) filter is applied.
Reload Annotations: Reloads only the (annotation) data, and prompts the viewer to update. Normally not needed.
Reset Viewer: Resets both the viewer model, and reloads annotations. Normally not needed.
Selected annotations can be exported to file or the clipboard, using the buttons “To File…” or “To Clipboard” (in the bottom export toolbar). A short-cut is available via the context-menu (right-click on a viewer annotation row).
Export format
Export sorting mode By default, exports use the GUI sorting order. This can be overridden by selecting a sorting mode in the bottom export toolbar.
Export to “Clipboard” Copies selected annotation data to the clipboard.
Export “to file…” Opens a file dialog for selecting an export file, and exports selected annotations to that file.
Notes, PocketBooks: Highlights edited on PB devices using that device’s Notes app, may lose their page and highlight location data.
Annotations from older PocketBook firmware versions, may miss page number information. The page column will then show “?”. Alternatives are being considered.
Disconnection will not (yet) cancel running operations: generally some warning will be shown; file transfers and DB access tend to fail gracefully.
Connecting multiple (supported) readers at once is still untested. Windows versions prior to 0.9.8. performed a full device scan upon detecting USB changes, and should be avoided in this case.
If you can help test this, please contact the author(s). For extra safety, first disable “loading annotations” (under settings), restart the program, and then try (this avoids database access).
With v0.9.7, program speed was improved markedly. If hard-pressed, these changes may improve the situation:
First, AVATeR only outputs debug messages when requested to do so from the CLI or GUI settings. Second, many distributions have blocked debug messages from being output, due to excessive default(!) debug messaging by applications.
If AVATeR debug output is enabled, and it detects the OS has blocked debug messages, it will attempt to restore debug output. Alternatively, on Linux you can issue export QT_LOGGING_RULES="*.debug=true;qt.*.debug=false from the shell/terminal, to force debug messages to show. See also: https://stackoverflow.com/questions/30583577/qt-qdebug-not-working-with-qconsoleapplication-or-qapplication
Given it’s general background (i.e. competing Markdown standards, especially w/r to tables; various extensions and Markdown converters), the Markdown export is likely to experience problems down the road.
Known issues: - a single dollar sign in the booktitle (rare), may be seen as starting a math formula, breaking the table export. Ghostwriter ignores this, but Apostrophe inserts a tag breaking the table.
Scroll CPU spikes Mouse-scrolling annotations can show up in visual CPU monitors as a CPU spike; note however the used CPU time (i.e. duration) is very short.
Book metadata on PocketBooks For the time being, book metadata is read from the books.db file in order to keep things simple. Note external apps that change metadata may only target the explorer.db, explaining any discrepancies.
AVATeR expects the OS (or you) to have mounted any supported e-readers, making the files accessible. Otherwise, the program will (silently) ignore the e-reader. (Future versions may indicate unmounted devices.)
Also ensure your device is supported by AVATeR. Newly introduced devices may use a new (USB) vendor ID, which must first be added to the program.
Note: some Linux distributions will show filesystems, but only mount them upon user (UI) action/access.
Steps you may try in addition are listed below. If the problem persists, enable debugging, inspect the program logfile and/or contact the author(s).
This seems to occurs when saving column-sizes for an empty table; it should normally not occur.
To fix: right-click the annotation viewer header row (with “Author”, “Title”, etc.) and select “Reset to Default”.
In case that fails:
Pre v0.10, a rare crash could be caused when switching between Qt5/6 versions. As of v0.10, column settings for either Qt version are stored separately. For older version, fix this by removing both “table_columnsettings” entries from the preference file: see the previous FAQ item for this.
This program’s read-only functions should not be able to cause damage. AVATeR doesn’t continuously access DBs, importing data instead, at the cost of some memory.
Functions that modify the device’s DBs do carry an inherent risk: these are the “merge/fix annotations” and “restore reading progress”. To minimize risks, consider the following points:
Preferably make a DB backup before using these tools (as prompted).
Ensure the device and USB cable are undisturbed during operations (with worn out USB ports even a nudge can cause a temporary disconnection).
Do note the SQLite database (DB) commonly used by (embedded appliances such as) e-readers is a robust DB format, widely used and well tested.
Lastly, AVATeR is not exempt from programming mistakes and/or compatibility problems, such as database changes after firmware updates. Generally, such operations will fail early (and safely), and additional checks are in development.
Some topics will come up repeatedly.
Note we provide no support for, and assume no responsibility for, trying out the discussed recovery aspects.
The various PB configuration settings files (.cfg, .dat, etc) cannot directly be modified. The cause is a hash signature, prompting the reader to revert to the backup file.
In case of problems, deletion can be considered (also remove the “.back” copy), but this rarely solves problems.
Fonts, dictionaries and apps may be restored by loading a backup archive with AVATeR’s “Send files” tool.
The backup archives produced by this program are standard .zip archives, so any .zip extraction utility should be able to extract the files. Files are stored in their original (device-relative) paths. These may change however between firmware versions.
Restoring database files is risky and should be avoided. However, if devices have identical firmware and e-book file paths, a chance of success exists. Often there is little to lose anyway: at worst the device can be reset again.
For PocketBooks, the main DBs of interest are:
Consider the sources of complications when replacing databases:
firmware updates: may change the database layout between versions. A manufacturer may(!) account for this during access or updates, but cannot always do so.
ebook file changes: these change the file’s ‘hash’, used as a identification help. Upon change, a book file will be perceived as a ‘new’ book, thus losing it’s link to previously associated reading progress and annotation data.
See changelog.txt
Originally this was a set of tools for PB readers. The annotation HTML exporter was made interactive, and made the UI centerpiece.
We mainly provide a basic set of tools, focused on being generic and adaptable to new situations.
Content can be best managed by the e-reading device or specialized programs.
The current implementation integrates several components: an underlying reader management layer (using standard library functions, and Qt SQL functions), a Qt based mainwindow, with an integrated annotation viewer, and several tools. Future versions may see these components being split out.
At the moment, you are encouraged to donate if you are able to do so. This may change in future versions, if there are valid reasons (support load, etc.).
Transmitted data. During the optional update check, only the program platform is communicated by means of the URL.
Program configuration files and logfiles Configuration files may contain a device’s serial, once a ‘devicelabel’ or ‘local DB mirror’ has been configured. In addition, the serial of the last used device is stored. Note: the serial can be ‘hashed’, but this gains little.
Program Backup files These are stored locally on your device. Largely self-explanatory: depending on the device, the archived files may include references to document title/author(s), genres, highlighted texts and user notes, if any.
Please see the website for more information.
Copyright (C) 2021-2022+ Authors (See “help > about” or authors.txt)
This software is provided ‘as-is’, without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software.