#19 continue insert process even if duplicates exist 5 months ago

Comment by ~savoy on ~savoy/holo

savoy referenced this ticket in commit e04c987.

REPORTED RESOLVED IMPLEMENTED

#19 continue insert process even if duplicates exist 7 months ago

feature added by ~savoy on ~savoy/holo

#19 continue insert process even if duplicates exist 7 months ago

~savoy assigned ~savoy to #19 on ~savoy/holo

#19 continue insert process even if duplicates exist 7 months ago

Ticket created by ~savoy on ~savoy/holo

If there is a lack of data scrubbing prior to getting the list of new people to insert into holo, many duplicates end up existing. The current behavior of holo is to not insert any of the set of people if there exists duplicates, which could be a lot of additional time-consuming work if 20 people are being added and one duplicate ruins the set.

If a duplicate is present, it should be ignored while the others are correctly inserted, and the duplicates should be printed out (and preferably logged #12)

#18 allow None values in `insert` 7 months ago

Comment by ~savoy on ~savoy/holo

savoy referenced this ticket in commit a7779c8.

REPORTED RESOLVED FIXED

#18 allow None values in `insert` 10 months ago

bug added by ~savoy on ~savoy/holo

#18 allow None values in `insert` 10 months ago

~savoy assigned ~savoy to #18 on ~savoy/holo

#18 allow None values in `insert` 10 months ago

Ticket created by ~savoy on ~savoy/holo

TOML does not have a way to indicate a None value in its syntax, meaning that setting inserting a "none" of "" is shown as an empty string instead of null. This is an issue that has come up with attempting to insert people into the DB that don't have an email address. As there has already been an entry with "", it won't insert as the email field requires a unique value (or null).

Fields should be able to be ommitted from the TOML insertion if nullable or holo should automatically change empty strings. There already exist (and are used) two functions that should be doing this: holo_data._get_nullable_values & holo_data.get_type_matchup, but the now intended functionality is not present.

It seems this:

type_match: dict[re.Pattern, tuple[str, type, pl.datatypes.DataTypeClass, Any]] = {
        re.compile("BIGINT UNSIGNED"): ("number", int, pl.UInt64, 0),
        re.compile("BIGINT"): ("number", int, pl.Int64, 0),
        re.compile("INT UNSIGNED"): ("number", int, pl.UInt32, 0),
        re.compile("INT"): ("number", int, pl.Int32, 0),
        re.compile("CHAR|TEXT"): ('text (enclosed in "")', str, pl.Utf8, ""),
        re.compile("BOOLEAN"): ("boolean (true/false)", bool, pl.Boolean, False),
        re.compile("DATE|TIME"): (
            "datetime (in ISO format)",
            dt.datetime,
            pl.Datetime,
            dt.datetime(1970, 1, 1, 0, 0, 0),
        ),
    }

should be changed so that the CHAR|TEXT key reads:

re.compile("CHAR|TEXT"): ('text (enclosed in "")', str, pl.Utf8, "")

instead. This may achieve the desired result of setting empty strings to None. OR somewhere along the way, "" should be set to None before being inserted into the database.

#15 process for editing existing DB data 11 months ago

bug added by ~savoy on ~savoy/holo

#15 process for editing existing DB data 11 months ago

Comment by ~savoy on ~savoy/holo

savoy referenced this ticket in commit 47beea1.