UpNote to Obsidian Migration Script

Hello Everyone,

New user of obsidian here.

I was previously using UpNote as my note taking app. Unfortunately, there is not a good inbuilt way to export and import notes into Obsidian whilst maintaining the notebook structure. This is a big problem if you have many notes like me.

I wrote this Python script that once run, moves all notes to notebooks in the exact structure that you see in UpNote.

The only thing that you need to do is ensure that you have made an UpNote backup (available via the app) to the desktop of your computer. Once you have done that, you will also need to edit the backup_folder_name variable in the script to the unique name of the folder within the ‘UpNote Backup’ folder that UpNote creates. I believe this is string that references your user on UpNote.

I recommend that if you are to rerun it, you should delete the ‘Vault’ folder that the script creates. If you don’t then duplicates will occur.

If anyone has any questions or troubles please let me know. I created this script for myself and have made some effort to make it as one-clickable as possible, but may have made some oversights.

import gzip
import json
import os
import platform
import shutil

# ----------------------------------------------------
#     This variable stores the name of the folder that 
#       UpNote creates when making a backup
# ----------------------------------------------------
backup_folder_name = "g3utVsgtSGUK2n9enn2IFBnqg1E2"

def get_system_info():
    # -------------------------------------------------------------------
    #   Get the current system platform (either MacOS/Linux or Windows)
    # -------------------------------------------------------------------
    current_system = platform.platform().lower()
    desktop_path = ""

    # -------------------------------------------------------------------
    #       Get the path to the currently logged in user's Desktop
    # -------------------------------------------------------------------
    if "macos" in current_system or "linux" in current_system:
        desktop_path = os.path.join(os.path.join(
            os.path.expanduser('~')), 'Desktop')
    else:
        desktop_path = os.path.join(os.path.join(
            os.environ['USERPROFILE']), 'Desktop')

    # -------------------------------------------------------------------
    #                       Return the desktop path
    # -------------------------------------------------------------------
    return desktop_path


def prepare_upnote_backup_file(desktop_path, backup_folder_name):
    # ----------------------------------------------------------------------------------------------------------
    # 
    # ----------------------------------------------------------------------------------------------------------
    backup_data_directory_path = f"{desktop_path}/UpNote Backup/{backup_folder_name}/data"
    backup_data_directory = os.walk(backup_data_directory_path)
    # ----------------------------------------------------------------------------------------------------------
    #                   Initialise the dictionary that will contain the information of the 
    #                               most up to date .upnx backup file
    # ----------------------------------------------------------------------------------------------------------
    newest_backup_file = {
        "File Name": "", 
        "Modified Time": 0, 
        "File Path": ""
        }

    for root, dirs, files in backup_data_directory:
        for file in files:
            newest_backup_file_mtime = newest_backup_file["Modified Time"]
            # The os.stat method returns a tuple. The 'mtime' value is stored at index 8
            modified_file_time = os.stat(f"{desktop_path}/UpNote Backup/{backup_folder_name}/data/{file}")[8]

            # --------------------------------------------------------------------------------------------------
            #             Check to see if the current file's mtime is more recent than the last.
            #           If it is, save the details of the file to the newest_backup_file dictionary
            # --------------------------------------------------------------------------------------------------
            if modified_file_time > newest_backup_file_mtime and not file.__contains__(".DS_Store") and not file.endswith("json"):
                newest_backup_file = {
                    "File Name": file, 
                    "Modified Time": modified_file_time, 
                    "File Path": f"{root}/{file}"
                }

    # ----------------------------------------------------------------------------------------------------------        
    #          Save the neweest backup file path and name to variables to use in the upcoming steps
    # ----------------------------------------------------------------------------------------------------------
    newest_backup_file_path = newest_backup_file["File Path"]
    newest_backup_file_name = newest_backup_file["File Name"]

    # ----------------------------------------------------------------------------------------------------------
    #       Decompress the .upnx backup file and create a new json file with the contents of the backup
    # ----------------------------------------------------------------------------------------------------------
    with gzip.open(f"{newest_backup_file_path}", 'rb') as f_in:
        with open(f"{backup_data_directory_path}/{newest_backup_file_name}.json", 'wb') as f_out:
            shutil.copyfileobj(f_in, f_out)
    
    # ----------------------------------------------------------------------------------------------------------
    #                           Open the json file to format with correct json syntax
    # ----------------------------------------------------------------------------------------------------------
    with open(f"{backup_data_directory_path}/{newest_backup_file_name}.json", "r") as file:
        file_lines = file.readlines()

        # ------------------------------------------------------------------------------------------------------
        #           Change the 'upnote_key:' to be the key portion of a dictionary key:value pair
        # ------------------------------------------------------------------------------------------------------
        for line in file_lines:
            last_element_index_number = file_lines.index(file_lines[-1])
            current_element_index_number = file_lines.index(line)
            upnote_key_with_newline = file_lines[file_lines.index(line)]
            upnote_key = upnote_key_with_newline[:-1]
            if line.__contains__("upnote_key:") and current_element_index_number != 0:
                file_lines[file_lines.index(line)] = f"],\"{upnote_key}\": ["
            elif line.__contains__("upnote_key:") and current_element_index_number == 0:
                file_lines[file_lines.index(line)] = "{" + f"\"{upnote_key}\": ["
            
            # --------------------------------------------------------------------------------------------------
            #           Add an enclosing square bracket and curly bracket to the end of the file
            #          The curly bracket is to enclose the entire file contents into a dictionary
            # --------------------------------------------------------------------------------------------------
            if current_element_index_number == last_element_index_number:
                file_lines[file_lines.index(line)] = file_lines[file_lines.index(line)] + "]}"

        # ------------------------------------------------------------------------------------------------------
        #      Replace all instances of '}\n{' with '},\n{'. This is required to correct the json syntax
        #                                       of the backup file
        # ------------------------------------------------------------------------------------------------------
        joined_file_list = "".join(file_lines)
        joined_file_list = joined_file_list.replace("}\n{", "},\n{")

    # ----------------------------------------------------------------------------------------------------------
    #                       Write all formatting changes to the newly created json file
    # ----------------------------------------------------------------------------------------------------------
    with open(f"{backup_data_directory_path}/{newest_backup_file_name}.json", "w") as file:
        file.writelines(joined_file_list)

    return newest_backup_file_name


def get_upnote_data(desktop_path, backup_folder_name, newest_backup_file_name):
    # ---------------------------------------------------------------------------------------------------------------------------
    #       Get the json data from the newly created json file. This file contents all data about notebooks and notes
    # ---------------------------------------------------------------------------------------------------------------------------
    with open(f"{desktop_path}/UpNote Backup/{backup_folder_name}/data/{newest_backup_file_name}.json", mode="r") as upnoteFile:
        return json.load(upnoteFile)


def gather_notebook_and_note_data(upnote_data, desktop_path):
    # ------------------------------------------------------------------------------
    #                           Initialise variables
    # ------------------------------------------------------------------------------
    # This dictionary will store all the relevant data for each notebook
    list_of_notebooks = {}
    notebooks_metadata = upnote_data["upnote_key:notebooks"]
    notes_metadata = upnote_data["upnote_key:notes"]
    notes_locations = upnote_data["upnote_key:organizers"]

    # ------------------------------------------------------------------------------
    # Loop through each notebook and save the relevant data to the above dictionary
    # ------------------------------------------------------------------------------
    for notebook in notebooks_metadata:
        # --------------------------------------------------------------------------
        #    Save important notebook information to dictionary
        # --------------------------------------------------------------------------
        list_of_notebooks.update({
            notebook["id"]:
            {
                "Notebook ID": notebook["id"],
                "Notebook Name": notebook["title"],
                "Parent Notebook ID": notebook["parent"],
                "Notebook Location": "",
                "Child Notebooks": {},
                "Child Notes": {}
            }
        })

    # ------------------------------------------------------------------------------
    #               Populate 'Child Notebooks' dictionary for all notebooks
    # ------------------------------------------------------------------------------
    for notebook_id in list_of_notebooks:
        try:
            current_notebook = list_of_notebooks[notebook_id]
            parent_notebook = list_of_notebooks.get(
                list_of_notebooks[notebook_id]["Parent Notebook ID"])
            has_parent_notebook = parent_notebook["Parent Notebook ID"] != ""

            if has_parent_notebook:
                parent_notebook['Child Notebooks'].update(
                    {current_notebook["Notebook Name"]: current_notebook["Notebook ID"]})
        except:
            continue

    # ------------------------------------------------------------------------------
    #               Populate 'Child Notes' dictionary for all notebooks
    # ------------------------------------------------------------------------------
    # Loop through the note to notebook relationship metadata structure
    for note_location in notes_locations:
        # Whilst in each note to notebook relationship section, loop through each note
        notebook = list_of_notebooks.get(note_location["notebookId"])
        for note in notes_metadata:
            # If the current note to notebook relationship note ID equals the current note ID
            if note_location["noteId"] == note["id"]:
                # Add the note name to the notebook Child Notes list
                notebook['Child Notes'].update(
                    {
                        note["id"]:
                        {
                            "Note Name": f"{note['title']}.md",
                            "Note ID": note['id'],
                            "Parent Notebook Name": notebook['Notebook Name'],
                            "Parent Notebook ID": notebook['Notebook ID']
                        }
                    })

    # ------------------------------------------------------------------------------
    #  Get paths for all notebooks and add notebook ID and note path to dictionary
    # ------------------------------------------------------------------------------
    vault_location = os.walk(f"{desktop_path}/Vault")

    for root, dirs, files in vault_location:
        for notebook_id in list_of_notebooks:
            if root.endswith(notebook_id):
                list_of_notebooks[notebook_id]["Notebook Location"] = root
            else:
                continue

    # ------------------------------------------------------------------------------
    #                           Write all note data to file
    # ------------------------------------------------------------------------------
    with open("/Users/alexfletcher/Desktop/Notebook Data.json", "w") as notebook_data_file:
        json.dump(list_of_notebooks, notebook_data_file)

    return list_of_notebooks


def create_folders(notebooks, desktop_path):
    # --------------------------------------------------------------------------------------------
    #                                   Create Vault directory
    # --------------------------------------------------------------------------------------------
    try:
        os.mkdir(f"{desktop_path}/Vault")
    except FileExistsError:
        print("An error was received when attempting to create the \"Vault\" directory: Directory already exists")

    # --------------------------------------------------------------------------------------------
#                                       Create all notebook folders
    # --------------------------------------------------------------------------------------------
    for notebook_id in notebooks:
        try:
            os.mkdir(f"{desktop_path}/Vault/{notebook_id}")
        except:
            continue


def move_notes_to_folders(notebooks, desktop_path):
    # ------------------------------------------------------------------------------
    #               Save the directory where the notes are located in
    # ------------------------------------------------------------------------------
    note_backup_directory = f"{desktop_path}/UpNote Backup"

    # ------------------------------------------------------------------------------
    #                       Move notes to their parent folder
    # ------------------------------------------------------------------------------
    for notebook_id in notebooks:
        child_notes = notebooks[notebook_id]["Child Notes"]
        for note_id in child_notes:
            note_data = child_notes[note_id]
            note_name = note_data["Note Name"]
            note_parent_notebook_id = note_data["Parent Notebook ID"]

            try:
                if "/" in note_name:
                    shutil.copy(
                        f"{note_backup_directory}/g3utVsgtSGUK2n9enn2IFBnqg1E2/Markdown/{note_id}.md",
                        f"{desktop_path}/Vault/{note_parent_notebook_id}/{note_name.replace('/', 'FORWARDSLASH')}"
                    )
                else:
                    shutil.copy(
                        f"{note_backup_directory}/g3utVsgtSGUK2n9enn2IFBnqg1E2/Markdown/{note_id}.md",
                        f"{desktop_path}/Vault/{note_parent_notebook_id}/{note_name}"
                    )
            except:
                continue


def move_folders(notebooks, desktop_path):
    updated_notebook_data = notebooks

    # Get parent notebook ID for each notebook
    for notebook_id in updated_notebook_data:
        parent_notebook_id = updated_notebook_data[notebook_id]["Parent Notebook ID"]
        current_notebook_path = updated_notebook_data[notebook_id]["Notebook Location"]

        # Move note from current location to it's parent folder
        try:
            parent_notebook_path = updated_notebook_data[parent_notebook_id]["Notebook Location"]
            shutil.move(current_notebook_path, parent_notebook_path)
        except:
            continue

        vault_location = os.walk(f"{desktop_path}/Vault")

        for root, dirs, files in vault_location:
            for notebook_id in updated_notebook_data:
                if root.endswith(notebook_id):
                    updated_notebook_data[notebook_id]["Notebook Location"] = root
                else:
                    continue
            

def change_folder_names(notebooks, desktop_path):
    notebook_locations = os.walk(f"{desktop_path}/Vault")
    for root, dirs, files in notebook_locations:
        for notebook_id in notebooks:
            notebook_name = notebooks[notebook_id]["Notebook Name"]
            notebook_id = notebooks[notebook_id]["Notebook ID"]
            if root.endswith(notebook_id):
                try:
                    os.rename(root, f"{root[:-36]}{notebook_name}")
                    change_folder_names(notebooks, desktop_path)
                except:
                    continue


def sort_notes():
    # --------------------------------------------------------
    #              Get note and notebook data
    # --------------------------------------------------------
    desktop_path = get_system_info()
    newest_backup_file_name = prepare_upnote_backup_file(desktop_path, backup_folder_name)
    upnote_data = get_upnote_data(desktop_path, backup_folder_name, newest_backup_file_name)
    notebooks = gather_notebook_and_note_data(upnote_data, desktop_path)
    # --------------------------------------------------------

    # --------------------------------------------------------
    #                     Logical processes
    # --------------------------------------------------------
    create_folders(notebooks, desktop_path)
    move_notes_to_folders(notebooks, desktop_path)
    notebooks = gather_notebook_and_note_data(upnote_data, desktop_path)
    move_folders(notebooks, desktop_path)
    change_folder_names(notebooks, desktop_path)

sort_notes()

Kind Regards,
Fletcher4256

4 Likes

I copied and pasted this and changed the folder namme to my upnote backup folder name. Everything is on the desktop including the sccript. I am running this via python script.py as that is the name of my file. I am getting this issue:

PS C:\Users\dudet\desktop> python script.py
Traceback (most recent call last):
  File "C:\Users\dudet\desktop\script.py", line 328, in <module>
    sort_notes()
  File "C:\Users\dudet\desktop\script.py", line 314, in sort_notes
    newest_backup_file_name = prepare_upnote_backup_file(desktop_path, backup_folder_name)
  File "C:\Users\dudet\desktop\script.py", line 86, in prepare_upnote_backup_file
    file_lines = file.readlines()
  File "C:\Python310\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 2207: character maps to <undefined>

Hey, had a similar issue - I’ve adapted another script to convert the notes into a format you can just drop into your Obsidian folder.