UTF-16 Encoding Corruption

  • Plugins disabled and safe mode enabled.
  • Searched for issue on forum; found this post that might be relevant.

On my work computer, can’t use Obsidian, so I use a basic PowerShell script to generate a daily note in markdown format. When opening these notes with Obsidian on home computer, the files become corrupted; looks like an encoding problem. I did find that if I just copy+paste the file contents to another file and open it in Obsidian, it does not become corrupted; but the other file consistently does. (I sync this vault via GitLab, so when it corrupts, I pull down a fresh copy and repeat the behavior).

I found that the files that get corrupted are the ones PowerShell generated in UTF-16 LE. UTF-8 encoded files view just fine.

Steps to reproduce

  1. Generate markdown file with PowerShell using redirection. Script contents below.
  2. Open the file in VS Code to verify content displays correctly.
  3. Open file in Obsidian vault and verify that content becomes corrupted.

Expected result

Upon opening file, view contents as usual. Plain text is retained.

Actual result

Content has lots of extra characters added to make it unreadable.

Environment

  • Operating system: Windows 10, MacOS Monterey
  • Debug info:

SYSTEM INFO:
Obsidian version: v0.13.14
Installer version: v0.12.19
Login status: not logged in
Insider build toggle: off
Live preview: on
Legacy editor: off
Base theme: dark
Community theme: none
Snippets enabled: 0
Safe mode: off
Plugins installed: 1
Plugins enabled: 0

RECOMMENDATIONS:
none


Additional information

Script Contents:

$filename = (Get-Date -Format yyyyMMdd)
$dayofweek = (Get-Date).DayOfWeek
$dateandtime = (Get-Date).DateTime

Write-Output "# $filename, $dayofweek
$dateandtime

## Tasks

**Today's Top 3**
- [ ] 
---
- [ ] Time card
- [ ] Daily notes uploaded

## Notes

## Links

## Daily Retro

**What went well?**

**What didn't go well?**

**How can I improve tomorrow?**
" > ".\$filename.md"

We only support UTF8. We don’t support UTF-16 and the file encoding is not easy to determine before opening the file.