How to Use getDisplayMedia() or desktopCapturer for Screen Sharing in a Community Plugin?

Developers: Plugin & API
1

Hi everyone :waving_hand:

I’m developing a community plugin that integrates with the Gemini 2.0 Live API, enabling users to engage in real-time voice conversations with an AI assistant inside Obsidian.

As an enhancement, I’d like to include screen sharing — capturing either a full screen or a specific window — and streaming visual content alongside mic audio for AI processing.

What I’ve tried so far:

  • Calling navigator.mediaDevices.getDisplayMedia() results in a NotSupportedError inside Obsidian (Electron).
  • Wrapping it in feature checks or try/catch doesn’t help — no prompt appears.
  • I’m aware that in Electron, desktopCapturer is typically used, but I don’t know if it’s accessible inside a plugin sandbox.

Questions:

  1. Is getDisplayMedia() intentionally unsupported or sandboxed in the Obsidian Electron environment?
  2. Can community plugins access desktopCapturer, ipcRenderer, or Electron.getCurrentWindow()?
  3. Are there any known plugins or examples that perform screen capture (even periodic screenshots) from within Obsidian?
  4. If not possible directly, is the recommended approach to use an external Electron companion app that talks to the plugin via WebSocket or local server?

I’m trying to keep everything within the plugin if possible, but I’m open to clean workarounds. Any guidance from those who have explored this would be appreciated!

Thanks :folded_hands: