Shell commands plugin

[Jare]

Hi all,

Shell commands lets you define custom shell commands in settings and then run them via hotkeys or command palette. You can use it to open external applications or for automated processing of files. A few built-in variables are supported, so you can supplement your commands with for example current file path/folder, current date/time, selected text, clipboard content etc.

Examples

A few example commands (for Windows*):

• start git-gui: Open a graphical Git client for committing changes in your vault to a repository.
• start cmd: A quick way to run other commands that you have not defined in the settings.
• mkdir {{date:YYYY-MM}}: Create a monthly folder like 2021-08.
• echo {{clipboard}} >> NewNote.md: Create a new file and paste content there from clipboard.
• start https://duckduckgo.com/?q={{selection}}: Search the web using text you have selected.

The plugin does not predefine any commands, those are only example ideas.

*) Supported platforms are Windows, Linux, and Mac. You need to customise the given examples if you want to use them on Linux or Mac. Mobile is not supported. I’d like to investigate if it’s possible to run shell commands in the mobile app. Currently, it’s really uncertain to me (help is welcome ).

Note that for the sake of simplicity, there is no escaping done for variable values. If you have a command and a quoted string parameter like mycommand "{{clipboard}}", it might break if your clipboard content contains " quote characters, because those are inserted into the command as-is. Your command might end up looking like this: mycommand "Text pasted from clipboard that contains a " character."

Still being tested

This plugin in still in its 0.x era, meaning that there might be things that do not work as intended or things that could be improved. But the basic functionality that I wanted it to have is already there. I’d like to receive feedback about this plugin. If you have ideas for new features, new variables, improvements or fixes, you can freely write them to:

• Shell commands’ GitHub issues - the easiest for me to work on these ideas.
• In this topic - in case you don’t have a GitHub account or if it’s just general feedback or questions how to use this plugin.

I’m open to all discussion about how to change and improve this plugin in the future.

A word of warning

Be careful with system commands! Only use commands that you know and trust. If you are copy pasting commands from the internet or from files written by other people, you need to understand precisely what those commands do! Otherwise, you might lose your files, or screw up your system! This plugin doesn’t come with any kind of warranty.

P.S.

I also noticed another plugin that has the ability to run shell commands: Text Expander Plugin. It lets you write shell commands in your MarkDown files and run them from there, which might be better suited for some situations, such as scripts that are long or that need to be previewed every time before running them. My plugin works best for simple, short commands, even though you can run multiple commands by concatenating them with & in windows or && in Linux and Mac.

[Jare]

I just released version 0.3.0. During my opening post the latest version was 0.1.0, so I’ll add here a short summary of notable updates between 0.1.0 and 0.3.0:

• Alias names for commands that can make technical looking shell commands more readable in the command palette (Thank you @Felipe for this idea!).
• Preview variables in command palette so that you know exactly what values your shell commands will use when executed.
• {{workspace}} variable (Thank you @Felipe for this idea too!).
• Improved settings UI.

[Jare]

I’ve continued developing the plugin during the past few months, so I’ll write a status update here.

Current release version: 0.9.0
Beta test version: 0.10.0-beta.1

Notable changes between 0.3.0 and 0.9.0

• Possibly backwards incompatible change: {{variable}} values are escaped when using PowerShell or Bash/Dash/Zsh (no escaping when using CMD). Check that your variables work correctly after this upgrade! This change should improve the safety of variables, but it’s experimental - although so far it has been working very well!
• You can now select a shell where your shell commands will be executed in. Supported shells:
  • For Windows: CMD, PowerShell 5, PowerShell Core
  • For Linux and macOS: Bash, Dash, Zsh
• Shell command output can now be directed to:
  • A notification balloon
  • Currently open note (top, bottom or caret position)
  • Status bar
  • Clipboard
• New variables:
  • {{caret_position}}
  • {{file_extension}}
  • {{tags}}
  • {{yaml_value}}
• Autocompletion menu for typing variables quickly in the settings. You can also add your custom list of e.g. commands or whatever keywords.
• Operating system specific versions of shell commands: If you sync your vault (and SC settings file) between multiple devices that use different OSes, you can write multiple versions of a shell command, and SC will execute a version suitable for the current OS. E.g. if you have shell command that lists files in the vault’s root, for Windows you can write dir, and for macOS/Linux you can write ls.
• You can choose not to display error messages for certain exit codes.
• Shell commands can be set to ask a confirmation before execution.

More details are available in changelog.

New features in 0.10.0 (edit 2022-02-06: Released now)

So far the Shell commands plugin has offered a way to execute shell commands via the command palette and hotkeys. The version 0.10.0 also allows you to define events that execute your shell commands automatically.

The first set of events include:

• After Obsidian starts
• Before Obsidian quits
• After switching the active pane
• Every n seconds
• You can make your shell commands appear in menus, too:
  • File navigation menu
  • Folder navigation menu
  • Editor menu

Cheers!

[Jare]

A shorter update on notable new features.

0.11.0 - 2022-02-26

• Output channel Open a file can be used to receive a file name/path as output from a shell command, and open the file in Obsidian. It supports opening an existing or a new file. It also allows placing carets and selecting text in the opened note, based on line/column coordinates.
• Less prominent changes

0.12.0 - 2022-05-07

• When a shell command is about to be executed, a prompt modal can be opened up to ask for a user to input text values that will be used as part of the shell command.
• Less prominent changes

[nestorito]

Hello @Jare I’m wondering if your plugin could be used to retrieve web contents from a webpage and paste this contents in the same note. I’m not a coder so sorry if it appears as a silly question…
The context is the pocket plugin (GitHub - nybbles/obsidian-pocket: Pocket integration for Obsidian); the Pocket API currently doesn’t allow to copy the full text of a saved page so the resulting note looks like this:

---
Title: "El nuevo film de ficción del realizador se ve en el CC de la Cooperación"
URL: https://www.pagina12.com.ar/425910-reloj-soledad-de-cesar-gonzalez-cimiento-realista-sin-etique
Pocket URL: https://getpocket.com/read/3635259506
Tags: [pocket, ]
Excerpt: >
    Poeta, ensayista, artista plástico y realizador, César González viene produciendo desde hace tiempo una obra cinematográfica frondosa y singular que permanecía en una invisibilidad casi plena.
---

The idea:

• get the url from the “Pocket URL” in frontmatter (better than the original URL because the pocket url is in a readability format)
• copy the contents to clipboard
• paste in the same note under the frontmatter

Any idea? I made a manual attempt (opened the pocket url copied the contents and pasted in the note and the result is good and basic formatting is mantained (and also images, if it’s the case)
(posted also the idea on git-hub: Make `ContentType` of Pocket item accessible · Issue #36 · nybbles/obsidian-pocket · GitHub)
Thanks!

[Jare]

Hi @nestorito!

It should be possible. First I’d like to ask what is your operating system: Windows, Linux or macOS? (No mobile OS’s are supported by the SC plugin). Then I’m able to give you precise instructions with a correct shell command that will work in your operating system.

[nestorito]

Thank you @Jare, I’m on macOS

[stepsal]

Nice plugin. Running shell commands is powerful, but as you have disclaimed… pretty dangerous. The safest way to run shell commands would be from inside a docker container hosting your Obsidian application. This would protect the main filesystem / OS from potential horrors

[Jare]

Ok, I don’t have macOS (I have Windows and Linux), but I’ll assume wget command works in macOS, as it works in Linux. You might need to install wget on your macOS. wget is a command that will do the actual web page downloading for us.

1. The shell command:

wget {{yaml_value:Pocket URL}} --output-document=- --quiet

• {{yaml_value:Pocket URL}}: This is a variable specific to the Shell commands plugin. It reads a value from the currently open note’s YAML frontmatter and inserts it into the shell command.
• --output-document=-: This part defines that wget should not write the downloaded content to any file, but to output it to the standard output (stdout) instead. This allows the content to be captured by the Shell commands plugin when the whole shell command finishes.
• --quiet: Tells wget not to output any side information regarding the download process. You can try to leave this argument off, and you’ll see that the Shell commands plugin shows a bit messy “error” message balloon containing just trivial information about the download.

2. Set the output channel to either Current file: caret position or Current file: bottom. (I don’t recommend Current file: top in this case, because you have YAML in your note and outputting on top of that would make the YAML frontmatter inactive.)

Please let me know how this works for you! There might come up things that need improving.

Copy to clipboard

You mentioned you’d also like to copy the downloaded content to clipboard. This can be done, but let’s first try if the basic downloading and inserting to note works. I can then try to make the shell command contain a part that copies the content to clipboard.

Should HTML be stripped?

I’m not familiar with the Pocket service, so don’t know the exact format how it outputs content. If it outputs a whole HTML page, including <!doctype> and <html> tags, it’s going to be a problem, and will require an additional part to the shell command where we would strip unnecessary html. Some formatting HTML could be left, because markdown allows to have inline HTML; or perhaps HTML formatting could be converted to markdown syntax for the sake of cleanliness.

Stripping HTML and/or converting to markdown is not familiar to me. I know how it would be done by hand, but I’m not familiar with a shell command aimed for this. Maybe there is some HTML to Markdown converter that can be executed from shell. I can look for something if you need, but cannot promise anything.

[nestorito]

Hi @Jare,
thanks a lot for your very detailed how-to! Unfortunately when I run the command from command palette I get this error in the note:

zsh:1: command not found: wget

The strange thing is that if I run the command from terminal I get all the html text starting from

<!DOCTYPE html>.......

So apparently the wget command works in terminal…

Coming to your notes on “clipboard” and “html stripping”: I thought about copying to clipboard because I’m not a coder and to me it was the most obvious idea, but not necessarily the better one… What I notice is that the output in terminal doesn’t contain the text of the article but just html code.
Sorry if I can’t give you more details
THX

[Jare]

Fix the command not found: wget error

@nestorito

Unfortunately when I run the command from command palette I get this error in the note:

zsh:1: command not found: wget

The strange thing is that if I run the command from terminal I get all the html text

I suspect that for some reason the shell does not know the path of wget when executing shell commands via Obsidian. This can probably be fixed:

1. In the normal terminal, please execute the following shell command: which wget . which is a command that tells the installation path where a specific executable is located at - in this case, the absolute path of wget.
2. Change the shell command in the plugin’s settings so that it contains the full path to wget: /path/to/wget {{yaml_value:Pocket URL}} --output-document=- --quiet . I don’t know what the path is for you, so you need to find it out in step 1 and then substitute /path/to/wget with it. If you encounter any problems with this, you can post the path here.

There will be a way to fix this in another way, when SC version 0.13.0 gets released (probably soon). It’s called Add directories to the PATH environment variable. The link goes to a documentation, which instructs you how to use it. You would need to add the absolute path of wget to this list of additional directories (retrieved from step 1). Then you don’t need to do the modification mentioned in step 2. But like I said, this is only available once SC 0.13.0 is released. There’s a beta test available also, but I’ll probably release 0.13.0 soon so you don’t need to do the beta installation.

Directing output to clipboard

@nestorito

Coming to your notes on “clipboard” and “html stripping”: I thought about copying to clipboard because I’m not a coder and to me it was the most obvious idea, but not necessarily the better one…

Ok, I first though you’d like the output to go automatically to both clipboard and to the note. In case you don’t need it to go automatically to both places, it will become a bit easier. The Shell commands plugin supports inserting into the current note, or to the clipboard. A current limitation is that you can only pick one destination for the output (I’m planning to support selecting multiple destinations in the future). To sum it up, if you wish, you can switch the output channel of stdout to Clipboard.

Wrong content downloaded

@nestorito

What I notice is that the output in terminal doesn’t contain the text of the article but just html code.

Can you find the text of the article somewhere inside that HTML code? Or is the textual content of the HTML code something totally different?

Hmm, after thinking a while, I guess this is because the content might require logging in to the Pocket service before it can be accessed. And there’s probably no easy way to make wget automatically log in to the website before downloading the actual content. I guess this workflow would only suit for pages that are publicly visible in the internet.

[Jare]

A new version of the Shell commands plugin is now released.

0.13.0

Feature (link to GitHub issue)	Documentation
Shell commands URI	Documentation
PATH setting	Documentation
New file and folder events	Documentation
Multiline shell commands	Documentation (scroll to Multiline shell commands)
Submit/accept modals by pressing enter	No documentation
Shortcut icon for defining hotkeys	No documentation
Display shell command id in extra options modal	No documentation

[nestorito]

Thank you @Jare for your detailed help and sorry for late response.
I’ve uninstalled and reinstalled home-brew and followed the instructions to move it in the right path but apparently nothing changes…even if I add the pat in which wget is installed I get the same error. I also added this directory to the path in environments:
/opt/homebrew/bin/wget
{{!environment:PATH}}
No luck.

Coming to the other point: what the wget command gets from the page:
You are right: the getpocket url needs a username and password so I made some attempts with the original URL: here the problem is that when I run the shell command from terminal I get the whole html code from the page which is not useful as the article text is really scattered and in some pages it neither follow a unique style (I mean: I can’t find a unique ‘tag’ where it begins and end the text, sometimes the first paragraph is treated as an intro and other paragraphs as other ‘styles’.) So I guess it’s not the right way to get what I thought it would be easier.
I can’t figure out how to get to get only the visible article text (as in Safari reality mode) and I 'm sorry if I made you loose some time… Thanks a lot!

[Jare]

@nestorito

Thank you @Jare for your detailed help and sorry for late response.

You are welcome! And no problem, we all have our busy lives every now and then. Some times it takes a long time for me to reply, much longer than a few days.

wget not found

@nestorito

I’ve uninstalled and reinstalled home-brew and followed the instructions to move it in the right path but apparently nothing changes…even if I add the pat in which wget is installed I get the same error. I also added this directory to the path in environments:
/opt/homebrew/bin/wget
{{!environment:PATH}}
No luck.

Can you try to use /opt/homebrew/bin instead of /opt/homebrew/bin/wget? Does it help at all? PATH usually contains directories instead of files, and wget is a file (or so I assume, but I don’t know homebrew at all), and I don’t know if PATH can work with file paths - I’m not experienced enough about PATH. So that’s why I’m suggesting to use the directory that contains the wget file instead of the full path to wget. Your shell should be able to pick wget up when you just point it to the directory.

Logging in and HTML format problems

@nestorito

Coming to the other point: what the wget command gets from the page:
You are right: the getpocket url needs a username and password so I made some attempts with the original URL:

The login at least is an issue. In theory, wget could be made to perform a login action, but it’s really not trivial and depends heavily on how the logging in is implemented in the target service (i.e. getpocket in this case). Any possible bot detectors will probably make this kind of automated login to fail, so I wouldn’t use too much your time on bangin head against this wall.

@nestorito

here the problem is that when I run the shell command from terminal I get the whole html code from the page which is not useful as the article text is really scattered and in some pages it neither follow a unique style (I mean: I can’t find a unique ‘tag’ where it begins and end the text, sometimes the first paragraph is treated as an intro and other paragraphs as other ‘styles’.) So I guess it’s not the right way to get what I thought it would be easier.
I can’t figure out how to get to get only the visible article text (as in Safari reality mode)

Too bad that I don’t know the getpocket service at all. I don’t know if it provides any better URLs for exporting data. Reading content from HTML automatically will be a pain if there is no common structure that would repeat in every URL. So, unfortunately this is another nail to the coffin of a completely automated content getter.

@nestorito

I 'm sorry if I made you loose some time… Thanks a lot!

No problem, I’m always interested to learn how people use the Shell commands plugin - or how they would want to use it - and searching for solutions to different needs can really open my mind to see things I wouldn’t come up with myself. It’s not waste of time even if a final solution is not found. If even some parts are solved, then those parts can be used later to build some other shell command in a different situation.

[Jare]

Coming back to your older post

@nestorito

the Pocket API currently doesn’t allow to copy the full text of a saved page

I’d suggest that you try to contact Pocket and ask if they can add the full text of a saved page to their API. Would that be a solution? Of course it’s possible that they might say no, and even if they say yes, implementing it can take some time.

[Jare]

I’ve just released version 0.17.0.

It can now receive output in realtime from shell commands that take a long time to execute.

Notable changes between 0.13.0 and 0.17.0

Feature (GitHub issue)	Documentation
Real time output handling (#275)	Documentation
Output wrappers (#262)	Documentation
New variable: {{caret_paragraph}} (#282)	Documentation
New variables: {{file_uri}} and {{event_file_uri}} (#258)	Documentation
New variables: {{new_note_folder_name}} and {{new_note_folder_path}} (#235).	Documentation

A complete list of changes

[sbell]

I have the following script which I am trying to get to work using this plugin… but when I execute it, nothing happens! Not even an error log. It works fine when run from a .cmd file. Any ideas?

echo [Startup]  >%temp%\fhini.ini
echo Window=FocusWin >>%temp%\fhini.ini
echo Record=I92 >>%temp%\fhini.ini
start "Run FH" "C:\Program Files (x86)\Family Historian\program\fh.exe" "/ini:%temp%\fhini.ini" "C:\Users\sarah\OneDrive\Genealogy\Family Historian Projects\2022 Family Tree\2022 Family Tree.fh_data\2022 Family Tree.ged"

[Jare]

Hi, what is fh.exe supposed to do? Should it write to a file, or should it open up a graphical window or something else?

[sbell]

Sorry, I should have explained better. This is a script to open a software called Family Historian, and specifically it opens to the focus window for an individual with record ID 92. But once it works using the Shell Commands plugin, I intend to replace that ‘92’ with a variable which can then be entered to change which ‘focus person’ the program opens on.

[ReaderGuy42]

Hey, love this plugin! Can it create a new folder within a directory and then place new files there? I use the plugin to convert .md to .tex file with pandoc:

pandoc {{file_path:absolute}} -o "/home/Obsidian/Output/"{{title}}.tex

And this works perfectly. The only issue with this is that LaTeX creates a lot of auxiliary files, so I’d really like the plugin to create a folder with the same name as {{title}} and then place the new file {{title}}.tex in that folder. Is this possible?

I’ve been playing around with {{new_note_folder_name}} but I don’t think I understand its usage correclty. Any advice?
Thanks!!