Folder locking/hiding (vs nested Vaults) to solve unrelated concerns?

Basement
1

Hi, I am new here (I am a researcher with intense hobby and they are quite connected). Sorry, this is a bit long an exciting speculation on planning the workflow. Currently, I log and plan in plant text files. My “everything” log (plain text file in BBEdit) has about 1.73 million words and 153k lines (just from the past in 3 years). From this everything log I regulary (diary or several times a week) copy and past text to different project-specific logs. The limitations of this approach are obvious :slight_smile: . I have started to evaluate options (Tinderbox, TheBrain, Obsidian, Devon, TaskPaper - yes, they are quite different.) At the moment, Obsidian is quite attractive for note management. However, after reading some comments on big vaults, one concern is how well Obsidian would serve me if I put all text notes into it (I don’t plan to embed files into the notes, rather just link to them). Will I experience speed issues, even if I do regular book-keeping (removing obsolete notes into project log files) and some organisation? In addition, what is a good robust way of hiding private information when showing and discussing content with other folks? If I use several independent Vaults to avoid one huge Vault, I will have the problem of too many notes related to more than one Vault that justifies this as a concern. Sugesstions that I considered:

  1. Allow some global folders accessible from all Vaults. For instance, let the Daly notes folder be global and allow some global notes visible from all Vaults. Actually I was surprised that the Daily Notes are Vault-specific. First I created 3 Vaults for different areas, but then I realised that each would have its own separate Daily Notes. This is not how my brain works. Too many of my thoughts/notes are ralated to more than one of these areas. Currently I am copy-pasting such notes from my “everything” text log to several project-specific text logs – takes a lot of time, and using several independent Vaults would not help in avoiding duplicated notes, but globally accessible folder would. I realise this is probably confronting the idea of self-consistent independent Vaults.
  2. Allow (Mac) OS or special Obsidian aliase (symbolic link) pointing into different Vault folders or notes. I tried to use Mac OS alias, it shows up but if I click on it, it will be opened vie the Finder, that is with BBEdit in my case. Placing in an alias to a folder behaves the same way. It would be nice to open any .md file in my computer in Obsidian even if it does not become part of the active Vault (just for viewing and if I want importing).
  3. If the above suggestions are not rational to implement, then perhaps the simplest solution (in my view) would be to use a single Vault but with project-specific sub-folders in combination with folder locking/hiding. The content of locked/hidden folders would not be visible for the application. In this way everything can be connected that needs to be, a uniform tagging system is used, and would be no need to duplicate notes, etc. Great benefits. It would also help against the size/speed issue, since blocking/hiding selected sub-folders in a Vault (when focusing on a single project) would gain speed and increase clarity. Once could of course filter out specific folders in searches (also from the graph), but the information is still loaded and managed in Obsidion. However, the contents of locked/hidden folders would not be seen to the app releasing memory and increasing speed. The links into a locked/hidden folder would be seen but could not be activated. Hence, it would also help in hiding private information in a presentation. When activating a link int to a locked/hidden folder Obsidian could show a virtual/dummy link and note such as (locked in)[name of folder], then you could click to go to that folder to unlock if you wanted. The dummy/virtual links and notes would be no real .md files and would be temporary, created only when a link is activated into a locked/hidden folder.
  4. I have seen some discussion on nested Vaults. Quite interesting option, and I may try it. I could have all private content and everything that is related to more than one specific area (or nested Vault) in the root Valt, and keep all specific knowledge in the nested Vaults (of smaller size). In addition, using a nested Vault with other folks would exclude the possibility of showing private info accidentally. But I would also have the option of seeing things globally. One concern is though that if (in the root Vault) I create a link in a note (in a nested Vault/folder) to a root or neighbouring Vault, that link will point to a non-existing note within the nested Vault. If I accidentaly click on it, will a new note be created (causing confusion in the root Vault) or only if I start editing it? Anyway, this can be done right now, whereas the others are basically feature suggestions.

Anyway, I think I like the 3rd option most - folder locking/hiding -, and please consider it as a suggestion for a new feature.

2

I would suggest to first try Obsidian, then see if and where there is an issue and then decide the best course of action.

Assuming there is a scaling problem somewhere in obsidian, the ideal solution would be to address that scaling problem not try to circumvent it with hacks like the ones you are proposing. Moreover some of these hacks require a significant engineering effort that would be better spent trying to address the underling problem.

I would also suggest to read the included help vault in its entirety.
We support folder symbolic links (and they are not going to help with your potential problem).

3

Dear WhiteNoise, thanks for quick reply!

I am having Obsidian up for a few days now. I admit that I have not yet read all the help docs in great detail (most of the pages I did read), but I gathered the basic picture how it works. I still don’t see how I could hide private info when discussing notes with other people, other than keeping private notes in a dedicated folder and excluding that folder from searchers and from the graph such as, e.g., “-path:private”. This is what I am doing now, but hiding/locking folders would be much more robust and safe (and would exclude that I accidentally show something private).
You are right, that only future can tell if I will experience speed issues with the amount of notes I have. However, I have to make a decision now about what platform/sw and workflow to choose. As I wrote, I have some 1.73 million words and 153k lines of plain text notes in the “everything” log and I have several other text log files (with ideas and info – I am reasearch physicist with ~110 peer reviewed publications and I have intense hobby too, hence I have a lot of text notes). I will need to extract the most important notes from these text files and transfer them to my new note management platform. This will take a significant amount of time (even if I automate some part of the process) and if then it turns out that it is not working in Obsidian, I will have to repeat it with some other platform. Again, I see that locking/hiding folders would for sure help to avod size/speed related potential problems, hence I proposed the request.

Anyway, I can understand that none of my suggestions are going to happen, and I neeed to keep this argument in mind, among other pro-contra arguments.

However, I have just tried the following: I moved out a folder from the Vault before starting Obsidian (since I have only a few dozens of notes, so there was no risk). As a result, I have seen that the colour of the notes in that folder changed in the graph (to that of missing notes I think) (perfect!), and when I clicked on such a note there was a message saying “Folder already exists.”, but apprently nothing else happend. Most importantly Obsidian did not create new note or folder (perfect!). Then when I moved that folder back into its place everything seemed normal (perfect!).
Could you confirm that this method of temporarily hiding/removing folders from the Vault makes no harm, whatsoever? Because this would be a perfect workaround solution to my two objectives!

4

From a document security, if other people have access to your computer where obsidian is running, every locking mechanism that you suggested will be “fake” (since they can bypass it by accessing the notes using Finder).

Obsidian is an editing tool. If you want to present your notes, I think you will be better served with a publishing tool:

  1. Obsidian Publish
  2. A third party markdown publishing system (something like Jekyll)

    Other approximations:
  3. As you mentioned, filtering.
  4. there is a plugin for encrypting some parts of your notes
  5. Nested vault, if you understand how they work.

Feel free to post “folder locking” in plugin ideas, maybe somebody will pick it up.

5

Thanks again!

The case of my concern is not about document security, or web publishing or public presentation. What I have in mind is when I discuss ideas, tasks, info from my Obsidian notes with other people. I am controlling the app on my computer. (I did this many times with other apps in the past with prepared documents). That is, just showing Obsidian in full-screen mode (with collapsed side panels - very nice for presentation of notes in a seminar btw) on a big screen or projector. (So, your 1st and 2nd suggestion is not relevant in this case.) One objective is to avoid that in the heat of the discussion I click on a link accidentally and display some private info.

As I mentioned already, filtering (your 3rd suggestion) is not “safe” enough becuse if I (accidentally) follow a link to a note in a private folder it will override the filter string, e.g., “-path:private”. That is, it will be shown despite the filter. (I tried it.) I guess it is a correct design decision, but in this case it is not good.

I will definitely check out the encryption option (your 4th suggestion) since it is most probably done within Obsidian, as opposed to just moving out a folder from the Vault (which is in the OS but does not need encryption). These are the most interesting options because both method could serve also the other important objective: temporarily reducing the number of notes in a single step. Can you or somebody comment on wheter simply moving out a folder from the Vault could cause any harm to my Obsidian data? I tried it yesterday, and I have seen no residual effect after putting the folder back in its place. After trying these two methods (folder encryption and folder removal) I will decide wether I need Vault nesting or suggest a plug-in for folder locking/hiding.

6
  1. Using separate vaults for different purposes. For example, I have an (almost) empty vault for Obsidian bug reporting with neither 3rd-party plugins nor themes.
  2. Keeping a healthy way organizing your folder structure in your file system, and reuse this structure within your vault or under a certain depth in your vault.
  3. Base on 2, you may find a harmonious way to merge different vaults, so that the note files can be logically connected within Obsidian in the merged new vault and at the same time the files themselves kept separated and self organized within file system (by maintaining their relative paths in file system)
7

Thank you GLight! For some decades I am having been having many projects folders and plaintext log files, and also a big “everything” log file (that takes several years of notes - see the amount of notes above). Too many of my notes are related to more than one project/topic (sometimes to 5-10). So, I was/am doing copy pasting these notes to project-specific logs. From now on, I don’t want my information fragmented in several folders (because exactly that is the problem with my current note management). I want the opposite: I want to be able to link notes freely, so that I will not need to copy-paste anymore. Hence I am here. Separating notes into completely independent Vaults is not an option for me because it will not address the current main problem. Locking/hiding folders temporarily, would help in two aspects:

  1. reduce the amount of notes and gain speed (when I know that I don’t need a folder). Basically, I could decide with projects/folders I need for the next session and lock/hide the others.
  2. Locking/hiding specific information (folders) that is not relevant/desirable to a discussion with other folks using my Obsidian notes.

Possible approaches:

  1. I should try nested Vaults but I am a bit worried about the developers strongly advising against it. Otherwise this approach would have some certain advantages. (Differnet working environment in different sub-vaults; common information could be on the root level, for two). But even with nested Vaults, folder locking/hiding would make sense in my view.
  2. Folder encryption (mentioned above). I don’t see it among the standard plug-ins and I have not yet turned off Safe mode, but I may try it.
  3. Removing selected folders from the Vault before starting Obsidien (and move them back when you will need the before starting Obsidian next time). This is what I am doing now and it worked so far, I have seen no problem. When I click on a note, which is in the removed folder, I get an error message “Folder already exists.”, which is not the right message, but that is OK. It is nice that notes that are in the removed folder have different colour in the graph. However, it would be nice to have a confirmation from the developers that I can make no harm to my Obsidian data if I temporary remove some folders from the Vault.
  4. Suggesting a community plug-in for folder locking/hiding inside a Vault in an intelligent way.
8

Thank you for the reply and further explanation.

I think the philosophy behind the Vault in Obsidian is that:

  1. the notes stored within the vault are mean to be logically connected.

  2. the notes stored in different vaults are mean to be not logically connected.

This is the fundamental rule, or working assumption of Obsidian.

By using vaults in vault(nested vaults) or by maintaining the files in vault via the file system, one can achieve something, looks like, beyond this working assumption, such as “hiding” the logical connections. However, all these methods are, actually, based on the very work assumption aforesaid.

And all these method have trade-offs, some may break the note system. For example, the problems of using Finder (file system tool) is discussed at: Using the Finder (or other file explorer) with Obsidian files? Problems?

IMHO, rather than dancing on the edge of the cliff, i mean, using some fancy tools but is unstable or even risky, I think we can have a review of our entire work flow, to figure out which part can be improved by the use of Obsidian. For those parts that violates the working assumption of Obsidian, the Obsidian may not be the best tool for them.

I know this sounds frustrating, especially for the people who had been struggling and looking for an ultimate tools for solving all problems on PKM for a long time. Obsidian is a very good tool, but it has limitation.

As the developers of Obsidian still sharpen it, we can expect some good features in the future. But right now, my suggestion is either upgrade the work flow to “adapt” Obsidian or use Obsidian only for what it is best for.

9

And I used to worry about the size of the vault may affect the performance, because I was planing to put all notes in one vault.

But the result shows that this worry is not necessary.

And I also realized, only for my case, connecting all notes may look fantastic, but it will actually distract my attention. The Cons is larger than the Pros. And I don’t have to put all the notes in one vault at all.

But based on what principle to split the vault is a very good topic to study. I used to read the Chinese version of a book named 「結果を出す人」はノートに何を書いているのか. It shows a very simple method: using both individual note book and the Carrier Notebook for different purposes. The former is for working in hand, the later is mainly for information. And the individual note book will be merged into the Carrier Notebook when the job in hand is done.

Hope this will be helpful.

10

Oh, one more thing, :slight_smile:

the search function in Obsidian is very powerful. I think it is very useful and flexible to help one to find what he wants. And hiding the unrelated concerns may be achieved by using search function, I guess.

11

Thank you for further comments and the links (which I did read). You are talking about general workflows and concepts and I am talking about very specific methods. I think I could not answer without repeating myself on few points you raised, so I won’t do that. My intention is not to rethink and adapt my worklow to any specific sw but find and test sw that will best serve my planned workflow. Since I am a researcher, I will experiment further (also with Obsidian) :slight_smile:

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.