Dynamic use while rendering Markdown pages

[pawamoy]

Hey @oprypin!

I'm working on a simple Markdown extension that allows to execute code blocks and inject their output in the page.
I'm building a small gallery of examples, and am currently trying to write a diagram example:

```python exec="true" show_source="tabbed-right"
import tempfile
from contextlib import suppress
import mkdocs_gen_files
from diagrams import Diagram, setdiagram
from diagrams.k8s.clusterconfig import HPA
from diagrams.k8s.compute import Deployment, Pod, ReplicaSet
from diagrams.k8s.network import Ingress, Service

with suppress(FileNotFoundError):
    with Diagram("Exposed Pod with 3 Replicas", show=False) as diagram:
        diagram.render = lambda: None
        net = Ingress("domain.com") >> Service("svc")
        net >> [Pod("pod1"),
                Pod("pod2"),
                Pod("pod3")] << ReplicaSet("rs") << Deployment("dp") << HPA("hpa")

        with mkdocs_gen_files.open(f"img/{diagram.filename}.png", "wb") as fd:
            fd.write(diagram.dot.pipe(format="png"))

output_html(f'<img src="../img/{diagram.filename}.png" />')
```

The issue is that, by the time this code is executed, FilesEditor._current is None again, so opening a file with mkdocs_gen_files.open triggers the creation of another instance of FilesEditor without a temporary directory attached. It means that files I create this way end up in my docs dir, which is unwanted.

So I attach a temporary directory myself:

        editor = mkdocs_gen_files.editor.FilesEditor.current()
        with tempfile.TemporaryDirectory(prefix="mkdocs_gen_files_") as tmp_dir:
            editor.directory = tmp_dir
            with editor.open(f"img/{diagram.filename}.png", "wb") as fd:
                fd.write(diagram.dot.pipe(format="png"))

But then it seems the file is not added to the final site because I'm getting a 404 😕

[pawamoy]

Ah, maybe the temporary file is deleted too early.
EDIT: nope, still 404 without using the temp dir as context manager.

[oprypin]

The clear issue here is that mkdocs_gen_files runs within the on_files event. But Markdown rendering is clearly outside of that event.

mkdocs-gen-files/mkdocs_gen_files/plugin.py

Lines 27 to 30 in 4d5d4ac

	def on_files(self, files: Files, config: Config) -> Files:
	self._dir = tempfile.TemporaryDirectory(prefix="mkdocs_gen_files_")
	with editor.FilesEditor(files, config, self._dir.name) as ed:

https://github.com/mkdocs/mkdocs/blob/dc35569ade5e101c8c48b78bbed8708dd0b1b49a/mkdocs/commands/build.py#L174

In terms of a solution, I don't have any suggestions yet.

[oprypin]

https://oprypin.github.io/mkdocs/dev-guide/plugins/#events

Ah and Markdown rendering is all done between on_page_markdown and on_page_content, and by that time at least the existence of all files is final. Well, it needs to be strictly final only for Markdown files (as we're iterating over them at that time!), other kinds of files might still be able to be added without bad consequences.
But, mkdocs_gen_files only modifies the set of files within the on_files event (by returning a new object as per the only official interface), and I'm not sure if mutating files is viable at later stages. Needs more thought....

[pawamoy]

Thanks, I get it now. That seems indeed a bit bold to modify files at such a stage of the build. And if you provide a way to do it, I guess users will try to generate Markdown files that way 😅 ... I wonder if there's a way for me to obtain and keep a reference to the Files instance...

[pawamoy]

There was a much simpler solution, which is to encode the PNG image as base64 and include it in the HTML with f'<img src="data:image/png;base64, {png}"/>'. Feel free to reopen 🙂

[pawamoy]

I'm reopening, because now I'm generate mp4 files, and inlining them in the HTML, base64 encoded, is not very efficient. It would be better to use an URL instead of base64 data, for smaller loading times and better browser perfs.