What is "python-markdown" and how to install it?

[TomLXXVI]

What I have done...

I have installed Jupterlab 4.0.10 in a virtual environment (managed by Poetry). I have installed the latest version of jupyterlab-lsp (5.0.2) and the latest version of python-lsp-server (1.9.0) with all extras, exactly in the way as it is described in the README file of the repository.

When jupyterlab has been launched and a notebook has been opened, a label appears in the status bar at the bottom of the window with the text "Initialized (additional servers needed)". When I click on this label a popup opens, saying that python-lsp-server is running, but one LSP server is missing, referring to markdown. When I hover over the label in the status bar the following text appears: "1/2 virtual documents connected (1 connections; waiting for: python-markdown).

What I am trying to do...

I want to install the additional server python-markdown.

How I would like to learn how to do it...

Since I have no clue where to look for this missing server python-markdown and since I can't find any clear answer after browsing for more than one hour in the docs about jupyterlab-lsp and also further on the web, I would like to ask this question to the developers: "Where can I get the LSP server python-markdown and how do I need to install this?"

[krassowski]

There is no python-markdown server, I but apparently the UI might be confusing there. More likely it is suggesting installation of a Markdown server. Can you post a screenshot?

[TomLXXVI]

[krassowski]

There above it says that Markdown is missing ;) It says python-markdown in the tooltip because a markdown document is embedded in a Python notebook.

Though I guess the statusbar item should be less verbose.

[TomLXXVI]

In earlier versions there was no message saying that a markdown server was missing. Also, now the python lsp server complains that there are too many blank lines between code lines. I suppose this is because it doesn't see the markdown cells?

[krassowski]

In earlier versions there was no message saying that a markdown server was missing

Oh yeah, this is a new feature since JupyterLab 4+ compatible versions. Previously there was no support for Markdown cells. Now you can get syntax errors, grammar checks etc when you have a suitable server installed.

Also, now the python lsp server complains that there are too many blank lines between code lines. I suppose this is because it doesn't see the markdown cells?

No, that would be unrelated. Can you post a specific example?

[TomLXXVI]

The first line of each code cell in my notebooks is now underlined with a curly line. When hovering over it, the message appears that there are too many blank lines. See this screenshot:

[grzegorz700]

The first line of each code cell in my notebooks is now underlined with a curly line. When hovering over it, the message appears that there are too many blank lines. See this screenshot:

The above error is discussed in the other issue, #1054 . Very irritating code highlighting.

[echarles]

Now you can get syntax errors, grammar checks etc when you have a suitable server installed.

Is there such a LSP server for markdown? On https://jupyterlab-lsp.readthedocs.io/en/latest/Language%20Servers.html it is suggested to use unified-language-server, installable as dev dependcy with npm/yarn. This would mean a rebuild of jupyterlab, right? Any other solution (the ideal would be a python markdown lsp sever).

[echarles]

@krassowski Just read on the doc

If you wish to install these someplace else, you may need to specify where you install them with extra_node_roots.

I have added that trait to my server --LanguageServerManagerAPI.extra_node_roots=<absolute_path> with various variant of the absolute path (the folder where node_module is, the node_module folder, the unified-language-server folder) without luck.

[krassowski]

Is there such a LSP server for markdown?

@echarles yes, unified-language-server works for markdown. There are probably other Markdown language servers, we should have a look to see if there is a good one which does not require Node.js (there was not one back when I looked few years ago).

This would mean a rebuild of jupyterlab, right?

No, this is not required.

various variant of the absolute path (the folder where node_module is, the node_module folder, the unified-language-server folder) without luck.

It might be that a new version of unified-language-server does not work. On CI we are testing "^0.3.0". In general the heuristic search for npm servers is defined here:

jupyterlab-lsp/python_packages/jupyter_lsp/jupyter_lsp/types.py

Lines 216 to 235 in b159ae2

	def find_node_module(self, *path_frag):
	"""look through the node_module roots to find the given node module"""
	all_roots = self.extra_node_roots + self.node_roots
	found = None
	for candidate_root in all_roots:
	candidate = pathlib.Path(candidate_root, "node_modules", *path_frag)
	self.log.debug("Checking for %s", candidate)
	if candidate.exists():
	found = str(candidate)
	break
	if found is None: # pragma: no cover
	self.log.debug(
	"{} not found in node_modules of {}".format(
	pathlib.Path(*path_frag), all_roots
	)
	)
	return found

so running with --debug you should see which paths get checked. For unified-language-server it searches for:

jupyterlab-lsp/python_packages/jupyter_lsp/jupyter_lsp/specs/unified_language_server.py

Lines 4 to 7 in b159ae2

	class UnifiedLanguageServer(NodeModuleSpec):
	node_module = key = "unified-language-server"
	script = ["src", "server.js"]
	args = ["--parser=remark-parse", "--stdio"]

[echarles]

we should have a look to see if there is a good one which does not require Node.js (there was not one back when I looked few years ago).

I quickly looks for python ones but did not find. @fcollonval pointed me to the vscode (nodejs) one https://github.com/microsoft/vscode/tree/main/extensions/markdown-language-features/server which would require additional config as documented on https://jupyterlab-lsp.readthedocs.io/en/latest/Configuring.html#language-servers

. On CI we are testing "^0.3.0". In...
so running with --debug you should see which paths get checked. For unified-language-server it searches for

Pinning and pointing to the folder above the node_modules made it work but clicking on TAB in a markdown cell or document does nothing...

[echarles]

When I click on TAB in a Markdown content, I see the following log, but nothing shows up (btw the LSP autocomplete for python works fine).

[D 2024-05-15 10:49:48.393 ServerApp] [unified-language-server] Handling a message

[krassowski]

When I click on TAB in a Markdown content

Not every language serer implements Tab completion. unified-language-server implements spell checking via linting (textDocument/publishDiagnostics), and two requests which jupyterlab-lsp does not yet support (textDocument/codeAction and textDocument/formatting - although I had code actions running locally) - the list of what they implement is here: https://github.com/unifiedjs/unified-language-server?tab=readme-ov-file#requests

[krassowski]

The VScode one does implement completions but only for links as per https://github.com/microsoft/vscode/tree/main/extensions/markdown-language-features/server#server-capabilities

[echarles]

Thx a lot @krassowski for the explanation. I get indeed spell check working and may try later the vscode one for completion.

[muhac]

It might be that a new version of unified-language-server does not work. On CI we are testing "^0.3.0".

It took me a hard time to figure out that the version should be 0.3.0. The release tags of unified-language-server start from 1.0.0 and none of them work. It is confusing and here is another post on it.

---

It turns out that the newer unified-language-server versions require a wrapper remark-language-server:

class RemarkLanguageServer(NodeModuleSpec):
    node_module = key = "remark-language-server"
    script = ["index.js"]
    args = ["--stdio"]

@krassowski Do you like the newer version to replace the old unified-language-server? Or we can pin the version in the documentation.

[krassowski]

@krassowski Do you like the newer version to replace the old unified-language-server?

Sounds sensible, thank you for looking into this!