PR: Add remote filesystem API to the Remote client plugin #23381

hlouzada · 2024-12-30T23:47:31Z

Description of Changes

Wrote at least one-line docstrings (for any new functions)
Added unit test(s) covering the changes (if testable)

Use a mixture of rest/websocket to comunicate with spyder-remote-services and provide a filesystem-like API to control remote file system.

This implements the client-side for spyder-ide/spyder-remote-services#11

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @hlouzada

subrepo: subdir: "external-deps/spyder-kernels" merged: "d7e4319b5" upstream: origin: "https://github.com/spyder-ide/spyder-kernels.git" branch: "master" commit: "d7e4319b5" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

pep8speaks · 2024-12-30T23:47:37Z

Hello @hlouzada! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file external-deps/spyder-remote-services/spyder_remote_services/app.py:

Line 115:80: E501 line too long (80 > 79 characters)

In the file external-deps/spyder-remote-services/spyder_remote_services/services/__init__.py:

Line 2:80: E501 line too long (85 > 79 characters)

In the file external-deps/spyder-remote-services/spyder_remote_services/services/files/base.py:

Line 28:80: E501 line too long (83 > 79 characters)
Line 36:80: E501 line too long (103 > 79 characters)
Line 394:80: E501 line too long (91 > 79 characters)
Line 427:65: E252 missing whitespace around parameter equals
Line 427:66: E252 missing whitespace around parameter equals

In the file external-deps/spyder-remote-services/spyder_remote_services/services/files/handlers.py:

Line 49:80: E501 line too long (100 > 79 characters)
Line 70:80: E501 line too long (86 > 79 characters)
Line 134:80: E501 line too long (88 > 79 characters)
Line 136:80: E501 line too long (86 > 79 characters)
Line 152:80: E501 line too long (81 > 79 characters)

In the file spyder/plugins/remoteclient/tests/test_files.py:

Line 114:80: E501 line too long (85 > 79 characters)

Comment last updated at 2025-01-23 21:32:18 UTC

…xternal-deps/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "8d51ee08a" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "feat/remote-file-service" commit: "8d51ee08a" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

…erAPIs

…file-service

…xternal-deps/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "e2f920d11" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "feat/remote-file-service" commit: "e2f920d11" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

…xternal-deps/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "83bfb48fd" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "feat/remote-file-service" commit: "83bfb48fd" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

spyder/plugins/remoteclient/api/modules/base.py

…xternal-deps/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "b77bdbfb2" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "feat/remote-file-service" commit: "b77bdbfb2" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

…xternal-deps/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "4599cda25" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "feat/remote-file-service" commit: "4599cda25" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

…xternal-deps/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "bb7624869" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "feat/remote-file-service" commit: "bb7624869" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

…/spyder-remote-services subrepo: subdir: "external-deps/spyder-remote-services" merged: "2b8dcf2aa" upstream: origin: "https://github.com/spyder-ide/spyder-remote-services" branch: "main" commit: "2b8dcf2aa" git-subrepo: version: "0.4.9" origin: "https://github.com/ingydotnet/git-subrepo" commit: "cce3d93"

ccordoba12

Thanks @hlouzada for your hard work to implement this functionality!

ccordoba12 · 2025-01-23T23:47:09Z

spyder/api/asyncdispatcher.py

+        early_return : bool, optional
+            Return the coroutine as a Future object before it is done
+            or wait for it to finish and return the result.
+        return_awaitable : bool, optional
+            Return the coroutine as an awaitable object instead of a Future.


Please document these new kwargs in our Changelog (see previous API changes sections to get an idea about that).

ccordoba12 · 2025-01-23T23:49:03Z

spyder/api/utils.py

+    NotImplementedError: Can't instantiate abstract class
+                         with abstract attributes.


Suggested change

NotImplementedError: Can't instantiate abstract class

with abstract attributes.

NotImplementedError

When it's not possible to instantiate an abstract class with abstract

attributes.

ccordoba12 · 2025-01-23T23:49:51Z

spyder/plugins/remoteclient/api/__init__.py

+
+
+class SpyderRemoteAPIManager:
+    """Class to manage a remote server and its apis."""


Suggested change

"""Class to manage a remote server and its apis."""

"""Class to manage a remote server and its APIs."""

ccordoba12 · 2025-01-23T23:50:17Z

spyder/plugins/remoteclient/api/__init__.py

+    GET_SERVER_INFO_COMMAND = (f"/${{HOME}}/.local/bin/micromamba run"
+                               f" -n {SERVER_ENV} spyder-server info")


Suggested change

GET_SERVER_INFO_COMMAND = (f"/${{HOME}}/.local/bin/micromamba run"

f" -n {SERVER_ENV} spyder-server info")

GET_SERVER_INFO_COMMAND = (

f"/${{HOME}}/.local/bin/micromamba run"

f" -n {SERVER_ENV} spyder-server info"

)

ccordoba12 · 2025-01-23T23:51:28Z

spyder/plugins/remoteclient/api/__init__.py

+
+    @property
+    def server_url(self):
+        """Return server URL


Suggested change

"""Return server URL

"""

Return server URL

ccordoba12 · 2025-01-24T03:56:07Z

spyder/plugins/remoteclient/tests/test_files.py

+            }
+
+    @AsyncDispatcher.dispatch(early_return=False)
+    async def test_list_dir(


This test and test_list_directories below are very similar. The latter seems more complete though, so why not remove this one?

ccordoba12 · 2025-01-24T03:58:38Z

spyder/plugins/remoteclient/tests/test_files.py

+        async with file_api_class() as file_api:
+            assert await file_api.rmdir(self.remote_temp_dir) == {
+                "success": True
+            }


I think you should also check that the directory is not present anymore with file_api.ls for completeness.

ccordoba12 · 2025-01-24T03:59:27Z

spyder/plugins/remoteclient/utils/installation.py

+SCRIPT_URL = ("https://raw.githubusercontent.com/spyder-ide/"
+              f"{PACKAGE_NAME}/master/scripts")


Suggested change

SCRIPT_URL = ("https://raw.githubusercontent.com/spyder-ide/"

f"{PACKAGE_NAME}/master/scripts")

SCRIPT_URL = (

f"https://raw.githubusercontent.com/spyder-ide/"

f"{PACKAGE_NAME}/master/scripts"

)

This is an easier to read formatting.

ccordoba12 · 2025-01-24T04:03:12Z

spyder/plugins/remoteclient/widgets/connectiondialog.py

This file doesn't contain any change related to your work, so you shouldn't have touched it.

Please edit the commit in which you introduced these changes and revert them. In other words, don't add another commit to do that (otherwise you'll break git blame for the rest of the team).

ccordoba12 · 2025-01-24T04:04:09Z

spyder/plugins/remoteclient/widgets/connectionstatus.py

The same comment I left for spyder/plugins/remoteclient/widgets/connectiondialog.py applies to this one: please edit your git history to revert your changes.

hlouzada added 3 commits December 30, 2024 20:22

feat: add fsspek client-side backend for remote files service

9020d1a

feat: add fsspec dependency

39a17e3

hlouzada changed the title ~~Add remote-files capability to remote-client plugin~~ [WIP] PR: Add remote-files capability to remote-client plugin Dec 30, 2024

hlouzada added 17 commits January 9, 2025 16:35

feat: refactor jupyter's rest api for plugin support

737300c

feat: add rest plugin for spyder remote files api

045c53e

feat: add custom metaclass for not implemented attributes

6db9945

fix: also handle oserror on remotefileservices api

7036b08

feat: inherit FileIO for remote file io api

30efa81

fix: standarize os methods name and add shutil common functions

f7074bd

fix: set jupyterpluginbaseapi context as optional

9388da9

fiix: handle sessions and websockets contexts properly

c021698

del: remove unused fsspec dependency

afb723b

refac: move remote plugins as modules and add public api registers

0b26e70

feat: bump version

dd3223c

feat: add support to await dispatched coroutine

be91fca

fix: don't get attribute value on checking for abstract attributes

8ce21de

fix: refactor client api and pass manager directly to SpyderBaseJupyt…

38b2581

…erAPIs

fix: ensure that the server is connected on remote client base api

55030dd

feat: add tests for files services

e8c8ef4

hlouzada marked this pull request as ready for review January 18, 2025 13:58

hlouzada changed the title ~~[WIP] PR: Add remote-files capability to remote-client plugin~~ PR: Add remote-files capability to remote-client plugin Jan 18, 2025

hlouzada added 6 commits January 18, 2025 11:00

Merge remote-tracking branch 'upstream/master' into add-remoteclient-…

7ec8dc4

…file-service

fix: update monkeypatch from refactored client api

167808a

fix: use AsyncDispatcher to call async tests

d4e60e6

fix: implement _raise_for_status to SpyderRemoteFileIOAPI

514691e

ccordoba12 reviewed Jan 18, 2025

View reviewed changes

spyder/plugins/remoteclient/api/modules/base.py Show resolved Hide resolved

hlouzada added 8 commits January 18, 2025 20:23

fix: add __init__.py to api modules

12b8d84

fix: use correct endpoint for file services

012f766

fix: set minimun version back to 1.0.0

1d7a6d2

refac: apply black to remoteclient

a4e0d2e

ccordoba12 assigned hlouzada Jan 22, 2025

ccordoba12 added type:Enhancement component:Remote client labels Jan 22, 2025

ccordoba12 added this to the v6.1.0 milestone Jan 22, 2025

ccordoba12 changed the title ~~PR: Add remote-files capability to remote-client plugin~~ PR: Add remote filesystem API to the Remote client plugin Jan 22, 2025

hlouzada added 2 commits January 22, 2025 13:26

refac: fix line length with black

5d8d553

refac: fix pep8

ab22af6

hlouzada force-pushed the add-remoteclient-file-service branch from 91ae29a to ab22af6 Compare January 23, 2025 13:40

fix: test copy file considering random order of ls contents

257d314

ccordoba12 reviewed Jan 24, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PR: Add remote filesystem API to the Remote client plugin #23381

PR: Add remote filesystem API to the Remote client plugin #23381

hlouzada commented Dec 30, 2024 •

edited by ccordoba12

Loading

pep8speaks commented Dec 30, 2024 •

edited

Loading

ccordoba12 left a comment

ccordoba12 Jan 23, 2025 •

edited

Loading

ccordoba12 Jan 23, 2025

ccordoba12 Jan 23, 2025

ccordoba12 Jan 23, 2025

ccordoba12 Jan 23, 2025

ccordoba12 Jan 24, 2025

ccordoba12 Jan 24, 2025

ccordoba12 Jan 24, 2025

ccordoba12 Jan 24, 2025

ccordoba12 Jan 24, 2025

		NotImplementedError: Can't instantiate abstract class
		with abstract attributes.



		class SpyderRemoteAPIManager:
		"""Class to manage a remote server and its apis."""

		GET_SERVER_INFO_COMMAND = (f"/${{HOME}}/.local/bin/micromamba run"
		f" -n {SERVER_ENV} spyder-server info")

		SCRIPT_URL = ("https://raw.githubusercontent.com/spyder-ide/"
		f"{PACKAGE_NAME}/master/scripts")

PR: Add remote filesystem API to the Remote client plugin #23381

Are you sure you want to change the base?

PR: Add remote filesystem API to the Remote client plugin #23381

Conversation

hlouzada commented Dec 30, 2024 • edited by ccordoba12 Loading

Description of Changes

Affirmation

pep8speaks commented Dec 30, 2024 • edited Loading

Comment last updated at 2025-01-23 21:32:18 UTC

ccordoba12 left a comment

Choose a reason for hiding this comment

ccordoba12 Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hlouzada commented Dec 30, 2024 •

edited by ccordoba12

Loading

pep8speaks commented Dec 30, 2024 •

edited

Loading

ccordoba12 Jan 23, 2025 •

edited

Loading