Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Partial Downloads Attempted to be Imported #61

Open
lvleph opened this issue Dec 28, 2024 · 4 comments
Open

Partial Downloads Attempted to be Imported #61

lvleph opened this issue Dec 28, 2024 · 4 comments

Comments

@lvleph
Copy link
Contributor

lvleph commented Dec 28, 2024 •
edited
Loading

I have been seeing an issue where an album is not yet completed, but soularr seems to think it is. It then tries to import a partial album, which fails. The partial album is placed in failed_imports then the rest of the album downloads, another album completes, and then the second part of the original album is attempted to be imported, but fails and is placed in failed_imports. The logs don't really say much:

  [INFO|soularr|L402] 2024-12-28T15:41:46+0100: Username: jeffycool Directory: @@pjcpd\music\complete\a-pasta-sea\Never Mind the Bollocks Here’s the Sex Pistols
  [INFO|soularr|L404] 2024-12-28T15:41:46+0100: -------------------
  [INFO|soularr|L405] 2024-12-28T15:41:46+0100: Waiting for downloads... monitor at: http://localhost:5030/downloads
  [ERROR|soularr|L694] 2024-12-28T15:41:46+0100: Traceback (most recent call last): /error      [19/97]
  ▏ File "/app/soularr.py", line 692, in <module>
  ▏ ▏ failed = grab_most_wanted(wanted_records)
  ▏ ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/app/soularr.py", line 413, in grab_most_wanted
  ▏ ▏ downloads = slskd.transfers.get_downloads(username)
  ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/slskd_api/apis/transfers.py", line 112, in get_downloads
  ▏ ▏ response = self.session.get(url)
  ▏ ▏ ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 602, in get
  ▏ ▏ return self.request("GET", url, **kwargs)
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
  ▏ ▏ resp = self.send(prep, **send_kwargs)
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 710, in send
  ▏ ▏ r = dispatch_hook("response", hooks, r, **kwargs)
  ▏ ▏ ▏ ▏ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
} ▏ File "/usr/local/lib/python3.11/site-packages/requests/hooks.py", line 30, in dispatch_hook
  ▏ ▏ _hook_data = hook(hook_data, **kwargs)
  ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/slskd_api/client.py", line 60, in <lambda>
  ▏ ▏ session.hooks = {'response': lambda r, *args, **kwargs: r.raise_for_status()}
  ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏                     ^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
  ▏ ▏ raise HTTPError(http_error_msg, response=self)
  requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost:5030/api/v0/transfers/downloads/crzyhrsmngse

  [ERROR|soularr|L695] 2024-12-28T15:41:46+0100:
   Fatal error! Exiting...
...
[INFO|soularr|L402] 2024-12-28T15:47:47+0100: Username: jeffycool Directory: @@pjcpd\music\complete\a-pasta-sea\Never Mind the Bollocks Here’s the Sex Pistols 
  [INFO|soularr|L404] 2024-12-28T15:47:47+0100: -------------------
  [INFO|soularr|L405] 2024-12-28T15:47:47+0100: Waiting for downloads... monitor at: http://localhost:5030/downloads
  [INFO|soularr|L436] 2024-12-28T15:47:47+0100: All tracks finished downloading!
  [INFO|soularr|L505] 2024-12-28T15:47:52+0100: Starting Lidarr import for: Never Mind the Bollocks Here’s the Sex Pistols ID: 195016
  [INFO|soularr|L520] 2024-12-28T15:47:54+0100: Downloaded Albums Scan Failed to import from: /Downloads/slsk/Never Mind the Bollocks Here’s the Sex Pistols
  [INFO|soularr|L546] 2024-12-28T15:47:54+0100: Failed import moved to: failed_imports/Never Mind the Bollocks Here’s the Sex Pistols
  ...
  [INFO|soularr|L436] 2024-12-28T16:14:57+0100: All tracks finished downloading!
  [INFO|soularr|L505] 2024-12-28T16:15:02+0100: Starting Lidarr import for: Never Mind the Bollocks Here’s the Sex Pistols ID: 195069
  [INFO|soularr|L520] 2024-12-28T16:15:04+0100: Downloaded Albums Scan Failed to import from: /Downloads/slsk/Never Mind the Bollocks Here’s the Sex Pistols
  [INFO|soularr|L546] 2024-12-28T16:15:04+0100: Failed import moved to: failed_imports/Never Mind the Bollocks Here’s the Sex Pistols_1
@lvleph
Copy link
Contributor Author

lvleph commented Dec 28, 2024 •
edited
Loading

I got another case:

  [INFO|soularr|L176] 2024-12-28T16:40:56+0100: Selected release for The Casualties: Official, United States, CD, Mediums: 1, Tracks: 19, ID: 18427
  [INFO|soularr|L349] 2024-12-28T16:40:56+0100: Searching album: The Casualties Until Death: Studio Sessions
  [INFO|soularr|L256] 2024-12-28T16:41:04+0100: Search returned 3 results
  [INFO|soularr|L259] 2024-12-28T16:41:04+0100: Serching for matches with selected attributes: flac 24/192
  [INFO|soularr|L263] 2024-12-28T16:41:04+0100: Parsing result from user: 9@$#VBZfd.Cl1eHpbMu4
  [INFO|soularr|L263] 2024-12-28T16:41:04+0100: Parsing result from user: informatiko
  [INFO|soularr|L263] 2024-12-28T16:41:04+0100: Parsing result from user: BeeDee3
  [INFO|soularr|L259] 2024-12-28T16:41:04+0100: Serching for matches with selected attributes: flac 16/44.1
  [INFO|soularr|L263] 2024-12-28T16:41:04+0100: Parsing result from user: 9@$#VBZfd.Cl1eHpbMu4
  [INFO|soularr|L263] 2024-12-28T16:41:12+0100: Parsing result from user: informatiko
  [INFO|soularr|L263] 2024-12-28T16:41:12+0100: Parsing result from user: BeeDee3
  [INFO|soularr|L259] 2024-12-28T16:41:12+0100: Serching for matches with selected attributes: flac
  [INFO|soularr|L263] 2024-12-28T16:41:12+0100: Parsing result from user: 9@$#VBZfd.Cl1eHpbMu4
  [INFO|soularr|L263] 2024-12-28T16:41:14+0100: Parsing result from user: informatiko
  [INFO|soularr|L263] 2024-12-28T16:41:14+0100: Parsing result from user: BeeDee3
  [ERROR|soularr|L694] 2024-12-28T16:41:14+0100: Traceback (most recent call last):
  [ERROR|soularr|L694] 2024-12-28T16:41:14+0100: Traceback (most recent call last):
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 199, in _new_conn
  ▏ ▏ sock = connection.create_connection(
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
  ▏ ▏ raise err
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
  ▏ ▏ sock.connect(sa)
  ConnectionRefusedError: [Errno 111] Connection refused
}
  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
  ▏ ▏ response = self._make_request(
  ▏ ▏ ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 495, in _make_request
  ▏ ▏ conn.request(
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 441, in request
  ▏ ▏ self.endheaders()
  ▏ File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
  ▏ ▏ self._send_output(message_body, encode_chunked=encode_chunked)
  ▏ File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
  ▏ ▏ self.send(msg)
  ▏ File "/usr/local/lib/python3.11/http/client.py", line 996, in send
  ▏ ▏ self.connect()
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 279, in connect
  ▏ ▏ self.sock = self._new_conn()
  ▏ ▏ ▏ ▏ ▏ ▏ ▏ ▏ ^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 214, in _new_conn
  ▏ ▏ raise NewConnectionError(
  urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8319007f10>: Failed to establish a new connection: [Errno 111] Connection refused

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
  ▏ ▏ resp = conn.urlopen(
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
  ▏ ▏ retries = retries.increment(
  ▏ ▏ ▏ ▏ ▏ ▏ ▏ ^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
  ▏ ▏ raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
  ▏ ▏ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5030): Max retries exceeded with url: /api/v0/searches/210aebc8-c532-11ef-8484-0242ac1d5c02 (Caused by NewConnectionError('<urllib3.c

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
  ▏ File "/app/soularr.py", line 692, in <module>
  ▏ ▏ failed = grab_most_wanted(wanted_records)
  ▏ ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/app/soularr.py", line 350, in grab_most_wanted
  ▏ ▏ success = search_and_download(grab_list, query, all_tracks, all_tracks[0], artist_name, release)
  ▏ ▏ ▏ ▏ ▏ ▏ ▏ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/app/soularr.py", line 312, in search_and_download
  ▏ ▏ slskd.searches.delete(search['id'])
  ▏ File "/usr/local/lib/python3.11/site-packages/slskd_api/apis/searches.py", line 116, in delete
  ▏ ▏ response = self.session.delete(url)
  ▏ ▏ ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 671, in delete
  ▏ ▏ return self.request("DELETE", url, **kwargs)
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
  ▏ ▏ resp = self.send(prep, **send_kwargs)
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
  ▏ ▏ r = adapter.send(request, **kwargs)
  ▏ ▏ ▏ ▏ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/slskd_api/client.py", line 32, in send
  ▏ ▏ return super().send(*args, **kwargs)
  ▏ ▏ ▏ ▏ ▏  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ▏ File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 700, in send
  ▏ ▏ raise ConnectionError(e, request=request)
  requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5030): Max retries exceeded with url: /api/v0/searches/210aebc8-c532-11ef-8484-0242ac1d5c02 (Caused by NewConnectionError('<urllib
  
[ERROR|soularr|L695] 2024-12-28T16:41:14+0100:
   Fatal error! Exiting... /Fatal error! Exiting        [2/2]
...
  [INFO|soularr|L402] 2024-12-28T16:47:55+0100: Username: 9@$#VBZfd.Cl1eHpbMu4 Directory: ExtMusic\Casualties, The\2019 - The Casualties - Until Death: Studio Sessions /Casualties        [3/9]
  [INFO|soularr|L404] 2024-12-28T16:47:55+0100: -------------------
  [INFO|soularr|L405] 2024-12-28T16:47:55+0100: Waiting for downloads... monitor at: http://localhost:5030/downloads
  [INFO|soularr|L436] 2024-12-28T16:47:55+0100: All tracks finished downloading!
  [INFO|soularr|L505] 2024-12-28T16:48:00+0100: Starting Lidarr import for: Ray Charles - 1994-The Very Best Of @FLAC ID: 195130
  [INFO|soularr|L505] 2024-12-28T16:48:00+0100: Starting Lidarr import for: 2019 - The Casualties - Until Death: Studio Sessions ID: 195131
  [INFO|soularr|L520] 2024-12-28T16:48:02+0100: Downloaded Albums Scan Failed to import from: /Downloads/slsk/Ray Charles - 1994-The Very Best Of @FLAC
  [INFO|soularr|L546] 2024-12-28T16:48:02+0100: Failed import moved to: failed_imports/Ray Charles - 1994-The Very Best Of @FLAC
  [INFO|soularr|L520] 2024-12-28T16:48:02+0100: Downloaded Albums Scan Failed to import from: /Downloads/slsk/2019 - The Casualties - Until Death: Studio Sessions
  [INFO|soularr|L546] 2024-12-28T16:48:02+0100: Failed import moved to: failed_imports/2019 - The Casualties - Until Death: Studio Sessions
  [INFO|soularr|L707] 2024-12-28T16:48:02+0100: 6: releases failed while downloading and are still wanted.
...

@lvleph
Copy link
Contributor Author

lvleph commented Dec 28, 2024

So from the logs it seems like there is a connection error that leads to an unhandled exception and so soularr crashes.

@EricH9958
Copy link

I have been having the same problem. I then have to go into the failed imports folder and stitch them together. Maybe a delay to ensure all downloads are complete before the import, or a query that all downloads are complete before the import begins.

@mrusse
Copy link
Owner

mrusse commented Dec 28, 2024

Looking at the logs above it seems that the connection to your slskd container is dropping mid way through a run so the albums don't fully download. Then on the next run it tries to import the partial downloads since they are in your downloads folder still. Not sure why your connection is dropping but I will try to catch that error. Although not much can be done if your connection to Slskd drops.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@EricH9958 @mrusse @lvleph