Impossible to add repos

[marcelb98]

Description:
I did a fresh installation of Augur (following the guide) and tried to add GitHub Repositories using the Web UI as well as the CLI. This fails with an exception.

How to reproduce:

1. Checkout current master from git, install augur
2. Start augur (augur backend start)
3. Prepare repos.txt:

   $cat ~/repos.txt
   https://github.com/cmatsuoka/asciiquarium,1
   $

4. try to add a repository: augur db add-repos ~/repos.txt, this will fail with the error mentioned below
5. Stop augur, checkout v0.76.2 in git, start augur again
6. try to add a repository: augur db add-repos ~/repos.txt, this will now succeed:

   # augur db add-repos ~/repos.txt 
   Inserting repo 0/1 with Git URL `https://github.com/cmatsuoka/asciiquarium` into repo group 1
   Success

Expected behavior:
After adding a repo it should be listed in the Augur Web UI.
With version v0.76.2 this works, with the current 0.76.6 not.
Following the error message, this might be introduced in #2929.

Log files
Exception printed to stdout when trying to add repos with augur 0.76.6:

Traceback (most recent call last):
  File "/opt/augur_new/venv/bin/augur", line 33, in <module>                                                                                                                                
    sys.exit(load_entry_point('augur', 'console_scripts', 'augur')())                                                                                                                       
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 1128, in __call__                                                                                             
    return self.main(*args, **kwargs)                                                                                                                                                       
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 1053, in main                                                                                                 
    rv = self.invoke(ctx)                                                                                                                                                                   
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 1659, in invoke                                                                                               
    return _process_result(sub_ctx.command.invoke(sub_ctx))                                                                                                                                 
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 1659, in invoke                                                                                               
    return _process_result(sub_ctx.command.invoke(sub_ctx))                                                                                                                                 
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 1395, in invoke                                                                                               
    return ctx.invoke(self.callback, **ctx.params)                                                                                                                                          
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 754, in invoke                                                                                                
    return __callback(*args, **kwargs)                                                                                                                                                      
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func                                                                                         
    return f(get_current_context(), *args, **kwargs)                                                                                                                                        
  File "/opt/augur_new/augur/application/cli/__init__.py", line 24, in new_func                                                                                                             
    return ctx.invoke(function_internet_connection, *args, **kwargs)                                                                                                                        
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 754, in invoke                                                                                                
    return __callback(*args, **kwargs)
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/augur_new/augur/application/cli/__init__.py", line 47, in new_func
    return ctx.invoke(function_db_connection, *args, **kwargs)
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/augur_new/augur/application/cli/__init__.py", line 95, in new_func
    return ctx.invoke(f, *args, **kwargs)
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/augur_new/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/augur_new/augur/application/cli/db.py", line 76, in add_repos
    succeeded, message = controller.add_cli_repo(repo_data)
  File "/opt/augur_new/augur/util/repo_load_controller.py", line 71, in add_cli_repo
    repo_id = Repo.insert_github_repo(self.session, url, repo_group_id, "CLI", repo_type)
TypeError: Repo.insert_github_repo() missing 1 required positional argument: 'repo_src_id'

Other logs are empty:

/opt/augur_new$ ls -l logs/
total 0
-rw-r--r-- 1 root root 0 Jan  6 10:51 augur.error
-rw-r--r-- 1 root root 0 Jan  6 10:51 augur.info

Software versions:

• Augur: 0.76.6
• OS: Ubuntu 22.04.5 LTS

[Ulincsys]

Hello there!

Could you provide the error message (if any) you received when attempting to add repos via the web UI? Please also provide the steps to reproduce so we can better locate the issue in the frontend.

As for the CLI, we will publish a hotfix soon to update it. It seems as though that interface was overlooked during a recent internal protocol revision.

[marcelb98]

Thanks for the CLI fix!

For the WebUI I've created an account, after login go to ”My Repos” and add a repo:

After clicking the Add button, it seems like the repo was added:

The process running augur backend start prints at the same time: Adding user repos.

I would expect that after some time the number of repos in the ”Your Repo Group” table increases. But this is not the case (also after waiting some hours).

In the log files I can find:

augur.info

[7713] augur [INFO] awaiting Gunicorn start
[7713] augur [INFO] Gunicorn webserver started...
[7713] augur [INFO] Augur is running at: https://0.0.0.0:5000
[7713] augur [INFO] The API is available at 'api/unstable'
[7713] augur [INFO] Starting core worker processes with concurrency=1
[7713] augur [INFO] Starting secondary worker processes with concurrency=1
[7713] augur [INFO] Starting facade worker processes with concurrency=1

augur_view.error

2025-01-08 10:30:06,060 [PID: 7758] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,061 [PID: 7758] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,062 [PID: 7762] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,062 [PID: 7762] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,060 [PID: 7761] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,061 [PID: 7759] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,063 [PID: 7761] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,061 [PID: 7760] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,063 [PID: 7760] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,060 [PID: 7763] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,063 [PID: 7759] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,066 [PID: 7761] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,066 [PID: 7761] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,064 [PID: 7763] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,068 [PID: 7758] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,068 [PID: 7758] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,068 [PID: 7762] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,069 [PID: 7762] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,066 [PID: 7760] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,069 [PID: 7760] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,073 [PID: 7763] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,073 [PID: 7763] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable
2025-01-08 10:30:06,074 [PID: 7759] augur_view [loadReports() in utils.py:L86] [ERROR]: An exception occurred reading reports endpoints from [reports.yml]:
2025-01-08 10:30:06,075 [PID: 7759] augur_view [loadReports() in utils.py:L87] [ERROR]: 'NoneType' object is not iterable

augur_view.info

[7758] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7758] augur_view [ERROR] 'NoneType' object is not iterable                                                                   
[7762] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7758] augur_view [INFO] Attempting to generate default reports.yml
[7762] augur_view [ERROR] 'NoneType' object is not iterable                                                                   
[7761] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7759] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7761] augur_view [ERROR] 'NoneType' object is not iterable                                                                   
[7760] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7762] augur_view [INFO] Attempting to generate default reports.yml
[7760] augur_view [ERROR] 'NoneType' object is not iterable                                                                   
[7760] augur_view [INFO] Attempting to generate default reports.yml  
[7763] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7761] augur_view [INFO] Attempting to generate default reports.yml
[7758] augur_view [INFO] Default reports file successfully generated.
[7760] augur_view [INFO] Default reports file successfully generated.
[7761] augur_view [INFO] Default reports file successfully generated.
[7759] augur_view [ERROR] 'NoneType' object is not iterable
[7763] augur_view [ERROR] 'NoneType' object is not iterable
[7761] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7761] augur_view [ERROR] 'NoneType' object is not iterable
[7759] augur_view [INFO] Attempting to generate default reports.yml
[7761] augur_view [INFO] Attempting to generate default reports.yml
[7761] augur_view [INFO] Default reports file successfully generated.
[7758] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7758] augur_view [ERROR] 'NoneType' object is not iterable
[7762] augur_view [INFO] Default reports file successfully generated.
[7758] augur_view [INFO] Attempting to generate default reports.yml
[7762] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7758] augur_view [INFO] Default reports file successfully generated.
[7762] augur_view [ERROR] 'NoneType' object is not iterable
[7762] augur_view [INFO] Attempting to generate default reports.yml
[7760] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7762] augur_view [INFO] Default reports file successfully generated.
[7760] augur_view [ERROR] 'NoneType' object is not iterable
[7763] augur_view [INFO] Attempting to generate default reports.yml
[7760] augur_view [INFO] Attempting to generate default reports.yml
[7760] augur_view [INFO] Default reports file successfully generated.
[7763] augur_view [INFO] Default reports file successfully generated.
[7759] augur_view [INFO] Default reports file successfully generated.
[7763] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7763] augur_view [ERROR] 'NoneType' object is not iterable
[7763] augur_view [INFO] Attempting to generate default reports.yml
[7763] augur_view [INFO] Default reports file successfully generated.
[7759] augur_view [ERROR] An exception occurred reading reports endpoints from [reports.yml]:
[7759] augur_view [ERROR] 'NoneType' object is not iterable
[7759] augur_view [INFO] Attempting to generate default reports.yml
[7759] augur_view [INFO] Default reports file successfully generated.

gunicorn.log is only the HTTP access log.
augur.error, server.error and server.info are empty.

[sgoggins]

Hi @marcelb98 : What you are observing there is Augur starting the frontend when it has not collected any data yet. Without data, our API endpoints are not interpreted beautifully ... as you can see. This should stop happening as data is collected.