Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upograde python from 3.9.5 to 3.13.1 #5533

Open
wants to merge 44 commits into
base: master
Choose a base branch
from

Conversation

gitofanindya
Copy link
Collaborator

@gitofanindya gitofanindya commented Jan 10, 2025

Summary by Sourcery

Upgrade Python from 3.9.5 to 3.12.3.

Enhancements:

  • Update time-related functions to use timezone-aware datetime objects.

Build:

  • Upgrade Python to 3.12.3 in the CI environment.

CI:

  • Update CI workflows to use Python 3.12.

Documentation:

  • Update documentation to reflect Python 3.12 usage.

Tests:

  • Remove a mocked path utility and update tests accordingly.

Copy link
Contributor

sourcery-ai bot commented Jan 10, 2025

Reviewer's Guide by Sourcery

This pull request upgrades the Python version used in the project from 3.9.5 to 3.12.3. Several files were modified to reflect this change, primarily by updating version numbers and resolving compatibility issues related to timezones and database operations.

Class diagram showing changes to AutoRetryCursor class

classDiagram
    class AutoRetryCursor {
        +adapt_datetime_iso(val)
        +execute(sql: str, parameters: Iterable[Any])
    }
    note for AutoRetryCursor "Modified to handle datetime
conversion for SQLite in Python 3.12"
Loading

State diagram showing datetime handling changes

stateDiagram-v2
    [*] --> DatetimeInput
    DatetimeInput --> TimezoneAware: datetime.now(tz=timezone.utc)
    DatetimeInput --> LegacyHandling: datetime.utcnow()
    TimezoneAware --> [*]: New Python 3.12 handling
    LegacyHandling --> [*]: Old Python 3.9 handling
    note right of TimezoneAware: Updated to use timezone-aware
datetime objects
Loading

File-Level Changes

Change Details Files
Upgrade Python version in GitHub Actions workflows
  • Updated Python version from 3.9 to 3.12 in all workflow files.
.github/workflows/functional_tests.yml
.github/workflows/functional_tests_2023.yml
.github/workflows/unit_tests.yml
.github/workflows/dead_code.yml
.github/workflows/integration_tests.yml
.github/workflows/lint.yml
.github/workflows/spell.yml
.github/workflows/style.yml
.github/workflows/translations.yml
.github/workflows/types.yml
Update Python version in documentation and deployment scripts
  • Updated documentation to reflect the Python 3.12.3 version.
  • Modified deployment scripts to use Python 3.12.3.
docs/deployment.md
docs/gnu_linux.md
docs/support.md
tools/posix/deploy_ci_agent.sh
tools/windows/deploy_ci_agent.ps1
Update Python version in requirements files
  • Updated requirements.txt to specify Python 3.12.3 and updated cffi and pywin32 package versions.
  • Removed many unnecessary hashes from requirements.txt.
  • Updated dev and test requirements files to reflect changes in main requirements.
tools/deps/requirements.txt
tools/deps/requirements-dev.txt
tools/deps/requirements-tests.txt
Address timezone and datetime compatibility in code
  • Updated datetime handling to explicitly use timezone-aware UTC timestamps.
  • Corrected potential timezone-related issues in database operations and file information retrieval.
nxdrive/dao/base.py
nxdrive/dao/engine.py
nxdrive/utils.py
nxdrive/client/local/base.py
nxdrive/engine/engine.py
nxdrive/engine/watcher/remote_watcher.py
Remove unused code and update tests
  • Removed unused MockedPath class from tests.
  • Updated test cleanup to handle directory removal.
  • Simplified database cleanup in tests.
tests/unit/test_utils.py
tests/unit/conftest.py
tests/unit/test_proxy.py
Update database dumping and add file metrics tracking
  • Improved database dumping process for better file handling.
  • Added file metrics tracking for direct edit functionality.
nxdrive/dao/utils.py
nxdrive/direct_edit.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time. You can also use
    this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@CLAassistant
Copy link

CLAassistant commented Jan 10, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ poojadaine
✅ gitofanindya
❌ unknown


unknown seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @gitofanindya - I've reviewed your changes - here's some feedback:

Overall Comments:

  • The SQLite datetime adapter modifications in base.py need careful review and testing to ensure no timezone-related regressions. Consider adding explicit tests for this functionality.
  • Remove the 'testing' comment in direct_edit.py
Here's what I looked at during the review
  • 🟡 General issues: 4 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +24 to +25
def adapt_datetime_iso(self, val):
return datetime.fromtimestamp(val.strftime('%s'), tz=timezone.utc)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Using strftime('%s') is platform-dependent and may not work on Windows

Consider using val.timestamp() instead to get a POSIX timestamp that works across all platforms.

Comment on lines +31 to +36
import sqlite3
# return super().execute(sql, parameters)
# new_param = tuple( datetime.fromtimestamp(param, tz=timezone.utc) if isinstance(param, datetime) else param for param in parameters )
new_param = tuple( sqlite3.register_adapter(param, self.adapt_datetime_iso) if isinstance(param, datetime) else param for param in parameters )

return super().execute(sql, new_param)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (performance): The tuple comprehension creates unnecessary intermediate objects for non-datetime parameters

Consider only transforming datetime objects and keeping other parameters as-is to avoid unnecessary tuple creation.

Suggested change
import sqlite3
# return super().execute(sql, parameters)
# new_param = tuple( datetime.fromtimestamp(param, tz=timezone.utc) if isinstance(param, datetime) else param for param in parameters )
new_param = tuple( sqlite3.register_adapter(param, self.adapt_datetime_iso) if isinstance(param, datetime) else param for param in parameters )
return super().execute(sql, new_param)
import sqlite3
# Only create a new tuple if we have datetime objects to transform
if any(isinstance(param, datetime) for param in parameters):
new_params = list(parameters)
for i, param in enumerate(parameters):
if isinstance(param, datetime):
new_params[i] = sqlite3.register_adapter(param, self.adapt_datetime_iso)
parameters = tuple(new_params)
return super().execute(sql, parameters)

@@ -1315,7 +1315,7 @@ def get_current_locale() -> str:
l10n_code = NSLocale.currentLocale()
l10n = NSLocale.localeIdentifier(l10n_code)
else:
l10n = locale.getdefaultlocale()[0] or ""
l10n = locale.getlocale()[1] or ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Incorrect index used for locale tuple and potential None handling issue

getlocale() returns (language code, encoding). Using index [1] returns only the encoding. Consider using [0] for language code and handle None case explicitly.

Comment on lines +17 to +18
pefile==2023.2.7 ; sys_platform == "win32" \
--hash=sha256:da185cd2af68c08a6cd4481f7325ed600a88f6a813bad9dea07ab3ef73d8d8d6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Please explain the reason for changing the pefile version.

A brief comment about compatibility or bug fixes would be helpful for future maintainers.

Suggested implementation:

# TODO: Document why pefile 2023.2.7 is required (compatibility/bug fixes?)
pefile==2023.2.7 ; sys_platform == "win32" \

The developer should replace the TODO comment with the actual reason for requiring pefile 2023.2.7, such as:

  • If it's due to a compatibility issue with other dependencies
  • If there are known bugs in newer versions
  • If it's required for specific Windows platform support

@gitofanindya gitofanindya force-pushed the Upograde-python-from-3.9.5-to-3.13.1 branch 2 times, most recently from 2eb4162 to 0509e6d Compare January 10, 2025 09:06
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

Attention: Patch coverage is 80.95238% with 4 lines in your changes missing coverage. Please review.

Project coverage is 49.26%. Comparing base (e57e0ab) to head (9b33ff8).

Files with missing lines Patch % Lines
nxdrive/client/local/base.py 66.66% 1 Missing ⚠️
nxdrive/dao/base.py 83.33% 1 Missing ⚠️
nxdrive/engine/engine.py 50.00% 1 Missing ⚠️
nxdrive/engine/watcher/remote_watcher.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5533      +/-   ##
==========================================
- Coverage   52.09%   49.26%   -2.84%     
==========================================
  Files          96       96              
  Lines       16094    16130      +36     
==========================================
- Hits         8384     7946     -438     
- Misses       7710     8184     +474     
Flag Coverage Δ
functional 40.96% <61.90%> (-1.93%) ⬇️
integration ?
unit 35.30% <66.66%> (-5.22%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gitofanindya gitofanindya force-pushed the Upograde-python-from-3.9.5-to-3.13.1 branch from c6ffe48 to 6958530 Compare January 10, 2025 09:21
@gitofanindya gitofanindya force-pushed the Upograde-python-from-3.9.5-to-3.13.1 branch 3 times, most recently from dff2071 to 796d25e Compare January 16, 2025 04:16
@gitofanindya gitofanindya force-pushed the Upograde-python-from-3.9.5-to-3.13.1 branch 6 times, most recently from 9f0ee91 to 9f5e9d8 Compare January 17, 2025 04:20
@gitofanindya gitofanindya force-pushed the Upograde-python-from-3.9.5-to-3.13.1 branch from 9f5e9d8 to 9b33ff8 Compare January 17, 2025 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants