Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect timestamp handling when filter_pushdown is enabled #81

Closed
2 tasks done
LMnet opened this issue Aug 26, 2024 · 1 comment · Fixed by #84
Closed
2 tasks done

Incorrect timestamp handling when filter_pushdown is enabled #81

LMnet opened this issue Aug 26, 2024 · 1 comment · Fixed by #84

Comments

@LMnet
Copy link

LMnet commented Aug 26, 2024

What happens?

It looks like the filter pushdown incorrectly converts dates into timestamps.

This is my DuckDB query:

set mysql_experimental_filter_pushdown=true;

select *
from mysql.infohash_update_log
where updated_on <= '2021-09-30'
limit 100;

The updated_on column has timestamp null type.

As a result I see this error message:

IO Error: Failed to run query "SELECT `updated_on`, `id`, `infohash`, `value` FROM `helpdesk`.`infohash_update_log` WHERE (`updated_on` <= '2021-09-29 11:00:00+00' AND `updated_on` IS NOT NULL) LIMIT 100": Incorrect TIMESTAMP value: '2021-09-29 11:00:00+00'

MySQL itself supports these syntaxes:

-- Date literal
`updated_on` <= '2021-09-29'

-- Timestamp literal
`updated_on` <= '2021-09-29 11:00:00'

I think it somehow relates to #36 But I tried the latest MySQL extension (with force install) and my problem persists:

+--------------+-----------------+--------------+
|extension_name|extension_version|installed_from|
+--------------+-----------------+--------------+
|mysql_scanner |4dd5963          |core          |
+--------------+-----------------+--------------+

To Reproduce

  1. Create a MySQL table with a column with timestamp type.
  2. Try to run a DuckDB query like
select *
from mysql.my_table
where timestamp_column <= '2021-09-30'; # any date here

OS:

linux

MySQL Version:

8.0.32

DuckDB Version:

1.0.0

DuckDB Client:

Python

Full Name:

Yury Badalyants

Affiliation:

Parrot Analytics

Have you tried this on the latest main branch?

  • I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

  • I agree
Mytherin added a commit that referenced this issue Sep 5, 2024
Fix #81: don't include timezone offset in timestamp literal when doing filter pushdown
@Mytherin
Copy link
Contributor

Mytherin commented Sep 5, 2024

Thanks for the report! I've pushed a fix in #84

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants
@LMnet @Mytherin and others