Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Map types not supported in FlintDataType #908

Open
engechas opened this issue Nov 14, 2024 · 1 comment
Open

[BUG] Map types not supported in FlintDataType #908

engechas opened this issue Nov 14, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@engechas
Copy link
Contributor

What is the bug?
FlintDataType doesn't support MAP types from Glue

24/11/14 18:18:54 ERROR DefaultOptimisticTransaction: Rolling back transient log due to transaction operation failure
java.lang.IllegalStateException: unsupported data type: MAP<STRING, STRING>
	at org.apache.spark.sql.flint.datatype.FlintDataType$.serializeField(FlintDataType.scala:166) ~[opensearch-spark-standalone_2.12-latest.jar:0.7.0-SNAPSHOT]
	at org.apache.spark.sql.flint.datatype.FlintDataType$.$anonfun$serializeJValue$1(FlintDataType.scala:121) ~[opensearch-spark-standalone_2.12-latest.jar:0.7.0-SNAPSHOT]

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Create a glue table with a map type
  2. Attempt to create a materialized view of the table

What is the expected behavior?
MAP types should be supported. Maps can be converted to OpenSearch objects

What is your host/environment?

  • OS: [e.g. iOS]
  • Version [e.g. 22]
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

@engechas engechas added bug Something isn't working untriaged labels Nov 14, 2024
@dblock dblock removed the untriaged label Dec 2, 2024
@dblock
Copy link
Member

dblock commented Dec 2, 2024

[Catch All Triage - 1, 2, 3]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants