You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Here is the schema which gives error: ["null",{"type":"map","values":["null","int","long","string","boolean","float","double"]},"int","long","string","boolean","float","double"]
See error
Expected behavior
This utility should handle such schema as well. While using the same avro file through some other online utilities, I was able to get proper JSON file with data. https://dataconverter.io/convert/avro-to-json
Screenshots
Test code written to debug the failure locally
Code section which gives error
Environment
OS: Mac and Ubuntu
Pulsar version: 3.3.0.5
Deployment: standalone and development env with pulsar and hudi connected
Additional context
This conversion is being done to source data from pulsar topic and to load it in HUDI. Another interesting thing is Pulsar storage sink connector is able to load this data in Bigquery without any issues.
The text was updated successfully, but these errors were encountered:
SimplytheVinay
changed the title
Issue with schema writer when encountered with specific type of schema
Issue with schema converter when encountered with specific type of schema
Aug 1, 2024
Bug
Issue with schema Converter when encountered with specific type of schema
Steps to reproduce the behavior:
https://github.com/streamnative/pulsar-io-lakehouse/blob/master/src/main/java/org/apache/pulsar/ecosystem/io/lakehouse/common/SchemaConverter.java#L153
["null",{"type":"map","values":["null","int","long","string","boolean","float","double"]},"int","long","string","boolean","float","double"]
Expected behavior
This utility should handle such schema as well. While using the same avro file through some other online utilities, I was able to get proper JSON file with data.
https://dataconverter.io/convert/avro-to-json
Screenshots
Test code written to debug the failure locally
Code section which gives error
Environment
Additional context
This conversion is being done to source data from pulsar topic and to load it in HUDI. Another interesting thing is Pulsar storage sink connector is able to load this data in Bigquery without any issues.
The text was updated successfully, but these errors were encountered: