测试特殊字符无法导入

[lm520hy]

语法：cat customer.txt | curl --location-trusted -u root: -H "timeout:1200" -H "column_separator:|" -T - http://192.168.194.138:8030/api/tpcds/customer/_stream_load
无法导入的文本格式
322|AAAAAAAACEBAAAAA|367590|256|93369|2449888|2449858|Miss|Sunshine|Solomon|Y|15|2|1928|RÉUNION||[email protected]|2452392|
里面的特殊字符是É
报错
Reason: Unable to display. src line [Unable to display];

[ixzc]

Hello, what is your version number, please? I tested it on 2.0.0 release and 1.2.6 release, but I didn't find the problem you mentioned.、

[lm520hy]

你好
我的版本是：版本是apache-doris-2.0.0-bin-x64.tar.gz
数据来源是：TPC-DS自动生成的（处理后去掉最后"|"）
测试中发现直接复制出来，使用echo 可以导入数据，但是文本不可以导入
我的建表语句是
create table customer
(
c_customer_sk INT not null,
c_customer_id char(16) not null,
c_current_cdemo_sk INT ,
c_current_hdemo_sk INT ,
c_current_addr_sk INT ,
c_first_shipto_date_sk INT ,
c_first_sales_date_sk INT ,
c_salutation char(10) ,
c_first_name char(20) ,
c_last_name char(30) ,
c_preferred_cust_flag char(1) ,
c_birth_day INT ,
c_birth_month INT ,
c_birth_year INT ,
c_birth_country varchar(20) ,
c_login char(13) ,
c_email_address char(50) ,
c_last_review_date char(10)
)
DUPLICATE KEY(c_customer_sk)
DISTRIBUTED BY HASH(c_customer_sk) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);

customer01.txt

[ixzc]

Hello, for this question. First, you'd better take a look at the error link, which will give you the details. Based on the syntax you provided, the test was repeated. The following is the error link:

In addition, I found new problems: column_name[c_login], the length of input is too long than schema. You should modify the corresponding fields.

[lm520hy]

你好
我最后一张发出来的图片中有报错信息如下：

是使用的curl打印出来的，并且我的错误和你并不一致
关于你说的字段[c_login]的位置不对我使用EXCLE对应起来如下：

并且我试过使用varchar替代char来作为类型进行导入依然报错

[ixzc]

can you add my wechat ? (17625672480) So I can better respond to your question.

[zhbdesign]

用json类型应该可以，我们是遇到特殊符号的就转为json，但是默认的csv性能是最好的

[lm520hy]

周一去试一下 电脑没带回来 主要是报错直接退出了 使用json中间多了一些过程势必会影响性能
另外测试一下其他库这样数据使用seaturn导入会报错吗

[zhbdesign]

如果不是并发大量数据，json的性能也可以接受吧