Connector not working with vertica 12.0.4

[malayt]

hi,

I am not able to save dataframe to vertica using the v2 connector, can see parquet files are getting saved in staging area but from hdfs to vertica copy command throwing error, tried same with vertica 23.3.0 same error there as well.

below are my connection string :
val options = Map(
"host" ->"verticahostip",
"user" -> "etluser",
"db" -> "vmart",
"staging_fs_url" ->"hdfs://hdfshost:8020/user/hadoop/localtest",
"password" ->"password"
)
val VERTICA_SOURCE="com.vertica.spark.datasource.VerticaSource"
df.write.format(VERTICA_SOURCE)
.options(options + ("table" -> tableName)+("prevent_cleanup"->"true")+("file_permissions"->"777"))
.mode(mode)
.save()

version of tech used:
scala : 2.12.12,
connector : "vertica-spark" % "3.3.5-slim"
spark:3.3.0
hadoop: 3.3.6 (also tried with 3.3.2)

below command gives success message
SELECT VERIFY_HADOOP_CONF_DIR(); //success
SELECT HDFS_CLUSTER_CONFIG_CHECK();//success

below is the error that I am getting:

Opening write to file: hdfs://hdfshost:8020/user/hadoop/localtest/f90e0d97_ff65_44b9_b677_58144d673ba5/0-1.snappy.parquet
23/12/16 02:11:16 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:

23/12/16 02:11:25 INFO VerticaDistributedFilesystemWritePipe: The copy statement is:
COPY "dftest" ("col1") FROM 'hdfs://hdfshost:8020/user/hadoop/localtest/f90e0d97_ff65_44b9_b677_58144d673ba5/*.parquet' ON ANY NODE parquet REJECTED DATA AS TABLE "dftest_f90e0d97_ff65_44b9_b677_58144d673ba5_COMMITS" NO COMMIT
23/12/16 02:11:34 INFO VerticaDistributedFilesystemWritePipe: Performing copy from file store to Vertica
23/12/16 02:11:34 INFO VerticaDistributedFilesystemWritePipe: Committing data into Vertica.
23/12/16 02:11:34 INFO VerticaDistributedFilesystemWritePipe: Timed operation: Copy and commit data into Vertica -- took 13083 ms.

23/12/16 02:11:38 ERROR VerticaBatchReader: commit: Failed to copy rows into target table
performCopy: JDBC error when trying to copy
Error in commit step of write to Vertica. There was a failure copying data from the intermediary into Vertica.
Error when sending query
Unexpected SQL Error.
A generic JDBC error occurred

Caused by:
java.sql.SQLException: [Vertica]VJDBC ERROR: Failed to glob [hdfs://hdfshost:8020/user/hadoop/localtest/f90e0d97_ff65_44b9_b677_58144d673ba5/*.parquet] because of error: Seen exception:
URL: [http://hdfshost:8020/webhdfs/v1/user/hadoop/localtest/f90e0d97_ff65_44b9_b677_58144d673ba5/?op=LISTSTATUS&user.name=etluser]
HTTP response code: 404
HTTP response message: It looks like you are making an HTTP request to a Hadoop IPC port. This is not the correct port for the web interface on this daemon.

[malayt]

o/p of config check commands:
Is DefaultFS : [true]
Nameservice : []
Namenodes : [hdfshost:8020]
High Availability : [false]
RPC Encryption : [false]
Kerberos Authentication : [false]
HTTPS Only : [false]
[INFO] Unable to get a Libhdfs++ handle, using Webhdfs
[INFO] Checking connections to [hdfs:///]
vertica : [OK]
dbadmin : [OK]

[INFO] Checking connections to [http://hdfshost:9870]
[OK] Can make authenticated external curl connection
[INFO] Checking webhdfs
vertica : [OK]
dbadmin : [OK]

[!] hdfs_cluster_config_check : [PASS]