[HUDI-8892] Introduce projection push down for payload mode

[TheR1sing3un]

In many wide table scenarios, there may be thousands of columns in a table, and there are multiple tasks processing different columns. When reading, the downstream only cares about some dimension columns. But now the payload mode does not support columns trimming, resulting in great performance regression when perform snapshot read on file slices with log files exist. This is because all columns of the base file are read, even though most of the columns are not needed by the user

Change Logs

1. support projection push down for payload mode

Describe context and summary for this change. Highlight if any code was copied.

Impact

Improve mor read performance

Risk level (write none, low medium or high below)

medium

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[hudi-bot]

CI report:

• b2fdb81 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build